Imaging support device, imaging device, imaging system, imaging support system, imaging support method, and program

Abstract

An imaging support device, imaging support system, imaging support method, imaging device, imaging system, and program are provided that support tracking of a target subject when imaging an imaging area including the target subject. [Solution] In a surveillance system, the functions of the CPU included in a surveillance camera attached to a rotation mechanism include an acquisition unit that acquires the amount of intra-image shift between a predetermined position in an image obtained by capturing an image of an imaging area using an imaging element and the position of a target subject image showing the target subject, and the focal length of the imaging device; a derivation unit that derives the amount of movement required to move the position of the target subject image to a specific position using a position adjustment unit that adjusts the position of the target subject image within the captured image based on the intra-image shift amount acquired by the acquisition unit, the focal length acquired by the acquisition unit, and information related to the pixel spacing of pixels in the imaging element; and an output unit that outputs the amount of movement derived by the derivation unit.

Description

[Technical field]

[0001] The present disclosure relates to an imaging support device, an imaging device, an imaging system, an imaging support system, an imaging support method, and a program. [Background technology]

[0002] JP 2017-215350 A discloses an image blur correction device including an output means for outputting a shake detection signal related to shake of the device, a detection means for detecting a subject from a captured image, a notification means for notifying the shooting state, and a control means for performing subject tracking and image blur correction using a correction means based on the tracking target position of the detected subject and the shake detection signal, wherein the control means determines whether to give priority to subject tracking or image blur correction depending on the shooting state.

[0003] JP 2017-126960 A discloses an image shake correction device comprising: a position detection means for detecting the position of a subject within a screen; a speed detection means for detecting the speed of the subject using the amount of movement of the subject position within the screen; a tracking amount calculation means for calculating a tracking amount which is a correction amount of a shift means for moving a subject image on a screen, which is necessary to move the subject to a target position within the screen, based on the subject position and subject speed detected immediately before shooting in continuous shooting; and a control means for driving the shift means based on the tracking amount.

[0004] Japanese Patent Application Laid-Open No. 2017-063340 discloses an image shake correction device that corrects image shake using a shake correction means based on a shake detection signal, and is characterized in that the image shake correction device includes an acquisition means that acquires subject information about a subject detected from a captured image, and a control means that sets a reference position of the subject within the shooting angle of view based on the subject information and controls the shake correction means so that the subject moves toward the reference position. Summary of the Invention

[0005] One embodiment of the technology disclosed herein provides an imaging support device, an imaging device, an imaging system, an imaging support system, an imaging support method, and a program that can assist in tracking a target subject when imaging an imaging area that includes the target subject. [Means for solving the problem]

[0006] A first aspect of the technology disclosed herein is an imaging support device that supports imaging by an imaging device including an imaging element, and includes: an acquisition unit that acquires an intra-image shift amount between a default position in an captured image obtained by imaging an imaging area including a target subject by the imaging element and a position of a target subject image indicating the target subject, and a focal length of the imaging device; a derivation unit that derives an amount of movement required to move the position of the target subject image to a specific position by a position adjustment unit that adjusts the position of the target subject image in the captured image, based on the intra-image shift amount acquired by the acquisition unit, the focal length acquired by the acquisition unit, and information regarding the pixel spacing of pixels in the imaging element; and an output unit that outputs the amount of movement derived by the derivation unit.

[0007] A second aspect of the technology of the present disclosure is an imaging support device according to the first aspect of the technology of the present disclosure, in which the position adjustment unit has a rotation mechanism to which an imaging device is attached and which can rotate the imaging device, and a shake correction unit that corrects shake caused by vibrations applied to the imaging device, and further includes a control unit that performs adjustment control to adjust the position of the target subject image within the captured image by operating at least one of the rotation mechanism and the shake correction unit based on the amount of movement.

[0008] A third aspect of the technology disclosed herein is an imaging support device related to the second aspect of the technology disclosed herein, in which the amount of movement is determined based on a first amount of movement required to adjust the position of the target subject image by the rotation mechanism and a second amount of movement required to adjust the position of the target subject image by the shake correction unit.

[0009] A fourth aspect of the technology of the present disclosure is an imaging support device according to the second or third aspect of the technology of the present disclosure, in which the shake correction unit has a shake correction element which is at least one of a lens and an imaging element that corrects shake by moving in response to vibration, and at a specific position, the shake correction element is located at the center of the movable range of the shake correction element.

[0010] A fifth aspect of the technology disclosed herein is an imaging support device related to the fourth aspect of the technology disclosed herein, in which the acquisition unit further acquires the sensitivity of the shake correction unit, and the derivation unit derives the shake correction element movement amount required to move the current position to the central position as a second movement amount required for adjusting the position of the target subject image by the shake correction unit based on the shake correction element deviation amount between the central position of the movable range and the current position of the shake correction element and the sensitivity acquired by the acquisition unit.

[0011] A sixth aspect of the technology disclosed herein is an imaging support device according to any one of the third to fifth aspects of the technology disclosed herein, wherein the first movement amount is determined based on a value obtained by dividing the product of the amount of shift within the image and the pixel spacing by the focal length.

[0012] A seventh aspect of the technology of the present disclosure is an imaging support device relating to any one of the third to sixth aspects of the technology of the present disclosure, wherein the movement amount is obtained by combining a first movement amount and a second movement amount derived by a derivation unit.

[0013] An eighth aspect of the technology disclosed herein is an imaging support device according to any one of the second to seventh aspects of the technology disclosed herein, in which the control unit performs shake correction by the shake correction unit and adjustment control in a time-division manner.

[0014] A ninth aspect of the technology of the present disclosure is an imaging support device according to the eighth aspect of the technology of the present disclosure, in which the control unit causes the shake correction unit to correct shake while the imaging device is being rotated by the rotation mechanism, and performs adjustment control while rotation of the imaging device by the rotation mechanism is stopped.

[0015] A tenth aspect of the technology of the present disclosure is an imaging support device according to the eighth or ninth aspect of the technology of the present disclosure, in which the adjustment control adjusts the position of the target subject image by a shake correction unit after adjusting the position of the target subject image by a rotation mechanism.

[0016] An eleventh aspect of the technology disclosed herein is an imaging support device relating to any one of the second to tenth aspects of the technology disclosed herein, in which the rotation mechanism is a two-axis rotation mechanism capable of rotating the imaging device in a first direction and a second direction intersecting the first direction, and the shake correction unit is at least one of an optical shake correction mechanism and an electronic shake correction unit.

[0017] A twelfth aspect of the technology disclosed herein is an imaging support device related to the eleventh aspect of the technology disclosed herein, in which the optical shake correction mechanism is at least one of a lens movement type shake correction mechanism and an image sensor movement type shake correction mechanism.

[0018] A thirteenth aspect of the technology of the present disclosure is an imaging support device according to any one of the first to twelfth aspects of the technology of the present disclosure, in which the acquisition unit further acquires information relating to pixel intervals.

[0019] A fourteenth aspect of the technology of the present disclosure is an imaging support device according to any one of the first to thirteenth aspects of the technology of the present disclosure, in which the output unit outputs the amount of movement to an outside.

[0020] A fifteenth aspect of the technology disclosed herein is an imaging support device according to any one of the first to fourteenth aspects of the technology disclosed herein, wherein the amount of movement is determined based on the movement speed of the target subject when the target subject is moving.

[0021] A sixteenth aspect of the technology of the present disclosure is an imaging support device according to the fifteenth aspect of the technology of the present disclosure, in which the moving speed is a plurality of speeds obtained by decomposing the moving speed into a plurality of different directions.

[0022] A 17th aspect of the technology of the present disclosure includes an imaging support device relating to any one of the first to sixteenth aspects of the technology of the present disclosure, and an imaging element, wherein the imaging element is an imaging device in which imaging support is provided by the imaging support device.

[0023] An 18th aspect of the technology of the present disclosure is an imaging system including an imaging device related to the 17th aspect of the technology of the present disclosure, and a control device that performs at least one of control to cause a display unit to display an image reflecting the adjustment result of the position of the target subject image based on the amount of movement derived by the derivation unit, and control to store image data showing the image reflecting the adjustment result in a memory unit.

[0024] A 19th aspect of the technique of the present disclosure is an imaging support device that supports imaging by an imaging device including an imaging element, and includes: an acquisition unit that acquires an intra-image shift amount between a default position in an captured image obtained by capturing an imaging area including a target subject by the imaging element and a position of a target subject image indicating the target subject, and a focal length of the imaging device; a derivation unit that derives an amount of movement required to move the position of the target subject image to a specific position by a position adjustment unit that adjusts the position of the target subject image in the captured image based on information about the intra-image shift amount acquired by the acquisition unit, the focal length acquired by the acquisition unit, and the pixel spacing of pixels in the imaging element; and an output unit that outputs the amount of movement derived by the derivation unit, wherein the position adjustment unit has a rotation mechanism to which the imaging device is attached and capable of rotating the imaging device, and a shake correction unit that corrects shake caused by vibrations applied to the imaging device, and the imaging system includes: an imaging support device that further includes a control unit that performs adjustment control to adjust the position of the target subject image in the captured image by operating at least one of the rotation mechanism and the shake correction unit based on the amount of movement; and an imaging element.

[0025] A twentieth aspect of the technology of the present disclosure includes an imaging support device relating to any one of the first to sixteenth aspects of the technology of the present disclosure and a position adjustment unit, and the derivation unit included in the imaging support device is an imaging support system that derives the amount of movement.

[0026] A 21st aspect of the technology of the present disclosure is an imaging support method for supporting imaging by an imaging device including an image sensor, the imaging support method including: acquiring an intra-image shift amount between a default position in an captured image obtained by capturing an imaging area including a target subject by the image sensor and a position of a target subject image showing the target subject, and a focal length; deriving an amount of movement required to move the position of the target subject image to a specific position by a position adjustment unit that adjusts the position of the target subject image in the captured image, based on the acquired intra-image shift amount, the acquired focal length, and the pixel spacing of pixels of the image sensor; and outputting the derived amount of movement.

[0027] A 22nd aspect of the technology of the present disclosure is a program for causing a computer to execute a process for assisting in imaging by an imaging device including an image sensor, the process including acquiring an amount of intra-image shift between a default position in an captured image obtained by imaging an imaging area including a target subject by the image sensor and a position of a target subject image showing the target subject, and a focal length, deriving an amount of movement required to move the position of the target subject image to a specific position by a position adjustment unit that adjusts the position of the target subject image in the captured image based on the acquired amount of intra-image shift, the acquired focal length, and the pixel spacing of pixels of the image sensor, and outputting the derived amount of movement. [Brief description of the drawings]

[0028] [Figure 1] 1 is a schematic configuration diagram showing an example of the configuration of a monitoring system according to an embodiment. [Diagram 2] 1 is a perspective view showing an example of the appearance of a surveillance camera according to an embodiment; [Diagram 3] 1 is a perspective view showing an example of the appearance of a surveillance camera according to an embodiment; [Figure 4] 1 is a block diagram showing an example of the configuration of an optical system and an electrical system of a surveillance camera according to an embodiment. [Diagram 5] 2 is a block diagram showing an example of a configuration of an electrical system of a management device and a turning mechanism according to the embodiment. FIG. [Figure 6]2 is a functional block diagram showing an example of functions of a CPU included in the surveillance camera according to the embodiment. FIG. [Figure 7] FIG. 2 is a conceptual diagram illustrating an example of a captured image according to the embodiment. [Figure 8] 5 is a conceptual diagram illustrating derivation of a first movement amount according to the embodiment. FIG. [Figure 9] 5A and 5B are conceptual diagrams illustrating a movable range of a shake correction element according to an embodiment. [Figure 10] 5A and 5B are conceptual diagrams illustrating a movable range of a shake correction element according to an embodiment. [Figure 11] FIG. 4 is a conceptual diagram showing an example of a screen display displayed by a display unit according to the embodiment. [Figure 12] 11A and 11B are conceptual diagrams illustrating a position adjustment process according to the embodiment. [Figure 13] 10 is a flowchart illustrating an example of the flow of a position adjustment process according to the embodiment. [Figure 14] FIG. 2 is a conceptual diagram illustrating an example of a display unit of the surveillance camera according to the embodiment. [Figure 15] FIG. 11 is a functional block diagram showing a modified example of the functions of the CPU included in the surveillance camera according to the embodiment. [Figure 16] 1 is a schematic configuration diagram showing an example of the configuration of a surveillance camera according to an embodiment. [Figure 17] FIG. 4 is a conceptual diagram showing an example of a screen display displayed by a display unit according to the embodiment. [Figure 18] FIG. 11 is a functional block diagram showing a modified example of the functions of the CPU included in the surveillance camera according to the embodiment. [Figure 19] 11A and 11B are conceptual diagrams illustrating a position adjustment process according to the embodiment. [Figure 20] 11A and 11B are conceptual diagrams illustrating a position adjustment process according to the embodiment. [Figure 21A] 10 is a flowchart illustrating an example of the flow of a position adjustment process according to the embodiment. [Figure 21B] 10 is a flowchart showing an example of the flow of a speed determination process according to the embodiment. [Figure 22]11 is a conceptual diagram showing an example of a manner in which a position adjustment program according to an embodiment is installed in a computer in a surveillance camera from a storage medium on which the position adjustment program is stored. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] An example of an embodiment of the technology of the present disclosure will be described with reference to the accompanying drawings.

[0030] First, the terms used in the following description will be explained.

[0031] CPU is an abbreviation for "Central Processing Unit". GPU is an abbreviation for "Graphics Processing Unit". ASIC is an abbreviation for "Application Specific Integrated Circuit". PLD is an abbreviation for "Programmable Logic Device". FPGA is an abbreviation for "Field-Programmable Gate Array". AFE is an abbreviation for "Analog Front End". DSP is an abbreviation for "Digital Signal Processor". SoC is an abbreviation for "System-on-a-chip". SSD is an abbreviation for "Solid State Drive". USB is an abbreviation for "Universal Serial Bus". HDD is an abbreviation for "Hard Disk Drive". EEPROM is an abbreviation for "Electrically Erasable and Programmable Read Only Memory". EL is an abbreviation for "Electro-Luminescence". A / D is an abbreviation for "Analog / Digital". I / F is an abbreviation for "Interface". UI is an abbreviation for "User Interface". WAN is an abbreviation for "Wide Area Network". ISP is an abbreviation for "Image Signal Processor". CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor". CCD is an abbreviation for "Charge Coupled Device". SWIR is an abbreviation for "Short-wavelength infrared".

[0032] In the description of this specification, "vertical" refers to vertical in the sense of including a generally acceptable error in the technical field to which the technology of the present disclosure belongs, in addition to completely vertical. In the description of this specification, "horizontal" refers to horizontal in the sense of including a generally acceptable error in the technical field to which the technology of the present disclosure belongs, in addition to completely horizontal. In the description of this specification, "parallel" refers to parallel in the sense of including a generally acceptable error in the technical field to which the technology of the present disclosure belongs, in addition to completely parallel. In the description of this specification, "vertical" refers to vertical in the sense of including a generally acceptable error in the technical field to which the technology of the present disclosure belongs, in addition to completely vertical. In the description of this specification, "same" refers to same in the sense of including a generally acceptable error in the technical field to which the technology of the present disclosure belongs, in addition to completely same.

[0033] [First embodiment] 1, a surveillance system 2 includes a surveillance camera 10 and a management device 11. The surveillance system 2 is an example of an "imaging system" and an "imaging support system" according to the technology of the present disclosure, and the surveillance camera 10 is an example of an "imaging device" according to the technology of the present disclosure.

[0034] The surveillance camera 10 is installed on a pillar, a wall, or a part of a building (e.g., a rooftop) indoors or outdoors via a rotation mechanism 16 described later, and captures an image of a subject to be monitored, thereby generating a moving image. The moving image includes a plurality of frames of images obtained by capturing the image. The surveillance camera 10 transmits the moving image obtained by capturing the image to the management device 11 via a communication line 12.

[0035] The management device 11 includes a display 13 and a secondary storage device 14. Examples of the display 13 include a liquid crystal display and an organic EL display. The display 13 is an example of a "display unit (display)" according to the technology of the present disclosure.

[0036] An example of the secondary storage device 14 is an HDD. The secondary storage device 14 may be a non-volatile memory such as a flash memory, an SSD, or an EEPROM, instead of an HDD. The secondary storage device 14 is an example of a "storage unit (storage device)" according to the technology of the present disclosure.

[0037] In the management device 11, the moving images transmitted by the surveillance camera 10 are received, and the received moving images are displayed on the display 13 and / or stored in the secondary storage device .

[0038] The surveillance camera 10 is attached to the rotation mechanism 16. The rotation mechanism 16 allows the surveillance camera 10 to rotate. Specifically, the rotation mechanism 16 is a two-axis rotation mechanism that allows the surveillance camera 10 to rotate in a first direction and a second direction intersecting the first direction. As shown in FIG. 2 as an example, the rotation mechanism 16 allows the surveillance camera 10 to rotate in a rotation direction (hereinafter also referred to as the "pitch direction") about the pitch axis PA as the central axis. As shown in FIG. 3 as an example, the rotation mechanism 16 allows the surveillance camera 10 to rotate in a rotation direction (hereinafter referred to as the "yaw direction") about the yaw axis YA as the central axis. The rotation mechanism 16 is an example of a "rotation mechanism" according to the technology of the present disclosure. The "pitch direction" is an example of a "first direction" according to the technology of the present disclosure, and the yaw direction is an example of a "second direction" according to the technology of the present disclosure. In this embodiment, a two-axis rotation mechanism is exemplified as the rotation mechanism 16, but the technology of the present disclosure is not limited to this, and the technology of the present disclosure can also be applied when a three-axis rotation mechanism is applied.

[0039] As an example, as shown in FIG. 4, the surveillance camera 10 includes an optical system 15 and an image sensor 25. The image sensor 25 is located behind the optical system 15. The optical system 15 includes an objective lens 15A and a lens group 15B. The objective lens 15A and the lens group 15B are arranged in this order along the optical axis OA of the optical system 15 from the monitored object side (object side) to the light receiving surface 25A side (image side) of the image sensor 25. The lens group 15B includes a zoom lens 15B2 and the like. The zoom lens 15B2 is supported by a moving mechanism 21 so as to be movable along the optical axis OA. The moving mechanism 21 moves the zoom lens 15B2 along the optical axis OA in response to the power applied from a motor (not shown) for the zoom lens. The lens group 15B also includes an anti-vibration lens 15B1. The anti-vibration lens 15B1 moves in a direction perpendicular to the optical axis of the anti-vibration lens 15B1 in response to the power applied.

[0040] The optical system 15 configured in this manner causes the monitored object light, which indicates the monitored object, to form an image on the light receiving surface 25A. The imaging element 25 is an example of the "imaging element" according to the technology of the present disclosure.

[0041] Incidentally, vibrations imparted to surveillance camera 10 include, if outdoors, vibrations due to passing cars, vibrations due to wind, vibrations due to road construction, etc., and if indoors, vibrations due to the operation of an air conditioner and vibrations due to people entering and exiting. Vibrations imparted to surveillance camera 10 also include vibrations while surveillance camera 10 is being rotated by rotation mechanism 16, and vibrations when the rotation operation by rotation mechanism 16 is started or stopped. Therefore, in surveillance camera 10, shaking occurs due to vibrations imparted to surveillance camera 10 (hereinafter also simply referred to as "vibration").

[0042] In this embodiment, "shake" refers to a phenomenon in which the subject image on the light receiving surface 25A of the surveillance camera 10 fluctuates due to a change in the positional relationship between the optical axis OA and the light receiving surface 25A. In other words, "shake" can also be said to be a phenomenon in which the optical image formed on the light receiving surface 25A fluctuates due to the optical axis OA tilting caused by vibrations applied to the surveillance camera 10. The fluctuation of the optical axis OA means, for example, that the optical axis OA tilts with respect to a reference axis (for example, the optical axis OA before the shake occurs). Hereinafter, the shake caused by vibrations will also be simply referred to as "shake".

[0043] Therefore, the surveillance camera 10 includes a shake correction unit 51. The shake correction unit 51 is an example of a "shake correction component" according to the technology of the present disclosure. The shake correction unit 51 includes a mechanical shake correction unit 29 and an electronic shake correction unit 33. The shake correction unit 51 corrects shake. The mechanical shake correction unit 29 is an example of an "optical shake correction mechanism" according to the technology of the present disclosure. The mechanical shake correction unit 29 is a mechanism that applies power generated by a driving source such as a motor (e.g., a voice coil motor) to the shake-proof lens to move the shake-proof lens in a direction perpendicular to the optical axis of the imaging optical system, thereby correcting shake. The electronic shake correction unit 33 corrects shake by performing image processing on the captured image based on the amount of shake. In other words, the shake correction unit 51 performs shake correction mechanically or electronically with a hardware configuration and / or software configuration. Here, mechanical shake correction refers to shake correction achieved by mechanically moving a shake correction element such as an anti-vibration lens and / or an image sensor using power generated by a driving source such as a motor (e.g., a voice coil motor), and electronic shake correction refers to shake correction achieved, for example, by image processing performed by a processor. Note that in this embodiment, "shake correction" includes the meaning of reducing shake in addition to eliminating shake.

[0044] The mechanical shake correction unit 29 includes an anti-vibration lens 15B1, an actuator 17, a driver 23, and a position detection sensor 39.

[0045] Various well-known methods can be used to correct shake by the mechanical shake correction unit 29. In this embodiment, the method of correcting shake is to move the vibration-proof lens 15B1 based on the amount of shake detected by a shake amount detection sensor 40 (described later). Specifically, the vibration is corrected by moving the vibration-proof lens 15B1 in a direction that cancels the shake and by an amount that cancels the shake.

[0046] An actuator 17 is attached to the anti-vibration lens 15B1. The actuator 17 is a shift mechanism equipped with a voice coil motor, and by driving the voice coil motor, the anti-vibration lens 15B1 is moved in a direction perpendicular to the optical axis of the anti-vibration lens 15B1. Note that, although a shift mechanism equipped with a voice coil motor is used as the actuator 17 here, the technology of the present disclosure is not limited to this, and other power sources such as a stepping motor or a piezoelectric element may be applied instead of the voice coil motor.

[0047] The actuator 17 is controlled by a driver 23. When the actuator 17 is driven under the control of the driver 23, the position of the vibration-proof lens 15B1 is mechanically changed with respect to the optical axis OA.

[0048] The position detection sensor 39 detects the current position of the anti-vibration lens 15B1 and outputs a position signal indicating the detected current position. Here, a device including a Hall element is used as an example of the position detection sensor 39. Here, the current position of the anti-vibration lens 15B1 refers to the current position in the anti-vibration lens two-dimensional plane. The anti-vibration lens two-dimensional plane refers to a two-dimensional plane perpendicular to the optical axis of the anti-vibration lens 15B1. Note that in this embodiment, a device including a Hall element is used as an example of the position detection sensor 39, but the technology of the present disclosure is not limited to this, and a magnetic sensor, a photo sensor, or the like may be used instead of the Hall element.

[0049] The surveillance camera 10 includes a computer 19, a DSP 31, an image memory 32, an electronic shake correction unit 33, a communication I / F 34, a shake amount detection sensor 40, and a UI device 43. The computer 19 includes a memory 35, a storage 36, and a CPU 37. The electronic shake correction unit 33 is an example of an "electronic shake correction unit" according to the technology of the present disclosure. The CPU 37 is an example of an "imaging support device" according to the technology of the present disclosure.

[0050] The imaging element 25, the DSP 31, the image memory 32, the electronic shake correction unit 33, the communication I / F 34, the memory 35, the storage 36, the CPU 37, the shake amount detection sensor 40, and the UI device 43 are connected to a bus 38. The driver 23 is also connected to the bus 38. Note that, in the example shown in Fig. 4, for convenience of illustration, one bus is shown as the bus 38, but multiple buses may be used. The bus 38 may be a serial bus, or may be a parallel bus including a data bus, an address bus, a control bus, and the like.

[0051] The memory 35 temporarily stores various information and is used as a work memory. An example of the memory 35 is a RAM, but is not limited thereto and may be another type of storage device. The storage 36 is a non-volatile storage device. Here, a flash memory is used as an example of the storage 36. The flash memory is merely an example, and the storage 36 may be, for example, various non-volatile memories such as a magnetoresistive memory and / or a ferroelectric memory instead of or in addition to the flash memory. The non-volatile storage device may be an EEPROM, an HDD, and / or an SSD. The storage 36 stores various programs for the surveillance camera 10. The CPU 37 reads the various programs from the storage 36 and executes the read programs on the memory 35 to control the entire surveillance camera 10.

[0052] The imaging element 25 is a CMOS image sensor. The imaging element 25 captures an image of a monitoring target at a default frame rate under the instruction of the CPU 37. The "default frame rate" here refers to, for example, several tens of frames per second to several hundreds of frames per second. The imaging element 25 itself may also have a built-in control device (imaging element control device), in which case the imaging element control device performs detailed control inside the imaging element 25 in response to an imaging instruction output by the CPU 37. The imaging element 25 may also capture an image of a target subject at a default frame rate under the instruction of the DSP 31, in which case the imaging element control device performs detailed control inside the imaging element 25 in response to an imaging instruction output by the DSP 31. The DSP 31 may also be called an ISP.

[0053] The light receiving surface 25A is formed by a plurality of photosensitive pixels (not shown) arranged in a matrix. In the image sensor 25, each photosensitive pixel is exposed to light, and photoelectric conversion is performed for each photosensitive pixel. The charge obtained by performing photoelectric conversion for each photosensitive pixel is an analog image signal indicating the monitoring target. Here, a plurality of photoelectric conversion elements (for example, photoelectric conversion elements with color filters arranged thereon) having sensitivity to visible light are adopted as the plurality of photosensitive pixels. In the image sensor 25, a photoelectric conversion element having sensitivity to R (red) light (for example, a photoelectric conversion element with an R filter corresponding to R arranged thereon), a photoelectric conversion element having sensitivity to G (green) light (for example, a photoelectric conversion element with a G filter corresponding to G arranged thereon), and a photoelectric conversion element having sensitivity to B (blue) light (for example, a photoelectric conversion element with a B filter corresponding to B arranged thereon) are adopted as the plurality of photoelectric conversion elements. In the surveillance camera 10, imaging based on visible light (for example, light on the short wavelength side of about 700 nanometers or less) is performed by using these photosensitive pixels. However, the present embodiment is not limited thereto, and imaging may be performed based on infrared light (for example, light having a wavelength longer than about 700 nanometers). In this case, a plurality of photoelectric conversion elements having sensitivity to infrared light may be used as a plurality of photosensitive pixels. In particular, for imaging of SWIR, for example, an InGaAs sensor and / or a Simulation of Type-II Quantum Well (T2SL) sensor may be used.

[0054] The imaging element 25 performs signal processing such as A / D conversion on the analog imaging signal to generate a digital image, which is a digital imaging signal. The imaging element 25 is connected to the DSP 31 via a bus 38, and outputs the generated digital image to the DSP 31 via the bus 38 in frame units. Here, the digital image is an example of a "captured image" according to the technology of the present disclosure.

[0055] Note that, although a CMOS image sensor is described here as an example of the imaging element 25, the technology of the present disclosure is not limited thereto, and a CCD image sensor may be applied as the imaging element 25. In this case, the imaging element 25 is connected to the bus 38 via an AFE (not shown) with a built-in CCD driver, and the AFE generates a digital image by performing signal processing such as A / D conversion on the analog imaging signal obtained by the imaging element 25, and outputs the generated digital image to the DSP 31. The CCD image sensor is driven by a CCD driver built into the AFE. Of course, the CCD driver may be provided separately.

[0056] The DSP 31 performs various types of digital signal processing on the digital image, such as demosaic processing, noise removal processing, tone correction processing, and color correction processing.

[0057] The DSP 31 outputs the digital image after digital signal processing for each frame to the image memory 32. The image memory 32 stores the digital image from the DSP 31. Note that, hereinafter, for convenience of explanation, the digital image stored in the image memory 32 is also referred to as a "captured image."

[0058] The shake amount detection sensor 40 is, for example, a device including a gyro sensor, and detects the amount of shake of the surveillance camera 10. In other words, the shake amount detection sensor 40 detects the amount of shake in each of a pair of axial directions. The gyro sensor detects the amount of rotational shake around each axis (see FIG. 1), the pitch axis PA, the yaw axis YA, and the roll axis RA (axis parallel to the optical axis OA). The shake amount detection sensor 40 detects the amount of shake of the surveillance camera 10 by converting the amount of rotational shake around the pitch axis PA and the amount of rotational shake around the yaw axis YA detected by the gyro sensor into the amount of shake in a two-dimensional plane parallel to the pitch axis PA and the yaw axis YA.

[0059] Here, a gyro sensor is given as an example of the shake amount detection sensor 40, but this is merely an example, and the shake amount detection sensor 40 may be an acceleration sensor. The acceleration sensor detects the amount of shake in a two-dimensional plane parallel to the pitch axis PA and the yaw axis YA. The shake amount detection sensor 40 outputs the detected shake amount to the CPU 37.

[0060] In addition, although an example in which the shake amount is detected by a physical sensor called the shake amount detection sensor 40 is given here, the technology of the present disclosure is not limited to this. For example, a motion vector obtained by comparing captured images stored in the image memory 32 one after the other in a time series may be used as the shake amount. Also, the shake amount to be finally used may be derived based on the shake amount detected by the physical sensor and the motion vector obtained by image processing.

[0061] The CPU 37 acquires the amount of shake detected by the shake amount detection sensor 40, and controls the mechanical shake correction unit 29 and the electronic shake correction unit 33 based on the acquired amount of shake. The amount of shake detected by the shake amount detection sensor 40 is used for shake correction by each of the mechanical shake correction unit 29 and the electronic shake correction unit 33. The mechanical shake correction unit 29 and the electronic shake correction unit 33 correct the shake according to the amount of shake detected by the shake amount detection sensor 40.

[0062] The electronic shake correction unit 33 is a device including an ASIC. The electronic shake correction unit 33 performs image processing on the captured image in the image memory 32 based on the amount of shake detected by the shake amount detection sensor 40, thereby correcting the shake.

[0063] Note that, here, a device including an ASIC is exemplified as the electronic shake correction unit 33, but the technology of the present disclosure is not limited thereto, and may be, for example, a device including an FPGA or a PLD. Also, for example, the electronic shake correction unit 33 may be a device including two or more of an ASIC, an FPGA, and a PLD. Also, a computer including a CPU, a storage, and a memory may be adopted as the electronic shake correction unit 33. The CPU may be a single one or a plurality of CPUs. Also, the electronic shake correction unit 33 may be realized by a combination of a hardware configuration and a software configuration.

[0064] The communication I / F 34 is, for example, a network interface, and controls the transmission of various information between the management device 11 and the monitoring camera 10 via a network. An example of the network is a WAN such as the Internet or a public communication network. The communication I / F 34 manages communication between the monitoring camera 10 and the management device 11.

[0065] The UI device 43 includes a reception device 43A and a display 43B. The reception device 43A is, for example, a hard key or a touch panel, and receives various instructions from a user of the monitoring system 2 (hereinafter, also simply referred to as a "user"). The CPU 37 acquires the various instructions accepted by the reception device 43A and operates according to the acquired instructions.

[0066] The display 43B displays various information under the control of the CPU 37. Examples of the various information displayed on the display 43B include the contents of various instructions accepted by the accepting device 43A, captured images, and the like.

[0067] 5 as an example, the rotation mechanism 16 includes a yaw axis rotation mechanism 71, a pitch axis rotation mechanism 72, a motor 73, a motor 74, a driver 75, and a driver 76. The yaw axis rotation mechanism 71 rotates the surveillance camera 10 in the yaw direction. The motor 73 generates power by being driven under the control of the driver 75. The yaw axis rotation mechanism 71 rotates the surveillance camera 10 in the yaw direction by receiving the power generated by the motor 73. The motor 74 generates power by being driven under the control of the driver 76. The pitch axis rotation mechanism 72 rotates the surveillance camera 10 in the pitch direction by receiving the power generated by the motor 74.

[0068] As an example, as shown in Fig. 5, the management device 11 includes a display 13, a secondary storage device 14, a control device 60, a receiving device 62, and communication I / Fs 66 to 68. The control device 60 includes a CPU 60A, a storage 60B, and a memory 60C. The receiving device 62, the display 13, the CPU 60A, the storage 60B, the memory 60C, and the communication I / Fs 66 to 68 are each connected to a bus 70. Note that, in the example shown in Fig. 5, for convenience of illustration, one bus is shown as the bus 70, but multiple buses may be used. The bus 70 may be a serial bus, or may be a parallel bus including a data bus, an address bus, and a control bus.

[0069] The memory 60C temporarily stores various information and is used as a work memory. An example of the memory 60C is a RAM, but is not limited thereto and may be another type of storage device. The storage 60B is a non-volatile storage device. Here, a flash memory is adopted as an example of the storage 60B. The flash memory is merely an example, and the storage 60B may be, for example, various non-volatile memories such as a magnetoresistive memory and / or a ferroelectric memory instead of or in addition to the flash memory. The non-volatile storage device may be an EEPROM, an HDD, and / or an SSD. The storage 60B stores various programs for the management device 11 (hereinafter simply referred to as "programs for the management device"). The CPU 60A reads out the programs for the management device from the storage 60B and executes the read programs for the management device on the memory 60C to control the entire management device 11.

[0070] The communication I / F 66 is, for example, a network interface. The communication I / F 66 is communicatively connected to the communication I / F 34 of the surveillance camera 10 via a network, and controls the transmission of various information between the surveillance camera 10. For example, the communication I / F 66 requests the surveillance camera 10 to transmit a captured image, and receives the captured image transmitted from the communication I / F 34 of the surveillance camera 10 in response to the request to transmit the captured image.

[0071] The communication I / Fs 67 and 68 are, for example, network interfaces. The communication I / F 67 is communicatively connected to a driver 75 via a network. The CPU 60A controls the motor 73 via the communication I / F 67 and the driver 75, thereby controlling the rotation operation of the yaw axis rotation mechanism 71. The communication I / F 68 is communicatively connected to a driver 76 via a network. The CPU 60A controls the motor 74 via the communication I / F 68 and the driver 76, thereby controlling the rotation operation of the pitch axis rotation mechanism 72.

[0072] The receiving device 62 is, for example, a keyboard, a mouse, a touch panel, etc., and receives various instructions from a user, etc. The CPU 60A acquires the various instructions accepted by the receiving device 62, and operates according to the acquired instructions.

[0073] The display 13 displays various information under the control of the CPU 60 A. Examples of the various information displayed on the display 13 include the contents of various instructions accepted by the accepting device 62 and captured images received by the communication I / F 66.

[0074] The secondary storage device 14 stores various types of information under the control of the CPU 60A. Examples of the various types of information stored in the secondary storage device 14 include captured images received by the communication I / F 66.

[0075] In this way, the control device 60 controls the display 13 to display the captured image received by the communication I / F 66, and controls the secondary storage device 14 to store the captured image received by the communication I / F 66. The captured image displayed on the display 13 is an example of an "image reflecting the result of adjusting the position of the target subject image" according to the technology of the present disclosure. The captured image stored in the secondary storage device 14 is an example of "image data" according to the technology of the present disclosure.

[0076] Note that, here, the captured image is displayed on the display 13, and the captured image received by the communication I / F 66 is stored in the secondary storage device 14, but the technology of the present disclosure is not limited to this. For example, the captured image may be either displayed on the display 13 or stored in the secondary storage device 14.

[0077] Meanwhile, the surveillance camera 10 is provided with a function for tracking a target subject (hereinafter also referred to as a "tracking function"). When tracking a target subject using the tracking function, it is preferable to align the position of a target subject image showing a target subject (e.g., a specific person) included in the imaging area to a preset position in the captured image in preparation for the movement of the target subject. For example, if the position of the target subject image is aligned to the center position of the captured image, it becomes possible to respond to changes in the position of the target subject image in various directions.

[0078] Therefore, in order to align the position of the target subject image in the captured image with a default position in the captured image, as shown in FIG. 6 as an example, a position adjustment program 36A is stored in the storage 36, and the position adjustment program 36A is executed by the CPU 37. Specifically, the CPU 37 reads the position adjustment program 36A from the storage 36 and executes the read position adjustment program 36A on the memory 35, thereby functioning as an imaging support device that supports imaging by the surveillance camera 10 including the imaging element 25. In this way, the CPU 37 functions as an imaging support device to support imaging under a condition in which the position of the target subject image by the surveillance camera 10 is set to the center position of the captured image (hereinafter also referred to as the "image center position"). The image center position is an example of the "default position" and "specific position" according to the technology of the present disclosure.

[0079] In order to realize support for imaging by the surveillance camera 10 under a condition in which the position of the target subject image is set to a specific position, the surveillance camera 10 is equipped with a position adjustment unit 52. The position adjustment unit 52 has a rotation mechanism 16 and a shake correction unit 51, and adjusts the position of the target subject image in the captured image. The CPU 37 controls the position adjustment unit 52 to support imaging by the surveillance camera 10. The position adjustment unit 52 is an example of a "position adjustment device" according to the technology of the present disclosure.

[0080] Furthermore, CPU 37 derives the amount of movement required for position adjustment unit 52 to move the position of the target subject image to the image center position based on various information. CPU 37 also outputs the derived amount of movement. By controlling position adjustment unit 52 based on the amount of movement, the position of the target subject image can be aligned to the image center position, and tracking of the target subject is supported.

[0081] The CPU 37 executes the position adjustment program 36A on the memory 35 to operate as an acquisition unit 37A, a derivation unit 37B, a control unit 37C, an output unit 37D, and a determination unit 37E. The acquisition unit 37A is an example of an "acquisition unit" according to the technology of the present disclosure. The derivation unit 37B is an example of a "derivation unit" according to the technology of the present disclosure. The control unit 37C is an example of a "control unit" according to the technology of the present disclosure. The output unit 37D is an example of an "output unit" according to the technology of the present disclosure. The CPU 37 is an example of a "processor" according to the technology of the present disclosure, and the memory 35 is an example of a "memory" according to the technology of the present disclosure.

[0082] The determination unit 37E acquires a captured image from the image memory 32, and performs image recognition on the target subject image for the acquired captured image. The image recognition dictionary 36B is stored in the storage 36. Target subject images (e.g., images showing specific objects) to be image recognition targets are registered in the image recognition dictionary 36B. The determination unit 37E refers to the image recognition dictionary 36B of the storage 36 to determine whether or not the captured image contains a target subject image. Furthermore, when the captured image contains a target subject image, the determination unit 37E determines whether or not the position of the target subject image is at the image center position.

[0083] When the determining unit 37E determines that the position of the target subject image is at the image center position, the acquiring unit 37A acquires a captured image from the image memory 32, and acquires the amount of deviation between the image center position and the position of the target subject image (hereinafter also referred to as "intra-image deviation amount") by referring to the acquired captured image. The acquiring unit 37A calculates the amount of deviation of the pixel coordinates of the target subject image from the image center position (see FIG. 7). The amount of deviation of the pixel coordinates is an example of the "intra-image deviation amount" according to the technology of the present disclosure.

[0084] The acquisition unit 37A acquires the focal length of the surveillance camera 10. Specifically, the acquisition unit 37A monitors the position of the zoom lens 15B2 on the optical axis OA, and derives the focal length based on the monitoring result. The focal length is derived, for example, by the acquisition unit 37A using a focal length derivation table in which the monitoring result and the focal length are associated with each other, or a focal length derivation arithmetic expression in which the monitoring result is an independent variable and the focal length is a dependent variable.

[0085] The storage 36 stores the sensitivity of the shake correction unit 51 (hereinafter also simply referred to as "sensitivity") and the pixel interval of the pixels of the image sensor 25 (hereinafter also simply referred to as "pixel interval"). The acquisition unit 37A acquires the sensitivity from the storage 36. Here, the sensitivity is the product of the movement amount of the imaging area on the light receiving surface 25A per unit deflection angle and the movable amount of the shake correction unit 51 required to move the imaging area on the light receiving surface 25A by one degree. The acquisition unit 37A also acquires the pixel interval from the storage 36.

[0086] The derivation unit 37B derives a movement amount required for the position of the target subject image to be moved to the image center position by the position adjustment unit 52. Specifically, the derivation unit 37B derives the movement amount based on the intra-image shift amount acquired by the acquisition unit 37A, the focal length acquired by the acquisition unit 37A, and the pixel interval. The pixel interval is an example of "information relating to pixel interval" according to the technology of the present disclosure.

[0087] Note that, although an example is given here in which the pixel interval is stored in the storage 36 and the acquisition unit 37A acquires the pixel interval from the storage 36, the technology of the present disclosure is not limited to this, and the pixel interval may be derived from the size and number of pixels of the captured image by the acquisition unit 37A or the derivation unit 37B. In this case, the information on the size and number of pixels of the captured image is an example of "information related to pixel interval" according to the technology of the present disclosure.

[0088] Furthermore, the acquisition unit 37A may acquire information relating to the pixel interval from the imaging element 25. In this case, the derivation unit 37B may use the information relating to the pixel interval acquired by the acquisition unit 37A as the information relating to the pixel interval when deriving the amount of movement.

[0089] The control unit 37C performs adjustment control to adjust the position of the target subject image in the captured image by operating the position adjustment unit 52 based on the amount of movement derived by the derivation unit 37B. The control unit 37C performs adjustment control to adjust the position of the target subject image in the captured image by operating the rotation mechanism 16 and the shake correction unit 51 included in the position adjustment unit 52 based on the amount of movement derived by the derivation unit 37B.

[0090] Note that, although an example in which control unit 37C operates both turning mechanism 16 and shake correction unit 51 of position adjustment unit 52 has been described here, the technology of the present disclosure is not limited to this. For example, control unit 37C may operate turning mechanism 16 or shake correction unit 51 based on the amount of movement derived by derivation unit 37B.

[0091] The control unit 37C performs the correction of the shake by the shake correction unit 51 and the adjustment control for adjusting the position of the target subject image in the captured image in a time-division manner. Specifically, the control unit 37C causes the shake correction unit 51 to perform the shake correction while the surveillance camera 10 is being rotated by the rotation mechanism 16. The control unit 37C performs the adjustment control while the rotation of the surveillance camera 10 by the rotation mechanism 16 (hereinafter also referred to as "rotation during non-adjustment") is stopped. Here, the rotation of the surveillance camera 10 by the rotation mechanism 16 is broadly divided into rotation during non-adjustment (rotation during normal times) and rotation during adjustment (rotation during times other than normal times). The rotation during adjustment refers to the rotation during adjustment of the position of the target subject image by the rotation mechanism 16 that is operated by performing the adjustment control. In other words, the adjustment time refers to a timing different from the non-adjustment time, and in other words, the timing when the shake correction by the shake correction unit 51 is not performed. As adjustment control, control unit 37C controls rotation mechanism 16 and shake correction unit 51 so that adjustment of the position of the target subject image is performed by shake correction unit 51 after adjustment of the position of the target subject image by rotation mechanism 16 is performed.

[0092] The output unit 37D outputs the movement amount derived by the derivation unit 37B. Specifically, the output unit 37D outputs the movement amount derived by the derivation unit 37B to the management device 11. The management device 11 is an example of the “outside” according to the technology of the present disclosure.

[0093] Next, an example of a method for deriving the movement amount (hereinafter also simply referred to as "movement amount") derived by derivation unit 37B will be described. The movement amount is determined based on a first movement amount required for adjusting the position of the target subject image by rotation mechanism 16 and a second movement amount required for adjusting the position of the target subject image by shake correction unit 51. Specifically, the movement amount derived by derivation unit 37B is determined by combining the first movement amount related to rotation mechanism 16 and the second movement amount related to shake correction unit 51.

[0094] As an example, as shown in Fig. 8, the first movement amount is determined based on a value obtained by dividing the product of the in-image shift amount and the pixel interval by the focal length. As an example, the derivation unit 37B derives the first movement amount by the following calculation formula (1). Note that the pan / tilt angle θ obtained by the following calculation formula (1) is an example of the "first movement amount" according to the technology of the present disclosure.

[0095] θ=arctan(p×t / L) (1) In formula (1), θ is the pan / tilt angle [deg], p is the amount of deviation [pixel] of the pixel coordinates of the target subject image position, t is the pixel spacing of the image sensor [mm / pixel], and L is the focal length [mm].

[0096] Moreover, the derivation unit 37B derives the second movement amount. Specifically, the second movement amount is derived by the derivation unit 37B as the movement amount required to move the shake correction element of the shake correction unit 51 from the current position to the center position. As shown in FIG. 9 as an example, for the shake correction lens 15B1 which is the shake correction element of the shake correction unit 51, a movable range is determined due to the structural constraints of the surveillance camera 10 and the like. Therefore, as shown in FIG. 10 as an example, if the shake correction lens 15B1 (hereinafter also referred to as the "shake correction element" without a reference number) is in a position shifted from the center position in the movable range, the tracking of the target subject by the shake correction unit 51 may be restricted depending on the correction amount required for the shake correction. That is, when the target subject moves in a direction outside the movable range of the shake correction element, it becomes difficult for the shake correction unit 51 to track the target subject. Therefore, the derivation unit 37B derives a second movement amount, which is a movement amount required to move the shake correction element to the center position of the movable range (hereinafter also simply referred to as the "movable range central position").

[0097] The derivation unit 37B derives the shake correction element movement amount required to move the shake correction element from the current position to the movable range central position, based on the shake correction element shift amount and information related to the sensitivity of the shake correction unit 51 acquired by the acquisition unit 37A. The shift amount of the shake correction element is the distance between the movable range central position and the current position of the shake correction element, and is acquired by the acquisition unit 37A based on information output from the position detection sensor 39.

[0098] As an example, the derivation unit 37B calculates the second movement amount by the following calculation formula (2): Note that the pan / tilt angle φ calculated by the following calculation formula is an example of the "second movement amount" according to the technique of the present disclosure.

[0099] φ=N / k (2) In formula (2), N is the deviation amount of the shake correction element [mm], and k is the sensitivity [mm / deg].

[0100] In this embodiment, the vibration is corrected by moving the vibration-proof lens 15B1, but the technology of the present disclosure is not limited to this. For example, instead of the vibration-proof lens 15B1, the vibration may be corrected by moving the image sensor 25 in a plane parallel to the two-dimensional plane of the vibration-proof lens. In this case, the vibration can be corrected within the movable range of the image sensor 25. The amount of deviation from the center position of the movable range of the image sensor 25 is acquired by a position detection sensor (not shown) of the image sensor 25. In addition, when the image sensor 25 is used instead of the vibration-proof lens 15B1 for vibration correction, the image sensor 25 is an example of a "shake correction element" according to the technology of the present disclosure. In this case, an image sensor moving type shake correction mechanism (not shown), which is an example of a "shake correction component" according to the technology of the present disclosure, moves the image sensor in a direction perpendicular to the optical axis of the imaging optical system by applying power generated by a driving source such as a motor (e.g., a voice coil motor) to the image sensor, thereby correcting the shake.

[0101] Furthermore, a mechanical shake correction mechanism (not shown), which is an example of a "shake correction component" and an "optical shake correction mechanism" according to the technology of the present disclosure, may correct shake by moving both the anti-shake lens 15B1 and the image sensor 25 using power generated by a drive source such as a motor (e.g., a voice coil motor). In this case, shake can be corrected within the respective movable ranges of the anti-shake lens 15B1 and the image sensor 25. In this case, the anti-shake lens 15B1 and the image sensor 25 are examples of a "shake correction element" according to the technology of the present disclosure.

[0102] The derivation unit 37B derives the movement amount by combining the first movement amount and the second movement amount. The movement amount derived by the derivation unit 37B is output by the output unit 37D. As shown in FIG. 11 as an example, the movement amount output by the output unit 37D is displayed on the display 13 of the management device 11. Specifically, the captured image is displayed on the display 13, and the pan-tilt angle is displayed as the movement amount in a graph on a movement amount display screen at a position adjacent to the captured image. The user, etc., rotates the surveillance camera 10 while referring to the movement amount displayed on the movement amount display screen.

[0103] In the example shown in FIG. 11, a movement amount display screen is displayed visually, however, an audible display such as audio output from an audio playback device (not shown), a permanent visual display such as printed matter output from a printer, or a tactile display using a vibrator may be used in place of or in combination with the visual display.

[0104] Here, as an example, as shown in the upper part of Fig. 12, in the initial state of the adjustment control performed by the control unit 37C (hereinafter, simply referred to as "initial state"), the position of the target subject image is assumed to be displaced from the image center position. Also, in the initial state, the current position of the shake correction element is assumed to be displaced from the movable range center position. In this case, the movement amount derived by the derivation unit 37B is displayed on the movement amount display screen.

[0105] The user or the like rotates the surveillance camera 10 by rotating the rotation mechanism 16 based on the displayed amount of movement (see the middle part of FIG. 12). The position of the target subject image in the captured image is adjusted by operating the rotation mechanism 16.

[0106] After the position of the target subject image is adjusted by the rotation mechanism 16 (see the middle part of FIG. 12), the position of the target subject image is adjusted by the shake correction unit 51 (see the lower part of FIG. 12). The resolution of the adjustment of the position of the target subject image by the shake correction unit 51 is higher than the resolution of the adjustment of the position of the target subject image by the rotation mechanism 16. That is, the user, etc., first adjusts the position of the target subject image roughly by the rotation mechanism 16, and then adjusts it finely by the shake correction unit 51. When the position of the target subject image is adjusted by the shake correction unit 51, the position of the target subject image moves to the image center position. That is, an image reflecting the adjustment result of the position of the target subject image is displayed as the captured image. Furthermore, when the position of the target subject image is at the image center position, the shake correction element of the shake correction unit 51 is located at the center of the movable range.

[0107] Next, the operation of the portion of the monitoring system 2 related to the technology of the present disclosure will be described with reference to Fig. 13. Fig. 13 shows an example of the flow of the position adjustment process executed by the CPU 37. Note that the flow of the position adjustment process shown in Fig. 13 is an example of the "imaging support method" related to the technology of the present disclosure.

[0108] In the position adjustment process shown in Fig. 13, first, in step ST10, the determination unit 37E determines whether or not a target subject image is included in the captured image. In step ST10, if a target subject image is included in the captured image, the determination is affirmative, and the position adjustment process proceeds to step ST12. In step ST10, if a target subject image is not included in the captured image, the determination is negative, and the position adjustment process proceeds to step ST32.

[0109] In step ST12, the determination unit 37E determines whether the position of the target subject image is at the image center position. If the position of the target subject image is at the image center position, the determination is affirmative, and the position adjustment process proceeds to step ST10. In step ST12, if the position of the target subject image is not at the image center position, the determination is negative, and the position adjustment process proceeds to step ST14.

[0110] In step ST14, the determination unit 37E determines whether position adjustment by the rotation mechanism 16 is necessary to move the position of the target subject image to the image center position. If position adjustment by the rotation mechanism 16 is not necessary in step ST14, the determination is negative, and the position adjustment process proceeds to step ST20. If position adjustment by the rotation mechanism 16 is necessary in step ST14, the determination is positive, and the position adjustment process proceeds to ST16.

[0111] In step ST16, the acquisition unit 37A acquires the in-image displacement amount, the focal length, and the pixel interval. After that, the position adjustment process proceeds to step ST18.

[0112] In step ST18, the derivation unit 37B derives the first movement amount based on the in-image displacement amount and the focal length acquired in step ST18, and the pixel interval. After that, the position adjustment process proceeds to step ST20.

[0113] In step ST20, the acquisition unit 37A determines whether the shake correction element is in the center position of the movable range. In step ST20, if the shake correction element is in the center position of the movable range, the determination is affirmative, and the position adjustment process proceeds to step ST26. In step ST20, if the shake correction element is not in the center position of the movable range, the determination is negative, and then the position adjustment process proceeds to step ST22.

[0114] In step ST22, the acquisition unit 37A acquires the deviation amount and sensitivity of the shake correction element. After that, the position adjustment process proceeds to step ST24.

[0115] In step ST24, the derivation unit 37B derives the second movement amount based on the deviation amount and the sensitivity of the shake correction element. After that, the position adjustment process proceeds to step ST26.

[0116] In step ST26, the derivation unit 37B derives the movement amount based on the first movement amount and the second movement amount. After that, the position adjustment process proceeds to step ST28.

[0117] In step ST28, the output section 37D outputs the amount of movement. After that, the position adjustment process proceeds to step ST30.

[0118] In step ST30, the determination unit 37E determines whether the shake correction element is at the center position of the movable range. In step ST30, if the shake correction element is not at the center position of the movable range, the determination is negative, and the position adjustment process then proceeds to step ST30. In step ST30, if the shake correction element is at the center position of the movable range, the determination is positive, and the position adjustment process proceeds to step ST32.

[0119] In step ST32, the determination unit 37E determines whether or not a condition for terminating the position adjustment process (hereinafter referred to as a "position adjustment process termination condition") has been satisfied. An example of the position adjustment process termination condition is that an instruction to terminate the position adjustment process has been accepted by the acceptance device 62. In step ST32, if the position adjustment process termination condition is not satisfied, the determination is negative, and the position adjustment process proceeds to step ST10. In step ST32, if the position adjustment process termination condition is satisfied, the determination is positive, and the position adjustment process ends.

[0120] As described above, the surveillance camera 10 may track a target subject when capturing an image of an imaging area including the target subject. In this case, it may be required to keep the position of the target subject image in a specific position in the captured image in preparation for the movement of the target subject.

[0121] Therefore, in the surveillance camera 10, the amount of movement required for the position adjustment unit 52 to move the position of the target subject image in the captured image to a specific position is derived based on information on the amount of displacement in the image, the focal length, and the pixel interval. Furthermore, the surveillance camera 10 outputs the derived amount of movement. Therefore, the surveillance camera 10 can assist in tracking the target subject when capturing an image of an imaging area including the target subject.

[0122] Moreover, in the surveillance camera 10, the rotation mechanism 16 and the shake correction unit 51 are used as the position adjustment unit 52. Therefore, compared to the case where the position adjustment unit 52 is separately provided, the configuration is simplified by using the existing rotation mechanism and the shake correction unit 51 as the position adjustment unit 52.

[0123] Furthermore, in surveillance camera 10, the amount of movement is determined based on a first amount of movement required for adjusting the position of the target subject image by rotation mechanism 16 and a second amount of movement required for adjusting the position of the target subject image by shake correction unit 51. Therefore, compared to the case where the amounts of movement of rotation mechanism 16 and shake correction unit 51 are default values, appropriate amounts of movement are set for rotation mechanism 16 and shake correction unit 51, respectively.

[0124] Furthermore, in the surveillance camera 10, the shake correction unit 51 has a shake correction element which is at least one of the anti-shake lens 15B1 and the image sensor 25, and when the position of the target subject image is at a specific position, the shake correction element is located at the center of the movable range of the shake correction element. Therefore, when the position of the target subject image is aligned with the specific position, a wider range in which the target subject image can be tracked can be secured compared to when the shake correction element is located at a position other than the center.

[0125] Furthermore, in the surveillance camera 10, the sensitivity of the shake correction unit 51 is acquired, and the amount of movement required to move the current position of the shake correction element to the center position of the movable range is derived as the second amount of movement based on the deviation amount and sensitivity of the shake correction element. Therefore, compared to the case where the amount of movement of the position of the target subject image by the shake correction unit 51 is an invariant value, since it is determined based on the deviation amount and sensitivity of the shake correction element, the amount of movement of the target subject image can be obtained accurately.

[0126] Furthermore, in the surveillance camera 10, the first movement amount is determined by a value obtained by dividing the product of the in-image displacement amount and the pixel interval by the focal length. Therefore, compared to the case where the movement amount of the position of the target subject image of the rotation mechanism 16 is an invariant value, the movement amount of the target subject image can be obtained accurately because it is determined based on the in-image displacement amount, the pixel interval, and the focal length.

[0127] In addition, in the surveillance camera 10, the amount of movement is determined by combining the first amount of movement and the second amount of movement. Therefore, since the amount of movement is derived as a value combining the first amount of movement and the second amount of movement, subsequent processing such as display on a display unit is easier than when the first amount of movement and the second amount of movement are derived separately.

[0128] In addition, in the surveillance camera 10, the shake correction and adjustment control are performed in a time-division manner in the shake correction unit 51. Therefore, compared to a case where the shake correction and adjustment control are performed in parallel by the shake correction unit 51, the influence of one of the shake correction and adjustment control by the shake correction unit 51 on the other can be suppressed.

[0129] Furthermore, in surveillance camera 10, shake correction is performed by shake correction unit 51 while rotation is being performed by rotation mechanism 16, and adjustment control is performed while rotation by rotation mechanism 16 is stopped. Therefore, while surveillance camera 10 is being rotated and while rotation of surveillance camera 10 by rotation mechanism 16 is stopped, it is possible to improve both the accuracy of shake correction by shake correction unit 51 and the accuracy of adjustment control, compared to a case in which shake correction and adjustment control are performed in parallel by shake correction unit 51.

[0130] Furthermore, in the surveillance camera 10, the position of the target subject image is adjusted by the rotation mechanism 16, and then the position of the target subject image is adjusted in a narrower range by the shake correction unit 51. Therefore, the accuracy of the adjustment of the position of the target subject image is improved compared to the case where only the position of the target subject image is adjusted by the rotation mechanism.

[0131] Furthermore, in the surveillance camera 10, the rotation mechanism 16 is a two-axis rotation mechanism, and the shake correction unit 51 is at least one of an optical shake correction mechanism and an electronic shake correction unit. Therefore, by combining the two-axis rotation mechanism with at least one of an optical shake correction mechanism and an electronic shake correction unit, it is possible to ensure the amount of movement required to adjust the position of the target subject image in each of the two axial directions.

[0132] Furthermore, in the surveillance camera 10, the optical shake correction mechanism is at least one of a lens movement type shake correction mechanism and an image sensor movement type shake correction mechanism. Therefore, by using at least one of the lens movement type shake correction mechanism and the image sensor movement type shake correction mechanism, the amount of movement required to adjust the position of the target subject image can be ensured in each of the two axial directions.

[0133] Furthermore, in the surveillance camera 10, the acquisition unit 37A acquires information relating to the pixel intervals of the pixels of the image sensor 25. Therefore, even if the information relating to the pixel intervals is updated, the amount of movement required to move the position of the target subject image to a specific position can be derived using the latest information relating to the pixel intervals.

[0134] Furthermore, the derived amount of movement is output to the outside in the surveillance camera 10. Therefore, it is possible for a user or the like to grasp the amount of movement required to move the position of the target subject image to a specific position.

[0135] In the above embodiment, the movement amount output to the outside is displayed on the display 13 of the management device 11, but the technology of the present disclosure is not limited to this. As an example, as shown in Fig. 14, the movement amount output from the output unit 37D may be displayed on a display 43B provided on the surveillance camera 10. The display 43B is an example of the "outside" according to the technology of the present disclosure.

[0136] 15, the amount of movement derived by derivation unit 37B is output from output unit 37D. Output unit 37D outputs the amount of movement to display unit 53. Examples of display unit 53 include display 13 and / or display 43B. This enables an operator of surveillance camera 10 or the like to grasp the amount of movement required to move the position of the target subject image to the image center position.

[0137] In the example shown in FIG. 14, a movement amount display screen is displayed visually, however, an audible display such as audio output from an audio playback device (not shown), a permanent visual display such as printed matter output from a printer, or a tactile display using a vibrator may be used in place of or in combination with the visual display.

[0138] In the above embodiment, the example in which the rotation by the rotation mechanism 16 is performed by the operator of the surveillance camera 10 or the like has been described, but the technology of the present disclosure is not limited thereto. As an example, as shown in FIG. 16, the surveillance camera 10 and the rotation mechanism 16 may be communicably connected via the communication line 12. In this embodiment, the control unit 37C outputs to the rotation mechanism 16 the amount of movement required to move the position of the target subject image from the surveillance camera 10 to the image center position. The rotation mechanism 16 rotates the surveillance camera 10 by the amount of movement input from the control unit 37C.

[0139] [Second embodiment] In the first embodiment, the case where the target subject does not move is described, but in the second embodiment, the case where the target subject moves is described. In the second embodiment, the same components as those in the first embodiment are given the same reference numerals, and the description thereof is omitted. The following describes the parts that are different from the first embodiment.

[0140] In image capture by the surveillance camera 10 according to the second embodiment, as shown in Fig. 17 as an example, the position of the target subject image in the captured image changes over time due to the movement of the target subject. That is, the amount of deviation in the image between the image center position and the position of the target subject image changes. For this reason, the amount of movement required to move the position of the target subject image to the image center position must also be an amount of movement that takes into account the movement of the target subject.

[0141] Specifically, as an example shown in Fig. 18, the determination unit 37E determines whether the position of the target subject image has changed within the captured image. The acquisition unit 37A acquires the movement speed of the target subject image by calculating the amount of movement per unit time of the target subject image between frames of the captured image stored in the image memory 32. When the position of the target subject image has changed within the captured image, the derivation unit 37B derives the amount of movement required to move the position of the target subject image to the image center position based on the movement speed.

[0142] The determination unit 37E separates the moving speed of the target subject image into a vertical direction and a horizontal direction in the captured image to obtain a plurality of speeds. The vertical direction and the horizontal direction in the captured image are an example of the "plurality of different directions" according to the technology of the present disclosure. The determination unit 37E obtains the speed of each directional component of the moving speed obtained by separating the moving speed of the target subject image into a vertical component and a horizontal component of the captured image. Furthermore, the determination unit 37E compares the speed of the horizontal component and the vertical component of the captured image. The determination unit 37E also determines whether or not the component in the direction with the greater speed is toward the center position of the image. The derivation unit 37B then determines whether or not to output the derived amount of movement based on the result of the determination of whether or not the component in the direction with the greater speed is toward the center position of the image.

[0143] As an example, as shown in FIG. 19, when the position of the target subject image in the captured image changes to the right toward the image center position, the speed of the horizontal component of the captured image is faster than that of the vertical component. The component with the faster speed (the horizontal component of the captured image in the example shown in FIG. 19) is the component toward the image center position. As a result, the derivation unit 37B does not output the amount of movement in the horizontal direction (pan direction) of the captured image. Therefore, in the initial state, the movement amount display screen displays only the amount of movement in the vertical direction (tilt direction) of the captured image (see the upper part of FIG. 19). After the position adjustment by the position adjustment unit 52 is completed, the position of the target subject image in the captured image changes in a direction approaching the image center position (see the lower part of FIG. 19).

[0144] On the other hand, as shown in FIG. 20 as an example, when the position of the target subject image in the captured image is changing to the left so as to move away from the image center position, the component with the greater speed is the direction moving away from the image center position. In this case, the derivation unit 37B outputs the amount of movement in the horizontal direction (pan direction) as well as the vertical direction (tilt direction) of the captured image. Therefore, in the initial state, the movement amount display screen displays the amount of movement (pan tilt angle) in the horizontal direction (pan direction) and vertical direction (tilt direction) of the captured image (see the upper part of FIG. 20). After the position adjustment by the position adjustment unit 52 is completed, the position of the target subject image in the captured image becomes a specific position (see the lower part of FIG. 20).

[0145] Next, the operation of the portion of the monitoring system 2 related to the technology of the present disclosure will be described with reference to Fig. 21A and Fig. 21B. Note that Fig. 21A and Fig. 21B show an example of the flow of the position adjustment process executed by the CPU 37. The flow of the position adjustment process shown in Fig. 21A and Fig. 21B is an example of the "imaging support method" related to the technology of the present disclosure.

[0146] In the position adjustment process shown in Fig. 21A, first, in step ST10, the determination unit 37E determines whether or not a target subject image is included in the captured image. In step ST10, if a target subject image is included in the captured image, the determination is affirmative, and the position adjustment process proceeds to step ST34. In step ST10, if a target subject image is not included in the captured image, the determination is negative, and the position adjustment process proceeds to step ST32.

[0147] In step ST34, the determination unit 37E executes a speed determination process shown in FIG. 21B as an example. In the speed determination process shown in FIG. 21B, first, in step ST40, the determination unit 37E determines whether the position of the target subject image has moved within the captured image. In step ST40, if the position of the target subject image has moved, the determination is affirmative, and the speed determination process proceeds to step ST42. In the determination in step ST40, if the position of the target subject image has not moved, the determination is negative, and the speed determination process proceeds to step ST14 of the position adjustment process shown in FIG. 21A.

[0148] In step ST42, the determination unit 37E resolves the moving speed of the target subject into two different directions. After that, the speed determination process proceeds to step ST44.

[0149] In step ST44, the determination unit 37E compares the speed of the components in each direction resolved in step ST42. After that, the speed determination process proceeds to step ST46.

[0150] In step ST46, the determination unit 37E determines whether or not the component with the greater speed among the directions compared in step ST44 is the direction toward the image center position. If the component with the greater speed is the direction toward the image center position in step ST46, the determination is affirmative, and the speed determination process proceeds to step ST48. If the component with the greater speed is not the direction toward the image center position in step ST46, the determination is negative, and the speed determination process proceeds to step ST14 of the position adjustment process shown in FIG. 21A.

[0151] In step ST48, the derivation unit 37B controls the output unit 37D so as not to output the movement amount of the component with the greater speed. After that, the speed determination process proceeds to step ST12 of the position adjustment process shown in FIG.

[0152] As described above, the surveillance camera 10 according to the second embodiment is capable of adjusting the position in consideration of the moving speed of the moving target subject, even while the target subject is moving.

[0153] Furthermore, in the surveillance camera 10 according to the second embodiment, it is possible to achieve highly accurate position adjustment even while the target subject is moving, compared to a case in which the moving speed is in only one direction.

[0154] In the above-described embodiments, the combined first and second movement amounts are displayed on the display 13, but the technology of the present disclosure is not limited to this. For example, the first and second movement amounts may be displayed by separate indicators.

[0155] In addition, in each of the above embodiments, the specific position and the default position are the center position of the image, but the technology of the present disclosure is not limited to this. The specific position and the default position may be set by the user or the like to any position in the captured image (for example, one of the four corners in the captured image).

[0156] Furthermore, in each of the above embodiments, devices including ASICs and FPGAs are exemplified, but the technology of the present disclosure is not limited to this, and various processes may be realized by a software configuration using a computer.

[0157] In this case, for example, as shown in Fig. 22, a computer 19 is built into the surveillance camera 10. A position adjustment program 36A for causing the computer 19 to execute the position adjustment process according to the above embodiment is stored in a storage medium 100, which is a non-transitory storage medium. An example of the storage medium 100 is any portable storage medium such as an SSD or a USB memory.

[0158] The computer 19 includes a CPU 37, a storage 36, and a memory 35. The storage 36 is a non-volatile storage device such as an EEPROM, and the memory 35 is a volatile storage device such as a RAM. A position adjustment program 36A stored in the storage medium 100 is installed in the computer 19. The CPU 37 executes a position adjustment process in accordance with the position adjustment program 36A.

[0159] The position adjustment program 36A may be stored in the storage 36 instead of the storage medium 100. In this case, the CPU 37 reads out the position adjustment program 36A from the storage 36, and executes the read out position adjustment program 36A on the memory 35. In this manner, the position adjustment program 36A is executed by the CPU 37 to realize the position adjustment process.

[0160] In addition, the position adjustment program 36A may be stored in a memory unit of another computer or server device connected to the computer 19 via a communication network (not shown), and the position adjustment program 36A may be downloaded and installed in the computer 19 in response to a request from the above-mentioned surveillance camera 10.

[0161] It is not necessary to store the entire position adjustment program 36A in the storage unit of another computer or server device connected to the computer 19, or in the storage 36; only a part of the position adjustment program 36A may be stored therein.

[0162] In the example shown in Figure 22, an example is shown in which the computer 19 is built into the surveillance camera 10, but the technology of the present disclosure is not limited to this, and for example, the computer 19 may be provided outside the surveillance camera 10.

[0163] 22, the CPU 37 is a single CPU, but it may be a plurality of CPUs. Also, a GPU may be used in place of the CPU 37.

[0164] 22 illustrates a computer 19, but the technology of the present disclosure is not limited to this, and a device including an ASIC, an FPGA, and / or a PLD may be applied instead of the computer 19. Also, instead of the computer 19, a combination of a hardware configuration and a software configuration may be used.

[0165] As the hardware resource for executing the position adjustment process described in each of the above embodiments, the following various processors can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the position adjustment process by executing software, i.e., a program. Another example of the processor is a dedicated electrical circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as an FPGA, PLD, or ASIC. Each processor has a built-in or connected memory, and each processor executes the position adjustment process by using the memory.

[0166] The hardware resource for executing the position adjustment process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource for executing the position adjustment process may be a single processor.

[0167] As an example of a configuration using one processor, first, one or more CPUs and software are combined to configure one processor, and this processor functions as a hardware resource that executes the position adjustment process. Second, as typified by SoC, a processor is used that realizes the functions of the entire system, including multiple hardware resources that execute the position adjustment process, in one IC chip. In this way, the position adjustment process is realized using one or more of the above various processors as hardware resources.

[0168] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. Also, the above position adjustment process is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be changed, without departing from the spirit of the invention.

[0169] 1 illustrates a surveillance camera 10, the technology of the present disclosure is not limited to this. That is, the technology of the present disclosure can be applied to various electronic devices with built-in imaging devices (e.g., interchangeable lens cameras, fixed lens cameras, smart devices, personal computers, and / or wearable terminal devices, etc.), and the same actions and effects as the above-mentioned surveillance camera 10 can be obtained with these electronic devices.

[0170] In addition, in each of the above embodiments, the display 43B is exemplified, but the technology of the present disclosure is not limited to this. For example, a separate display connected to the imaging device may be used as the "display unit" according to the technology of the present disclosure.

[0171] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[0172] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0173] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or standard was specifically and individually indicated to be incorporated by reference.

[0174] The following supplementary notes are further disclosed regarding the above embodiment.

[0175] (Additional Note) A processor; A memory embedded in or connected to the processor; The processor is acquiring an in-image shift amount between a predetermined position in a captured image obtained by capturing an image of an imaging area including a target subject by an imaging element and a position of a target subject image showing the target subject, and a focal length of an imaging device; deriving a movement amount required for moving the position of the target subject image to a specific position by a position adjustment unit that adjusts the position of the target subject image within the captured image based on the intra-image shift amount acquired by the acquisition unit, the focal length acquired by the acquisition unit, and information regarding a pixel interval between pixels in the image sensor; The movement amount derived by the derivation unit is output. Information processing device.

Claims

1. A processor; A memory embedded in or connected to the processor; An imaging support device that supports imaging by an imaging device having an imaging element, The processor, a rotation mechanism for rotating the imaging device and a shake correction component for correcting shake caused by vibrations applied to the imaging device, the rotation mechanism moving a position of a target subject image showing the target subject to a specific position; a resolution of the adjustment of the position of the target subject image by the shake correction component is higher than a resolution of the adjustment of the position of the target subject image by the rotation mechanism; the processor performs, in a time-division manner, the correction of the shake by the shake correction component, and an adjustment control for adjusting the position of the target subject image in the captured image obtained by imaging an imaging area including the target subject by the imaging element, by operating the shake correction component based on a movement amount required for moving the position of the target subject image to a specific position by the rotation mechanism and the shake correction component; The adjustment control is a control for adjusting the position of the target subject image by the shake correction component after the position of the target subject image is adjusted by the rotation mechanism. Imaging support device.

2. The shake correction component includes at least one of an optical shake correction mechanism and an electronic shake correction component. The imaging support device according to claim 1 .

3. A first movement amount required for adjusting the position of the target subject image by the rotation mechanism is larger than a second movement amount required for adjusting the position of the target subject image by the shake correction unit. The imaging support device according to claim 1 or 2.

4. The processor further operates the rotation mechanism and the shake correction component based on a movement amount required for the rotation mechanism and the shake correction component to move the position of the target subject image to a specific position, thereby performing adjustment control to adjust the position of the target subject image in the captured image obtained by capturing an image of an imaging area including the target subject by the imaging element. The imaging support device according to claim 1 .

5. The amount of movement is determined based on a first amount of movement required to adjust the position of the target-subject image by the rotation mechanism and a second amount of movement required to adjust the position of the target-subject image by the shake correction component. The imaging support device according to claim 4.

6. the shake correction component has a shake correction element which is at least one of a lens and the imaging element that corrects the shake by moving in response to the vibration, At the specific position, the shake correction element is located at the center of the movable range of the shake correction element. The imaging support device according to any one of claims 1 to 5.

7. The processor, Further obtaining a sensitivity of the shake correction component; Based on the shake correction element shift amount between the center position of the movable range and the current position of the shake correction element and the acquired sensitivity, a shake correction element movement amount required to move the current position to the center position is derived as a second movement amount required to adjust the position of the target subject image by the shake correction component. The imaging support device according to claim 6.

8. A movement amount required for moving the position of the target subject image to a specific position by the rotation mechanism and the shake correction component is obtained by combining a first movement amount required for adjusting the position of the target subject image by the rotation mechanism and a second movement amount required for adjusting the position of the target subject image by the shake correction component. The imaging support device according to any one of claims 1 to 7.

9. The processor causes the shake correction component to correct the shake while the imaging device is being rotated by the rotation mechanism, and performs the adjustment control while the rotation of the imaging device by the rotation mechanism is stopped. The imaging support device according to claim 1 .

10. the rotation mechanism is a two-axis rotation mechanism capable of rotating the imaging device in a first direction and a second direction intersecting the first direction, The shake correction component is at least one of an optical shake correction mechanism and an electronic shake correction component. The imaging support device according to any one of claims 1 to 9.

11. The optical shake correction mechanism is at least one of a lens movement type shake correction mechanism and an image sensor movement type shake correction mechanism. The imaging support device according to claim 10.

12. The processor outputs to an external device a movement amount required for the rotation mechanism and the shake correction component to move the position of the target subject image to a specific position. The imaging support device according to any one of claims 1 to 11.

13. A movement amount required for the rotation mechanism and the shake correction component to move the position of the target-subject image to a specific position is determined based on a moving speed of the target subject when the target subject is moving. The imaging support device according to any one of claims 1 to 12.

14. The moving speed is a plurality of speeds obtained by decomposing the moving speed into a plurality of different directions. The imaging support device according to claim 13.

15. An imaging support device according to any one of claims 1 to 14; The imaging element. Imaging device.

16. An imaging device according to claim 15; a control device that performs at least one of control to display an image on a display reflecting an adjustment result of the position of the target subject image based on a movement amount required for moving the position of the target subject image to a specific position by the rotation mechanism and the shake correction component, and control to store image data representing the image reflecting the adjustment result in a storage device; and 2. An imaging system comprising:

17. An imaging element; an imaging support device including a processor and a memory built in or connected to the processor, the imaging support device supporting imaging by an imaging device having the imaging element; a rotation mechanism capable of rotating the imaging device, and a shake correction component that corrects shake caused by vibration applied to the imaging device, The processor, moving a position of a target subject image showing a target subject to a specific position by the pivot mechanism and the shake correction component; a resolution of the adjustment of the position of the target subject image by the shake correction component is higher than a resolution of the adjustment of the position of the target subject image by the rotation mechanism; the processor performs, in a time-division manner, the correction of the shake by the shake correction component, and an adjustment control for adjusting the position of the target subject image in the captured image obtained by imaging an imaging area including the target subject by the imaging element, by operating the shake correction component based on a movement amount required for moving the position of the target subject image to a specific position by the rotation mechanism and the shake correction component; The adjustment control is a control for adjusting the position of the target subject image by the shake correction component after the position of the target subject image is adjusted by the rotation mechanism. Imaging system.

18. An imaging support device according to any one of claims 1 to 14; the pivot mechanism and the shake correction component, The processor included in the imaging support device moves a position of a target subject image showing a target subject to a specific position by the rotation mechanism and the shake correction component, a resolution of the adjustment of the position of the target subject image by the shake correction component is higher than a resolution of the adjustment of the position of the target subject image by the rotation mechanism; the processor performs, in a time-division manner, the correction of the shake by the shake correction component, and an adjustment control for adjusting the position of the target subject image in the captured image obtained by imaging an imaging area including the target subject by the imaging element, by operating the shake correction component based on a movement amount required for moving the position of the target subject image to a specific position by the rotation mechanism and the shake correction component; The adjustment control is a control for adjusting the position of the target subject image by the shake correction component after the position of the target subject image is adjusted by the rotation mechanism. Imaging support system.

19. An imaging support method for supporting imaging by an imaging device including an imaging element, comprising: a rotation mechanism for rotating the imaging device and a shake correction component for correcting shake caused by vibrations applied to the imaging device, the rotation mechanism moving a position of a target subject image showing the target subject to a specific position; a resolution of the adjustment of the position of the target subject image by the shake correction component is higher than a resolution of the adjustment of the position of the target subject image by the rotation mechanism; and performing, in a time-division manner, a correction of the shake by the shake correction component, and an adjustment control for adjusting a position of the target subject image in an image obtained by imaging an imaging area including the target subject by the imaging element, by operating the shake correction component based on a movement amount required for moving the position of the target subject image to a specific position by the rotation mechanism and the shake correction component, The adjustment control is a control for adjusting the position of the target subject image by the shake correction component after the position of the target subject image is adjusted by the rotation mechanism. Imaging assistance method.

20. A program for causing a computer to execute a process for supporting imaging by an imaging device having an imaging element, The process comprises: a rotation mechanism for rotating the imaging device and a shake correction component for correcting shake caused by vibrations applied to the imaging device, the rotation mechanism moving a position of a target subject image showing the target subject to a specific position; a resolution of the adjustment of the position of the target subject image by the shake correction component is higher than a resolution of the adjustment of the position of the target subject image by the rotation mechanism; The process comprises: and performing, in a time-division manner, a correction of the shake by the shake correction component, and an adjustment control for adjusting a position of the target subject image in an image obtained by imaging an imaging area including the target subject by the imaging element, by operating the shake correction component based on a movement amount required for moving the position of the target subject image to a specific position by the rotation mechanism and the shake correction component, The adjustment control is a control for adjusting the position of the target subject image by the shake correction component after the position of the target subject image is adjusted by the rotation mechanism. program.

Images