Imaging support device, imaging support system, imaging system, imaging support method, and program

Abstract

To provide an imaging support device that can accurately align a subject image position to a specific position within a captured image when a vibration isolation control unit moves an imaging range for wide-angle imaging.SOLUTION: An imaging support device includes: an acquisition unit that acquires a focal length of an imaging device having a lens movement mechanism and an image area movement unit; a detection unit that detects a subject image position of a target subject image showing a target subject within a captured image obtained by imaging an imaging area including the target subject with the imaging device; and a control unit that performs positioning control to align the subject image position detected by the detection unit to a specific position within the captured image by increasing a ratio of applying the image area movement unit compared to the lens movement mechanism when the focal length acquired by the acquisition unit is equal to or greater than a threshold value, and increasing a ratio of applying the lens movement mechanism compared to the image area movement unit when the focal length is less than the threshold value.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The technology disclosed herein relates to an imaging support device, an imaging support system, an imaging system, an imaging support method, and a program. [Background technology]

[0002] Japanese Patent Application Publication No. 2016-90973 discloses an imaging device including a first vibration isolation control unit that controls a first vibration isolation unit that moves a lens to correct camera shake; a second vibration isolation control unit that controls a second vibration isolation unit that moves an image sensor to correct camera shake; a wide-angle image generation unit that moves the shooting range for wide-angle shooting by at least either the first vibration isolation control unit or the second vibration isolation control unit and generates a wide-angle image based on a plurality of images captured sequentially in the moved shooting range; and a setting unit that sets whether the first vibration isolation unit and the second vibration isolation unit are to be used for the camera shake correction or for moving the shooting range. Summary of the Invention

[0003] One embodiment of the technology disclosed herein provides an imaging support device, an imaging support system, an imaging system, an imaging support method, and a program that can align a subject image position to a specific position within a captured image with high precision, regardless of focal length, compared to a case where alignment control relies solely on a lens movement mechanism and an image area movement unit. [Means for solving the problem]

[0004] A first aspect of the technology disclosed herein is an imaging support device that includes an imaging device having a lens and an image sensor, the imaging device including a lens movement mechanism that moves the lens in a direction intersecting the optical axis of the lens and an image area movement unit that moves an area of ​​an image captured on the image sensor in a direction intersecting the optical axis, an acquisition unit that acquires a focal length of the imaging device, a detection unit that detects a subject image position of a target subject image indicating a target subject in an image captured by the imaging device by capturing an imaging area including the target subject, and a control unit that performs alignment control to align the subject image position detected by the detection unit to a specific position in the captured image by increasing the rate at which the image area movement unit is applied compared to the lens movement mechanism when the focal length acquired by the acquisition unit is equal to or greater than a threshold, and increasing the rate at which the lens movement mechanism is applied compared to the image area movement unit when the focal length is less than the threshold.

[0005] A second aspect of the technology disclosed herein is an imaging support device according to the first aspect, in which the lens movement mechanism is a lens-side shake correction mechanism that corrects shake caused by vibrations applied to the imaging device by moving the lens, the image area movement unit is an image sensor-side shake correction unit that corrects shake by moving the image sensor or by image processing of the captured image obtained by capturing an image by the image sensor, and the control unit performs alignment control by increasing the proportion of application of the image sensor-side shake correction unit compared to the lens-side shake correction mechanism when the focal length is equal to or greater than a threshold, and by increasing the proportion of application of the lens-side shake correction mechanism compared to the image sensor-side shake correction unit when the focal length is less than the threshold.

[0006] A third aspect of the technology disclosed herein is an imaging support device related to the second aspect, in which a control unit causes the lens side shake correction mechanism to correct shake when the focal length acquired by the acquisition unit is greater than or equal to a threshold value.

[0007] A fourth aspect of the technology disclosed herein is an imaging support device according to the second or third aspect, in which the control unit causes the image sensor side shake correction unit to correct shake when the focal length acquired by the acquisition unit is less than a threshold value.

[0008] A fifth aspect of the technology disclosed herein is an imaging support device relating to any one of the first to fourth aspects, in which the imaging device has a magnification change mechanism that changes the focal length, and the control unit performs alignment control by using an image sensor side shake correction unit and a lens side shake correction mechanism in combination when the focal length acquired by the acquisition unit changes.

[0009] A sixth aspect of the technology disclosed herein is an imaging support device relating to any one of the first to fifth aspects, in which the control unit causes at least one of the image sensor side shake correction unit and the lens side shake correction mechanism to correct shake when the focal length acquired by the acquisition unit has not changed.

[0010] A seventh aspect of the technology disclosed herein is an imaging support device relating to any one of the first to sixth aspects, in which the control unit performs alignment control by using a lens side shake correction mechanism and an image sensor side shake correction unit in combination when the focal length acquired by the acquisition unit is within a predetermined range including a threshold value.

[0011] An eighth aspect of the technology of the present disclosure is an imaging support device according to the seventh aspect, in which the alignment control includes control that, when the focal length acquired by the acquisition unit is within a predetermined range, gradually increases the proportion of application of the image sensor side shake compensation unit compared to the lens side shake compensation mechanism as the focal length becomes longer.

[0012] A ninth aspect of the technology of the present disclosure is an imaging support device according to the seventh or eighth aspect, in which the control unit performs alignment control using the lens side shake compensation mechanism of the lens side shake compensation mechanism and the image sensor side shake compensation unit when the focal length acquired by the acquisition unit is less than the lower limit of a predetermined range.

[0013] A tenth aspect of the technology disclosed herein is an imaging support device relating to any one of the seventh to ninth aspects, in which the control unit performs alignment control using the image sensor side shake correction unit of the lens side shake correction mechanism and the image sensor side shake correction unit when the focal length acquired by the acquisition unit exceeds an upper limit value of a predetermined range.

[0014] An eleventh aspect of the technology disclosed herein is an imaging support device relating to any one of the first to tenth aspects, in which the control unit performs alignment control by using an alignment unit including a rotation mechanism capable of rotating the imaging device, an image sensor side shake correction unit, and a lens side shake correction mechanism, and the alignment accuracy for adjusting the subject image position within the captured image is higher for the image sensor side shake correction unit and the lens side shake correction mechanism than for the rotation mechanism.

[0015] A twelfth aspect of the technology of the present disclosure is an imaging support system that includes an imaging support device of the eleventh aspect and a rotation mechanism, in which the imaging support device supports imaging by the imaging device when the rotation mechanism rotates the imaging device.

[0016] A thirteenth aspect of the technology of the present disclosure is an imaging system that includes an imaging support device relating to any one of the first to eleventh aspects and an imaging device, in which imaging support is provided by the imaging support device.

[0017] A fourteenth aspect of the technique of the present disclosure is the imaging system according to the thirteenth aspect, further including a rotation mechanism for rotating the imaging device.

[0018] A fifteenth aspect of the technique of the present disclosure is an imaging device having a lens and an imaging element, An imaging support method including: acquiring a focal length of an imaging device having a lens movement mechanism that moves a lens in a direction intersecting with the optical axis of the lens and an image area movement unit that moves an area of ​​an image captured on an imaging element in a direction intersecting with the optical axis; detecting a subject image position of a target subject image that indicates a target subject in an captured image obtained by imaging an imaging area including the target subject by the imaging device; and performing alignment control to align the detected subject image position to a specific position in the captured image by increasing the rate at which the image area movement unit is applied compared to the lens movement mechanism when the acquired focal length is equal to or greater than a threshold, and increasing the rate at which the lens movement mechanism is applied compared to the image area movement unit when the focal length is less than the threshold.

[0019] A sixteenth aspect of the technology of the present disclosure is a program for causing a computer to execute processing including: acquiring a focal length of an imaging device having a lens and an imaging element, the imaging device having a lens movement mechanism that moves the lens in a direction intersecting with the optical axis of the lens, and an image area movement unit that moves an area of ​​an imaging image on the imaging element in a direction intersecting with the optical axis; detecting a subject image position of a target subject image indicating a target subject in an imaging image obtained by imaging an imaging area including the target subject by the imaging device; and performing alignment control to align the detected subject image position to a specific position in the imaging image by increasing the rate at which the image area movement unit is applied compared to the lens movement mechanism when the acquired focal length is equal to or greater than a threshold, and increasing the rate at which the lens movement mechanism is applied compared to the image area movement unit when the focal length is less than the threshold. [Brief description of the drawings]

[0020] [Figure 1] 1 is a schematic configuration diagram showing an example of the configuration of a monitoring system according to a first embodiment. [Diagram 2] 1 is a perspective view showing an example of the appearance of a surveillance camera according to a first embodiment in a state where the surveillance camera has been rotated in a pitch direction. FIG. [Diagram 3] 1 is a perspective view showing an example of the appearance of a surveillance camera according to a first embodiment in a state where the surveillance camera has been rotated in a yaw direction. [Figure 4] 1 is a block diagram showing an example of the configuration of an optical system and an electrical system of a surveillance camera according to a first embodiment. [Diagram 5] 2 is a block diagram showing an example of a configuration of an electrical system of a management device and a turning mechanism according to the first embodiment. FIG. [Figure 6] 2 is a functional block diagram showing an example of functions of a CPU included in the surveillance camera according to the first embodiment. FIG. [Figure 7] 4 is a functional block diagram showing an example of functions of a CPU included in the management device according to the first embodiment; FIG. [Figure 8] 4 is a functional block diagram showing an example of functions of a CPU included in the management device according to the first embodiment; FIG. [Figure 9] 10 is a functional block diagram showing an example of the functions of a management device in a case where the management device according to the first embodiment uses a ratio derivation table. FIG. [Figure 10A] 10 is a flowchart showing an example of the flow of alignment control processing according to the first embodiment. [Figure 10B] This is a continuation of the flowchart shown in FIG. 10A. [Figure 11A] 4 is a conceptual diagram showing an example of an aspect in which a target subject is present in the center of an imaging area according to the first embodiment. FIG. [Figure 11B] 11B is a schematic image diagram showing an example of a captured image obtained by capturing an image of the imaging area shown in FIG. 11A with a surveillance camera. FIG. [Figure 12A] 4 is a conceptual diagram showing an example of a state in which a target subject is present at an edge of an imaging region according to the first embodiment. FIG. [Figure 12B] 12B is a schematic image diagram showing an example of a captured image obtained by capturing an image of the imaging area shown in FIG. 12A by a surveillance camera. FIG. [Figure 13A] 1A is a conceptual diagram showing an example of a state in which a target subject is moved out of an imaging area as a result of increasing the focal length of the surveillance camera according to the first embodiment. FIG. [Figure 13B] 13B is a schematic image diagram showing an example of a captured image obtained by capturing an image of the imaging area shown in FIG. 13A by a surveillance camera. FIG. [Figure 14] 10 is a schematic image diagram showing an example of a captured image in which a target subject image is placed within a central region of the captured image by operating the shake correction unit according to the first embodiment. FIG. [Figure 15] FIG. 11 is a functional block diagram showing an example of functions of a CPU included in a management device according to a second embodiment. [Figure 16A] 13 is a flowchart showing an example of the flow of alignment control processing according to the second embodiment. [Figure 16B] This is a continuation of the flowchart shown in FIG. 16A. [Figure 17] 10A to 10C are schematic image diagrams showing an example of a captured image obtained by performing rough adjustment in the alignment control process and an example of a captured image obtained by performing fine adjustment after the rough adjustment. [Figure 18] 10 is a graph showing an example, different from that in FIG. 9, of a ratio derivation table of the application ratio of the lens side shake compensation mechanism and the image sensor side shake compensation mechanism in the imaging support device. [Figure 19] A conceptual diagram showing an example of how a display control processing program and a zoom mechanism control processing program according to an embodiment are installed on a computer within a surveillance camera from a storage medium on which the alignment control processing program, display control processing program, and zoom mechanism control processing program are stored. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0021] An example of an embodiment of the technology of the present disclosure will be described with reference to the accompanying drawings.

[0022] First, the terms used in the following description will be explained.

[0023] CPU is an abbreviation for "Central Processing Unit". RAM is an abbreviation for "Random Access Memory". ROM is an abbreviation for "Read Only Memory". ASIC is an abbreviation for "Application Specific Integrated Circuit". PLD is an abbreviation for "Programmable Logic Device". FPGA is an abbreviation for "Field-Programmable Gate Array". AFE is an abbreviation for "Analog Front End". DSP is an abbreviation for "Digital Signal Processor". ISP is an abbreviation for "Image Signal Processor". SoC is an abbreviation for "System-on-a-chip". CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor". CCD is an abbreviation for "Charge Coupled Device". SWIR is an abbreviation for "Short-wavelength infrared".

[0024] SSD is an abbreviation of "Solid State Drive". USB is an abbreviation of "Universal Serial Bus "HDD is an abbreviation for "Hard Disk Drive".EEPROM is an abbreviation for "Electrically Erasable and Programmable Read Only Memory".EL is an abbreviation for "Electro-Luminescence".A / D is an abbreviation for "Analog / Digital".I / F is an abbreviation for "Interface".UI is an abbreviation for "User Interface".WAN is an abbreviation for "Wide Area Network".CRT is an abbreviation for "Cathode Ray Tube".OIS is an abbreviation for "Optical Image Stabilizer".BIS is an abbreviation for "Body Image Stabilizer".

[0025] In the description of this specification, "horizontal" refers to horizontal in the sense of including a generally acceptable error in the technical field to which the technology of the present disclosure belongs, in addition to being completely horizontal. In the description of this specification, "parallel" refers to parallel in the sense of including a generally acceptable error in the technical field to which the technology of the present disclosure belongs, in addition to being completely parallel. In the description of this specification, "vertical" refers to vertical in the sense of including a generally acceptable error in the technical field to which the technology of the present disclosure belongs, in addition to being completely vertical. In the description of this specification, "same" refers to same in the sense of including a generally acceptable error in the technical field to which the technology of the present disclosure belongs, in addition to being completely same.

[0026] [First embodiment] 1, a surveillance system 2 includes a surveillance camera 10, a rotation mechanism 16, and a management device 11. The surveillance system 2 is an example of an "imaging system" or an "imaging support system" according to the technology of the present disclosure, and the surveillance camera 10 is an example of an "imaging device" according to the technology of the present disclosure.

[0027] The surveillance camera 10 is installed on a pillar, a wall, or a part of a building (e.g., a rooftop) indoors or outdoors via a rotation mechanism 16 described later, and captures an image of a subject to be monitored (hereinafter also referred to as an "imaging area"), thereby generating a moving image. The moving image includes a plurality of frames of images obtained by imaging. The surveillance camera 10 transmits the moving image obtained by imaging to the management device 11 via a communication line 12.

[0028] The management device 11 includes a display 13 and a secondary storage device 14. Examples of the display 13 include a liquid crystal display, a plasma display, an organic EL display, and a CRT display.

[0029] An HDD is one example of the secondary storage device 14. The secondary storage device 14 may be a non-volatile memory such as a flash memory, an SSD, or an EEPROM, instead of an HDD.

[0030] In the management device 11, the moving images transmitted by the surveillance camera 10 are received, and the received moving images are displayed on the display 13 and / or stored in the secondary storage device .

[0031] The surveillance camera 10 is attached to the rotation mechanism 16. The rotation mechanism 16 allows the surveillance camera 10 to rotate. Specifically, as shown in FIG. 2 as an example, the rotation mechanism 16 is a two-axis rotation mechanism that can rotate the surveillance camera 10 in a rotation direction (hereinafter referred to as "pitch direction") that intersects with the yaw direction and has a pitch axis PA as a central axis, and in a rotation direction (hereinafter referred to as "yaw direction") that has a yaw axis YA as a central axis as shown in FIG. 3 as an example. The rotation mechanism 16 is an example of a "rotation mechanism" according to the technology of the present disclosure. Note that, although the rotation mechanism 16 according to the present embodiment is an example of a two-axis rotation mechanism, the technology of the present disclosure is not limited thereto, and may be a three-axis rotation mechanism.

[0032] 4, the surveillance camera 10 includes an optical system 15 and an imaging element 25. The imaging element 25 is located after the optical system 15. The imaging element 25 includes a light receiving surface 25A. Light representing an imaging area is focused on the light receiving surface 25A by the optical system 15, and the imaging area is imaged by the imaging element 25.

[0033] The surveillance camera 10 includes a computer 19, a displacement driver 22, a displacement driver 23, a zoom driver 28, a DSP 31, an image memory 32, an electronic shake correction unit 33, a communication I / F 34, position detection sensors 39 and 47, a shake amount detection sensor 40, and a UI device 43. The computer 19 includes a memory 35, a storage 36, and a CPU 37. The computer 19 is an example of a "computer" according to the technology of the present disclosure. The electronic shake correction unit 33 and the CPU 37 are examples of an "electronic shake correction unit (electronic shake correction component)" according to the technology of the present disclosure.

[0034] The displacement driver 22, the displacement driver 23, the image sensor 25, the zoom driver 28, the DSP 31, the image memory 32, the electronic shake correction unit 33, the communication I / F 34, the memory 35, the storage 36, the CPU 37, the position detection sensors 39 and 47, the shake amount detection sensor 40, and the UI device 43 are connected to a bus 38. Note that, in the example shown in Fig. 4, for convenience of illustration, one bus is shown as the bus 38, but multiple buses may be used. The bus 38 may be a serial bus, or may be a parallel bus including a data bus, an address bus, a control bus, and the like.

[0035] The memory 35 temporarily stores various information and is used as a work memory. An example of the memory 35 is a RAM, but the memory 35 may be another type of storage device. The storage 36 stores various programs for the surveillance camera 10. The CPU 37 reads the various programs from the storage 36 and executes the read programs on the memory 35 to control the entire surveillance camera 10. Examples of the storage 36 include a flash memory, an SSD, an EEPROM, and an HDD. Also, for example, various non-volatile memories such as a magnetoresistive memory and a ferroelectric memory may be used instead of or in combination with the flash memory.

[0036] The imaging element 25 is a CMOS image sensor. The imaging element 25 captures an image of a target subject at a default frame rate under the instruction of the CPU 37. The "default frame rate" here refers to, for example, several tens of frames per second to several hundreds of frames per second. The imaging element 25 itself may also have a built-in control device (imaging element control device), in which case the imaging element control device performs detailed control inside the imaging element 25 in response to an imaging instruction output by the CPU 37. The imaging element 25 may also capture an image of a target subject at a default frame rate under the instruction of the DSP 31, in which case the imaging element control device performs detailed control inside the imaging element 25 in response to an imaging instruction output by the DSP 31. The DSP 31 may also be called an ISP.

[0037] The light receiving surface 25A of the image sensor 25 is formed by a plurality of photosensitive pixels (not shown) arranged in a matrix. In the image sensor 25, each photosensitive pixel is exposed to light, and photoelectric conversion is performed for each photosensitive pixel. The charge obtained by performing photoelectric conversion for each photosensitive pixel is an analog image signal indicating the target subject. Here, a plurality of photoelectric conversion elements (for example, photoelectric conversion elements in which color filters are arranged) having sensitivity to visible light are adopted as the plurality of photosensitive pixels. In the image sensor 25, a photoelectric conversion element having sensitivity to R (red) light (for example, a photoelectric conversion element in which an R filter corresponding to R is arranged), a photoelectric conversion element having sensitivity to G (green) light (for example, a photoelectric conversion element in which a G filter corresponding to G is arranged), and a photoelectric conversion element having sensitivity to B (blue) light (for example, a photoelectric conversion element in which a B filter corresponding to B is arranged) are adopted as the plurality of photoelectric conversion elements. In the surveillance camera 10, these photosensitive pixels are used to capture images based on visible light (for example, light on the short wavelength side of approximately 700 nanometers or less). However, the present embodiment is not limited to this, and images may be captured based on infrared light (for example, light on the long wavelength side of approximately 700 nanometers). In this case, a plurality of photoelectric conversion elements having sensitivity to infrared light may be used as the plurality of photosensitive pixels. In particular, for SWIR imaging, for example, an InGaAs sensor and / or a Simulation of Type-II Quantum Well (T2SL) sensor may be used.

[0038] The imaging element 25 performs signal processing such as A / D conversion on the analog imaging signal to generate a digital image, which is a digital imaging signal. The imaging element 25 is connected to the DSP 31 via a bus 38, and outputs the generated digital image to the DSP 31 via the bus 38 in frame units. Here, the digital image is an example of a "captured image" according to the technology of the present disclosure.

[0039] Note that, although a CMOS image sensor is described here as an example of the imaging element 25, the technology of the present disclosure is not limited thereto, and a CCD image sensor may be applied as the imaging element 25. In this case, the imaging element 25 is connected to the bus 38 via an AFE (not shown) with a built-in CCD driver, and the AFE generates a digital image by performing signal processing such as A / D conversion on the analog imaging signal obtained by the imaging element 25, and outputs the generated digital image to the DSP 31. The CCD image sensor is driven by a CCD driver built into the AFE. Of course, the CCD driver may be provided separately.

[0040] The DSP 31 performs various types of digital signal processing on the digital image, such as demosaic processing, noise removal processing, tone correction processing, and color correction processing.

[0041] The DSP 31 outputs the digital image after digital signal processing for each frame to the image memory 32. The image memory 32 stores the digital image from the DSP 31. Note that, hereinafter, for convenience of explanation, the digital image stored in the image memory 32 is also referred to as a "captured image."

[0042] The shake amount detection sensor 40 is, for example, a device including a gyro sensor, and detects the amount of shake of the surveillance camera 10. In other words, the shake amount detection sensor 40 detects the amount of shake in each of a pair of axial directions. The gyro sensor detects the amount of rotational shake around each axis (see FIG. 1), the pitch axis PA, the yaw axis YA, and the roll axis RA (axis parallel to the optical axis OA). The shake amount detection sensor 40 detects the amount of shake of the surveillance camera 10 by converting the amount of rotational shake around the pitch axis PA and the amount of rotational shake around the yaw axis YA detected by the gyro sensor into the amount of shake in a two-dimensional plane parallel to the pitch axis PA and the yaw axis YA.

[0043] Here, a gyro sensor is given as an example of the shake amount detection sensor 40, but this is merely an example, and the shake amount detection sensor 40 may be an acceleration sensor. The acceleration sensor detects the amount of shake in a two-dimensional plane parallel to the pitch axis PA and the yaw axis YA. The shake amount detection sensor 40 outputs the detected shake amount to the CPU 37.

[0044] In addition, although an example in which the shake amount is detected by a physical sensor called the shake amount detection sensor 40 is given here, the technology of the present disclosure is not limited to this. For example, a motion vector obtained by comparing captured images stored in the image memory 32 one after the other in a time series may be used as the shake amount. Also, the shake amount to be finally used may be derived based on the shake amount detected by the physical sensor and the motion vector obtained by image processing.

[0045] The CPU 37 acquires the amount of shake detected by the shake amount detection sensor 40, and controls the lens side shake compensation mechanism 29, the image sensor side shake compensation mechanism 45, and the electronic shake compensation unit 33 based on the acquired amount of shake. The amount of shake detected by the shake amount detection sensor 40 is used for shake compensation by each of the lens side shake compensation mechanism 29 and the electronic shake compensation unit 33.

[0046] The electronic shake correction unit 33 is a device including an ASIC. The electronic shake correction unit 33 corrects shake by performing image processing on the captured image in the image memory 32 obtained by imaging by the imaging element 25 based on the shake amount detected by the shake amount detection sensor 40. Note that, although an example in which shake is corrected by the electronic shake correction unit 33 based on the shake amount detected by the shake amount detection sensor 40 is given here, the technology of the present disclosure is not limited to this. For example, shake may be corrected by the electronic shake correction unit 33 according to a motion vector calculated based on the difference between frames of the captured image.

[0047] Further, here, a device including an ASIC is exemplified as the electronic shake correction unit 33, but the technology of the present disclosure is not limited thereto, and may be, for example, a device including an FPGA or a PLD. Furthermore, for example, the electronic shake correction unit 33 may be a device including two or more of an ASIC, an FPGA, and a PLD. Furthermore, a computer including a CPU, a storage, and a memory may be adopted as the electronic shake correction unit 33. The CPU may be a single one or a plurality of CPUs. Furthermore, the electronic shake correction unit 33 may be realized by a combination of a hardware configuration and a software configuration.

[0048] The communication I / F 34 is, for example, a network interface, and controls the transmission of various information between the management device 11 and the monitoring camera 10 via a network. An example of the network is a WAN such as the Internet or a public communication network. The communication I / F 34 manages communication between the monitoring camera 10 and the management device 11.

[0049] The UI device 43 includes a reception device 43A and a display 43B. The reception device 43A is, for example, a hard key or a touch panel, and receives various instructions from a user. The CPU 37 acquires the various instructions accepted by the reception device 43A and operates according to the acquired instructions.

[0050] The display 43B displays various information under the control of the CPU 37. Examples of the various information displayed on the display 43B include the contents of various instructions accepted by the accepting device 43A, captured images, and the like.

[0051] The optical system 15 includes an objective lens 15A and a lens group 15B. The objective lens 15A and the lens group 15B are arranged along the optical axis OA of the optical system 15 from the target subject side to the light receiving surface 25A side of the image sensor 25 in the order of the objective lens 15A and the lens group 15B. The lens group 15B includes an anti-vibration lens 15B1, a focus lens (not shown), and a zoom lens 15B2. The optical system 15 may include various lenses (not shown) in addition to the objective lens 15A and the lens group 15B. Furthermore, the optical system 15 may include an aperture. The positions of the lenses, lens group, and aperture included in the optical system 15 are not limited, and the technology of the present disclosure is valid even if they are located at positions different from those shown in FIG. 4, for example.

[0052] The surveillance camera 10 includes actuators 17 and 21, and an actuator 27. The zoom lens 15B2 is supported by the actuator 21, which will be described later, so as to be movable along the optical axis OA. The vibration-proof lens 15B1 is supported by the actuator 17 so as to be movable in a direction perpendicular to the optical axis OA. The imaging element 25 is supported by the actuator 27 so as to be movable in a direction perpendicular to the optical axis OA.

[0053] The actuator 17 applies power to the anti-vibration lens 15B1 to move it in a direction perpendicular to the optical axis of the anti-vibration lens 15B1. The actuator 17 is controlled by the CPU 37 via a displacement driver 23. The displacement driver 23 operates the actuator 17 in accordance with instructions from the CPU 37 to move the position of the anti-vibration lens 15B1 in a direction perpendicular to the optical axis OA.

[0054] The actuator 21 applies power to the zoom lens 15B2 to move it along the optical axis OA of the optical system 15. The actuator 21 is controlled by the CPU 37 via a zoom driver 28. The zoom driver 28 operates the actuator 21 in accordance with instructions from the CPU 37, thereby moving the position of the zoom lens 15B2 along the optical axis OA.

[0055] The actuator 27 applies power to the imaging element 25 to move the imaging element 25 in a direction perpendicular to the optical axis of the vibration-proof lens 15B1. The actuator 27 is controlled by the CPU 37 via the displacement driver 22. The displacement driver 22 operates the actuator 27 in accordance with instructions from the CPU 37 to move the position of the imaging element 25 in a direction perpendicular to the optical axis OA.

[0056] The surveillance camera 10 has a variable magnification mechanism 18. The variable magnification mechanism 18 includes a zoom lens 15B2, an actuator 21, and a zoom driver 28, and is a mechanism capable of changing the focal length. The CPU 37 operates the actuator 21 via the zoom driver 28 to change the focal length. In the surveillance camera 10, the zoom lens 15B2 is positioned closer to the telephoto side than the current position by making the focal length longer than the current position, so that the angle of view becomes narrower. The narrower angle of view means that the imaging range becomes narrower. In addition, the surveillance camera 10 is configured so that the zoom lens 15B2 is positioned closer to the wide-angle side than the current position by making the focal length shorter than the current position, so that the angle of view becomes wider. The wider angle of view means that the imaging range becomes wider. Note that the diagram of the variable magnification mechanism 18 illustrated in the technology disclosed herein is merely a conceptual diagram, and the variable magnification mechanism 18 may have various configurations.

[0057] Incidentally, vibrations given to the surveillance camera 10 include, if outdoors, vibrations due to passing cars, vibrations due to wind, vibrations due to road construction, etc., and if indoors, vibrations due to the operation of an air conditioner, vibrations due to people entering and exiting, etc. Therefore, in the surveillance camera 10, shaking occurs due to vibrations given to the surveillance camera 10 (hereinafter also simply referred to as "vibration"). Hereinafter, shaking caused by vibrations is also simply referred to as "shake".

[0058] In this embodiment, "shake" refers to a phenomenon in which the subject image on the light receiving surface 25A of the image sensor 25 of the surveillance camera 10 fluctuates due to a change in the positional relationship between the optical axis OA and the light receiving surface 25A. In other words, "shake" can be said to be a phenomenon in which the optical image formed on the light receiving surface 25A fluctuates due to the optical axis OA tilting caused by vibrations applied to the surveillance camera 10. The fluctuation of the optical axis OA means, for example, that the optical axis OA tilts with respect to a reference axis (for example, the optical axis OA before the occurrence of the shake).

[0059] The shake is included in the captured image as a noise component and affects the image quality of the captured image. Therefore, in order to remove the noise component included in the captured image due to the shake, the surveillance camera 10 is provided with a shake correction unit 24. The shake correction unit 24 is an example of an "alignment unit (alignment component)" according to the technology of the present disclosure. The shake correction unit 24 has a lens side shake correction mechanism 29, an image sensor side shake correction mechanism 45, and an electronic shake correction unit 33. The lens side shake correction mechanism 29, the image sensor side shake correction mechanism 45, and the electronic shake correction unit 33 are all used to correct shake. The lens side shake correction mechanism 29 is a mechanism that applies power generated by a driving source such as a motor (e.g., a voice coil motor) to the vibration-proof lens to move the vibration-proof lens in a direction perpendicular to the optical axis of the imaging optical system, thereby correcting shake. The image sensor side shake correction mechanism 45 is a mechanism that applies power generated by a driving source such as a motor (e.g., a voice coil motor) to the image sensor to move the image sensor in a direction perpendicular to the optical axis of the imaging optical system, thereby correcting shake. The electronic shake correction unit 33 corrects shake by performing image processing on the captured image based on the amount of shake. That is, the shake correction unit 24 performs shake correction mechanically or electronically with a hardware configuration and / or software configuration. Here, mechanical shake correction refers to shake correction realized by mechanically moving a shake correction element such as an anti-vibration lens and / or an image sensor using power generated by a driving source such as a motor (e.g., a voice coil motor), and electronic shake correction refers to shake correction realized by, for example, image processing performed by a processor. In this embodiment, "shake correction" includes the meaning of reducing shake in addition to eliminating shake. The lens side shake correction mechanism 29 is a mechanism that moves the vibration-proof lens 15B1 in a direction perpendicular to the optical axis OA, and is an example of a "lens moving mechanism" according to the technology of the present disclosure. The image sensor side shake correction mechanism 45 and the electronic shake correction unit 33 are mechanisms that move the area of ​​the captured image on the image sensor 25 in a direction perpendicular to the optical axis OA.The image sensor side shake correction mechanism 45 and the electronic shake correction unit 33 are examples of an "image area moving unit (image area moving component)" and an "image sensor side shake correction unit (image sensor side shake correction component)" according to the technique of the present disclosure.

[0060] As an example, as shown in FIG. 4, the lens side shake correction mechanism 29 includes an anti-vibration lens 15B1, an actuator 17, a displacement driver 23, and a position detection sensor 39.

[0061] Various well-known methods can be adopted as a method of correcting shake by the lens side shake correction mechanism 29. In this embodiment, the method of correcting shake is to move the vibration-proof lens 15B1 based on the amount of shake detected by the shake amount detection sensor 40. Specifically, the vibration is corrected by moving the vibration-proof lens 15B1 in a direction that cancels the shake and by an amount that cancels the shake.

[0062] An actuator 17 is attached to the anti-vibration lens 15B1. The actuator 17 is a shift mechanism equipped with a voice coil motor, and by driving the voice coil motor, the anti-vibration lens 15B1 is moved in a direction perpendicular to the optical axis of the anti-vibration lens 15B1. Note that, although a shift mechanism equipped with a voice coil motor is used as the actuator 17 here, the technology of the present disclosure is not limited to this, and other power sources such as a stepping motor or a piezoelectric element may be applied instead of the voice coil motor.

[0063] The actuator 17 is controlled by a displacement driver 23. When the actuator 17 is driven under the control of the displacement driver 23, the position of the vibration-proof lens 15B1 is mechanically displaced within a two-dimensional plane perpendicular to the optical axis OA.

[0064] The position detection sensor 39 detects the current position of the anti-vibration lens 15B1 and outputs a position signal indicating the detected current position. Here, a device including a Hall element is used as an example of the position detection sensor 39. Here, the current position of the anti-vibration lens 15B1 refers to the current position in the anti-vibration lens two-dimensional plane. The anti-vibration lens two-dimensional plane refers to a two-dimensional plane perpendicular to the optical axis of the anti-vibration lens 15B1. Note that in this embodiment, a device including a Hall element is used as an example of the position detection sensor 39, but the technology of the present disclosure is not limited to this, and a magnetic sensor, a photo sensor, or the like may be used instead of the Hall element.

[0065] The lens side shake correction mechanism 29 corrects shake by moving the vibration-proof lens 15B1 along at least one of the pitch axis PA direction and the yaw axis YA direction. In other words, the lens side shake correction mechanism 29 corrects shake by moving the vibration-proof lens 15B1 in a two-dimensional vibration-proof lens plane by an amount of movement corresponding to the amount of shake detected by the shake amount detection sensor 40.

[0066] The image sensor side shake correction mechanism 45 includes the image sensor 25 , a displacement driver 22 , an actuator 27 , and a position detection sensor 47 .

[0067] As with the shake correction method by the lens side shake correction mechanism 29, the shake correction method by the image sensor side shake correction mechanism 45 can employ various well-known methods. In this embodiment, the shake correction method employs a method in which the displacement driver 22 corrects the shake by moving the image sensor 25 via the actuator 27 according to an instruction from the CPU 37 based on the amount of shake detected by the shake amount detection sensor 40. Specifically, the shake correction is performed by moving the image sensor 25 in a direction that cancels the shake and by an amount that cancels the shake.

[0068] The actuator 27 is a shift mechanism equipped with a voice coil motor, and by driving the voice coil motor, the imaging element 25 is moved in a direction perpendicular to the optical axis of the vibration-proof lens 15B1. Note that, although a shift mechanism equipped with a voice coil motor is used as the actuator 27 here, the technology of the present disclosure is not limited to this, and other power sources such as a stepping motor or a piezoelectric element may be applied instead of the voice coil motor.

[0069] The position detection sensor 47 detects the current position of the image sensor 25 and outputs a position signal indicating the detected current position. Here, a device including a Hall element is used as an example of the position detection sensor 47. Here, the current position of the image sensor 25 refers to the current position in a two-dimensional plane of the image sensor. The two-dimensional plane of the image sensor refers to a two-dimensional plane perpendicular to the optical axis of the vibration-proof lens 15B1. Note that, in this embodiment, a device including a Hall element is used as an example of the position detection sensor 47, but the technology of the present disclosure is not limited to this, and a magnetic sensor, a photo sensor, or the like may be used instead of the Hall element.

[0070] 5, the management device 11 includes a display 13, a control device 60, a receiving device 62, and a communication I / F 66. The rotation mechanism 16 includes a yaw axis rotation mechanism 71, a pitch axis rotation mechanism 72, a motor 73, a motor 74, a driver 75, and a driver 76.

[0071] The control device 60 includes a CPU 60A, a storage 60B, and a memory 60C. The accepting device 62, the display 13, the secondary storage device 14, the CPU 60A, the storage 60B, the memory 60C, and the communication I / F 66 are each connected to a bus 70. Note that, in the example shown in Fig. 5, for convenience of illustration, one bus is shown as the bus 70, but multiple buses may be used. The bus 70 may be a serial bus, or may be a parallel bus including a data bus, an address bus, a control bus, and the like.

[0072] The memory 60C temporarily stores various information and is used as a work memory. An example of the memory 60C is a RAM, but the memory 60C may be another type of storage device. The storage 60B stores various programs for the management device 11 (hereinafter simply referred to as "programs for the management device"). The CPU 60A reads out the programs for the management device from the storage 60B and executes the read programs for the management device on the memory 60C to control the entire management device 11.

[0073] The communication I / F 66 is, for example, a network interface. The communication I / F 66 is communicatively connected to the communication I / F 34 of the surveillance camera 10 via a network, and controls the transmission of various information between the surveillance camera 10. For example, the communication I / F 66 requests the surveillance camera 10 to transmit a captured image, and receives the captured image transmitted from the communication I / F 34 of the surveillance camera 10 in response to the request to transmit the captured image.

[0074] The communication I / Fs 67 and 68 are, for example, network interfaces. The communication I / F 67 is communicatively connected to a driver 75 of the turning mechanism 16 via a network. The CPU 60A controls the turning operation of the yaw axis turning mechanism 71 by controlling the motor 73 via the communication I / F 67 and the driver 75. The communication I / F 68 is communicatively connected to a driver 76 of the turning mechanism 16 via a network. The CPU 60A controls the turning operation of the pitch axis turning mechanism 72 by controlling the motor 74 via the communication I / F 68 and the driver 76.

[0075] The receiving device 62 is, for example, a keyboard, a mouse, a touch panel, etc., and receives various instructions from a user. The CPU 60A acquires the various instructions accepted by the receiving device 62 and operates according to the acquired instructions. For example, when the receiving device 62 accepts processing content for the surveillance camera 10 and / or the rotation mechanism 16, the CPU 60A operates the surveillance camera 10 and / or the rotation mechanism 16 according to the instruction content accepted by the receiving device 62.

[0076] The display 13 displays various information under the control of the CPU 60 A. Examples of the various information displayed on the display 13 include the contents of various instructions accepted by the accepting device 62 and captured images received by the communication I / F 66.

[0077] The management device 11 includes a secondary storage device 14. The secondary storage device 14 is, for example, a non-volatile memory, and stores various information under the control of the CPU 60A. Examples of the various information stored in the secondary storage device 14 include captured images received by the communication I / F 66.

[0078] In this manner, the control device 60 controls the display 13 to display the captured image received by the communication I / F 66 , and controls the secondary storage device 14 to store the captured image received by the communication I / F 66 .

[0079] In this embodiment, the control device 60 causes the captured image to be displayed on the display 13 and also causes the captured image received by the communication I / F 66 to be stored in the secondary storage device 14, but the technology of the present disclosure is not limited to this. For example, the captured image may be either displayed on the display 13 or stored in the secondary storage device 14.

[0080] The motor 73 generates power by being driven under the control of a driver 75. The yaw axis rotation mechanism 71 receives the power generated by the motor 73 and rotates the surveillance camera 10 in the yaw direction. The motor 74 generates power by being driven under the control of a driver 76. The pitch axis rotation mechanism 72 receives the power generated by the motor 74 and rotates the surveillance camera 10 in the pitch direction.

[0081] As an example, as shown in FIG. 6, the storage 60B of the management device 11 stores an alignment control processing program 36A, a display control processing program 36B, and a magnification mechanism control processing program 36C.

[0082] The CPU 60A reads out the magnification change mechanism control processing program 36C from the storage 60B, and executes the read out magnification change mechanism control processing program 36C on the memory 60C (see FIG. 5) to operate as a magnification change mechanism control unit 37S.

[0083] The accepting device 62 accepts an instruction to cause the magnification change mechanism 18 to change the magnification (hereinafter also referred to as a "magnification change instruction"), and the magnification change mechanism control unit 37S controls the magnification change mechanism 18 in accordance with the magnification change instruction accepted by the accepting device 62. Furthermore, the memory 35 of the surveillance camera 10 has a focal length storage area 35A. The latest focal length is overwritten and saved in the focal length storage area 35A.

[0084] The CPU 60A reads the display control processing program 36B from the storage 60B, and executes the read display control processing program 36B on the memory 60C to operate as a display control unit 37T.

[0085] The display control unit 37T reads out a captured image from the image memory 32 of the surveillance camera 10, and displays the read out captured image as a live view image on the display 13. Note that, although a live view image is exemplified here as an image displayed on the display 13, the technology of the present disclosure is not limited thereto, and the image displayed on the display 13 may be a moving image for recording or a still image for recording.

[0086] The CPU 60A is an example of a "processor" according to the technology of the present disclosure, and the memory 60C is an example of a "memory" according to the technology of the present disclosure. The CPU 60A reads out the alignment control processing program 36A from the storage 60B, and executes the read out alignment control processing program 36A on the memory 60C. Specifically, the CPU 60A operates as an acquisition unit 37P, a focal length change determination unit 37Q, a subject image detection unit 37A, an image position determination unit 37B, a deviation amount calculation unit 37D, a deviation determination unit 37E, a shake correction execution unit 37F, and an alignment control unit 37G. The subject image detection unit 37A and the image position determination unit 37B are examples of a "detection unit" according to the technology of the present disclosure.

[0087] 7, the acquisition unit 37P acquires the focal length from the focal length storage area 35A. When the focal length acquired by the acquisition unit 37P becomes longer, the angle of view that can be captured by the surveillance camera 10 becomes narrower than before the focal length was increased.

[0088] The focal length change determination unit 37Q determines whether the focal length acquired by the acquisition unit 37P has changed. Specifically, the focal length acquired by the acquisition unit 37P at the previous acquisition timing (hereinafter also referred to as the "previous focal length") is stored in the internal memory 37M of the CPU 60A in the management device 11. The focal length change determination unit 37Q compares the previous focal length stored in the internal memory 37M with the latest focal length acquired by the acquisition unit 37P (hereinafter referred to as the "latest focal length"). Then, if the previous focal length and the latest focal length differ, it determines that the focal length has changed, and if they do not differ, it determines that the focal length has not changed.

[0089] After making the determination, focal length change determination unit 37Q overwrites and saves the value of the latest focal length in internal memory 37M, thereby updating the value of the previous focal length in internal memory 37M. The updated previous focal length is used by focal length change determination unit 37Q as a comparison target with the latest focal length the next time a determination is made by focal length change determination unit 37Q.

[0090] The subject image detection unit 37A acquires one frame of a captured image from the image memory 32. Then, the subject image detection unit 37A detects a target subject image in the captured image acquired from the image memory 32. Furthermore, when the subject image detection unit 37A detects a target subject image, it detects the position of the target subject image (hereinafter also referred to as "subject image position").

[0091] The image position determination unit 37B determines whether or not the subject image position detected by the subject image detection unit 37A is in the central region in the captured image. As an example, as shown in Fig. 11A and Fig. 11B, the subject image position may be in the central region in the captured image. In contrast, as an example, as shown in Fig. 12A and Fig. 12B, the surveillance camera 10 may be rotated by the rotation mechanism 16, or the target subject itself may move, so that the target subject may be present in the captured image but not in the central region.

[0092] If the target subject image is present in the central region, the determination by the image position determination unit 37B is positive. Here, "existence" means that at least a part of the target subject image is covered in the central region. In other words, if at least a part of the target subject image is covered in the central region, the image position determination unit 37B determines that the target subject image is "present". For example, at least a part of the target subject image may be a specific part in the subject image (for example, the face of the person if the target subject is a person). The surveillance camera 10 is equipped with an image recognition function (for example, a face recognition function), and if a specific part is recognized in the central region by the image recognition function, the image position determination unit 37B may determine that the target subject image is present in the central region. The central region is an example of a "specific position" according to the technology of the present disclosure.

[0093] The deviation amount calculation unit 37D calculates the deviation amount when the image position determination unit 37B determines that the subject image position is not within the central area CE. The deviation amount is an amount indicating the deviation of the subject image position with respect to the central area CE, i.e., the difference in relative position, and is a vector having two values ​​in the pitch direction and the yaw direction. An example of the deviation amount is the deviation amount between the center of the target subject image and the center of the central area CE. As an example, the center of the target subject image can be obtained as two values, the central coordinate in the pitch direction and the central coordinate in the yaw direction of the target subject image.

[0094] The deviation determination unit 37E determines whether or not there is deviation of the subject image position from the central region based on the deviation amount calculated by the deviation amount calculation unit 37D. The presence or absence of deviation can be determined, for example, by whether or not the deviation amount calculated by the deviation amount calculation unit 37D exceeds a predetermined threshold value. That is, if the deviation amount does not exceed the threshold value, it is determined that there is no deviation. On the other hand, if the deviation amount is equal to or greater than the threshold value, it is determined that there is deviation.

[0095] The threshold value may be a fixed value derived in advance as an allowable upper limit of the amount of deviation by a sensory test using an actual device and / or a computer simulation, etc. The threshold value may be a variable value that can be changed according to an instruction accepted by the accepting device 62, etc., or a variable value that can be changed according to an imaging scene, etc.

[0096] The shake correction execution unit 37F operates the shake correction unit 24 based on the shake amount detected by the shake amount detection sensor 40 when the deviation determination unit 37E determines that there is no deviation.

[0097] As an example, as shown in Fig. 8, a ratio derivation table 60L is stored in the storage 60B. As an example, as shown in Fig. 9, the ratio derivation table 60L is used by the alignment control unit 37G in the alignment control. The ratio derivation table 60L is a table indicating whether to increase the ratio of application of the image sensor side shake compensation mechanism 45 compared to the lens side shake compensation mechanism 29, or to increase the ratio of application of the lens side shake compensation mechanism 29 compared to the image sensor side shake compensation mechanism 45, depending on the focal length acquired by the acquisition unit 37P.

[0098] For ease of explanation, hereinafter, the ratio of application of the image sensor side shake compensation mechanism 45 compared to the lens side shake compensation mechanism 29 will also be referred to as the "first application ratio," and the ratio of application of the lens side shake compensation mechanism 29 compared to the image sensor side shake compensation mechanism 45 will also be referred to as the "second application ratio." When there is no need to distinguish between the first application ratio and the second application ratio, they will be referred to as the "application ratio." Here, the image sensor side shake compensation mechanism 45 is exemplified as an element that defines the application ratio, but the technology of the present disclosure is not limited thereto, and the electronic shake compensation unit 33 may be applied instead of the image sensor side shake compensation mechanism 45 or together with the image sensor side shake compensation mechanism 45.

[0099] In the examples shown in Figures 8 and 9, when the focal length is less than threshold value th, the lens side shake compensation mechanism 29 is used more frequently than the image sensor side shake compensation mechanism 45, and when the focal length is greater than or equal to threshold value th, the image sensor side shake compensation mechanism 45 is used more frequently than the lens side shake compensation mechanism 29.

[0100] In particular, when the focal length is within a predetermined range including the threshold th, the first application ratio gradually decreases and the second application ratio gradually increases as the focal length increases. As an example of the predetermined range, as shown in Figs. 8 and 9, a range of x on the negative and positive sides of the threshold th, that is, a range from "threshold th-x" to "threshold th+x" (hereinafter also referred to as "predetermined range"). The "threshold th-x" is an example of the "lower limit of the specified range" in the technology of the present disclosure, and the "threshold th+x" is an example of the "upper limit of the specified range" in the technology of the present disclosure. In the predetermined range, the first application ratio decreases linearly and the second application ratio increases linearly. Note that, here, an example in which the first application ratio decreases linearly and the second application ratio increases linearly is described, but the technology of the present disclosure is not limited to this, and within the above-mentioned specified range, the first application ratio may decrease nonlinearly (for example, exponentially) and the second application ratio may also increase nonlinearly (for example, exponentially). Also, within the above-mentioned specified range, one of the first application ratio and the second application ratio may change nonlinearly and the other may change linearly. In this case, it is sufficient that the first application ratio decreases and the second application ratio increases.

[0101] When the focal length is within a predetermined range, only the lens-side shake correction mechanism 29 is used in the alignment control, and when the focal length is in a range exceeding the "threshold value th-x", only the image sensor-side shake correction mechanism 45 is used in the alignment control.

[0102] Of the lens-side shake compensation mechanism 29 and the image sensor-side shake compensation mechanism 45, the one with the smaller application ratio is used for shake compensation by the shake compensation execution unit 37F, and can exert the effect of correcting shake. For example, when the focal length exceeds the "threshold th+x", the lens-side shake compensation mechanism 29 is used for shake compensation by the shake compensation execution unit 37F, and when the focal length is less than the "threshold th-x", the image sensor-side shake compensation mechanism 45 is used for shake compensation by the shake compensation execution unit 37F.

[0103] Note that, although a ratio derivation table 60L is illustrated here as an example, the technology of the present disclosure is not limited to this, and instead of the ratio derivation table 60L, a ratio calculation formula in which the focal length is an independent variable and the application ratio is a dependent variable may be used.

[0104] The positioning control unit 37G performs positioning control. The positioning control includes a positioning control in which the first application ratio is increased when the focal length acquired by the acquisition unit 37P is equal to or greater than a threshold, and the second application ratio is increased when the focal length is less than the threshold, thereby aligning the subject image position detected by the subject image detection unit 37A to a specific position in the captured image. Specifically, in the positioning control, the positioning control unit 37G derives an application ratio corresponding to the amount of deviation calculated by the deviation amount calculation unit 37D from the ratio derivation table 60L stored in the storage 60B. Then, in the positioning control, the lens side shake correction mechanism 29 and / or the image sensor side shake correction mechanism 45 are driven by the positioning control unit 37G according to the derived application ratio, thereby aligning the subject image position to a specific position in the captured image.

[0105] Next, the operation of the monitoring system 2 according to the technology of the present disclosure will be described with reference to Fig. 10A and Fig. 10B. Fig. 10A and Fig. 10B show an example of the flow of the alignment control process executed by the CPU 60A according to the alignment control process program 36A. The flow of the alignment control process shown in Fig. 10A and Fig. 10B is an example of the "imaging support method" in the technology of the present disclosure.

[0106] In the alignment control process shown in FIG. 10A and FIG. 10B, first, in step ST10, the acquisition unit 37P determines whether or not the timing to acquire the focal length (hereinafter also referred to as the "focal length acquisition timing") has arrived. The focal length acquisition timing may be, for example, the timing to image the target subject at a default frame rate in the image sensor 25. More specifically, in the case where the frame rate is 60 frames / second and the acquisition unit 37P acquires the focal length for each frame, the focal length is acquired at a frequency of 60 times per second. The acquisition unit 37P may acquire the focal length for each plurality of frames, for example, may acquire the focal length once every two frames. In step ST10, if the focal length acquisition timing has not arrived, the determination is negative, and the alignment control process proceeds to step ST30. In step ST10, if the focal length acquisition timing has arrived, the determination is positive, and the alignment control process proceeds to step ST12.

[0107] In step ST12, the acquisition unit 37P acquires the latest focal length of the surveillance camera 10. Then, the positioning control process proceeds to step ST14.

[0108] In step ST14, the focal length change determination unit 37Q determines whether the focal length has changed based on the latest focal length acquired in step ST12 and the previous focal length in the internal memory 37M. If the focal length has not changed in step ST14, the determination is negative, and the alignment control process proceeds to step ST32 shown in Fig. 10B. If the focal length has changed in step ST14, the determination is positive, and the alignment control process proceeds to step ST16.

[0109] In step ST16, the focal length change determination unit 37Q updates the value of the previous focal length stored in the internal memory 37M to the value of the latest focal length.

[0110] In the next step ST18, the subject image detection unit 37A judges whether or not a target subject image has been detected from the captured image. In step ST18, if a target subject image has not been detected from the captured image, the judgment is negative, and the alignment control process proceeds to step ST30. In step ST18, if a target subject image has been detected from the captured image, the judgment is positive, and the alignment control process proceeds to step ST20.

[0111] In step ST20, the image position determination unit 37B detects the subject image position in the captured image. Then, the positioning control process proceeds to step ST21.

[0112] In the next step ST21, the deviation amount calculation section 37D calculates the deviation amount. Then, the alignment control process proceeds to step ST22.

[0113] In step ST22, the deviation determination unit 37E determines whether or not there is a deviation between the subject image position and the central region based on the deviation amount calculated in step ST21. If there is no deviation in step ST22, the determination is negative, and the alignment control process proceeds to step ST32. If there is a deviation in step ST22, the determination is positive, and the alignment control process proceeds to step ST24.

[0114] In step ST24, the shake correction execution unit 37F derives the application ratio according to the latest focal length acquired in step ST12 from the ratio derivation table 60L.

[0115] In the next step ST25, the shake correction execution unit 37F judges whether or not shake correction is in progress. That is, it judges whether or not shake correction is being performed by the shake correction unit 24 (hereinafter also referred to as "shake correction in progress"). In step ST25, if shake correction is not in progress, the judgment is negative, and the alignment control process proceeds to step ST28. In step ST25, if shake correction is in progress, the judgment is positive, and the alignment control process proceeds to step ST26.

[0116] In step ST26, the shake correction execution unit 37F causes the shake correction unit 24 to end the shake correction. Then, the alignment control process proceeds to step ST28.

[0117] In step ST28, the alignment control unit 37G executes alignment control in accordance with the deviation amount calculated in step ST21 and the application rate acquired in step ST12.

[0118] In the next step ST30, the alignment control unit 37G determines whether or not a condition for terminating the alignment control process (hereinafter referred to as the "termination condition") has been satisfied. An example of the termination condition is that an instruction to terminate the alignment control process has been accepted by the acceptance device 62. In step ST30, if the termination condition has not been satisfied, the determination is negative, and the alignment control process proceeds to step ST10. In step ST30, if the termination condition has been satisfied, the determination is positive, and the alignment control process ends.

[0119] In step ST32, the shake correction execution unit 37F judges whether shake correction is being performed. In step ST32, if shake correction is being performed, the judgment is affirmative, and the alignment control process proceeds to step ST30. In step ST32, if shake correction is not being performed, the judgment is negative, and the alignment control process proceeds to step ST34.

[0120] In step ST34, the shake correction execution unit 37F starts shake correction, and then the process proceeds to step ST30.

[0121] In the surveillance system 2, positioning control is performed to align the subject image position to the central area of ​​the captured image. By locating the subject image in the central area of ​​the captured image, it is possible to realize a state in which the subject image is easily visible.

[0122] In this embodiment, when performing alignment control, the application ratio of the lens side shake compensation mechanism 29 and the image sensor side shake compensation mechanism 45 is derived according to the focal length of the surveillance camera 10. Specifically, when the focal length is less than a threshold value th, the ratio of using the lens side shake compensation mechanism 29 is increased over the image sensor side shake compensation mechanism 45, and when the focal length is equal to or greater than the threshold value th, the ratio of using the image sensor side shake compensation mechanism 45 is increased over the lens side shake compensation mechanism 29, thereby performing alignment control.

[0123] 11A and 11B show an example of the relationship between the imaging range for imaging a target subject and the position of the target subject image in the captured image when the focal length of the surveillance camera 10 is on the wide-angle side of the telephoto side and the wide-angle side. In the example shown in Fig. 11A and 11B, the target subject image is in the central region of the captured image.

[0124] In this state, when the target subject moves, the subject image position shifts from the central region of the captured image, as shown in Figures 12A and 12B as an example, although in the examples shown in Figures 12A and 12B, the subject image position is within the range of the captured image.

[0125] When the focal length of the surveillance camera 10 is changed from the state shown in Figures 12A and 12B to the telephoto side of the wide-angle side, the angle of view becomes narrower. As an example, as shown in Figures 13A and 13B, the subject image position may be outside the range of the captured image.

[0126] In this manner, when the subject image position is outside the range of the captured image or outside the central region, in this embodiment, a positioning control process is performed, and the lens side shake compensation mechanism 29 and the image sensor side shake compensation mechanism 45 operate at an application ratio according to the focal length. As a result, it is possible to position the subject image position in the central region of the captured image, as shown in Fig. 14 as an example.

[0127] 8 and 9, when performing the positioning control process, if the focal length is equal to or greater than the threshold value th, the application rate of the image sensor side shake compensation mechanism 45 is increased compared to the application rate of the lens side shake compensation mechanism 29. The image sensor side shake compensation mechanism 45 moves the light receiving surface 25A on which the light from the subject is focused, so that the subject image position can be aligned to the central region regardless of the focal length.

[0128] On the other hand, when the focal length is less than the threshold value th, the ratio of application of the lens side image stabilization mechanism 29 is increased compared to the ratio of application of the image sensor side image stabilization mechanism 45. The lens side image stabilization mechanism 29 tilts the optical axis OA compared to before operation, thereby adjusting the position of the subject image position to the central region. Therefore, the position of the subject image position can be adjusted to the central region by moving the anti-vibration lens 15B1 in a direction perpendicular to the optical axis OA.

[0129] As described above, the surveillance camera 10 includes the imaging support device 44. When the focal length of the surveillance camera 10 is equal to or greater than the threshold th, the imaging support device 44 increases the proportion of application of the image sensor side shake correction mechanism 45, and when the focal length is less than the threshold th, increases the proportion of application of the lens side shake correction mechanism 29, thereby performing alignment control. Therefore, according to this configuration, the subject image position can be aligned to the central region (an example of a specific position) in the captured image, compared to when alignment control is performed solely on the image sensor side shake correction mechanism 45 or the lens side shake correction mechanism 29, regardless of the focal length.

[0130] In this embodiment, when the focal length is equal to or greater than the threshold value th, the application rate of the lens side shake compensation mechanism 29 in the alignment control is lower than the application rate of the image sensor side shake compensation mechanism 45. In this case, the lens side shake compensation mechanism 29 is caused to execute shake compensation control. This makes it possible to correct shake even while the alignment control is being performed.

[0131] In particular, when the focal length exceeds the "threshold th+x", the lens side shake correction mechanism 29 is not used for positioning control, so that it is possible to correct shake by using the lens side shake correction mechanism 29 only for shake correction.

[0132] On the other hand, when the focal length is less than the threshold value th, in the positioning control, the application rate of the image sensor side shake compensation mechanism 45 is lower than the application rate of the lens side shake compensation mechanism 29. In this case, by having the image sensor side shake compensation mechanism 45 execute shake compensation control, it is possible to correct shake even while the positioning control is being performed.

[0133] In particular, when the focal length is less than the “threshold th-x”, the image sensor side shake correction mechanism 45 is not used for positioning control, so that it is possible to correct shake by using the image sensor side shake correction mechanism 45 only for shake correction.

[0134] The surveillance camera 10 of this embodiment has a variable magnification mechanism. When the focal length is changing, alignment control is performed by using both the lens side shake correction mechanism 29 and the image sensor side shake correction mechanism 45. When the focal length is changing, the subject image position can be aligned to the central area of ​​the captured image with high accuracy, compared to when alignment control processing is performed using only the lens side shake correction mechanism 29 or the image sensor side shake correction mechanism 45.

[0135] In particular, in this embodiment, in a surveillance camera 10 having such a variable magnification mechanism, when the focal length is in the range of "threshold th-x" or more and "threshold th+x" or less, alignment control processing is performed using both the lens side shake compensation mechanism 29 and the image sensor side shake compensation mechanism 45. Therefore, the subject image position can be aligned to a specific position in the captured image with high accuracy compared to a case where alignment control is always performed using only the image sensor side shake compensation mechanism 45 or the lens side shake compensation mechanism 29 regardless of whether the focal length is in the range of "threshold th-x" or more and "threshold th+x" or less.

[0136] Moreover, in the focal length range between "threshold th-x" and "threshold th+x", the application rate of the lens side shake compensation mechanism 29 decreases linearly, and the application rate of the image sensor side shake compensation mechanism 45 increases linearly. When the focal length of the surveillance camera 10 is continuously changed by the magnification change mechanism 18, there is no abrupt switching from the lens side shake compensation mechanism 29 to the image sensor side shake compensation mechanism 45. This makes it possible to prevent the timing of switching from one of the alignment control process using only the lens side shake compensation mechanism 29 and the alignment control using only the image sensor side shake compensation mechanism 45 to the other from being visually recognized from the captured image.

[0137] In this embodiment, when the focal length of the surveillance camera 10 has not changed, the shake correction unit 24 executes shake correction control on either the lens side shake correction mechanism 29 or the image sensor side shake correction mechanism 45. Therefore, compared to a configuration in which shake is not corrected when the focal length has not changed, it is possible to obtain a captured image in which the effects of shake are reduced.

[0138] When the focal length of the surveillance camera 10 is less than the "threshold th-x", the shake correction unit 24 performs alignment control by applying the lens-side shake correction mechanism 29. When the focal length is less than the "threshold th-x", the image sensor-side shake correction mechanism 45 is not applied to the alignment control, so that the shake correction control can be performed using the image sensor-side shake correction mechanism 45, i.e., a margin for correcting shake can be secured.

[0139] When the focal length exceeds the "threshold th+x", the shake correction unit performs alignment control by applying the image sensor side shake correction mechanism 45. When the focal length exceeds the "threshold th+x", the lens side shake correction mechanism 29 is not applied to the alignment control, so that shake correction control can be performed using the lens side shake correction mechanism 29, that is, a margin for correcting shake can be secured.

[0140] [Second embodiment] Next, a second embodiment will be described. In the second embodiment, the same elements, members, etc. as those in the first embodiment are given the same reference numerals as those in the first embodiment, and detailed description thereof will be omitted. In addition, the overall configuration of a surveillance camera, which is an example of an imaging device in the second embodiment, is also the same as the surveillance camera 10 in the first embodiment, and therefore will not be illustrated.

[0141] In the second embodiment, when performing alignment control processing, the imaging support device also rotates the surveillance camera 10 by activating the rotation mechanism 16 in addition to the lens side shake correction mechanism 29 and / or the image sensor side shake correction mechanism 45.

[0142] For example, when the focal length is on the wide-angle side of the telephoto side and the wide-angle side, and the subject image position is not within the central area, it may be possible to bring the subject image position into the central area by operating the rotation mechanism 16 to rotate the surveillance camera 10. Furthermore, even if the subject image position is not within the captured image, it may be possible to bring the subject image position into the captured image by operating the rotation mechanism 16 to rotate the surveillance camera 10.

[0143] 15, in addition to the ratio derivation table 60L, a rotation amount derivation table 60M is stored in the storage 60B. The rotation amount derivation table 60M associates the deviation amount with the rotation amount of the surveillance camera 10. Here, the rotation amount is information used when moving the subject image position closer to the central area in the captured image based on the deviation amount between the subject image position and the central area.

[0144] The amount of rotation is determined according to each of the pair of axial directions. That is, the amount of rotation is determined according to each of the pitch axis PA direction and the yaw axis YA direction. Therefore, in the captured image, the subject image position can be brought closer to the central area CE in both the pitch axis PA direction and the yaw axis YA direction. Note that although the rotation amount derivation table 60M is illustrated here, the technology of the present disclosure is not limited to this. For example, instead of the rotation amount derivation table 60M, a calculation formula for deriving the amount of rotation may be applied in which the deviation amount is an independent variable and the amount of rotation is a dependent variable.

[0145] The alignment control unit 37G obtains the amount of rotation from the rotation amount derivation table 60M. Then, the alignment control unit 37G operates the rotation mechanism 16 with the derived amount of rotation, thereby rotating the surveillance camera 10 in the pitch direction and / or the yaw direction.

[0146] Figures 16A and 16B show an example of the flow of the alignment control process executed by the CPU 60A (see Figure 5) in the second embodiment. Figures 16A and 16B differ from the flow of the alignment control process shown in Figures 10A and 10B in that steps ST100 to ST106 are provided between step ST22 and step ST24. The flow of the alignment control process shown in Figures 16A and 16B is an example of an "imaging support method" in the technology of the present disclosure.

[0147] In step ST100, the alignment control unit 37G derives a rotation amount corresponding to the deviation amount from the rotation amount derivation table 60M stored in the storage 60B.

[0148] In step ST102, the alignment control unit 37G operates the rotation mechanism 16 to rotate the surveillance camera 10.

[0149] In step ST104, the alignment control unit 37G calculates the amount of deviation.

[0150] In step ST106, the alignment control unit 37G judges whether or not there is a misalignment. If there is no misalignment in step ST106, the judgment is negative, and the alignment control process proceeds to step ST32 (see FIG. 10B). If there is a misalignment in step ST106, the judgment is positive, and the alignment control process proceeds to step ST24.

[0151] Therefore, in the second embodiment, when there is a deviation between the subject image position and the central region, the rotation mechanism 16 is actuated to rotate the surveillance camera 10 by an amount of rotation corresponding to the amount of deviation. By rotating the surveillance camera 10, when the target subject image is located outside the captured image, it is possible to realize a state in which the target subject image is located within the captured image.

[0152] The accuracy of alignment by the lens side shake correction mechanism 29 and the image sensor side shake correction mechanism 45 is higher than that of alignment by rotating the surveillance camera 10. In contrast, alignment by rotating the surveillance camera 10 can align a wide range in a short time compared to alignment by the lens side shake correction mechanism 29 and the image sensor side shake correction mechanism 45. Therefore, as shown in Fig. 17 as an example, first, rough adjustment of alignment is performed by rotating the surveillance camera 10, and then fine adjustment of alignment can be performed with higher accuracy than that of alignment by rotating the surveillance camera 10 by the lens side shake correction mechanism 29 and the image sensor side shake correction mechanism 45.

[0153] In particular, in the case of alignment control using the rotation mechanism 16, if there is variation in the rotation angle caused by the operation of the rotation mechanism 16, it is difficult to position the subject image position in the central region with the alignment control using the rotation mechanism 16. In the alignment control of this embodiment, the rotation mechanism 16 is operated to perform coarse adjustment of the subject image position, and then the lens side shake compensation mechanism 29 and / or the image sensor side shake compensation mechanism 45 are operated to perform fine adjustment of the subject image position, thereby achieving high accuracy in alignment control.

[0154] In each of the above embodiments, an example was given in which the surveillance camera 10 has the image area moving unit including the image sensor side shake correction mechanism 45 and the electronic shake correction unit 33, and the image area moving unit applied in the positioning control is the image sensor side shake correction mechanism 45. However, the image area moving unit applied in the positioning control may be the electronic shake correction unit 33, or the image sensor side shake correction mechanism 45 and the electronic shake correction unit 33 may be used in combination.

[0155] In the above embodiments, the position alignment control process calculates the amount of deviation of the subject image position, and performs position alignment control when there is a deviation. Alternatively, a threshold value (hereinafter, referred to as the "deviation threshold value") may be set for the amount of deviation of the subject image position, and when the amount of deviation is equal to or greater than the deviation threshold value, the image stabilization unit 24 may be operated to perform position alignment control.

[0156] In this case, for example, the amount of deviation may be calculated at a fixed timing regardless of whether the focal length of the surveillance camera 10 has changed. For example, even if the focal length of the surveillance camera 10 has not changed and is maintained on the wide-angle side, the target subject position may be outside the central area even if it is inside the captured image due to the target subject itself moving, etc. If the amount of deviation is large in this state, when the magnification is changed to the telephoto side by the magnification change mechanism 18, the subject image may move outside the captured image while being enlarged. Therefore, the deviation threshold can be set based on whether the subject image position outside the central area moves outside the captured image due to the magnification change from the wide-angle side to the telephoto side.

[0157] In particular, when the image stabilizer is operated, the center of the captured image is misaligned with the optical axis, so that even if the subject image is located in the central area, changing the focal length to the telephoto side in this state may cause the subject image to move to the outside of the captured image. In this case, by performing positioning control, it is possible to align the subject image to the central area with high accuracy compared to a configuration that does not perform positioning control.

[0158] In this embodiment, the lens side shake correction mechanism 29 is given as an example of the "lens moving mechanism" according to the technology of the present disclosure, and the image sensor side shake correction mechanism 45 and the electronic shake correction unit 33 are given as an example of the "image area moving unit (image area moving component)" according to the technology of the present disclosure. The lens side shake correction mechanism 29, the image sensor side shake correction mechanism 45, and the electronic shake correction unit 33 are all examples of shake correction units, and are applied to correct the shift of the subject image position. However, the lens movement mechanism according to the technology of the present disclosure is not limited to a mechanism for correcting the shift of the subject image position, and may be any mechanism capable of moving the lens in a direction intersecting with the optical axis OA. The image area moving unit according to the technology of the present disclosure is also not limited to a configuration for correcting the shift of the subject image position, as long as it can move the area of ​​the captured image in the image sensor 25 in a direction intersecting with the optical axis OA.

[0159] The central region described in the above embodiment is an example of a "specific position" according to the technology of the present disclosure. The "specific position" is not limited to the central region of the captured image, and may be a position shifted from the center of the captured image in the pitch direction and yaw direction. By setting the specific position to the central region of the captured image, it is possible to align the subject image position to the center of the captured image.

[0160] Furthermore, in the surveillance camera 10, shake is corrected by moving the anti-vibration lens 15B1. Therefore, shake can be corrected within the movable range of the anti-vibration lens 15B1, and the subject image position can be aligned to the central region.

[0161] In the above embodiment, as shown in Fig. 9 as an example, in the positioning control, when the focal length is in a predetermined range, the application rate of the lens side shake compensation mechanism 29 decreases linearly and the application rate of the image sensor side shake compensation mechanism 45 increases linearly. In contrast to this, as shown in Fig. 18, for example, the control may be such that the application rate of the lens side shake compensation mechanism 29 is 100% when the focal length is less than a threshold th, and the application rate of the image sensor side shake compensation mechanism 45 is 100% when the focal length is equal to or greater than the threshold th. In the control shown in Fig. 18 as an example, only one of the lens side shake compensation mechanism 29 and the image sensor side shake compensation mechanism 45 is applied depending on whether the focal length is equal to or greater than the threshold th, making the positioning control easy.

[0162] In each of the above-described embodiments, the alignment control processing program 36A, the display control processing program 36B, and the zoom mechanism control processing program 36C are stored in the storage 60B of the management device 11. An example of a form in which these programs are executed in the memory 60C of the management device 11 has been given. The technology of the present disclosure is not limited to this. For example, these programs may be stored in the storage 36 of the surveillance camera 10, and the CPU 37 of the surveillance camera 10 may execute these programs in the memory 35. Furthermore, as an example, as shown in FIG. 19, the alignment control processing program 36A, the display control processing program 36B, and the zoom mechanism control processing program 36C may be stored in a storage medium 100, which is a non-transitory storage medium. In the example shown in FIG. 19, the alignment control processing program 36A, the display control processing program 36B, and the zoom mechanism control processing program 36C stored in the storage medium 100 are installed in the control device 60, and the CPU 60A executes each of the above-mentioned processes in accordance with the alignment control processing program 36A, the display control processing program 36B, and the zoom mechanism control processing program 36C.

[0163] 5, the CPU 60A is a single CPU, but the technology of the present disclosure is not limited to this, and multiple CPUs may be adopted. Note that an example of the storage medium 100 is any portable storage medium such as an SSD or a USB memory.

[0164] The positioning control processing program 36A may be stored in a storage unit of another computer or a server device connected to the computer 19 via a communication network (not shown), and the positioning control processing program 36A may be downloaded to the computer 19 in response to a request from the surveillance camera 10. In this case, the downloaded positioning control processing program 36A is executed by the CPU 37 of the computer 19.

[0165] In the example shown in FIG. 16, the positioning control processing program 36A, the display control processing program 36B, and the magnification mechanism control processing program 36C are installed in the control device 60 of the management device 11, but the technology of the present disclosure is not limited to this. The positioning control processing program 36A, the display control processing program 36B, and the magnification mechanism control processing program 36C may be installed in the computer 19 of the surveillance camera 10. In this case, the CPU 37 executes the above-mentioned positioning control processing according to the positioning control processing program 36A, executes the above-mentioned display control processing according to the display control processing program 36B, and executes the above-mentioned magnification mechanism control processing according to the magnification mechanism control processing program 36C. In addition, the positioning control processing, the display control processing, and the magnification mechanism control processing may be distributed between the surveillance camera 10 and the management device 11. For example, the positioning control may be executed by the CPU 60A of the management device 11, and the display control and the magnification mechanism control may be executed by the CPU 37 of the surveillance camera 10.

[0166] In the above embodiment, the zoom mechanism control unit 37S, the display control unit 37T, the acquisition unit 37P, the focal length change determination unit 37Q, the subject image detection unit 37A, the image position determination unit 37B, the shift amount calculation unit 37D, the shift determination unit 37E, and the alignment control unit 37G are implemented by software configuration of the control device 60. However, the technology of the present disclosure is not limited to this. For example, the zoom mechanism control unit 37S, the display control unit 37T, the acquisition unit 37P, the focal length change determination unit 37Q, the subject image detection unit 37A, the image position determination unit 37B, the shift amount calculation unit 37D, the shift determination unit 37E, and the alignment control unit 37G may be implemented by software configuration of the control device 60. The magnification mechanism control unit 37S, the display control unit 37T, the acquisition unit 37P, the focal length change determination unit 37Q, the subject image detection unit 37A, the image position determination unit 37B, the deviation amount calculation unit 37D, the deviation determination unit 37E, and / or the alignment control unit 37G may be realized by a device including, for example, an ASIC, an FPGA, and / or a PLD. Also, the magnification mechanism control unit 37S, the display control unit 37T, the acquisition unit 37P, the focal length change determination unit 37Q, the subject image detection unit 37A, the image position determination unit 37B, the deviation amount calculation unit 37D, the deviation determination unit 37E, and / or the alignment control unit 37G may be realized by a combination of a hardware configuration and a software configuration.

[0167] As hardware resources for executing the above-mentioned alignment control process, display control process, and zoom mechanism control process (hereinafter, these processes are collectively referred to as "control processes"), various processors as shown below can be used. As an example of a processor, as described above, there is a CPU, which is a general-purpose processor that functions as a hardware resource for executing control processes by executing software, i.e., a program. As another example of a processor, there is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing a specific process, such as an FPGA, a PLD, or an ASIC. Each processor has a built-in or connected memory, and each processor executes the control process by using the memory.

[0168] The hardware resource that executes the control process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the control process may be a single processor.

[0169] As an example of a single processor, first, one or more CPUs and software are combined to form one processor, and the processor functions as a hardware resource that executes the processes of the magnification mechanism control unit 37S, the display control unit 37T, the acquisition unit 37P, the focal length change determination unit 37Q, the subject image detection unit 37A, the image position determination unit 37B, the shift amount calculation unit 37D, the shift determination unit 37E, and the alignment control unit 37G. Second, as typified by SoC, a processor that realizes the functions of the entire system including multiple hardware resources that execute the alignment control process in one IC chip is used. In this way, the processes of the magnification mechanism control unit 37S, the display control unit 37T, the acquisition unit 37P, the focal length change determination unit 37Q, the subject image detection unit 37A, the image position determination unit 37B, the shift amount calculation unit 37D, the shift determination unit 37E, and the alignment control unit 37G are realized by using one or more of the above-mentioned various processors as hardware resources.

[0170] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.

[0171] Furthermore, the above-described alignment control process is merely an example, and it goes without saying that unnecessary steps may be deleted, new steps may be added, and the order of processes may be changed without departing from the spirit of the present invention.

[0172] In the above embodiment, the surveillance camera 10 is given as an example of an imaging device according to the technology of the present disclosure, but the technology of the present disclosure is not limited thereto. For example, instead of the surveillance camera 10, the technology of the present disclosure can be applied to various electronic devices such as a portable interchangeable lens camera, a portable fixed lens camera, a personal computer, a smart device, or a wearable terminal device. Even with these electronic devices, the same actions and effects as those of the surveillance camera 10 described in each of the above embodiments can be obtained.

[0173] Furthermore, the above control process is merely an example, and it goes without saying that unnecessary steps may be deleted, new steps may be added, and the process order may be changed without departing from the spirit of the present invention.

[0174] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[0175] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0176] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference.

[0177] The following supplementary notes are further disclosed regarding the above embodiment.

[0178] (Additional Note) A processor; A memory embedded in or connected to the processor; The processor is Acquire a focal length of an imaging device having a lens and an imaging element, the imaging device having a lens movement mechanism that moves the lens in a direction intersecting with an optical axis of the lens and an image area movement unit that moves an area of ​​an image captured on the imaging element in a direction intersecting with the optical axis; Detecting a subject image position of a target subject image showing the target subject in a captured image obtained by capturing an image of an imaging area including the target subject by an imaging device; When the acquired focal length is equal to or greater than a threshold value, the ratio of application of the image area moving unit is increased compared to the lens moving mechanism, and when the focal length is less than the threshold value, the ratio of application of the lens moving mechanism is increased compared to the image area moving unit, thereby performing positioning control to align the detected subject image position with a specific position within the captured image. Information processing device.

Claims

1. A processor; A memory embedded in or connected to the processor; The processor, Acquire a focal length of an imaging device having a lens and an imaging element, the imaging device including a lens movement mechanism that moves the lens in a direction intersecting an optical axis of the lens, and an image area movement component that moves an area of ​​an image captured on the imaging element in a direction intersecting the optical axis; detecting a subject image position of a target subject image showing the target subject within a captured image obtained by capturing an image of an imaging area including the target subject by the imaging device; A control table is stored in a storage device, and the control table defines an application ratio of the lens movement mechanism and the image area movement component corresponding to the focal length, and based on the control table, a positioning control is performed to align the detected subject image position with a specific position within the captured image. Imaging support device.

2. The control table includes: a first control region in which control is performed only by the lens moving mechanism; a second control area controlled by the lens moving mechanism and the image area moving component; and a third control area controlled only by the image area moving component. The imaging support device according to claim 1 .

3. the first control region corresponds to a focal length equal to or less than a first threshold value, the third control region corresponds to a focal length equal to or greater than a second threshold value, The second control region is a region corresponding to a region between the first threshold value and the second threshold value. The imaging support device according to claim 2 .

4. In the second control region, As the focal length increases, the proportion of the image region movement component applied increases; As the focal length becomes smaller, the application rate of the lens moving mechanism increases. The imaging support device according to claim 3 .

5. the lens movement mechanism is a lens-side image stabilization mechanism that corrects image shake caused by a vibration applied to the imaging device by moving the lens, The image area movement component is an image sensor side shake correction component that corrects the shake by moving an image sensor or by image processing of a captured image obtained by capturing an image with the image sensor. The imaging support device according to any one of claims 1 to 4.

6. the imaging device has a zoom mechanism that changes the focal length, The processor performs the alignment control by using the image sensor side shake correction component and the lens side shake correction mechanism in combination when the acquired focal length has changed. The imaging support device according to claim 5 .

7. The processor causes at least one of the image sensor side image stabilization component and the lens side image stabilization mechanism to correct the shake when the acquired focal length has not changed. The imaging support device according to claim 5 or 6.

8. The processor performs the alignment control by using the lens side shake correction mechanism and the image sensor side shake correction component in combination according to the application ratio. The imaging support device according to any one of claims 5 to 7.

9. The alignment control includes a control for gradually increasing a ratio of application of the image sensor side shake correction component compared to the lens side shake correction mechanism as the focal length becomes longer, the ratio being used as the application ratio when the focal length acquired by the processor is within a predetermined range. The imaging support device according to claim 8.

10. The processor may further comprise a step of: detecting the focal length of the lens when the obtained focal length is less than a lower limit value of a predetermined range; The alignment control is performed using the lens-side shake correction mechanism of the image sensor side shake correction component and the image sensor side shake correction component. The imaging support device according to claim 8 or 9.

11. When the acquired focal length exceeds an upper limit value of a predetermined range, the processor performs the alignment control using the image sensor side shake correction component of the lens side shake correction mechanism and the image sensor side shake correction component. The imaging support device according to any one of claims 8 to 10.

12. the processor performs the alignment control by using an alignment component including a rotation mechanism capable of rotating the imaging device, the image sensor side image stabilization component, and the lens side image stabilization mechanism; The image sensor side image stabilization component and the lens side image stabilization mechanism have a higher alignment accuracy for adjusting the subject image position within the captured image than the rotation mechanism. The imaging support device according to any one of claims 5 to 11.

13. An imaging support device according to claim 12; The pivot mechanism, The imaging support device is an imaging support system that supports imaging by the imaging device when the rotation mechanism rotates the imaging device.

14. An imaging support device according to any one of claims 1 to 12, The imaging device, The imaging support device is an imaging system that supports the imaging device in imaging.

15. The imaging device further includes a rotation mechanism for rotating the imaging device. The imaging system of claim 14.

16. Acquiring a focal length of an imaging device having a lens and an imaging element, the imaging device including a lens moving mechanism that moves the lens in a direction intersecting an optical axis of the lens, and an image area moving component that moves an area of ​​an image captured on the imaging element in a direction intersecting the optical axis; Detecting a subject image position of a target subject image showing the target subject within a captured image obtained by capturing an image of an imaging area including the target subject by the imaging device; and performing positioning control for aligning the detected subject image position to a specific position within the captured image based on a control table stored in a storage, the control table defining an application ratio of the lens movement mechanism and the image area movement component corresponding to the focal length; An imaging assistance method comprising:

17. On the computer, Acquiring a focal length of an imaging device having a lens and an imaging element, the imaging device including a lens moving mechanism that moves the lens in a direction intersecting an optical axis of the lens, and an image area moving component that moves an area of ​​an image captured on the imaging element in a direction intersecting the optical axis; Detecting a subject image position of a target subject image showing the target subject within a captured image obtained by capturing an image of an imaging area including the target subject by the imaging device; and performing positioning control for aligning the detected subject image position to a specific position within the captured image based on a control table stored in a storage, the control table defining an application ratio of the lens movement mechanism and the image area movement component corresponding to the focal length; A program for executing a process including the above.

Images