Information processor, method for processing information, and program

Abstract

To provide an information processor, a method for processing information, and a program which can move images along at least one of a first axis and a second axis of an image sensor even in a case where at least one of first and second driving axes of a driving mechanism are inclined with respect to the first axis and the second axis of the image sensor, respectively.SOLUTION: The processor of the information processor acquires inclination information on at least one of the inclination of a first driving axis of a driving mechanism with respect to a first axis of an image sensor and the inclination of a second driving axis of the driving mechanism with respect to a second axis of the image sensor, and calculates a movement amount of moving a movable lens along at least one of the first axis and the second axis on the basis of the inclination information.SELECTED DRAWING: Figure 16

Description

Technical Field

[0001] The technology of the present disclosure relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] Patent Document 1 discloses an imaging apparatus in which a lens device is detachable, and includes an imaging unit having a plurality of photoelectric conversion units that receive light beams incident through different pupil regions in the imaging optical system of the lens device and output a plurality of signals, a calculation unit that acquires the signals output by the plurality of photoelectric conversion units and calculates a defocus amount, and a correction unit that corrects the defocus amount calculated by the calculation unit. The correction unit performs imaging plane correction by correcting the defocus amount using correction information related to the optical characteristics of the lens device and correction information related to the inclination of the imaging plane of the imaging unit.

[0003] Patent Document 2 discloses an imaging apparatus including a lens barrel having a lens, an imaging element that receives light transmitted through the lens and generates an imaging signal, a driving unit that inclines either the imaging element or the lens with respect to a plane orthogonal to the optical axis of the lens, a diaphragm unit that adjusts the amount of light passing through the lens barrel, and a control unit that controls the driving unit to incline at least one of the lens and the imaging element based on the diaphragm value after the change or the luminance of the image captured by the imaging element when the diaphragm value of the diaphragm unit or the luminance of the image captured by the imaging element changes.

[0004] Patent Document 3 discloses an image blur correction apparatus including an imaging optical system and an imaging element that converts a subject image guided from the imaging optical system into an electrical signal, and corrects image blur by moving the imaging element. The image blur correction apparatus includes a fixed unit, an imaging element holding unit that holds the imaging element and moves together with the imaging element in a predetermined plane substantially orthogonal to the optical axis of the imaging optical system, a position restricting unit that restricts the position of the imaging element holding unit in the optical axis direction of the imaging optical system, a guide unit that guides the imaging element holding unit to be movable in a predetermined plane substantially orthogonal to the optical axis of the imaging optical system, and a driving unit that applies a biasing force to the imaging element holding unit.

Prior Art Documents

Patent Documents

[0005]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

[0006] One embodiment of the technology according to the present disclosure provides, as an example, an information processing apparatus, an information processing method, and a program capable of moving an image along at least one of a first axis and a second axis even when there is at least one of an inclination of a first drive axis of a drive mechanism with respect to a first axis of an image sensor and an inclination of a second drive axis of the drive mechanism with respect to a second axis of the image sensor.

Means for Solving the Problems

[0007] A first aspect of the technology according to the present disclosure includes a processor that acquires inclination information regarding at least one of an inclination of a first drive axis of a moving lens with respect to a first axis of an image sensor as viewed along an optical axis and an inclination of a second drive axis of the moving lens with respect to a second axis of the image sensor as viewed along the optical axis, and calculates a movement amount for moving the moving lens along at least one of the first axis and the second axis based on the inclination information.

[0008] A second aspect of the technology according to the present disclosure is the information processing apparatus according to the first aspect, wherein the inclination information is information calculated based on a control command for moving the moving lens along at least one of a first drive axis and a second drive axis and a plurality of images obtained by imaging with an image sensor before and after the moving lens moves based on the control command.

[0009] A third aspect of the technology according to the present disclosure is an information processing apparatus according to the first aspect or the second aspect, wherein the tilt information is information calculated based on a plurality of images obtained by imaging with an image sensor under imaging conditions in which an image with less noise than an image obtained by normal imaging is obtained.

[0010] A fourth aspect of the technology according to the present disclosure is an information processing apparatus according to any one of the first aspect to the third aspect, wherein the tilt information is information calculated based on a plurality of images obtained by imaging with an image sensor to which a sensitivity lower than the sensitivity of an image sensor performing normal imaging is applied.

[0011] A fifth aspect of the technology according to the present disclosure is an information processing apparatus according to any one of the first aspect to the fourth aspect, wherein the tilt information is information regarding at least one of a first tilt angle of a first drive axis with respect to a first axis viewed along the optical axis and a second tilt angle of a second drive axis with respect to a second axis viewed along the optical axis.

[0012] A sixth aspect of the technology according to the present disclosure is an information processing apparatus according to the fifth aspect, wherein the processor calculates a first movement amount for moving the movable lens along the first drive axis and a second movement amount for moving the movable lens along the second drive axis based on the tilt information.

[0013] A seventh aspect of the technology according to the present disclosure is an information processing apparatus according to any one of the first aspect to the sixth aspect, wherein the tilt information includes a first movement amount for moving the movable lens along the first drive axis and a second movement amount for moving the movable lens along the second drive axis, and the first movement amount and the second movement amount are movement amounts calculated based on at least one of the tilt of the first drive axis with respect to the first axis viewed along the optical axis and the tilt of the second drive axis with respect to the second axis viewed along the optical axis.

[0014] The eighth aspect according to the technology of the present disclosure includes obtaining tilt information regarding at least one of the tilt of the first drive axis of the movable lens with respect to the first axis of the image sensor as viewed along the optical axis, and the tilt of the second drive axis of the movable lens with respect to the second axis of the image sensor as viewed along the optical axis, and calculating a movement amount for moving the movable lens along at least one of the first axis and the second axis based on the tilt information.

[0015] The ninth aspect according to the technology of the present disclosure is a program for causing execution of a process including obtaining tilt information regarding at least one of the tilt of the first drive axis of the movable lens with respect to the first axis of the image sensor as viewed along the optical axis, and the tilt of the second drive axis of the movable lens with respect to the second axis of the image sensor as viewed along the optical axis, and calculating a movement amount for moving the movable lens along at least one of the first axis and the second axis based on the tilt information.

Brief Description of the Drawings

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

[0017] Hereinafter, an example of an embodiment of an information processing apparatus, an information processing method, and a program according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be described.

[0019] CPU refers to the abbreviation of "Central Processing Unit". GPU refers to the abbreviation of "Graphics Processing Unit". NVM refers to the abbreviation of "Non-Volatile Memory". RAM refers to the abbreviation of "Random Access Memory". IC refers to the abbreviation of "Integrated Circuit". ASIC refers to the abbreviation of "Application Specific Integrated Circuit". PLD refers to the abbreviation of "Programmable Logic Device". FPGA refers to the abbreviation of "Field-Programmable Gate Array". SoC refers to the abbreviation of "System-on-a-Chip". SSD refers to the abbreviation of "Solid State Drive". HDD refers to the abbreviation of "Hard Disk Drive". EEPROM refers to the abbreviation of "Electrically Erasable and Programmable Read Only Memory". SRAM refers to the abbreviation of "Static Random Access Memory". I / F refers to the abbreviation of "Interface". UI refers to the abbreviation of "User Interface". USB refers to the abbreviation of "Universal Serial Bus". CMOS refers to the abbreviation of "Complementary Metal Oxide Semiconductor". CCD refers to the abbreviation of "Charge Coupled Device". LAN refers to the abbreviation of "Local Area Network". WAN refers to the abbreviation of "Wide Area Network". BPF refers to the abbreviation of "Band Pass Filter". Ir refers to the abbreviation of "Infrared Rays".

[0020] In the description of this specification, "vertical" refers to vertical in the sense that, in addition to perfect verticality, it includes errors generally acceptable in the technical field to which the technology of the present disclosure belongs and that do not go against the spirit of the technology of the present disclosure. In the description of this specification, "horizontal" refers to horizontal in the sense that, in addition to perfect horizontality, it includes errors generally acceptable in the technical field to which the technology of the present disclosure belongs and that do not go against the spirit of the technology of the present disclosure. In the description of this specification, "parallel" refers to parallel in the sense that, in addition to perfect parallelism, it includes errors generally acceptable in the technical field to which the technology of the present disclosure belongs and that do not go against the spirit of the technology of the present disclosure. In the description of this specification, "orthogonal" refers to orthogonal in the sense that, in addition to perfect orthogonality, it includes errors generally acceptable in the technical field to which the technology of the present disclosure belongs and that do not go against the spirit of the technology of the present disclosure. In the description of this specification, "coincident" refers to coincident in the sense that, in addition to perfect coincidence, it includes errors generally acceptable in the technical field to which the technology of the present disclosure belongs and that do not go against the spirit of the technology of the present disclosure. In the description of this specification, "equidistant" refers to equidistant in the sense that, in addition to perfect equidistance, it includes errors generally acceptable in the technical field to which the technology of the present disclosure belongs and that do not go against the spirit of the technology of the present disclosure.

[0021] [First Embodiment] First, the first embodiment will be described.

[0022] (Surveillance System) As an example, as shown in FIG. 1, the surveillance system S includes a surveillance camera 10 and a management device 200. The surveillance camera 10 is an example of an "imaging device" according to the technology of the present disclosure, and the management device 200 is an example of an "external control device".

[0023] The monitoring camera 10 is installed, for example, on columns or walls inside and outside the building. The management device 200 is provided outside the imaging device and is installed, for example, in the management room in the management building. The monitoring camera 10 includes a monitoring camera body 20 and a lens device 70. The monitoring camera body 20 is an example of the "imaging device body" according to the technology of the present disclosure. The monitoring camera body 20 includes a lens mount 22. The lens device 70 is separate from the monitoring camera body 20 and is detachably attached to the lens mount 22.

[0024] The monitoring camera body 20 includes a controller 40 and a communication I / F 60. The lens device 70 includes a controller 90 and a communication I / F 100. The management device 200 includes a controller 210 and a communication I / F 220. The communication I / F 220 of the management device 200 is communicably connected to the communication I / F 60 of the monitoring camera body 20 and the communication I / F 100 of the lens device 70 by wire or wirelessly. Also, in a state where the lens device 70 is attached to the lens mount 22 of the monitoring camera body 20, a connector (not shown) provided on the monitoring camera body 20 and a connector (not shown) provided on the lens device 70 are connected. Then, the communication I / F 60 of the monitoring camera body 20 and the communication I / F 100 of the lens device 70 are communicably connected.

[0025] Note that the X-axis shown in FIG. 1 corresponds to the pitch axis of the monitoring camera 10, the Y-axis corresponds to the yaw axis of the monitoring camera 10, and the Z-axis corresponds to the roll axis of the monitoring camera 10. Hereinafter, the direction along the X-axis is referred to as the X-axis direction, the direction along the Y-axis is referred to as the Y-axis direction, and the direction along the Z-axis is referred to as the Z-axis direction. The X-axis direction, the Y-axis direction, and the Z-axis direction are orthogonal to each other.

[0026] (Monitoring camera 10) The monitoring camera body 20 includes an image sensor 24. The image sensor 24 is, for example, a CMOS image sensor, which photoelectrically converts the received light and outputs an electrical signal corresponding to the received light. The CMOS image sensor is merely an example, and the image sensor 24 may be an image sensor having an operation method different from that of a CMOS image sensor such as a CCD image sensor.

[0027] The image sensor 24 has a light receiving surface 24A. The imaging area light incident on the lens device 70 is imaged on the light receiving surface 24A by the lens device 70. An image is obtained when the imaging area light is imaged on the light receiving surface 24A. A plurality of photodiodes are arranged in a matrix on the light receiving surface 24A. Each photodiode receives the imaging area light. The image sensor 24 images the imaging area by receiving the imaging area light. As an example, the plurality of photodiodes include silicon photodiodes having sensitivity to visible light and indium gallium arsenide photodiodes having sensitivity to near-infrared light. The image sensor 24 performs imaging on each of the visible light and near-infrared light included in the imaging area light imaged on the light receiving surface 24A.

[0028] The lens device 70 has an optical axis OA. The optical axis OA is, for example, an axis that passes through the center of the light receiving surface 24A and is perpendicular to the light receiving surface 24A. The optical axis OA is parallel to the Z axis. The lens device 70 includes, as an example, an objective lens 72, a zoom lens 74, an image stabilization lens 76, a diaphragm 78, a filter unit 80, and a master lens 82. The objective lens 72, the zoom lens 74, the image stabilization lens 76, the diaphragm 78, the filter unit 80, and the master lens 82 are arranged in order from the subject side to the image side along the optical axis OA.

[0029] The image stabilization lens 76 is an example of the "moving lens" according to the technology of the present disclosure. Also, the plurality of lenses including the objective lens 72, the zoom lens 74, the image stabilization lens 76, the filter unit 80, and the master lens 82 are an example of the "lens" according to the technology of the present disclosure. The optical axis OA is an axis that passes through the centers of the objective lens 72, the zoom lens 74, the image stabilization lens 76, and the master lens 82. The optical axis OA is also the optical axis OA of the objective lens 72, the zoom lens 74, the image stabilization lens 76, and the master lens 82. The optical axis OA is an example of the "optical axis of the lens" according to the technology of the present disclosure.

[0030] The imaging area light is incident on the objective lens 72. The objective lens 72 guides the incident imaging area light to the zoom lens 74. The zoom lens 74 is composed of a lens group having a plurality of lenses movable along the optical axis OA, and is used for zooming the imaging area.

[0031] The shake correction lens 76 is a lens for correcting the blur of an image obtained by forming the imaging area light on the image sensor 24 as described later, and is also a lens for shifting the image along the light receiving surface 24A of the image sensor 24.

[0032] The aperture 78 has an aperture 78A. The imaging area light guided by the zoom lens 74 passes through the aperture 78A. The aperture 78 is a movable aperture 78 capable of changing the diameter of the aperture 78A. That is, the amount of light of the imaging area light is changed by the aperture 78.

[0033] The filter unit 80 is disposed on the subject side with respect to the image sensor 24. As an example, the filter unit 80 is disposed between the aperture 78 and the master lens 82. The imaging area light that has passed through the aperture 78 is incident on the filter unit 80. Although details will be described later, the filter unit 80 has a plurality of optical filters having translucency, and by switching the optical filter that transmits light among the plurality of optical filters, light in a plurality of wavelength bands included in the imaging area light (for example, visible light and near-infrared light in different wavelength bands within the near-infrared wavelength band) is selectively transmitted.

[0034] The imaging area light that has passed through the filter unit 80 is incident on the master lens 82, and the imaging area light incident on the master lens 82 is formed on the light receiving surface 24A. In this way, the imaging area light incident on the lens device 70 is guided to the image sensor 24 by a plurality of lenses provided in the lens device 70 and is formed on the light receiving surface 24A of the image sensor 24.

[0035] Note that the arrangement order of the objective lens 72, zoom lens 74, shake correction lens 76, aperture 78, filter unit 80, and master lens 82 may be other than the above. Also, each of the objective lens 72, zoom lens 74, shake correction lens 76, and master lens 82 may be a single lens or a lens group having a plurality of lenses. Further, the lens device 70 may include other lenses in addition to the objective lens 72, zoom lens 74, shake correction lens 76, and master lens 82.

[0036] (Filter Unit) As an example, as shown in FIG. 2, the filter unit 80 includes a disk 84. As an example, a plurality of optical filters, i.e., an Ir cut filter 86, a first BPF 88A, a second BPF 88B, a third BPF 88C, and a fourth BPF 88D, are provided at equal intervals along the circumferential direction on the disk 84. Hereinafter, when there is no need for particular distinction in description, the Ir cut filter 86, the first BPF 88A, the second BPF 88B, the third BPF 88C, and the fourth BPF 88D are referred to as optical filters. Also, hereinafter, when there is no need for particular distinction in description, the first BPF 88A, the second BPF 88B, the third BPF 88C, and the fourth BPF 88D are referred to as BPF 88.

[0037] The filter unit 80 selectively inserts and removes a plurality of optical filters in a turret manner with respect to the optical path of the imaging area light in the lens device 70 (hereinafter simply referred to as the "optical path"). Specifically, as the disk 84 rotates along the circumferential direction (for example, the arc dashed arrow direction shown in FIG. 2), the Ir cut filter 86, the first BPF 88A, the second BPF 88B, the third BPF 88C, and the fourth BPF 88D are selectively inserted and removed with respect to the optical path (in the example shown in FIG. 2, the optical axis OA). Thereby, the Ir cut filter 86, the first BPF 88A, the second BPF 88B, the third BPF 88C, and the fourth BPF 88D transmit light in different wavelength bands, respectively.

[0038] When the optical filter is inserted into the optical path, the optical axis OA passes through the center of the optical filter, and the center of the optical filter inserted into the optical path coincides with the center of the light receiving surface 24A. In the example shown in FIG. 2, since the Ir cut filter 86 is inserted into the optical path, the optical axis OA passes through the center of the Ir cut filter 86, and the center of the Ir cut filter 86 coincides with the center of the light receiving surface 24A.

[0039] The Ir cut filter 86 is an optical filter that cuts infrared rays and transmits only light other than infrared rays. The BPF 88 is an optical filter that transmits near-infrared light. The first BPF 88A, the second BPF 88B, the third BPF 88C, and the fourth BPF 88D transmit near-infrared light in different wavelength bands.

[0040] The first BPF 88A is an optical filter corresponding to a band near 1000 nm (nanometers). That is, the first BPF 88A transmits only near-infrared light in the band near 1000 nm. The second BPF 88B is an optical filter corresponding to a band near 1250 nm. That is, the second BPF 88B transmits only near-infrared light in the band near 1250 nm. The third BPF 88C is an optical filter corresponding to a band near 1550 nm. That is, the third BPF 88C transmits only near-infrared light in the band near 1550 nm. The fourth BPF 88D is an optical filter corresponding to a band near 2150 nm. That is, the fourth BPF 88D transmits only near-infrared light in the band near 2150 nm. Note that each band mentioned here includes an error generally allowed in the technical field to which the technology of the present disclosure belongs and that does not deviate from the gist of the technology of the present disclosure. In addition, each wavelength band mentioned here is merely an example, and any different wavelength bands may be used.

[0041] (Image sensor 24) As shown in FIG. 3 as an example, the image sensor 24 has a light receiving unit 26 and a color filter unit 28. The light receiving unit 26 has a plurality of first light receiving elements 30 and a plurality of second light receiving elements 32. As an example of the first light receiving element 30, an indium gallium arsenide photodiode can be mentioned. As an example of the second light receiving element 32, a silicon photodiode can be mentioned.

[0042] The color filter unit 28 is disposed on the plurality of first light receiving elements 30 and the plurality of second light receiving elements 32. The color filter unit 28 has an Ir filter, an R filter, a G filter, and a B filter. The Ir filter is a filter that transmits light of a near-infrared (Ir) component. The R filter is a filter that transmits light of a red (R) component. The G filter is a filter that transmits light of a green (G) component. The B filter is a filter that transmits light of a blue (B) component.

[0043] The first light receiving element 30 is a light receiving element having sensitivity to light of the Ir component. The second light receiving element 32 is roughly classified into a light receiving element 32R having sensitivity to light of the R component, a light receiving element 32G having sensitivity to light of the G component, and a light receiving element 32B having sensitivity to light of the B component.

[0044] The Ir filter is disposed on the first light receiving element 30. The R filter is disposed on the light receiving element 32R. The G filter is disposed on the light receiving element 32G. The B filter is disposed on the light receiving element 32B. In addition, a filter that blocks near-infrared light is further disposed on each of the light receiving elements 32R, 36G, and 36B.

[0045] In the image sensor 24 configured as described above, the plurality of first light receiving elements 30 receive the near-infrared light transmitted through any of the plurality of BPF88, generate and output a near-infrared light image 64 based on the received near-infrared light, and the plurality of second light receiving elements 32 receive the visible light transmitted through the Ir cut filter 86, generate and output a visible light image 62 based on the received visible light.

[0046] (Surveillance camera body) As an example, as shown in FIG. 4, the surveillance camera main body 20 includes a controller 40 and a UI system device 50. The controller 40 controls the operation of the surveillance camera main body 20. The controller 40 includes a CPU 42, an NVM 44, and a RAM 46. The CPU 42, the NVM 44, and the RAM 46 are connected to a bus 48. The CPU 42 of the surveillance camera main body 20 and the CPU 212 (see FIG. 6) of the management device 200 described later are communicably connected via the communication I / F 60 of the surveillance camera main body 20 and the communication I / F 220 (see FIG. 6) of the management device 200. The CPU 42 of the surveillance camera main body 20 controls the operation of the surveillance camera main body 20 according to an instruction given from the CPU 212 of the management device 200.

[0047] The NVM 44 stores various parameters and various programs. As an example of the NVM 44, an EEPROM (for example, a flash type EEPROM) can be mentioned. The EEPROM is only an example of the NVM 44. The NVM 44 may be various non-volatile storage devices such as an SSD and / or an HDD. The RAM 46 temporarily stores various information and is used as a work memory. As an example of the RAM 46, a DRAM can be mentioned. The DRAM is only an example of the RAM 46. The RAM 46 may be an SRAM, and may be various volatile storage devices.

[0048] Various programs are stored in the NVM 44. The CPU 42 reads out a necessary program from the NVM 44 and executes the read program on the RAM 46. The CPU 42 executes various processes according to the program executed on the RAM 46.

[0049] The UI system device 50 is also connected to the bus 48. Under the control of the CPU 42, the UI system device 50 receives an instruction given from the user or presents various information obtained by being processed by the surveillance camera main body 20 to the user.

[0050] In addition, the monitoring camera body 20 includes an image sensor driver 52, a signal processing device 54, a shake amount detection sensor 56, and a communication I / F 60. The image sensor driver 52, the signal processing device 54, the shake amount detection sensor 56, and the communication I / F 60 are connected to the bus 48.

[0051] As shown in FIG. 1 as an example, the image sensor 24 is located on the optical axis OA at a stage posterior to the master lens 82, that is, on the image side with respect to the master lens 82. As shown in FIG. 2 as an example, in a state where the Ir cut filter 86 is disposed on the optical axis OA, the image sensor 24 captures an imaging region based on visible light imaged on the light receiving surface 24A by the master lens 82, thereby generating a visible light image 62 shown in FIG. 3 and outputting the generated visible light image 62 to a subsequent stage. The visible light image 62 is an image showing the imaging region by visible light.

[0052] In a state where the BPF 88 (see FIG. 2) is disposed on the optical axis OA, the image sensor 24 captures an imaging region based on near-infrared light imaged on the light receiving surface 24A by the master lens 82, thereby generating a near-infrared light image 64 shown in FIG. 3 and outputting the generated near-infrared light image 64 to a subsequent stage. The near-infrared light image 64 is an image showing the imaging region by near-infrared light. In the following, when it is not necessary to distinguish between the near-infrared light image 64 and the visible light image 62, they are referred to as "imaging images" without reference numerals.

[0053] As shown in FIG. 4 as an example, the image sensor driver 52 and the signal processing device 54 are connected to the image sensor 24. The image sensor driver 52 outputs a timing control signal to the image sensor 24 under the control of the CPU 42. The timing control signal is a signal for controlling imaging by the image sensor 24. The frame rate of imaging by the image sensor 24 is defined by the timing control signal.

[0054] The timing control signal includes a vertical synchronization signal and a horizontal synchronization signal. The vertical synchronization signal is a signal that defines the timing to start transmitting an analog image for one frame. The horizontal synchronization signal is a signal that defines the timing to start outputting an analog image for one horizontal line. The image sensor 24 starts outputting the captured image in frame units to the signal processing device 54 according to the vertical synchronization signal input from the image sensor driver 52. Also, the image sensor 24 starts outputting the captured image in horizontal line units to the signal processing device 54 according to the horizontal synchronization signal input from the image sensor driver 52.

[0055] Under the control of the CPU 42, the signal processing device 54 performs signal processing such as demosaicing processing, noise removal processing, gradation correction processing, and color correction processing on the captured image input from the image sensor 24. The signal-processed captured image is output to the CPU 42 by the signal processing device 54. The CPU 42 stores the captured image input from the signal processing device 54 in a predetermined storage area (for example, NVM 44 and / or RAM 46, etc.).

[0056] The shake amount detection sensor 56 detects the amount of shake (hereinafter, also simply referred to as "shake amount") of the monitoring camera 10 shown in FIG. 1, for example. The shake of the monitoring camera 10 refers to a phenomenon in which the positional relationship between the optical axis OA and the light receiving surface 24A varies in the monitoring camera 10. When shake occurs in the monitoring camera 10, shake of the image occurs. Examples of the image include an image obtained by being captured by the image sensor 24 and / or an optical image (hereinafter, also simply referred to as "image" or "subject image") obtained by being formed on the light receiving surface 24A. "Shake of the image" refers to a phenomenon in which the subject image is displaced from the reference position due to the optical axis OA tilting along with a vibration phenomenon, that is, a phenomenon in which the subject image is displaced from the reference position along with the relative movement of the optical axis OA with respect to the subject. The vibration phenomenon refers to a phenomenon in which the lens device 70 vibrates due to vibration being transmitted from the outside of the monitoring camera 10 (for example, hands, wind, and / or a vehicle, etc.) and / or the inside of the monitoring camera 10 (for example, a motor mounted on the monitoring camera 10) to the lens device 70. Also, "the optical axis OA tilts" means, for example, that the optical axis OA tilts with respect to a reference axis (for example, the optical axis OA before the vibration phenomenon occurs (that is, the optical axis OA when the monitoring camera 10 is stationary)). Also, the "reference position" refers to, for example, the position of the subject image obtained in a state where no vibration is applied to the lens device 70 (for example, the position of the subject image within the light receiving surface 24A).

[0057] The shake amount detection sensor 56 shown in FIG. 4 is, for example, a gyro sensor. The gyro sensor detects the amount of rotational shake around each of the X-axis, Y-axis, and Z-axis. The shake amount detection sensor 56 detects the shake amount of the monitoring camera 10 by converting the amount of rotational shake around the X-axis and the amount of rotational shake around the Y-axis detected by the gyro sensor into the shake amount in a two-dimensional plane parallel to the X-axis and the Y-axis. Note that the meaning of parallel includes not only the meaning of complete parallel but also the meaning of substantially parallel including errors allowed in design and manufacturing.

[0058] Here, a gyro sensor is given as an example of the shake amount detection sensor 56, but this is merely an example, and the shake amount detection sensor 56 may be an acceleration sensor. The acceleration sensor detects the shake amount within a two-dimensional plane parallel to the X-axis and the Y-axis. The shake amount detection sensor 56 outputs the detected shake amount to the CPU 42.

[0059] Also, here, an example form in which the shake amount is detected by a physical sensor called the shake amount detection sensor 56 is given, but the technology of the present disclosure is not limited to this. For example, a motion vector obtained by comparing imaging images that are temporally before and after stored in the NVM 44 or the RAM 46 may be used as the shake amount. Further, the finally used shake amount may be derived based on the shake amount detected by a physical sensor and the motion vector obtained by image processing.

[0060] The communication I / F 60 has, for example, a network interface and performs transmission control of various information with the communication I / F 220 (see FIG. 6) of the management device 200 via a network. As an example of the network, a WAN such as the Internet or a public communication network can be cited. Also, the communication I / F 60 performs transmission control of various information with the communication I / F 100 (see FIG. 5) of the lens device 70.

[0061] The surveillance camera 10 configured as described above images a surveillance target that is a subject and generates a moving image by imaging. The moving image includes images of a plurality of frames obtained by imaging. The moving image obtained by being imaged by the surveillance camera 10 is transmitted to the management device 200 shown in FIG. 1, and the management device 200 receives the moving image transmitted by the surveillance camera 10 and displays the received moving image on a display or stores it in an image storage device.

[0062] (Lens device) As an example, as shown in FIG. 5, the controller 90 of the lens device 70 includes a CPU 92, an NVM 94, and a RAM 96. The controller 90 controls the operation of the lens device 70. The controller 90 is an example of a "computer applied to the lens device", the CPU 92 is an example of a "processor" according to the technology of the present disclosure, the NVM 94 is an example of a "non-volatile memory" according to the technology of the present disclosure, and the RAM 96 is an example of a "memory" according to the technology of the present disclosure. The CPU 92, the NVM 94, and the RAM 96 are connected to a bus 98.

[0063] The CPU 92 of the lens device 70 and the CPU 42 of the monitoring camera main body 20 (see FIG. 4) are communicably connected via the communication I / F 100 of the lens device 70 and the communication I / F 60 of the monitoring camera main body 20 (see FIG. 4), etc. The CPU 92 of the lens device 70 and the CPU 212 of the management device 200 (see FIG. 6) described later are communicably connected via the communication I / F 100 of the lens device 70 and the communication I / F 220 of the management device 200 (see FIG. 6), etc. The CPU 92 of the lens device 70 controls the operation of the lens device 70 according to the instructions given from the CPU 42 of the monitoring camera main body 20 and the information given from the CPU 212 of the management device 200.

[0064] The NVM 94 stores various parameters and various programs. As an example of the NVM 94, an EEPROM (for example, a flash type EEPROM) can be mentioned. The EEPROM is only an example of the NVM 94. The NVM 94 may be various non-volatile storage devices such as an SSD and / or an HDD. The RAM 96 temporarily stores various information and is used as a work memory. As an example of the RAM 96, a DRAM can be mentioned. The DRAM is only an example of the RAM 96. The RAM 96 may be an SRAM, and may be various volatile storage devices.

[0065] The NVM94 stores various programs. The CPU92 reads out the necessary programs from the NVM94 and executes the read programs on the RAM96. The CPU92 executes various processes according to the programs executed on the RAM96. Also, the "various programs" mentioned here include the blur correction and shift processing program 110 (see FIG. 15) described later.

[0066] The communication I / F100 has, for example, a network interface and controls the transmission of various information to and from the communication I / F220 (see FIG. 6) of the management device 200 via the network. Also, the communication I / F100 controls the transmission of various information to and from the communication I / F60 (see FIG. 4) of the monitoring camera body 20.

[0067] As shown in FIG. 5 as an example, the lens device 70 includes a first motor driver 114, an X-axis motor driver 116A, a Y-axis motor driver 116B, a second motor driver 118, a third motor driver 120, and a fourth motor driver 122. Also, the lens device 70 includes a first motor 134, an X-axis motor 136A, a Y-axis motor 136B, a second motor 138, a third motor 140, and a fourth motor 142. Further, the lens device 70 includes a first position sensor 154, an X-axis position sensor 156A, a Y-axis position sensor 156B, a second position sensor 158, a third position sensor 160, and a fourth position sensor 162.

[0068] The first motor driver 114, the X-axis motor driver 116A, the Y-axis motor driver 116B, the second motor driver 118, the third motor driver 120, the fourth motor driver 122, the first position sensor 154, the X-axis position sensor 156A, the Y-axis position sensor 156B, the second position sensor 158, the third position sensor 160, and the fourth position sensor 162 are connected to the bus 98.

[0069] Examples of each of the first position sensor 154, the X-axis position sensor 156A, the Y-axis position sensor 156B, the second position sensor 158, the third position sensor 160, and the fourth position sensor 162 include potentiometers.

[0070] The first position sensor 154 detects the position of the zoom lens 74 in the Z-axis direction. The X-axis position sensor 156A detects the position of the shake correction lens 76 in the X-axis direction, and the Y-axis position sensor 156B detects the position of the shake correction lens 76 in the Y-axis direction. The second position sensor 158 detects the aperture diameter of the aperture 78A formed in the diaphragm 78. The third position sensor 160 detects the rotational position of the filter unit 80 with respect to the optical axis OA. The fourth position sensor 162 detects the position of the master lens 82 in the Z-axis direction.

[0071] The detection result by the first position sensor 154 is output to the CPU 92 by the first position sensor 154. The detection result by the X-axis position sensor 156A is output to the CPU 92 by the X-axis position sensor 156A. The detection result by the Y-axis position sensor 156B is output to the CPU 92 by the Y-axis position sensor 156B. The detection result by the second position sensor 158 is output to the CPU 92 by the second position sensor 158. The detection result by the third position sensor 160 is output to the CPU 92 by the third position sensor 160. The detection result by the fourth position sensor 162 is output to the CPU 92 by the fourth position sensor 162.

[0072] The zoom lens 74 is attached to a first slide mechanism (not shown). The first slide mechanism is mechanically connected to the drive shaft of the first motor 134, and moves the zoom lens 74 along the Z-axis direction by receiving the power of the first motor 134. The first motor driver 114 is connected to the first motor 134 and controls the first motor 134 according to an instruction from the CPU 92. The CPU 92 controls the position of the zoom lens 74 in the Z-axis direction by controlling the first motor 134 via the first motor driver 114 based on the detection result by the first position sensor 154.

[0073] The shake correction lens 76 is attached to an X-axis slide mechanism (not shown). The X-axis slide mechanism is mechanically connected to the movable member of the X-axis motor 136A, and receives the power of the X-axis motor 136A to move the shake correction lens 76 along the X-axis direction. The X-axis motor driver 116A is connected to the X-axis motor 136A and controls the X-axis motor 136A according to an instruction from the CPU 92. The CPU 92 controls the position of the shake correction lens 76 in the X-axis direction by controlling the X-axis motor 136A via the X-axis motor driver 116A based on the detection result by the X-axis position sensor 156A. The X-axis motor 136A may be, for example, a voice coil motor or a small DC motor. Further, the X-axis motor 136A may include a voice coil motor that moves the shake correction lens 76 in the direction of correcting image blur and a piezoelectric element or a small DC motor that moves the shake correction lens 76 in the direction of shifting the image.

[0074] Also, the shake correction lens 76 is attached to a Y-axis slide mechanism (not shown). The Y-axis slide mechanism is mechanically connected to the movable member of the Y-axis motor 136B, and receives the power of the Y-axis motor 136B to move the shake correction lens 76 along the Y-axis direction. The Y-axis motor driver 116B is connected to the Y-axis motor 136B and controls the Y-axis motor 136B according to an instruction from the CPU 92. The CPU 92 controls the position of the shake correction lens 76 in the Y-axis direction by controlling the Y-axis motor 136B via the Y-axis motor driver 116B based on the detection result by the Y-axis position sensor 156B. The Y-axis motor 136B may be, for example, a voice coil motor or a small DC motor. Further, the Y-axis motor 136B may include a voice coil motor that moves the shake correction lens 76 in the direction of correcting image blur and a piezoelectric element or a small DC motor that moves the shake correction lens 76 in the direction of shifting the image. The X-axis motor 136A and the Y-axis motor 136B are an example of the "drive mechanism" according to the technology of the present disclosure.

[0075] The aperture 78 has a plurality of vanes (not shown) capable of opening and closing the aperture 78A. The plurality of vanes are mechanically connected to the drive shaft of the second motor 138 and open and close the aperture 78A by receiving the power of the second motor 138. The second motor driver 118 is connected to the second motor 138 and controls the second motor 138 according to an instruction from the CPU 92. The CPU 92 controls the second motor 138 via the second motor driver 118 shown in FIG. 5 based on the detection result by the second position sensor 158 and the amount of received light on the light receiving surface 24A shown in FIG. 4, thereby adjusting the opening degree of the aperture 78A.

[0076] The filter unit 80 is attached to a rotation mechanism (not shown). The rotation mechanism is mechanically connected to the drive shaft of the third motor 140 and rotates the filter unit 80 (see FIG. 3) in the circumferential direction by receiving the power of the third motor 140, thereby inserting and removing a plurality of optical filters into and out of the optical path. The third motor driver 120 is connected to the third motor 140 and controls the third motor 140 according to an instruction from the CPU 92. The CPU 92 controls the third motor 140 via the third motor driver 120 based on the detection result by the third position sensor 160, thereby controlling the rotational position of the filter unit 80 with respect to the optical axis OA.

[0077] The master lens 82 is attached to a fourth slide mechanism (not shown). The fourth slide mechanism is mechanically connected to the drive shaft of the fourth motor 142 and moves the master lens 82 along the Z-axis direction by receiving the power of the fourth motor 142. The fourth motor driver 122 is connected to the fourth motor 142 and controls the fourth motor 142 according to an instruction from the CPU 92. The CPU 92 controls the fourth motor 142 via the fourth motor driver 122 based on the detection result by the fourth position sensor 162, thereby controlling the position of the master lens 82 in the Z-axis direction.

[0078] (Management device) As shown in FIG. 6 as an example, the controller 210 of the management device 200 controls the operation of the management device 200. The controller 210 includes a CPU 212, an NVM 214, and a RAM 216. The CPU 212, the NVM 214, and the RAM 216 are connected to a bus 218.

[0079] The NVM 214 stores various parameters and various programs. As an example of the NVM 214, an EEPROM (for example, a flash type EEPROM) can be mentioned. The EEPROM is only an example of the NVM 214. The NVM 214 may be various non-volatile storage devices such as an SSD and / or an HDD. The RAM 216 temporarily stores various information and is used as a work memory. As an example of the RAM 216, a DRAM can be mentioned. The DRAM is only an example of the RAM 216. The RAM 216 may be an SRAM, and may be various volatile storage devices.

[0080] Various programs are stored in the NVM 214. The CPU 212 reads out necessary programs from the NVM 214 and executes the read programs on the RAM 216. The CPU 212 executes various processes according to the programs executed on the RAM 216. Also, the “various programs” mentioned here also includes the inclination information output processing program 230 (see FIG. 8) described later.

[0081] Also, as shown in FIG. 6 as an example, the management device 200 includes a display 222, a keyboard 224, a mouse 226, and an input / output I / F 228. The display 222, the keyboard 224, and the mouse 226 are connected to the input / output I / F 228. The input / output I / F 228 is connected to the bus 218. Information input by the keyboard 224 and the mouse 226 is given to the CPU 212 via the input / output I / F 228. The image information output from the CPU 212 is given to the display 222 via the input / output I / F 228, and the display 222 displays an image based on the given image information.

[0082] The communication I / F 220 has, for example, a network interface. The network interface of the communication I / F 220 is communicably connected to the communication I / F 60 (see FIG. 4) of the surveillance camera main body 20 via a network (not shown), and controls the exchange of various information between the CPU 212 and the CPU 40 (see FIG. 4) of the surveillance camera main body 20. The network interface of the communication I / F 220 is communicably connected to the communication I / F 100 (see FIG. 5) of the lens device 70 via a network, and controls the exchange of various information between the CPU 212 and the CPU 92 (see FIG. 5) of the lens device 70.

[0083] (Regarding the inclination of the X-axis and Y-axis of the lens device) Incidentally, when the lens device 70 is attached to the lens mount 22 of the surveillance camera main body 20, the X-axis and Y-axis of the lens device 70 may be inclined with respect to the X-axis and Y-axis of the surveillance camera main body 20 due to the looseness and / or deformation of the lens mount 22. Hereinafter, when distinguishing between the X-axis and Y-axis of the lens device 70 and the X-axis and Y-axis of the surveillance camera main body 20, the X-axis and Y-axis of the surveillance camera main body 20 are referred to as the X1-axis and Y1-axis, respectively, and the X-axis and Y-axis of the lens device 70 are referred to as the X2-axis and Y2-axis, respectively. Also, the X-axis and Y-axis of the image sensor 24 are the X-axis and Y-axis of the surveillance camera main body 20. Hereinafter, the X-axis and Y-axis of the image sensor 24 are referred to as the X1-axis and Y1-axis, respectively.

[0084] For example, in the example shown in FIG. 7, the X2-axis is inclined at an inclination angle θx with respect to the X1-axis, and the Y2-axis is inclined at an inclination angle θy with respect to the Y1-axis. The value of the inclination angle θx may be the same as or different from the value of the inclination angle θy. Both the inclination angle θx and the inclination angle θy are angles viewed along the Z-axis.

[0085] Hereinafter, as an example, an example in which the X2-axis and Y2-axis of the lens device 70 are inclined with respect to the X1-axis and Y1-axis of the surveillance camera main body 20 will be described. For the sake of convenience, it is assumed that the Z-axis of the lens device 70 coincides with the Z-axis of the surveillance camera main body 20, and the optical axis OA is parallel to the Z-axis of the lens device 70.

[0086] As shown in FIG. 7, when the X2 axis and the Y2 axis of the lens device 70 are inclined with respect to the X1 axis and the Y1 axis of the monitoring camera body 20, respectively, the direction in which the X-axis motor 136A (see FIG. 5) moves the shake correction lens 76 is the direction along the X2 axis, and the direction in which the Y-axis motor 136B (see FIG. 5) moves the shake correction lens 76 is the direction along the Y2 axis. When the X-axis motor 136A moves the shake correction lens 76 along the X2 axis, the image obtained by the light being imaged on the image sensor 24 moves along the X2 axis, and when the Y-axis motor 136B moves the shake correction lens 76 along the Y2 axis, the image moves along the Y2 axis. Therefore, although originally it is desired to move the image along the X1 axis and the Y1 axis of the image sensor 24 as the shake correction lens 76 moves, the image will move along the X2 axis and the Y2 axis. Therefore, even when the X2 axis and the Y2 axis of the lens device 70 are inclined with respect to the X1 axis and the Y1 axis of the monitoring camera body 20, respectively, it is desired to move the image along the X1 axis and the Y1 axis of the image sensor 24 as the shake correction lens 76 moves.

[0087] Hereinafter, a technique for moving the image along the X1 axis and the Y1 axis of the image sensor 24 as the shake correction lens 76 moves will be described even when the X2 axis and the Y2 axis of the lens device 70 are inclined with respect to the X1 axis and the Y1 axis of the monitoring camera body 20, respectively.

[0088] Note that the X1 axis is an example of the "first axis" according to the technology of the present disclosure, the Y1 axis is an example of the "second axis" according to the technology of the present disclosure, the X2 axis is an example of the "first drive axis that intersects the optical axis of the lens" according to the technology of the present disclosure, and the Y2 axis is an example of the "second drive axis that intersects each of the optical axis of the lens and the first drive axis" according to the technology of the present disclosure. Also, the inclination angle θx is an example of the "first inclination angle" according to the technology of the present disclosure, and the inclination angle θy is an example of the "second inclination angle" according to the technology of the present disclosure.

[0089] (Functional configuration of the CPU of the management device) As an example, as shown in FIG. 8, the tilt information output process (see FIG. 21) described later is realized by the CPU 212 of the management device 200 executing the tilt information output process program 230. In the example shown in FIG. 8, the tilt information output process program 230 is stored in the NVM 214, and the CPU 212 reads the tilt information output process program 230 from the NVM 214 and executes it on the RAM 216.

[0090] The CPU 212 performs the tilt information output process according to the tilt information output process program 230 executed on the RAM 216. By executing the tilt information output process program 230 on the RAM 216, the CPU 212 operates as the tilt information generation unit 232 and the tilt information output unit 234.

[0091] As an example, as shown in FIG. 9, the center of the X2-Y2 coordinate system is the point where the X2 axis and the Y2 axis intersect. The tilt information generation unit 232 controls the X-axis motor driver 116A and the Y-axis motor driver 116B via the communication I / F 220 of the management device 200, the communication I / F 100 of the lens device 70, and the CPU 92 of the lens device 70, thereby moving the shake correction lens 76 to the center of the X2-Y2 coordinate system. In this case, for example, the tilt information generation unit 232 outputs a first control command, which is a command to move the shake correction lens 76 to the center of the X2-Y2 coordinate system, to the CPU 92 of the lens device 70 via the communication I / F 220 of the management device 200 and the communication I / F 100 of the lens device 70. The CPU 92 of the lens device 70 controls the X-axis motor 136A and the Y-axis motor 136B via the X-axis motor driver 116A and the Y-axis motor driver 116B according to the first control command input from the tilt information generation unit 232, thereby moving the shake correction lens 76 to the center of the X2-Y2 coordinate system.

[0092] As a result, the shake correction lens 76 moves to the center of the X2-Y2 coordinate system. Note that the position of the shake correction lens 76 is defined based on the center 76A of the shake correction lens 76. Therefore, when the shake correction lens 76 moves to the center of the X2-Y2 coordinate system, the center 76A of the shake correction lens 76 is located at the center of the X2-Y2 coordinate system. In a state where the center 76A of the shake correction lens 76 is located at the center of the X2-Y2 coordinate system, the subject image 25A is located at the coordinates (0, 0) of the X1-Y1 coordinate system. The coordinates (0, 0) of the X1-Y1 coordinate system are the center of the X1-Y1 coordinate system, and the center of the X1-Y1 coordinate system is the point where the X1 axis and the Y1 axis intersect.

[0093] In addition, the inclination information generation unit 232 controls the image sensor driver 52 via the communication I / F 220 of the management device 200, the communication I / F 60 of the monitoring camera body 20, and the CPU 42 of the monitoring camera body 20, thereby causing the image sensor 24 to perform imaging. In this case, for example, the inclination information generation unit 232 outputs a first imaging command, which is a command to cause the image sensor 24 to perform imaging, to the CPU 42 of the monitoring camera body 20 via the communication I / F 220 of the management device 200 and the communication I / F 60 of the monitoring camera body 20. The CPU 42 of the monitoring camera body 20 controls the image sensor driver 52 in accordance with the first imaging command input from the inclination information generation unit 232, thereby causing the image sensor 24 to perform imaging.

[0094] As a result, an image is captured by the image sensor 24, and the first image 262 is obtained. The signal processing device 54 acquires the first image 262 from the image sensor 24. The signal processing device 54 performs signal processing on the first image 262 acquired from the image sensor 24, and outputs the signal-processed first image 262 to the CPU 42. The CPU 42 stores the first image 262 input from the signal processing device 54 in the NVM 44 and / or the RAM 46 (see FIG. 4).

[0095] As an example, as shown in FIG. 10, the tilt information generation unit 232 performs control to move the shake correction lens 76 along the X2 axis. The tilt information generation unit 232 controls the X-axis motor driver 116A to move the shake correction lens 76 along the X2 axis by a predetermined first movement amount. In this case, for example, the tilt information generation unit 232 outputs a second control command, which is a command to move the shake correction lens 76 along the X2 axis by a predetermined first movement amount, to the CPU 92 of the lens device 70 via the communication I / F 220 of the management device 200 and the communication I / F 100 of the lens device 70. The CPU 92 of the lens device 70 controls the X-axis motor 136A via the X-axis motor driver 116A in accordance with the second control command input from the tilt information generation unit 232, thereby moving the shake correction lens 76 along the X2 axis by the first movement amount.

[0096] As a result, the center 76A of the shake correction lens 76 moves along the X2 axis by the first movement amount from the center of the X2 - Y2 coordinate system, and in the X1 - Y1 coordinate system, the subject image 25A moves from the coordinates (0, 0) to the coordinates (a1, b1).

[0097] The tilt information generation unit 232 controls the image sensor driver 52 via the communication I / F 220 of the management device 200, the communication I / F 60 of the monitoring camera body 20, and the CPU 42 of the monitoring camera body 20 to cause the image sensor 24 to perform imaging. In this case, for example, the tilt information generation unit 232 outputs a second imaging command, which is a command to cause the image sensor 24 to perform imaging, to the CPU 42 of the monitoring camera body 20 via the communication I / F 220 of the management device 200 and the communication I / F 60 of the monitoring camera body 20. The CPU 42 of the monitoring camera body 20 controls the image sensor driver 52 in accordance with the second imaging command input from the tilt information generation unit 232 to cause the image sensor 24 to perform imaging.

[0098] As a result, an image is captured by the image sensor 24 to obtain a second image 264. The signal processing device 54 acquires the second image 264 from the image sensor 24. The signal processing device 54 performs signal processing on the second image 264 acquired from the image sensor 24 and outputs the signal-processed second image 264 to the CPU 42. The CPU 42 stores the second image 264 input from the signal processing device 54 in the NVM 44 and / or the RAM 46 (see FIG. 4).

[0099] As an example, as shown in FIG. 11, the inclination information generation unit 232 compares the first image 262 obtained based on the above-described first imaging command with the second image 264 obtained based on the above-described second imaging command. Then, based on the first image 262 and the second image 264, the inclination information generation unit 232 calculates, by first image analysis, to which position the image corresponding to the subject image 25A located at the position corresponding to the coordinate (0, 0) of the first image 262 in the X1 - Y1 coordinate system has moved in the second image 264. Various image analysis methods can be applied to the first image analysis. Hereinafter, for convenience, it will be described on the premise that the X1 - Y1 coordinate system is also applied to the first image 262 and the second image 264. In the example shown in FIG. 11, as an example, the image corresponding to the subject image 25A moves from the coordinate (0, 0) of the first image 262 to the coordinate (a1, b1) of the second image 264. Therefore, the inclination information generation unit 232 calculates the inclination angle θx of the X2 axis with respect to the X1 axis by the following formula (1). θx = tan -1 (b1 / a1) ··· (1)

[0100] As an example, as shown in FIG. 12, the tilt information generation unit 232 performs control to move the shake correction lens 76 along the Y2 axis. The tilt information generation unit 232 controls the Y-axis motor driver 116B to move the shake correction lens 76 along the Y2 axis by a predetermined second movement amount. In this case, for example, the tilt information generation unit 232 outputs a third control command, which is a command to move the shake correction lens 76 along the Y2 axis by a predetermined second movement amount, to the CPU 92 of the lens device 70 via the communication I / F 220 of the management device 200 and the communication I / F 100 of the lens device 70. The CPU 92 of the lens device 70 controls the Y-axis motor 136B via the Y-axis motor driver 116B in accordance with the third control command input from the tilt information generation unit 232, thereby moving the shake correction lens 76 along the Y2 axis by the second movement amount.

[0101] As a result, the center 76A of the shake correction lens 76 moves by the second movement amount along the Y2 axis from the position on the X2 axis, and the subject image 25A moves from the coordinates (a1, b1) to the coordinates (a2, b2) in the X1 - Y1 coordinate system.

[0102] The tilt information generation unit 232 controls the image sensor driver 52 via the communication I / F 220 of the management device 200, the communication I / F 60 of the monitoring camera main body 20, and the CPU 42 of the monitoring camera main body 20 to cause the image sensor 24 to perform imaging. In this case, for example, the tilt information generation unit 232 outputs a third imaging command, which is a command to cause the image sensor 24 to perform imaging, to the CPU 42 of the monitoring camera main body 20 via the communication I / F 220 of the management device 200 and the communication I / F 60 of the monitoring camera main body 20. The CPU 42 of the monitoring camera main body 20 controls the image sensor driver 52 in accordance with the third imaging command input from the tilt information generation unit 232 to cause the image sensor 24 to perform imaging.

[0103] As a result, an image is captured by the image sensor 24 to obtain a third image 266. The signal processing device 54 acquires the third image 266 from the image sensor 24. The signal processing device 54 performs signal processing on the third image 266 acquired from the image sensor 24 and outputs the signal-processed third image 266 to the CPU 42. The CPU 42 stores the third image 266 input from the signal processing device 54 in the NVM 44 and / or the RAM 46 (see FIG. 4).

[0104] As an example, as shown in FIG. 13, the tilt information generation unit 232 compares the second image 264 obtained based on the second imaging command described above with the third image 266 obtained based on the third imaging command described above. Then, the tilt information generation unit 232 calculates, by second image analysis, where the image corresponding to the subject image 25A located at the position corresponding to the coordinates (a1, b1) of the first image 262 in the X1 - Y1 coordinate system has moved in the third image 266. Various image analysis methods can be applied to the second image analysis. Hereinafter, for convenience, it will be described on the premise that the X1 - Y1 coordinate system is also applied to the first image 262 and the second image 264. In the example shown in FIG. 13, as an example, the image corresponding to the subject image 25A moves from the coordinates (a1, b1) of the second image 264 to the coordinates (a2, b2) of the third image 266. Therefore, the tilt information generation unit 232 calculates the tilt angle θy of the X2 axis with respect to the X1 axis by the following formula (2). θy = tan -1 {(b2 - b1) / (a2 - a1)} ··· (2)

[0105] For example, the first imaging command, the second imaging command, and the third imaging command are commands for causing the image sensor 24 with a sensitivity lower than that of the image sensor 24 performing normal imaging to perform imaging. Normal imaging is imaging performed based on a normal imaging command different from the first imaging command, the second imaging command, and the third imaging command. As an example of normal imaging, imaging when surveillance activities are performed using the surveillance camera 10 can be mentioned. Thereby, the first image 262, the second image 264, and the third image 266 are obtained by performing imaging with the image sensor 24 under imaging conditions where an image with less noise than the image obtained by normal imaging is obtained. When the sensitivity of the image sensor 24 is lowered, it is necessary to relatively increase the exposure time. However, since the surveillance camera 10 and the subject are stationary when calculating the tilt angles θx and θy, the influence of increasing the exposure time is small.

[0106] In the above manner, the tilt information generation unit 232 generates tilt information regarding the tilt angle θx of the X2 axis with respect to the X1 axis and tilt information regarding the tilt angle θy of the Y2 axis with respect to the Y1 axis. The tilt information regarding the tilt angle θx is information calculated based on a second control command for moving the shake correction lens 76 along the X2 axis and the first image 262 and the second image 264 obtained by performing imaging with the image sensor 24 before and after the shake correction lens 76 moves based on the second control command. Further, the tilt information regarding the tilt angle θy is information calculated based on a third control command for moving the shake correction lens 76 along the Y2 axis and the second image 264 and the third image 266 obtained by performing imaging with the image sensor 24 before and after the shake correction lens 76 moves based on the third control command. Note that the second control command and the third control command are examples of the "control command" according to the technology of the present disclosure, and the first image 262, the second image 264, and the third image 266 are examples of the "plurality of images" according to the technology of the present disclosure.

[0107] As an example, as shown in FIG. 14, the tilt information output unit 234 outputs tilt information regarding the tilt angle θx and the tilt angle θy to the CPU 92 of the lens device 70 via the communication I / F 220 of the management device 200 and the communication I / F 100 of the lens device 70.

[0108] (Functional configuration of the CPU of the lens device) As an example, as shown in FIG. 15, the shake correction and shift processing (see FIGS. 22 and 23) described later is realized by executing a shake correction and shift processing program 110 by the CPU 92 of the lens device 70. The shake correction and shift processing program 110 is an example of the "program" according to the technology of the present disclosure. In the example shown in FIG. 15, the shake correction and shift processing program 110 is stored in the NVM 94, and the CPU 92 reads the shake correction and shift processing program 110 from the NVM 94 and executes it on the RAM 96.

[0109] The CPU 92 performs shake correction and shift processing according to the shake correction and shift processing program 110 executed on the RAM 96. The CPU 92 operates as an acquisition unit 172, a calculation unit 174, and a control unit 176 by executing the shake correction and shift processing program 110 on the RAM 96. Although details will be described later, the shake correction and shift processing is a process including shift processing (see FIG. 22) and shake correction processing (see FIG. 23).

[0110] As an example, as shown in FIG. 16, the CPU 42 of the monitoring camera body 20 outputs an image shift command and frame period information to the CPU 42 of the monitoring camera body 20 via the communication I / F 60 of the monitoring camera body 20 and the communication I / F 100 of the lens device 70. The acquisition unit 172 acquires the image shift command and frame period information output from the CPU 42 of the monitoring camera body 20. The image shift command is command information requesting a shift of the image. The image shift command is classified into an X-axis image shift command representing the shift of the image in the X-axis direction and the shift amount, a Y-axis image shift command representing the shift of the image in the Y-axis direction and the shift amount, and an XY-axis image shift command representing the shift of the image in the X-axis direction and the Y-axis direction and the shift amount.

[0111] The shift amount of the image is defined, for example, by a pitch equal to or greater than the pixel pitch of the image sensor 24, or a pitch less than the pixel pitch of the image sensor 24. The pitch equal to or greater than the pixel pitch of the image sensor 24 is, for example, 1 pitch, 1.5 pitches, 2.5 pitches, or 3.5 pitches, etc. When the pixel pitch of the image sensor 24 is p, a natural number is n, and a pure decimal is d, the pitch greater than the pixel pitch of the image sensor 24 is defined as (n + d) × p. Also, the pitch less than the pixel pitch of the image sensor 24 is, for example, 0.25 pitch, 0.5 pitch, or 0.75 pitch, etc. When the pixel pitch of the image sensor 24 is p and a decimal less than 1 is D, the pitch less than the pixel pitch of the image sensor 24 is defined as D × p.

[0112] The frame period information is information that defines a frame period synchronized with a timing control signal output from the CPU 42 to the image sensor driver 52. The frame period is the period during which imaging is performed in frame units.

[0113] Furthermore, the CPU 212 of the management device 200 outputs the tilt information to the CPU 42 of the surveillance camera body 20 via the communication I / F 220 of the management device 200 and the communication I / F 100 of the lens device 70. The acquisition unit 172 acquires the tilt information output from the CPU 212 of the management device 200. The acquisition unit 172 stores the tilt information acquired from the CPU 212 of the management device 200 in the NVM 94 (see FIG. 5). The tilt information includes tilt information regarding the tilt angle θx of the X2 axis with respect to the X1 axis and tilt information regarding the tilt angle θy of the Y2 axis with respect to the Y1 axis. Also, the acquisition unit 172 acquires the position detection result by the X-axis position sensor 156A and the position detection result by the Y-axis position sensor 156B.

[0114] When the acquisition unit 172 acquires an X-axis image shift command, the calculation unit 174 calculates the movement amount A1 of the shake correction lens 76 for each frame period based on the shift amount of the image represented by the X-axis image shift command, the frame period represented by the frame period information, and the position detection result by the X-axis position sensor 156A. For example, when the shift amount of the image represented by the X-axis image shift command is the same pitch as the pixel pitch of the image sensor 24, the calculation unit 174 calculates the movement amount A1 of the shake correction lens 76 that shifts the image by the same pitch as the pixel pitch of the image sensor 24. Further, when the shift amount of the image represented by the X-axis image shift command is a pitch larger than the pixel pitch of the image sensor 24, the calculation unit 174 calculates the movement amount A1 of the shake correction lens 76 that shifts the image in the X1-axis direction by (n + d) × p. Further, when the shift amount of the image represented by the X-axis image shift command is a pitch less than the pixel pitch of the image sensor 24, the calculation unit 174 calculates the movement amount A1 of the shake correction lens 76 that shifts the image in the X1-axis direction by D × p.

[0115] By the way, as described above, when the lens device 70 is attached to the lens mount 22 of the monitoring camera main body 20, the X2 axis of the lens device 70 may tilt with respect to the X1 axis of the monitoring camera main body 20 due to the effects of rattling and / or deformation of the lens mount 22. For example, in the example shown in FIG. 16, the X2 axis of the lens device 70 is tilted at an inclination angle θx with respect to the X1 axis of the monitoring camera main body 20. Therefore, in order to move the shake correction lens 76 along the X1 axis by the movement amount A1, it is required to move the shake correction lens 76 along the X2 axis and the Y2 axis by the X-axis motor 136A and the Y-axis motor 136B, respectively.

[0116] Therefore, the calculation unit 174 calculates the movement amount A1 based on the image shift amount represented by the above-described image shift command, and based on the calculated movement amount A1 and the tilt angle θx represented by the tilt information acquired by the acquisition unit 172, calculates the movement amount Ax along the X2 axis of the shake correction lens 76 and the movement amount Ay along the Y2 axis of the shake correction lens 76, respectively. That is, the calculation unit 174 calculates the movement amount Ax along the X2 axis of the shake correction lens 76 by the following formula (3), and calculates the movement amount Ay along the Y2 axis of the shake correction lens 76 by the following formula (4). Ax = A1 × cosθy / cos(θy - θx) ··· (3) Ay = -A1 × sinθx / cos(θy - θx) ··· (4)

[0117] The movement amount Ax is calculated as a positive value when the shake correction lens 76 is moved in the positive direction of the X2 axis, and is calculated as a negative value when the shake correction lens 76 is moved in the negative direction of the X2 axis. Similarly, the movement amount Ay is calculated as a positive value when the shake correction lens 76 is moved in the positive direction of the Y2 axis, and is calculated as a negative value when the shake correction lens 76 is moved in the negative direction of the Y2 axis. The movement amount Ax is an example of the "first movement amount" according to the technology of the present disclosure, and the movement amount Ay is an example of the "second movement amount" according to the technology of the present disclosure.

[0118] Similarly, as an example, as shown in FIG. 17, when a Y-axis image shift command is acquired by the acquisition unit 172, the calculation unit 174 calculates the movement amount B1 of the shake correction lens 76 for each frame period based on the shift amount of the image represented by the Y-axis image shift command, the frame period represented by the frame period information, and the position detection result by the Y-axis position sensor 156B. For example, when the shift amount of the image represented by the Y-axis image shift command is the same pitch as the pixel pitch of the image sensor 24, the calculation unit 174 calculates the movement amount B1 of the shake correction lens 76 that shifts the image by the same pitch as the pixel pitch of the image sensor 24. Further, when the shift amount of the image represented by the Y-axis image shift command is a pitch larger than the pixel pitch of the image sensor 24, the calculation unit 174 calculates the movement amount B1 of the shake correction lens 76 that shifts the image in the Y1-axis direction by (n + d) × p. Further, when the shift amount of the image represented by the Y-axis image shift command is a pitch less than the pixel pitch of the image sensor 24, the calculation unit 174 calculates the movement amount B1 of the shake correction lens 76 that shifts the image in the Y1-axis direction by D × p.

[0119] Incidentally, as described above, when the lens device 70 is attached to the lens mount 22 of the monitoring camera body 20, the Y2-axis of the lens device 70 may tilt with respect to the Y1-axis of the monitoring camera body 20 due to the influence of rattling and / or deformation of the lens mount 22. For example, in the example shown in FIG. 17, the Y2-axis of the lens device 70 is tilted at an angle θy with respect to the Y1-axis of the monitoring camera body 20. Therefore, in order to move the shake correction lens 76 along the Y1-axis by the movement amount B1, it is required to move the shake correction lens 76 in the X2-axis and Y2-axis directions by the X-axis motor 136A and the Y-axis motor 136B, respectively.

[0120] Therefore, the calculation unit 174 calculates the movement amount B1 based on the image shift amount represented by the above-described image shift command, and based on the calculated movement amount B1 and the tilt angle θy represented by the tilt information acquired by the acquisition unit 172, calculates the movement amount Bx along the X2 axis of the shake correction lens 76 and the movement amount By along the Y2 axis of the shake correction lens 76, respectively. That is, the calculation unit 174 calculates the movement amount Bx along the X2 axis of the shake correction lens 76 by the following formula (5), and calculates the movement amount By along the Y2 axis of the shake correction lens 76 by the following formula (6). Bx = B1 × sin θy / cos(θy - θx) ··· (5) By = B1 × cos θx / cos(θy - θx) ··· (6)

[0121] The movement amount Bx is calculated as a positive value when the shake correction lens 76 is moved in the positive direction of the X2 axis, and is calculated as a negative value when the shake correction lens 76 is moved in the negative direction of the X2 axis. Similarly, the movement amount By is calculated as a positive value when the shake correction lens 76 is moved in the positive direction of the Y2 axis, and is calculated as a negative value when the shake correction lens 76 is moved in the negative direction of the Y2 axis. The movement amount Bx is an example of the "first movement amount" according to the technology of the present disclosure, and the movement amount By is an example of the "second movement amount" according to the technology of the present disclosure.

[0122] Further, when the acquisition unit 172 acquires an XY-axis image shift command, the calculation unit 174 calculates the movement amount A1 of the shake correction lens 76 in the same manner as when the acquisition unit 172 acquires an X-axis image shift command, and calculates the movement amount B1 of the shake correction lens 76 in the same manner as when the acquisition unit 172 acquires a Y-axis image shift command. Then, based on the calculated movement amount A1 and movement amount B1, the calculation unit 174 calculates the movement amount along the X2 axis of the shake correction lens 76 and the movement amount along the Y2 axis of the shake correction lens 76. At this time, the calculation unit 174 calculates the movement amount along the X2 axis of the shake correction lens 76 by adding a movement amount Ax that is a positive or negative value and a movement amount Bx that is a positive or negative value, and calculates the movement amount along the Y2 axis of the shake correction lens 76 by adding a movement amount Ay that is a positive or negative value and a movement amount By that is a positive or negative value.

[0123] The control unit 176 generates an X-axis control command corresponding to the movement amount along the X2 axis of the shake correction lens 76 calculated by the calculation unit 174 for each frame period. The X-axis control command is output by the control unit 176 to the X-axis motor driver 116A. Similarly, the control unit 176 generates a Y-axis control command corresponding to the movement amount along the Y2 axis of the shake correction lens 76 calculated by the calculation unit 174 for each frame period. The Y-axis control command is output by the control unit 176 to the Y-axis motor driver 116B.

[0124] The X-axis motor driver 116A generates an X-axis operation signal based on the X-axis control command input from the control unit 176. The Y-axis motor driver 116B generates a Y-axis operation signal based on the Y-axis control command input from the control unit 176. The X-axis motor 136A operates with an operation amount corresponding to the X-axis operation signal, and the Y-axis motor 136B operates with an operation amount corresponding to the Y-axis operation signal. Thereby, for each frame period, the shake correction lens 76 moves in the direction of shifting the image along the X1 axis and / or the Y1 axis, and the image is shifted along the X1 axis and / or the Y1 axis.

[0125] The control for shifting the image by the control unit 176 described above is not based on the shake amount detection result by the shake amount detection sensor 56 (i.e., the shake amount of the monitoring camera 10), but is sequence control based on a predetermined shift order.

[0126] Then, the image is shifted every frame period in this way, and every time the image is shifted, control is performed to cause the CPU 42 of the monitoring camera main body 20 to perform imaging on the image sensor 24. As a result, as shown in FIG. 18 as an example, images 182 of a plurality of frames corresponding to each frame period are obtained. Then, the images 182 of the plurality of frames are synthesized by the CPU 42 of the monitoring camera main body 20, and a synthesized image 184 is obtained.

[0127] The synthesized image 184 is obtained, for example, in the following manner. That is, when the shift amount of the image is the same pitch as the pixel pitch of the image sensor 24, among the images 182 of the plurality of frames, a plurality of image pixels forming one image and a plurality of image pixels forming another image are superimposed, and the synthesized image 184 is obtained from the images 182 of the plurality of frames. The synthesized image 184 obtained in this way is an image that does not require demosaicing processing. Also, when the shift amount of the image is a pitch larger than the pixel pitch of the image sensor 24 or when the shift amount of the image is a pitch less than the pixel pitch of the image sensor 24, among the images 182 of the plurality of frames, a plurality of image pixels forming another image are assigned between the plurality of image pixels forming one image, and the synthesized image 184 is obtained from the images 182 of the plurality of frames. The synthesized image 184 obtained in this way is an image with a higher resolution than the images 182 of the plurality of frames.

[0128] Also, as shown in FIG. 19 as an example, the CPU 42 of the monitoring camera main body 20 outputs a shake correction command to the CPU 42 of the monitoring camera main body 20 via the communication I / F 60 of the monitoring camera main body 20 and the communication I / F 100 of the lens device 70. The acquisition unit 172 acquires the shake correction command output from the CPU 42 of the monitoring camera main body 20 and the shake amount detection result by the shake amount detection sensor 56. The shake correction command is command information indicating a request for shake correction, and the shake amount detection result by the shake amount detection sensor 56 is information representing the result of detecting the shake amount of the monitoring camera 10.

[0129] Furthermore, the CPU 212 of the management device 200 outputs the inclination information to the CPU 212 of the management device 200 via the communication I / F 220 of the management device 200 and the communication I / F 100 of the lens device 70. The acquisition unit 172 acquires the inclination information output from the CPU 212 of the management device 200. The inclination information includes inclination information regarding the inclination angle θx of the X2 axis with respect to the X1 axis and inclination information regarding the inclination angle θy of the Y2 axis with respect to the Y1 axis. Also, the acquisition unit 172 acquires the position detection result by the X-axis position sensor 156A and the position detection result by the Y-axis position sensor 156B.

[0130] When the acquisition unit 172 acquires a shake correction command, the calculation unit 174 calculates a movement amount C1 for correcting the shake of the image in the X1-axis direction of the shake correction lens 76 based on the shake amount detection result by the shake amount detection sensor 56. Specifically, the calculation unit 174 calculates, for the shake correction lens 76, a movement amount C1 for returning the position of the image shaken by the shake of the monitoring camera 10 in the X1-axis direction to the position of the image in the X1-axis direction before the shake of the monitoring camera 10 occurred. The movement amount C1 for correcting the shake of the image in the X1-axis direction may be predetermined according to the shake amount detection result by the shake amount detection sensor 56, or may be calculated using various calculation formulas.

[0131] Incidentally, as described above, when the lens device 70 is attached to the lens mount 22 of the surveillance camera body 20, the X2 axis of the lens device 70 may tilt with respect to the X1 axis of the surveillance camera body 20 due to the effects of rattling and / or deformation of the lens mount 22. For example, in the example shown in FIG. 19, the X2 axis of the lens device 70 is tilted at an inclination angle θx with respect to the X1 axis of the surveillance camera body 20. Therefore, in order to move the shake correction lens 76 by a movement amount C1 along the X1 axis, it is necessary to move the shake correction lens 76 along the X2 axis and the Y2 axis by the X-axis motor 136A and the Y-axis motor 136B, respectively.

[0132] Therefore, the calculation unit 174 calculates the movement amount C1 based on the shake amount detection result by the above-described shake amount detection sensor 56, and based on the calculated movement amount C1 and the inclination angle θx represented by the inclination information acquired by the acquisition unit 172, it calculates the movement amount Cx along the X2 axis of the shake correction lens 76 and the movement amount Cy along the Y2 axis of the shake correction lens 76, respectively. That is, the calculation unit 174 calculates the movement amount Cx along the X2 axis of the shake correction lens 76 by the following formula (7), and calculates the movement amount Cy along the Y2 axis of the shake correction lens 76 by the following formula (8). Cx = C1 × cosθy / cos(θy - θx) ··· (7) Cy = -C1 × sinθx / cos(θy - θx) ··· (8)

[0133] The movement amount Cx is calculated as a positive value when moving the shake correction lens 76 in the positive direction of the X2 axis, and is calculated as a negative value when moving the shake correction lens 76 in the negative direction of the X2 axis. Similarly, the movement amount Cy is calculated as a positive value when moving the shake correction lens 76 in the positive direction of the Y2 axis, and is calculated as a negative value when moving the shake correction lens 76 in the negative direction of the Y2 axis. The movement amount Cx is an example of the "first movement amount" according to the technology of the present disclosure, and the movement amount Cy is an example of the "second movement amount" according to the technology of the present disclosure.

[0134] Similarly, as shown in FIG. 20 as an example, when a shake correction command is acquired by the acquisition unit 172, the calculation unit 174 calculates a movement amount D1 for correcting the shake in the Y1-axis direction of the image for the shake correction lens 76 based on the shake amount detection result by the shake amount detection sensor 56. Specifically, the calculation unit 174 calculates a movement amount D1 for returning the position in the Y1-axis direction of the image shaken by the shake of the monitoring camera 10 to the position in the Y1-axis direction of the image before the shake of the monitoring camera 10 for the shake correction lens 76. The movement amount D1 for correcting the shake in the Y1-axis direction of the image may be determined in advance according to the shake amount detection result by the shake amount detection sensor 56, or may be calculated using various calculation formulas.

[0135] By the way, as described above, when the lens device 70 is attached to the lens mount 22 of the monitoring camera body 20, the Y2 axis of the lens device 70 may be inclined with respect to the Y1 axis of the monitoring camera body 20 due to the rattling and / or deformation of the lens mount 22. For example, in the example shown in FIG. 20, the Y2 axis of the lens device 70 is inclined at an inclination angle θy with respect to the Y1 axis of the monitoring camera body 20. Therefore, in order to move the shake correction lens 76 by the movement amount D1 along the Y1 axis, it is required to move the shake correction lens 76 along the X2 axis and the Y2 axis by the X-axis motor 136A and the Y-axis motor 136B, respectively.

[0136] Therefore, the calculation unit 174 calculates the movement amount D1 based on the shake amount detection result by the shake amount detection sensor 56 described above, and based on the calculated movement amount D1 and the inclination angle θy represented by the inclination information acquired by the acquisition unit 172, calculates the movement amount Dx along the X2 axis of the shake correction lens 76 and the movement amount Dy along the Y2 axis of the shake correction lens 76, respectively. That is, the calculation unit 174 calculates the movement amount Dx along the X2 axis of the shake correction lens 76 by the following formula (9), and calculates the movement amount Dy along the Y2 axis of the shake correction lens 76 by the following formula (10). Dx = D1×sinθy / cos(θy - θx) ···(9) Dy = D1×cosθx / cos(θy - θx) ···(10)

[0137] The movement amount Dx is calculated as a positive value when the shake correction lens 76 is moved in the positive direction of the X2 axis, and is calculated as a negative value when the shake correction lens 76 is moved in the negative direction of the X2 axis. Similarly, the movement amount Dy is calculated as a positive value when the shake correction lens 76 is moved in the positive direction of the Y2 axis, and is calculated as a negative value when the shake correction lens 76 is moved in the negative direction of the Y2 axis. The movement amount Dx is an example of the "first movement amount" according to the technology of the present disclosure, and the movement amount Dy is an example of the "second movement amount" according to the technology of the present disclosure.

[0138] Also, in order to correct the shake of the image in the X1 axis direction and the Y1 axis direction simultaneously, when the shake correction lens 76 is moved simultaneously along the X1 axis and the Y1 axis, the calculation unit 174 adds the movement amount Cx, which is a positive value or a negative value calculated based on the movement amount C1 of the shake correction lens 76, and the movement amount Dx, which is a positive value or a negative value calculated based on the movement amount D1 of the shake correction lens 76, to calculate the movement amount of the shake correction lens 76 along the X2 axis. Further, the calculation unit 174 adds the movement amount Cy, which is a positive value or a negative value calculated based on the movement amount C1 of the shake correction lens 76, and the movement amount Dy, which is a positive value or a negative value calculated based on the movement amount D1 of the shake correction lens 76, to calculate the movement amount of the shake correction lens 76 along the Y2 axis.

[0139] Furthermore, when performing the above-described image shift and image shake correction simultaneously, the calculation unit 174 adds a plurality of movement amounts selected according to the direction of shifting the image and the direction of correcting the image shake among the movement amount Ax, which is a positive value or a negative value calculated based on the movement amount A1 of the shake correction lens 76, the movement amount Bx, which is a positive value or a negative value calculated based on the movement amount B1 of the shake correction lens 76, the movement amount Cx, which is a positive value or a negative value calculated based on the movement amount C1 of the shake correction lens 76, and the movement amount Dx, which is a positive value or a negative value calculated based on the movement amount D1 of the shake correction lens 76, to calculate the movement amount of the shake correction lens 76 along the X2 axis.

[0140] Also, when simultaneously performing the above-described image shift and correction of image blur, the calculation unit 174 calculates a positive or negative movement amount Ay calculated based on the movement amount A1 of the blur correction lens 76, a positive or negative movement amount By calculated based on the movement amount B1 of the blur correction lens 76, a positive or negative movement amount Cy calculated based on the movement amount C1 of the blur correction lens 76, and a positive or negative movement amount Dy calculated based on the movement amount D1 of the blur correction lens 76. The calculation unit 174 adds a plurality of movement amounts selected according to the direction of shifting the image and the direction of correcting the image blur to calculate the movement amount along the Y2 axis of the blur correction lens 76.

[0141] The control unit 176 sets the movement amount along the X2 axis of the blur correction lens 76 calculated by the calculation unit 174 as a target value, and generates an X-axis control command based on the position detection result by the X-axis position sensor 156A. The X-axis control command is output to the X-axis motor driver 116A. Similarly, the control unit 176 sets the movement amount along the Y2 axis of the blur correction lens 76 calculated by the calculation unit 174 as a target value, and generates a Y-axis control command based on the position detection result by the Y-axis position sensor 156B. The Y-axis control command is output to the Y-axis motor driver 116B.

[0142] The X-axis motor driver 116A generates an X-axis operation signal based on the X-axis control command, and the Y-axis motor driver 116B generates a Y-axis operation signal based on the Y-axis control command. The X-axis motor 136A operates with an operation amount corresponding to the X-axis operation signal, and the Y-axis motor 136B operates with an operation amount corresponding to the Y-axis operation signal. Thereby, the blur correction lens 76 moves in the direction in which the image blur is corrected, and the image blur is corrected.

[0143] The control for correcting the image blur by the control unit 176 described above is feedback control based on the blur amount detection result by the blur amount detection sensor 56 (that is, the blur amount of the monitoring camera 10).

[0144] Next, the operation of the monitoring system S according to the first embodiment (that is, the operation of the monitoring system S) will be described.

[0145] First, with reference to FIG. 21, the slope information output process executed by the CPU 212 of the management device 200 will be described.

[0146] First, in step ST100, the slope information generation unit 232 (see FIG. 9) moves the shake correction lens 76 to the center of the X2 - Y2 coordinate system. That is, the slope information generation unit 232 outputs a first control command to the CPU 92 of the lens device 70. When the CPU 92 of the lens device 70 receives the first control command output from the management device 200, it controls the X - axis motor 136A and the Y - axis motor 136B via the X - axis motor driver 116A and the Y - axis motor driver 116B, respectively, to move the shake correction lens 76 to the center of the X2 - Y2 coordinate system. As a result, the shake correction lens 76 moves to the center of the X2 - Y2 coordinate system.

[0147] In the next step ST102, the slope information generation unit 232 causes the image sensor 24 to perform imaging to obtain a first image 262. That is, the slope information generation unit 232 outputs a first imaging command to the CPU 42 of the monitoring camera main body 20. When the image sensor driver 52 receives the first imaging command, it controls the image sensor 24 to perform imaging. As a result, an image is captured by the image sensor 24 to obtain the first image 262.

[0148] In the next step ST104, the slope information generation unit 232 (see FIG. 10) moves the shake correction lens 76 along the X2 axis by a first movement amount. That is, the slope information generation unit 232 outputs a second control command to the CPU 92 of the lens device 70. When the CPU 92 of the lens device 70 receives the second control command output from the management device 200, it controls the X - axis motor 136A via the X - axis motor driver 116A to move the shake correction lens 76 along the X2 axis by the first movement amount. As a result, the center 76A of the shake correction lens 76 moves along the X2 axis by the first movement amount from the center of the X2 - Y2 coordinate system, and the subject image 25A moves from the coordinates (0, 0) to the coordinates (a1, b1) in the X1 - Y1 coordinate system.

[0149] In the next step ST106, the tilt information generation unit 232 causes the image sensor 24 to perform imaging to obtain a second image 264. That is, the tilt information generation unit 232 outputs a second imaging command to the CPU 42 of the monitoring camera main body 20. When the image sensor driver 52 receives the second imaging command, it controls the image sensor 24 to perform imaging. As a result, the second image 264 is obtained by imaging an image with the image sensor 24.

[0150] In the next step ST108, the tilt information generation unit 232 (see FIG. 11) calculates the tilt angle θx of the X2 axis with respect to the X1 axis. That is, the tilt information generation unit 232 performs a first image analysis based on the first image 262 and the second image 264. When the subject image 25A located at the coordinates (0, 0) of the first image 262 in the X1 - Y1 coordinate system has moved to the coordinates (a1, b1) of the second image 264, the tilt angle θx of the X2 axis with respect to the X1 axis is calculated by the following formula (1). θx = tan -1 (b1 / a1) ··· (1)

[0151] In the next step ST110, the tilt information generation unit 232 (see FIG. 12) moves the shake correction lens 76 along the Y2 axis by a second movement amount. That is, the tilt information generation unit 232 outputs a third control command to the CPU 92 of the lens device 70. When the CPU 92 of the lens device 70 receives the third control command output from the management device 200, it controls the Y - axis motor 136B via the Y - axis motor driver 116B to move the shake correction lens 76 along the Y2 axis by the second movement amount. As a result, the center 76A of the shake correction lens 76 moves along the Y2 axis by the second movement amount from the position on the X2 axis, and the subject image 25A moves from the coordinates (a1, b1) to the coordinates (a2, b2) in the X1 - Y1 coordinate system.

[0152] In the next step ST112, the tilt information generation unit 232 causes the image sensor 24 to perform imaging to obtain a third image 266. That is, the tilt information generation unit 232 outputs a third imaging command to the CPU 42 of the monitoring camera body 20. When the image sensor driver 52 receives the third imaging command, it controls the image sensor 24 to perform imaging. As a result, the third image 266 is obtained by imaging an image with the image sensor 24.

[0153] In the next step ST114, the tilt information generation unit 232 (see FIG. 13) calculates the tilt angle θy of the Y2 axis with respect to the Y1 axis. That is, the tilt information generation unit 232 performs a second image analysis based on the second image 264 and the third image 266. When the subject image 25A located at the coordinates (a1, b1) of the second image 264 in the X1-Y1 coordinate system has moved to the coordinates (a2, b2) of the third image 266, the tilt angle θy of the Y2 axis with respect to the Y1 axis is calculated by the following formula (2). θy = tan -1 {(b2 - b1) / (a2 - a1)} ··· (2)

[0154] In the next step ST116, the tilt information output unit 234 (see FIG. 14) outputs tilt information regarding the tilt angle θx and the tilt angle θy to the CPU 92 of the lens device 70 via the communication I / F 220 of the management device 200 and the communication I / F 100 of the lens device 70.

[0155] Next, the shake correction and shift process executed by the CPU 92 of the lens device 70 will be described. The shake correction and shift process includes a shift process (see FIG. 22) and a shake correction process (see FIG. 23). First, the shift process will be described with reference to FIG. 22.

[0156] First, in step ST200, the acquisition unit 172 (see FIG. 16) acquires an image shift command output from the CPU 42 of the monitoring camera body 20.

[0157] In the next step ST202, the acquisition unit 172 acquires the frame period information output from the CPU 42 of the monitoring camera main body 20.

[0158] In the next step ST204, the acquisition unit 172 acquires the tilt information output from the CPU 212 of the management device 200.

[0159] In the next step ST206, the calculation unit 174 calculates the movement amount for shifting the image with respect to the shake correction lens 76. That is, when the X-axis image shift command is acquired by the acquisition unit 172 (see FIG. 16), the calculation unit 174 calculates, based on the image shift amount represented by the X-axis image shift command, the frame period represented by the frame period information, and the position detection result by the X-axis position sensor 156A, the movement amount A1 of the shake correction lens 76 for each frame period. Further, when the Y-axis image shift command is acquired by the acquisition unit 172 (see FIG. 17), the calculation unit 174 calculates, based on the image shift amount represented by the Y-axis image shift command, the frame period represented by the frame period information, and the position detection result by the Y-axis position sensor 156B, the movement amount B1 of the shake correction lens 76 for each frame period. Also, when the XY-axis image shift command is acquired by the acquisition unit 172 (see FIGS. 16 and 17), the calculation unit 174 calculates the movement amount A1 of the shake correction lens 76 for each frame period in the same manner as when the X-axis image shift command is acquired by the acquisition unit 172, and calculates the movement amount B1 of the shake correction lens 76 for each frame period in the same manner as when the Y-axis image shift command is acquired by the acquisition unit 172.

[0160] In the next step ST208, the calculation unit 174 calculates the movement amount along the X2 axis of the shake correction lens 76 and the movement amount along the Y2 axis of the shake correction lens 76, respectively. That is, when the X-axis image shift command is acquired by the acquisition unit 172 (see FIG. 16), the calculation unit 174 calculates the movement amount Ax along the X2 axis of the shake correction lens 76 by the following formula (3), and calculates the movement amount Ay along the Y2 axis of the shake correction lens 76 by the following formula (4). Ax = A1 × cosθy / cos(θy - θx) ···(3) Ay = -A1 × sinθx / cos(θy - θx) ··· (4)

[0161] Further, when a Y-axis image shift command is acquired by the acquisition unit 172 (see FIG. 17), the calculation unit 174 calculates the movement amount Bx along the X2 axis of the blur correction lens 76 by the following formula (5), and calculates the movement amount By along the Y2 axis of the blur correction lens 76 by the following formula (6). Bx = B1 × sinθy / cos(θy - θx) ··· (5) By = B1 × cosθx / cos(θy - θx) ··· (6)

[0162] Also, when an XY-axis image shift command is acquired by the acquisition unit 172 (see FIGS. 16 and 17), the calculation unit 174 calculates the movement amount A1 of the blur correction lens 76 calculated in the same manner as when an X-axis image shift command is acquired by the acquisition unit 172, and the movement amount B1 of the blur correction lens 76 calculated in the same manner as when a Y-axis image shift command is acquired by the acquisition unit 172. Based on this, the movement amount along the X2 axis of the blur correction lens 76 and the movement amount along the Y2 axis of the blur correction lens 76 are calculated. At this time, the calculation unit 174 calculates the movement amount along the X2 axis of the blur correction lens 76 by adding the movement amount Ax, which is a positive or negative value, and the movement amount Bx, which is a positive or negative value. The movement amount along the Y2 axis of the blur correction lens 76 is calculated by adding the movement amount Ay, which is a positive or negative value, and the movement amount By, which is a positive or negative value.

[0163] In the next step ST210, the control unit 176 moves the blur correction lens 76 to shift the image. That is, the control unit 176 generates an X-axis control command corresponding to the movement amount along the X2 axis of the blur correction lens 76 calculated by the calculation unit 174 for each frame period. The X-axis control command is output to the X-axis motor driver 116A. Similarly, the control unit 176 generates a Y-axis control command corresponding to the movement amount along the Y2 axis of the blur correction lens 76 calculated by the calculation unit 174 for each frame period. The Y-axis control command is output to the Y-axis motor driver 116B.

[0164] The X-axis motor driver 116A generates an X-axis operation signal based on an X-axis control command, and the Y-axis motor driver 116B generates a Y-axis operation signal based on a Y-axis control command. The X-axis motor 136A operates with an operation amount corresponding to the X-axis operation signal, and the Y-axis motor 136B operates with an operation amount corresponding to the Y-axis operation signal. Thus, for each frame period, the shake correction lens 76 moves in the direction of shifting the image along the X1 axis and / or the Y1 axis, and the image is shifted along the X1 axis and / or the Y1 axis.

[0165] Next, with reference to FIG. 23, the shake correction process will be described.

[0166] First, in step ST300, the acquisition unit 172 (see FIGS. 19 and 20) acquires a shake correction command output from the CPU 42 of the monitoring camera body 20.

[0167] In the next step ST302, the acquisition unit 172 acquires a shake amount detection result by the shake amount detection sensor 56 output from the CPU 42 of the monitoring camera body 20.

[0168] In the next step ST304, the acquisition unit 172 acquires tilt information output from the CPU 212 of the management device 200.

[0169] In the next step ST306, the calculation unit 174 calculates a movement amount for correcting the shake of the image with respect to the shake correction lens 76. That is, the calculation unit 174 calculates a movement amount C1 (see FIG. 19) for correcting the shake of the image in the X1-axis direction with respect to the shake correction lens 76. Further, the calculation unit 174 calculates a movement amount D1 (see FIG. 20) for correcting the shake of the image in the Y1-axis direction with respect to the shake correction lens 76 based on the shake amount detection result by the shake amount detection sensor 56.

[0170] In the next step ST308, the calculation unit 174 calculates the amount of movement along the X2 axis of the shake correction lens 76 and the amount of movement along the Y2 axis of the shake correction lens 76, respectively. That is, the calculation unit 174 calculates the amount of movement Cx along the X2 axis of the shake correction lens 76 according to the following formula (7), and calculates the amount of movement Cy along the Y2 axis of the shake correction lens 76 according to the following formula (8). Cx = C1 × cosθy / cos(θy - θx) ··· (7) Cy = -C1 × sinθx / cos(θy - θx) ··· (8)

[0171] Further, the calculation unit 174 calculates the amount of movement Dx along the X2 axis of the shake correction lens 76 according to the following formula (9), and calculates the amount of movement Dy along the Y2 axis of the shake correction lens 76 according to the following formula (10). Dx = D1 × sinθy / cos(θy - θx) ··· (9) Dy = D1 × cosθx / cos(θy - θx) ··· (10)

[0172] Also, when the shake correction lens 76 is simultaneously moved along the X1 axis and the Y1 axis in order to simultaneously correct the shake in the X1 axis direction and the shake in the Y1 axis direction of the image, the calculation unit 174 adds the amount of movement Cx, which is a positive or negative value calculated based on the amount of movement C1 of the shake correction lens 76, and the amount of movement Dx, which is a positive or negative value calculated based on the amount of movement D1 of the shake correction lens 76, to calculate the amount of movement along the X2 axis of the shake correction lens 76. Further, the calculation unit 174 adds the amount of movement Cy, which is a positive or negative value calculated based on the amount of movement C1 of the shake correction lens 76, and the amount of movement Dy, which is a positive or negative value calculated based on the amount of movement D1 of the shake correction lens 76, to calculate the amount of movement along the Y2 axis of the shake correction lens 76.

[0173] Furthermore, when simultaneously performing the above-described image shift and correction of image blur, the calculation unit 174 adds a plurality of movement amounts selected according to the direction of shifting the image and the direction of correcting image blur among the movement amount Ax, which is a positive or negative value calculated based on the movement amount A1 of the blur correction lens 76, the movement amount Bx, which is a positive or negative value calculated based on the movement amount B1 of the blur correction lens 76, the movement amount Cx, which is a positive or negative value calculated based on the movement amount C1 of the blur correction lens 76, and the movement amount Dx, which is a positive or negative value calculated based on the movement amount D1 of the blur correction lens 76, to calculate the movement amount along the X2 axis of the blur correction lens 76.

[0174] Also, when simultaneously performing the above-described image shift and correction of image blur, the calculation unit 174 adds a plurality of movement amounts selected according to the direction of shifting the image and the direction of correcting image blur among the movement amount Ay, which is a positive or negative value calculated based on the movement amount A1 of the blur correction lens 76, the movement amount By, which is a positive or negative value calculated based on the movement amount B1 of the blur correction lens 76, the movement amount Cy, which is a positive or negative value calculated based on the movement amount C1 of the blur correction lens 76, and the movement amount Dy, which is a positive or negative value calculated based on the movement amount D1 of the blur correction lens 76, to calculate the movement amount along the Y2 axis of the blur correction lens 76.

[0175] In the next step ST310, the control unit 176 sets the movement amount along the X2 axis of the blur correction lens 76 calculated by the calculation unit 174 as a target value, and generates an X-axis control command based on the position detection result by the X-axis position sensor 156A. The X-axis control command is output to the X-axis motor driver 116A. Similarly, the control unit 176 sets the movement amount along the Y2 axis of the blur correction lens 76 calculated by the calculation unit 174 as a target value, and generates a Y-axis control command based on the position detection result by the Y-axis position sensor 156B. The Y-axis control command is output to the Y-axis motor driver 116B.

[0176] The X-axis motor driver 116A generates an X-axis actuation signal based on an X-axis control command, and the Y-axis motor driver 116B generates a Y-axis actuation signal based on a Y-axis control command. The X-axis motor 136A operates with an amount of actuation corresponding to the X-axis actuation signal, and the Y-axis motor 136B operates with an amount of actuation corresponding to the Y-axis actuation signal. Thereby, the shake correction lens 76 moves in the direction in which the image blur is corrected, and the image blur is corrected.

[0177] The method of operating the surveillance camera 10 described while referring to FIGS. 22 and 23 above is an example of the "method of operating an imaging device" according to the technology of the present disclosure. Further, the method of operating the lens device 70 included in the method of operating the surveillance camera 10 described while referring to FIGS. 22 and 23 above is an example of the "method of operating a lens device" according to the technology of the present disclosure.

[0178] Next, the effects of the first embodiment will be described.

[0179] As shown in FIGS. 16 and 19, the CPU 92 of the lens device 70 acquires tilt information regarding the tilt of the X2 axis of the lens device 70 with respect to the X1 axis of the image sensor 24, and controls the X-axis motor 136A and the Y-axis motor 136B to move the shake correction lens 76 along the X1 axis of the image sensor 24 based on the tilt information. Therefore, even when there is a tilt of the X2 axis of the lens device 70 with respect to the X1 axis of the image sensor 24, the shake correction lens 76 moves along the X1 axis of the image sensor 24 by receiving the power of the X-axis motor 136A and the Y-axis motor 136B, so that the image can be moved along the X1 axis of the image sensor 24.

[0180] Similarly, as shown in FIGS. 17 and 20, the CPU 92 of the lens device 70 acquires tilt information regarding the tilt of the Y2 axis of the lens device 70 with respect to the Y1 axis of the image sensor 24, and performs control to move the shake correction lens 76 along the Y1 axis of the image sensor 24 based on the tilt information with respect to the X-axis motor 136A and the Y-axis motor 136B. Therefore, even when there is a tilt of the Y2 axis of the lens device 70 with respect to the Y1 axis of the image sensor 24, the shake correction lens 76 moves along the Y1 axis of the image sensor 24 by receiving the power of the X-axis motor 136A and the Y-axis motor 136B, so that the image can be moved along the Y1 axis of the image sensor 24.

[0181] Also, as shown in FIG. 16, when the CPU 92 of the lens device 70 receives, for example, an X-axis image shift command, it performs control to move the shake correction lens 76 in the direction in which the image shifts in the X1-axis direction with respect to the X-axis motor 136A and the Y-axis motor 136B. Therefore, the image can be shifted in the X1-axis direction by the shake correction lens 76 moving in the X1-axis direction.

[0182] Similarly, as shown in FIG. 17, when the CPU 92 of the lens device 70 receives, for example, a Y-axis image shift command, it performs control to move the shake correction lens 76 in the direction in which the image shifts in the Y1-axis direction with respect to the X-axis motor 136A and the Y-axis motor 136B. Therefore, the image can be shifted in the Y1-axis direction by the shake correction lens 76 moving in the Y1-axis direction.

[0183] Also, as shown in FIG. 19, when, for example, shake in the X1-axis direction of the image occurs, the CPU 92 of the lens device 70 performs control to move the shake correction lens 76 in the direction in which the shake in the X1-axis direction of the image is corrected with respect to the X-axis motor 136A and the Y-axis motor 136B. Therefore, the shake in the X1-axis direction of the image can be corrected by the shake correction lens 76 moving in the X1-axis direction.

[0184] Similarly, as shown in FIG. 20, when shake occurs in the Y1-axis direction of the image, for example, the CPU 92 of the lens device 70 controls the X-axis motor 136A and the Y-axis motor 136B to move the shake correction lens 76 in the direction in which the shake in the Y1-axis direction of the image is corrected. Therefore, the shake in the Y1-axis direction of the image can be corrected by moving the shake correction lens 76 in the Y1-axis direction.

[0185] The lens device 70 also includes a communication I / F 100 that communicates with the management device 200, and the CPU 92 of the lens device 70 acquires the inclination information transmitted from the management device 200 and received by the communication I / F 100. Therefore, the CPU 92 of the lens device 70 can control the movement of the shake correction lens 76 based on the inclination information generated by the management device 200.

[0186] The CPU 92 of the lens device 70 also stores the acquired inclination information in the NVM 94 (see FIG. 5). Therefore, even when the power supply to the lens device 70 is stopped, the inclination information can be maintained stored in the NVM 94, and the inclination information stored in the NVM 94 can be used until new inclination information is obtained.

[0187] Also, since the inclination information stored in the NVM 94 can be used, it is not necessary to calculate the inclination information every time the power is supplied to the lens device 70 and the lens device 70 is started up.

[0188] As shown in FIG. 11, the inclination information representing the inclination angle θx is information calculated based on a second control command for moving the shake correction lens 76 along the X2 axis of the lens device 70 and the first image 262 and the second image 264 obtained by imaging with the image sensor 24 before and after the shake correction lens 76 moves based on the second control command. Therefore, for example, the inclination angle θx can be calculated without using an inclination angle detection sensor for detecting the inclination angle θx.

[0189] Similarly, as shown in FIG. 13, the tilt information representing the tilt angle θy is information calculated based on a third control command for moving the shake correction lens 76 along the Y2 axis of the lens device 70, and the second image 264 and the third image 266 obtained by imaging with the image sensor 24 before and after the shake correction lens 76 moves based on the third control command. Therefore, for example, even without using a tilt angle detection sensor that detects the tilt angle θy, the tilt angle θy can be calculated.

[0190] Further, the tilt information is information calculated based on a plurality of images obtained by imaging with the image sensor 24 under imaging conditions where an image with less noise than an image obtained by normal imaging is obtained. Therefore, for example, compared to the case where the tilt information is calculated based on a plurality of images obtained under normal imaging conditions, the tilt angles θx and θy represented by the tilt information can be calculated with higher accuracy.

[0191] Further, the tilt information is information regarding the tilt angle θx of the X2 axis of the lens device 70 with respect to the X1 axis of the image sensor 24 and the tilt angle of the Y2 axis of the lens device 70 with respect to the Y1 axis of the image sensor 24. Therefore, compared to the case where the tilt information is information that does not include a specific tilt angle (for example, information regarding the rotational position of the lens device 70 with respect to the monitoring camera body 20, etc.), the movement amount of the shake correction lens 76 along the X2 axis and the movement amount of the shake correction lens 76 along the Y2 axis can be calculated with high accuracy.

[0192] Further, the CPU 92 of the lens device 70 calculates the movement amount of the shake correction lens 76 along the X2 axis and the movement amount of the shake correction lens 76 along the Y2 axis based on the tilt information regarding the tilt angle θx, and controls the X-axis motor 136A and the Y-axis motor 136B to move the shake correction lens 76 with the calculated movement amount of the shake correction lens 76 along the X2 axis and the movement amount of the shake correction lens 76 along the Y2 axis. Therefore, even when there is a tilt of the X2 axis of the lens device 70 with respect to the X1 axis of the image sensor 24, the shake correction lens 76 can be moved along the X1 axis of the image sensor 24.

[0193] Similarly, based on the tilt information regarding the tilt angle θy, the CPU 92 of the lens device 70 calculates the amount of movement of the shake correction lens 76 along the X2 axis and the amount of movement of the shake correction lens 76 along the Y2 axis, and controls the X-axis motor 136A and the Y-axis motor 136B to move the shake correction lens 76 by the calculated amounts of movement along the X2 axis and the Y2 axis of the shake correction lens 76. Therefore, even when there is a tilt of the Y2 axis of the lens device 70 with respect to the Y1 axis of the image sensor 24, the shake correction lens 76 can be moved along the Y1 axis of the image sensor 24.

[0194] Next, a modification of the first embodiment will be described.

[0195] In the first embodiment, the shift of the image and the correction of the image blur are performed using the shake correction lens 76. However, the lens device 70 may separately include a shift lens that performs the shift of the image and a shake correction lens that corrects the image blur. Further, when the lens device 70 separately includes a shift lens that performs the shift of the image and a shake correction lens that corrects the image blur, the shift lens may be moved by the above-described shift process, and the shake correction lens may be moved by the above-described shake correction process. Note that the shift lens and the shake correction lens in this case respectively correspond to an example of the "moving lens" according to the technology of the present disclosure.

[0196] Also, in the first embodiment, both the control to move the shake correction lens 76 along the X1 axis based on the tilt information regarding the tilt angle θx and the control to move the shake correction lens 76 along the Y1 axis based on the tilt information regarding the tilt angle θy are executed. However, only one of the control to move the shake correction lens 76 along the X1 axis based on the tilt information regarding the tilt angle θx and the control to move the shake correction lens 76 along the Y1 axis based on the tilt information regarding the tilt angle θy may be executed.

[0197] Also, in the first embodiment, the tilt information includes the tilt angle, but may include a detection value other than the tilt angle.

[0198] In the first embodiment, in both the shift process of shifting the image by moving the shake correction lens 76 and the shake correction process of correcting the image shake by moving the shake correction lens 76, control is executed to move the shake correction lens 76 based on the tilt information. However, control may be executed to move the shake correction lens 76 based on the tilt information in only one of the shift process and the shake correction process.

[0199] In the first embodiment, the management device 200 generates tilt information and outputs the generated tilt information to the lens device 70. However, the monitoring camera main body 20 may generate tilt information and output the generated tilt information to the lens device 70. Further, the lens device 70 may generate tilt information. Also, when the monitoring camera main body 20 generates tilt information or when the lens device 70 generates tilt information, the communication I / F 220 of the management device 200 and the communication I / F 100 of the lens device 70 may not be communicably connected.

[0200] In the first embodiment, the communication I / F 60 of the monitoring camera main body 20 and the communication I / F 100 of the lens device 70 are communicably connected. However, the communication I / F 60 of the monitoring camera main body 20 and the communication I / F 100 of the lens device 70 may not be communicably connected. Also, when the communication I / F 60 of the monitoring camera main body 20 and the communication I / F 100 of the lens device 70 are not communicably connected, the information generated by the monitoring camera main body 20 may be output to the lens device 70 via the management device 200.

[0201] In the first embodiment, the CPU 92 of the lens device 70 stores the acquired tilt information in the NVM 94. However, the acquired tilt information may be stored in the RAM 96 without being stored in the NVM 94.

[0202] Also, in the first embodiment, a second control command for moving the shake correction lens 76 along the X2 axis, a third control command for moving the shake correction lens 76 along the Y2 axis, and based on the second control command and the third control command, the inclination angle θx and the inclination angle θy are calculated based on the first image 262, the second image 264, and the third image 266 obtained by imaging with the image sensor 24 before and after the shake correction lens 76 moves. However, for example, the inclination angle θx is calculated based on the second control command and the first image 262 and the second image 264 obtained by imaging with the image sensor 24 before and after the shake correction lens 76 moves based on the second control command, and the inclination angle θy may be set to the same value as the inclination angle θx by assuming that the Y2 axis is perpendicular to the X2 axis.

[0203] Also, in the first embodiment, for example, the inclination angle θy is calculated based on a plurality of images obtained by imaging with the image sensor 24 before and after moving the shake correction lens 76 along the Y2 axis from the center of the X2 - Y2 coordinate system, and the inclination angle θx may be set to the same value as the inclination angle θy by assuming that the X2 axis is perpendicular to the Y2 axis.

[0204] Also, in the first embodiment, the inclination angle θy is calculated based on the second image 264 and the third image 266 obtained by imaging with the image sensor 24 before and after moving the shake correction lens 76 along the Y2 axis from the position where it was moved along the X2 axis from the center of the X2 - Y2 coordinate system. However, the inclination angle θy may also be calculated based on a plurality of images obtained by imaging with the image sensor 24 before and after moving the shake correction lens 76 along the Y2 axis from the center of the X2 - Y2 coordinate system.

[0205] Also, in the first embodiment, the inclination information is calculated based on a plurality of images obtained by imaging with the image sensor 24 to which a sensitivity lower than that of the image sensor 24 performing normal imaging is applied. However, the inclination information may also be calculated based on a plurality of images obtained by imaging with the image sensor 24 to which the sensitivity of the image sensor 24 performing normal imaging is applied.

[0206] Also, in the first embodiment, when only the control for moving the shake correction lens 76 along the X1 axis is executed, the tilt angle θy may not be calculated. Similarly, when only the control for moving the shake correction lens 76 along the Y1 axis is executed, the tilt angle θx may not be calculated.

[0207] Also, in the first embodiment, the CPU 92 of the lens device 70 calculates the amount of movement for moving the shake correction lens 76 along the X2 axis and the amount of movement for moving the shake correction lens 76 along the Y2 axis based on the tilt information. However, the CPU 212 of the management device 200 may calculate the amount of movement for moving the shake correction lens 76 along the X2 axis and the amount of movement for moving the shake correction lens 76 along the Y2 axis based on the tilt information. Further, the CPU 42 of the monitoring camera body 20 may calculate the amount of movement for moving the shake correction lens 76 along the X2 axis and the amount of movement for moving the shake correction lens 76 along the Y2 axis based on the tilt information.

[0208] Also, among the plurality of modification examples according to the first embodiment, the combinable modification examples may be combined as appropriate.

[0209] [Second Embodiment] Next, the second embodiment will be described. In the second embodiment, the configuration of the monitoring system S is changed as follows with respect to the first embodiment. In the second embodiment, the same elements, members, etc. as those in the first embodiment are denoted by the same reference numerals as those in the first embodiment, and detailed descriptions thereof are omitted.

[0210] As an example, as shown in FIG. 24, related information is stored in the NVM 214 of the management device 200. The related information is information associating first registration information, second registration information, and inclination information. The first registration information is information regarding the surveillance camera main body 20, for example, information registering the individual number of the surveillance camera main body 20. The second registration information is information regarding the lens device 70, for example, information registering the individual number of the lens device 70. The inclination information is information calculated in a state where the lens device 70 registered with the second registration information is attached to the surveillance camera main body 20 registered with the first registration information. The method for calculating the inclination information is as described in the first embodiment.

[0211] When the CPU 212 of the management device 200 acquires a combination of the first registration information, the second registration information, and the inclination information, it stores, as related information, the information associating the first registration information, the second registration information, and the inclination information in the NVM 214.

[0212] First identification information is stored in the NVM 44 of the surveillance camera main body 20. The first identification information is information regarding the surveillance camera main body 20, similar to the first registration information, for example, information registering the individual number of the surveillance camera main body 20. Second identification information is stored in the NVM 94 of the lens device 70. The second identification information is information regarding the lens device 70, similar to the second registration information, for example, information registering the individual number of the lens device 70. When the lens device 70 is attached to the surveillance camera main body 20, the first identification information is output from the surveillance camera main body 20, and the second identification information is output from the lens device 70.

[0213] The CPU 212 of the management device 200 operates as an acquisition unit 242, a determination unit 244, an extraction unit 246, an output unit 248, and a notification control unit 250 by executing the inclination information output processing program 240 stored in the NVM 214 on the RAM 216.

[0214] The acquisition unit 242 acquires the first identification information output from the monitoring camera main body 20 via the communication I / F 220 of the management device 200 and the communication I / F 60 of the monitoring camera main body 20. Further, the acquisition unit 242 acquires the second identification information output from the lens device 70 via the communication I / F 220 of the management device 200 and the communication I / F 100 of the lens device 70. Furthermore, the acquisition unit 242 acquires the related information stored in the NVM 214.

[0215] Based on the first identification information, the second identification information, and the related information acquired by the acquisition unit 242, the determination unit 244 determines whether the first registration information included in the related information matches the first identification information acquired by the acquisition unit 242, and whether the second registration information included in the related information matches the second identification information acquired by the acquisition unit 242.

[0216] When the determination by the determination unit 244 is affirmative, the extraction unit 246 extracts tilt information from the related information acquired by the acquisition unit 242. The output unit 248 outputs the tilt information extracted by the extraction unit 246. The tilt information output from the output unit 248 is transmitted to the lens device 70 via the communication I / F 220 of the management device 200. In the lens device 70, as described in the first embodiment, control is executed to move the shake correction lens 76 (see FIGS. 16, 17, 19, and 20) based on the tilt information.

[0217] When the determination by the determination unit 244 is negative, that is, when the first registration information included in the related information is different from the first identification information of the monitoring camera main body 20 acquired by the acquisition unit 242, or when the second registration information included in the related information is different from the second identification information of the lens device 70 acquired by the acquisition unit 242, the notification control unit 250 performs control to notify, for example, as processing contributing to the update of the tilt information.

[0218] The notification may be, for example, a notification indicating that it is necessary to execute a process for updating the tilt information, or a notification indicating that the monitoring camera main body 20 and the lens device 70 are not in a known combination. Further, the notification may be, for example, a notification displayed on the display 222 (see FIG. 6) of the management device 200, a notification by sound output from the speaker of the management device 200, or a notification by the warning lamp of the management device 200 emitting light. Thereby, an operation for updating the tilt information is performed on the monitoring system S by the administrator who has received the notification.

[0219] Next, with reference to FIG. 25, the tilt information output process according to the second embodiment will be described.

[0220] First, in step ST400, the acquisition unit 242 acquires the first identification information output from the monitoring camera main body 20. Further, the acquisition unit 172 acquires the second identification information output from the lens device 70. Furthermore, the acquisition unit 242 acquires the related information stored in the NVM 214.

[0221] In the next step ST402, the determination unit 244 determines whether the first registration information included in the related information matches the first identification information acquired by the acquisition unit 242, and the second registration information included in the related information matches the second identification information acquired by the acquisition unit 242, based on the first identification information, the second identification information, and the related information acquired by the acquisition unit 242. If the determination by the determination unit 244 is affirmative, step ST404 is executed.

[0222] In the next step ST404, the extraction unit 246 extracts the tilt information from the related information acquired by the acquisition unit 242.

[0223] In the next step ST406, the output unit 248 outputs the tilt information extracted by the extraction unit 246. The tilt information output from the output unit 248 is transmitted to the lens device 70. In the lens device 70, as described in the first embodiment, control for moving the shake correction lens 76 (see FIGS. 16, 17, 19, and 20) based on the tilt information is executed.

[0224] On the other hand, if the determination by the determination unit 244 is negative in step ST402 described above, step ST408 is executed. In step ST408, the notification control unit 250 performs control such as notification as a process contributing to the update of the inclination information.

[0225] Next, the effects of the second embodiment will be described.

[0226] The CPU 212 of the management device 200 causes the NVM 214 to store the related information associating the first registration information regarding the monitoring camera main body 20, the second registration information regarding the lens device 70, and the inclination information. Therefore, even when the power supply to the management device 200 is stopped, the related information can be maintained stored in the NVM 214, and the related information stored in the NVM 214 can be used until new related information is obtained.

[0227] Also, since the related information stored in the NVM 214 can be used, it is not necessary to generate the related information every time the lens device 70 is attached to the monitoring camera main body 20.

[0228] In addition, when the first registration information included in the related information matches the first identification information of the monitoring camera main body 20 and the second registration information included in the related information matches the second identification information of the lens device 70, the CPU 212 of the management device 200 extracts the inclination information from the related information. Therefore, the inclination information included in the related information can be applied to a known combination of the monitoring camera main body 20 and the lens device 70.

[0229] In addition, when the first registration information included in the related information is different from the first identification information of the monitoring camera main body 20 acquired by the acquisition unit 242, or when the second registration information included in the related information is different from the second identification information of the lens device 70 acquired by the acquisition unit 242, the CPU 212 of the management device 200 performs control such as notification as a process contributing to the update of the inclination information. Therefore, the administrator who has received the notification can be prompted to perform an operation to update the inclination information.

[0230] Next, a modification of the second embodiment will be described.

[0231] In the second embodiment, a plurality of pieces of related information may be stored in the NVM 214 of the management device 200. Further, the CPU 212 of the management device 200 may extract related information corresponding to the combination of the target surveillance camera body 20 and the lens device 70 from the plurality of pieces of related information stored in the NVM 214 of the management device 200, and output the tilt information included in the extracted related information.

[0232] Also, in the second embodiment, related information associating the first registration information regarding the surveillance camera body 20, the second registration information regarding the lens device 70, and the tilt information is stored in the NVM 214 of the management device 200. However, the related information may be stored in the NVM 44 of the surveillance camera body 20, or may be stored in the NVM 94 of the lens device 70.

[0233] Also, in the second embodiment, the tilt information output process is executed by the CPU 212 of the management device 200. However, the tilt information output process may be executed by the CPU 42 of the surveillance camera body 20, or may be executed by the CPU 92 of the lens device 70.

[0234] Also, a plurality of modifications according to the second embodiment may be combined as appropriate. Also, a plurality of modifications of the first embodiment described above may be applied to the second embodiment.

[0235] [Third Embodiment] Next, the third embodiment will be described.

[0236] FIG. 26 shows an example of the optical characteristics of the shake correction lens 76. In FIG. 26, the shake correction lens 76 indicated by the two-dot chain line represents the shake correction lens before shifting the image, and the shake correction lens 76 indicated by the solid line represents the shake correction lens that has moved to the position where the image is shifted. The optical axis OA represents the optical axis OA that passes through the center of the light receiving surface 24A of the image sensor 24 and is perpendicular to the light receiving surface 24A.

[0237] The shift amount S1 is the shift amount of the central ray F1 passing through the shake correction lens 76 after movement on the imaging surface 24A of the image sensor 24 with respect to the movement of the shake correction lens 76 by the movement amount E, and the shift amount S2 is the shift amount of the peripheral ray F2 passing through the shake correction lens 76 after movement on the imaging surface 24A of the image sensor 24 at a position other than on the optical axis OA with respect to the movement of the shake correction lens 76 by the movement amount E. When compared with the same movement amount E, the shift amount S2 is larger than the shift amount S1. Therefore, generally, it is difficult to obtain the same shift amount over the entire image obtained by imaging light on the image sensor 24.

[0238] Therefore, in the third embodiment, the monitoring system S is configured as follows. In the third embodiment, the same elements, members, etc. as in the first embodiment are denoted by the same reference numerals as in the first embodiment, and detailed descriptions thereof are omitted.

[0239] As an example, as shown in FIG. 27, the CPU 212 of the management device 200 outputs image height position designation information and tilt information. The image height position designation information is information for designating the image height position on the light-receiving surface 24A (see FIG. 26) of the image sensor 24. The image height position is the position of the height of the image obtained by imaging light on the image sensor 24, and the height of the image is the distance from the optical axis OA to the center of the image. That is, the image height position designation information is information for designating the image height position indicating the height position on the light-receiving surface 24A of the center of the image obtained by imaging light on the image sensor 24. The position of the center of the image with respect to the optical axis OA is designated by the image height position designation information. For example, when object detection processing is performed by the management device 200, the position where the main subject appears in the captured image is specified, and the specified position is set as the image height position (that is, the reference image position). Note that the image height position may be designated by an administrator who manages the management device 200.

[0240] The tilt information is as described in the first embodiment. The image height position designation information and the tilt information are transmitted from the communication I / F 220 of the management device 200 and received by the communication I / F 100 of the lens device 70.

[0241] The CPU 42 of the surveillance camera body 20 outputs image shift amount designation information. The image shift amount designation information is information for designating the shift amount for shifting the image. The image shift amount designation information is the same information as the image shift command in the first embodiment. The movement amount of the center of the image accompanying the movement of the shake correction lens 76 is designated by the image shift amount designation information. The image shift amount designation information is transmitted from the communication I / F 60 of the surveillance camera body 20 and received by the communication I / F 100 of the lens device 70.

[0242] The conversion coefficient is stored in the NVM 94 of the lens device 70. The conversion coefficient is a coefficient for converting the shift amount for shifting the image into the movement amount of the shake correction lens 76. The conversion coefficient is represented by a value obtained by dividing the movement amount [mm] of the correction lens by the shift amount [p (pitch)] for shifting the image. The conversion coefficient is determined in advance according to the optical characteristic values (i.e., design values) of the shake correction lens 76. Further, the conversion coefficient is determined according to the image height position. The conversion coefficient is stored in the NVM in, for example, a look-up table format. Note that the conversion coefficient may be stored in the NVM in a format other than the look-up table format.

[0243] The CPU 92 of the lens device 70 operates as an acquisition unit 172, a calculation unit 174, and a control unit 176, similar to the first embodiment. The acquisition unit 172 acquires the image height position designation information, the tilt information, and the image shift amount designation information received by the communication I / F 100 of the lens device 70. Note that, for example, when the object detection function of the management device 200 is stopped or the image height position is not designated by the administrator who manages the management device 200, and thus the management device 200 does not output the image height position designation information, the CPU 92 of the lens device 70 may use a previously designated position as the image height position.

[0244] The calculation unit 174 determines, based on the image height position specification information and the image shift amount specification information, the shift amount of the image at the image height position specified by the image height position specification information to the shift amount specified by the image shift amount specification information. Further, the calculation unit 174 calculates, using the conversion coefficient, the movement amount of the blur correction lens 76 at which the shift amount of the image at the determined image height position is obtained.

[0245] For example, when the shift amount specified by the image shift amount specification information is x [p (pitch)], the image height position specified by the image height position specification information is r [mm], the conversion coefficient at the image height r is β, and the movement amount of the blur correction lens 76 corresponding to the shift amount specified by the image shift amount specification information is y, the movement amount y [mm] of the blur correction lens 76 is obtained by the following formula (11). y = x × β ··· (11)

[0246] Further, the calculation unit 174 calculates, based on the movement amount and tilt information of the blur correction lens 76 calculated using the conversion coefficient, the movement amount along the X2 axis of the blur correction lens 76 and the movement amount along the Y2 axis of the blur correction lens 76, respectively. The method of calculating, based on the movement amount and tilt information of the blur correction lens 76 calculated using the conversion coefficient, the movement amount along the X2 axis of the blur correction lens 76 and the movement amount along the Y2 axis of the blur correction lens 76, respectively, is the same as that of the first embodiment.

[0247] The control unit 176 performs control to move the blur correction lens 76 in the direction of shifting the image along the X1 axis and / or the Y1 axis on the X-axis motor 136A and the Y-axis motor 136B, in the same manner as in the first embodiment. By the above processing, the shift amount of the image at the image height position specified by the image height position specification information is made the shift amount specified by the image shift amount specification information.

[0248] In this way, the CPU 92 of the lens device 70 acquires image height position specifying information for specifying the image height position indicating the height position of the image on the image sensor 24, image shift amount specifying information for specifying the shift amount for shifting the image, and tilt information. Then, based on the image height position specifying information, the image shift amount specifying information, and the tilt information, control is performed on the X-axis motor 136A and the Y-axis motor 136B to move the moving lens by the movement amount at which the shift amount specified by the image shift amount specifying information is obtained at the image height position specified by the image height position specifying information. Therefore, the shift amount of the image at the image height position specified by the image height position specifying information can be made the shift amount specified by the image shift amount specifying information.

[0249] Also, in this way, the image is shifted every frame period, and every time the image is shifted, control is performed to cause the CPU 42 of the monitoring camera main body 20 to perform imaging on the image sensor 24 (see FIGS. 16 and 17). As a result, as shown in FIG. 18 as an example, a plurality of frame images 182 corresponding to each of the frame periods are obtained. Then, the plurality of frame images 182 are synthesized by the CPU 42 of the monitoring camera main body 20 to obtain a synthesized image 184.

[0250] Also, in the third embodiment, since the shift amount of the image at the image height position specified by the image height position specifying information can be made the shift amount specified by the image shift amount specifying information, a synthesized image 184 with the best image quality at the image height position specified by the image height position specifying information can be obtained.

[0251] Also, in the third embodiment as well, similar to the first embodiment, by being based on the inclination information, even when there is an inclination of the X2 axis of the lens device 70 with respect to the X1 axis of the image sensor 24, the blur correction lens 76 can be moved along the X1 axis of the image sensor 24 by the X-axis motor 136A and the Y-axis motor 136B. Similarly, even when there is an inclination of the Y2 axis of the lens device 70 with respect to the Y1 axis of the image sensor 24, the blur correction lens 76 can be moved along the Y1 axis of the image sensor 24 by the X-axis motor 136A and the Y-axis motor 136B.

[0252] Note that the processing of the calculation unit 174 in the third embodiment may be performed by the CPU 212 of the management device 200, or may be performed by the CPU 42 of the surveillance camera main body 20.

[0253] Also, in the third embodiment, a plurality of modification examples of the above-described first embodiment may be applied, and in the third embodiment, a plurality of modification examples of the above-described second embodiment may be applied. Further, the first embodiment, the second embodiment, and the third embodiment may be implemented in appropriate combination.

[0254] Next, modification examples common to the first embodiment, the second embodiment, and the third embodiment (hereinafter referred to as the above embodiments) will be described.

[0255] In the above embodiments, a controller 90 different from the controller 40 of the surveillance camera main body 20 is provided in the lens device 70, but the lens device 70 may not be provided with the controller 90. Further, the functions of the controller 90 of the lens device 70 may be aggregated in the controller 40 of the surveillance camera main body 20, and the lens device 70 may be controlled by the controller 40 of the surveillance camera main body 20. In this case, the controller 40 is an example of "a computer applied to the imaging device". Also, when the functions of the controller 90 of the lens device 70 are aggregated in the controller 40 of the surveillance camera main body 20, the communication I / F 100 of the lens device 70 may be omitted.

[0256] In the above-described embodiment, an example has been given in which imaging processing is executed by the controller 40 of the monitoring camera body 20. However, the technology of the present disclosure is not limited to this. For example, the imaging processing may be executed by the controller 210 of the management device 200. Further, the management device 200 and the monitoring camera body 20 may execute the imaging processing in a distributed manner, or a plurality of devices including the management device 200 and the monitoring camera body 20 may execute the imaging processing in a distributed manner.

[0257] In the above-described embodiment, the monitoring camera 10 has been described as an example of the imaging device. However, the technology of the present disclosure is not limited to this, and the technology shown in the above-described embodiment is applicable to various imaging devices. Examples of the imaging device here include, for example, a digital camera that is interchangeable-lens type and does not use a reflex mirror, a digital camera with a fixed lens, a digital camera that uses a reflex mirror, a smart device, a wearable terminal, a cell observation device, an ophthalmic observation device, or a digital camera incorporated in various electronic devices such as a surgical microscope. Further, the technology shown in the above-described embodiment may be applied to an imaging device including an image sensor 24 that is sensitive to light having a wavelength band other than the wavelength band of near-infrared light.

[0258] In the above-described embodiment, an example has been given in which the inclination information output processing program 230 is stored in the NVM 214 of the management device 200. However, the inclination information output processing program 230 may be stored in a portable storage medium such as an SSD or a USB memory, and the inclination information output processing program 230 only needs to be stored in a non-transitory storage medium. The inclination information output processing program 230 stored in the non-transitory storage medium is installed and used in, for example, the management device 200 or the like.

[0259] In the above embodiment, an example has been described in which the blur correction and shift processing program 110 is stored in the NVM 94 of the lens device 70. However, the blur correction and shift processing program 110 may be stored in a portable storage medium such as an SSD or a USB memory, and the blur correction and shift processing program 110 only needs to be stored in a non-transitory storage medium. The blur correction and shift processing program 110 stored in the non-transitory storage medium is installed and used in, for example, the lens device 70 or the like.

[0260] In the above embodiment, an example has been shown in which the controller 40 is built into the monitoring camera body 20. However, the technology of the present disclosure is not limited to this, and for example, the controller 40 may be provided outside the monitoring camera body 20.

[0261] In the above embodiment, an example has been shown in which the controller 90 is built into the lens device 70. However, the technology of the present disclosure is not limited to this, and for example, the controller 90 may be provided outside the lens device 70.

[0262] In the above embodiment, the CPU 42 of the monitoring camera body 20 is a single CPU, but it may also be a plurality of CPUs. Further, a GPU may be applied instead of the CPU 42. Similarly, the CPU 92 of the lens device 70 is a single CPU, but it may also be a plurality of CPUs. Further, a GPU may be applied instead of the CPU 92.

[0263] In the above embodiment, the monitoring camera body 20 includes the controller 40. However, the technology of the present disclosure is not limited to this, and instead of the controller 40, a device including an ASIC, an FPGA, and / or a PLD may be applied. Further, instead of the controller 40, a combination of a hardware configuration and a software configuration may be used.

[0264] In the above-described embodiment, the lens device 70 includes the controller 90. However, the technology of the present disclosure is not limited to this, and instead of the controller 90, a device including an ASIC, an FPGA, and / or a PLD may be applied. Further, instead of the controller 90, a combination of a hardware configuration and a software configuration may be used.

[0265] As the hardware resources for executing the tilt information output process described in the above embodiment, various types of processors shown below can be used. Examples of the processor include, for example, a general-purpose processor such as a CPU that functions as a hardware resource for executing the tilt information output process by executing software, that is, a program. Further, examples of the processor include a dedicated electric circuit that is a processor having a circuit configuration designed specifically for executing a specific process such as an FPGA, a PLD, or an ASIC. A memory is built in or connected to any of the processors, and any of the processors executes the tilt information output process by using the memory.

[0266] The hardware resources for executing the tilt information output process may be configured by one of these various types of processors, or may be configured by a combination of two or more processors of the same type or different types (for example, a combination of a plurality of FPGAs, or a combination of a CPU and an FPGA). Further, the hardware resources for executing the tilt information output process may be a single processor.

[0267] As an example of configuring with a single processor, first, there is a form in which one processor is configured by a combination of one or more CPUs and software, and this processor functions as a hardware resource for executing the tilt information output process. Second, there is a form in which a processor that realizes the functions of the entire system including a plurality of hardware resources for executing the tilt information output process with one IC chip, as represented by an SoC, is used. Thus, the tilt information output process is realized as a hardware resource by using one or more of the above various types of processors.

[0268] Furthermore, as the hardware structure of these various processors, more specifically, an electric circuit combining circuit elements such as semiconductor elements can be used. Also, the above-described slope information output process is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be changed within the scope not departing from the gist.

[0269] The description content and illustrated content shown above are detailed descriptions of the part related to the technology of the present disclosure and are merely examples of the technology of the present disclosure. For example, the descriptions regarding the above-described configuration, function, action, and effect are descriptions regarding an example of the configuration, function, action, and effect of the part related to the technology of the present disclosure. Therefore, it goes without saying that within the scope not departing from the gist of the technology of the present disclosure, the description content and illustrated content shown above may be modified by deleting unnecessary parts, adding new elements, or making replacements. Also, in order to avoid complication and facilitate understanding of the part related to the technology of the present disclosure, descriptions regarding common technical knowledge and the like that do not particularly require explanation for implementing the technology of the present disclosure are omitted from the description content and illustrated content shown above.

[0270] In this specification, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0271] All documents, patent applications, and technical standards described in this specification are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually stated to be incorporated by reference.

Description of Reference Numerals

[0272] S Monitoring system 10 Monitoring camera 20 Surveillance camera body 22 Lens mount 24 Image sensor 24A Light-receiving surface 25A Subject image 26 Light-receiving section 28 Color filter section 30 First light-receiving element 32 Second light-receiving element 32R Light-receiving element 32G Light-receiving element 32B Light-receiving element 40 Controller 42 CPU 44 NVM 46 RAM 48 Bus 50 UI device 52 Image sensor driver 54 Signal processing device 56 Shake amount detection sensor 60 Communication I / F 62 Visible light image 64 Near-infrared light image 70 Lens device 72 Objective lens 74 Zoom lens 76 Shake correction lens 76A Center 78 Diaphragm 78A Aperture 80 Filter unit 82 Master lens 84 Disk 86 Ir cut filter 90 Controller 92 CPU 94 NVM 96 RAM 98 Bus 100 Communication I / F 110 Shake correction and shift processing program 114 Motor driver 116A X-axis motor driver 116B Y-axis motor driver 118 Second motor driver 120 Third Motor Driver 122 Fourth Motor Driver 134 First Motor 136A X-axis Motor 136B Y-axis Motor 138 Second Motor 140 Third Motor 142 Fourth Motor 154 First Position Sensor 156A X-axis Position Sensor 156B Y-axis Position Sensor 158 Second Position Sensor 160 Third Position Sensor 162 Fourth Position Sensor 172 Acquisition Unit 174 Calculation Unit 176 Control Unit 182 Image 184 Composite Image 200 Management Device 210 Controller 212 CPU 214 NVM 216 RAM 218 Bus 220 Communication I / F 222 Display 224 Keyboard 226 Mouse 228 Input / Output I / F 230 Tilt Information Output Processing Program 232 Tilt Information Generation Unit 234 Tilt Information Output Unit 240 Tilt Information Output Processing Program 242 Acquisition Unit 244 Judgment Unit 246 Extraction Unit 248 Output Unit 250 Notification Control Unit 262 First Image 264 Second Image 266 Third Image θx Tilt Angle θy Tilt Angle A1 Movement Amount Ax movement amount Ay movement amount B1 movement amount Bx movement amount By movement amount C1 movement amount Cx movement amount Cy movement amount D1 movement amount Dx movement amount Dy movement amount E movement amount F1 central ray F2 peripheral ray OA optical axis S1 shift amount S2 shift amount

Claims

1. An information processing apparatus comprising a processor, wherein the processor acquires inclination information regarding at least one of an inclination of a first drive axis of a moving lens with respect to a first axis of an image sensor as viewed along an optical axis, and an inclination of a second drive axis of the moving lens with respect to a second axis of the image sensor as viewed along the optical axis, and calculates a movement amount for moving the moving lens along at least one of the first axis and the second axis based on the inclination information.

2. The inclination information is information calculated based on a control command for moving the moving lens along at least one of the first drive axis and the second drive axis, and a plurality of images obtained by imaging by the image sensor before and after the moving lens moves based on the control command. The information processing apparatus according to claim 1.

3. The inclination information is information calculated based on a plurality of images obtained by imaging by the image sensor under imaging conditions in which an image with less noise than an image obtained by normal imaging is obtained. The information processing apparatus according to claim 1.

4. The inclination information is information calculated based on a plurality of images obtained by imaging by the image sensor to which a sensitivity lower than the sensitivity of the image sensor for normal imaging is applied. The information processing apparatus according to claim 1.

5. The inclination information is information regarding at least one of a first inclination angle of the first drive axis with respect to the first axis as viewed along the optical axis, and a second inclination angle of the second drive axis with respect to the second axis as viewed along the optical axis. The information processing apparatus according to claim 1.

6. The processor calculates a first movement amount for moving the moving lens along the first drive axis and a second movement amount for moving the moving lens along the second drive axis based on the inclination information. The information processing apparatus according to claim 5.

7. The inclination information includes a first movement amount for moving the moving lens along the first drive axis and a second movement amount for moving the moving lens along the second drive axis, wherein the first movement amount and the second movement amount are movement amounts calculated based on at least one of an inclination of the first drive axis with respect to the first axis as viewed along the optical axis, and an inclination of the second drive axis with respect to the second axis as viewed along the optical axis. ​ ​ The information processing apparatus according to claim 1.

8. Obtaining tilt information regarding at least one of the tilt of the first drive axis of the moving lens with respect to the first axis of the image sensor as viewed along the optical axis, and the tilt of the second drive axis of the moving lens with respect to the second axis of the image sensor as viewed along the optical axis, and Calculating a movement amount for moving the moving lens along at least one of the first axis and the second axis based on the tilt information An information processing method including the above.

9. Obtaining tilt information regarding at least one of the tilt of the first drive axis of the moving lens with respect to the first axis of the image sensor as viewed along the optical axis, and the tilt of the second drive axis of the moving lens with respect to the second axis of the image sensor as viewed along the optical axis, and Calculating a movement amount for moving the moving lens along at least one of the first axis and the second axis based on the tilt information A program for causing execution of a process including the above.

Images