Imaging Apparatus, Control Method Thereof, and Storage Medium

Abstract

The present invention provides an imaging device, a control method thereof, and a storage medium. In order to provide an imaging device capable of simply correcting unnecessary rotation of an image caused by rotation of a movable unit, the imaging device has the following configuration: an imaging unit configured to acquire an image; a movable unit configured to change a direction of the imaging unit by rotation of the movable unit about a predetermined axis; and at least one processor or circuit configured to serve as a correction processing unit, the correction processing unit being configured to perform correction processing for correcting rotation of the image caused by rotation of the movable unit based on information regarding a relationship between a rotation angle about the axis and a rotation angle of the image acquired by the imaging unit.

Description

Technical Field

[0001] The present invention relates to an imaging device and the like, and particularly to an imaging device capable of changing the direction of an imaging unit. Background Art

[0002] In an imaging device including a plurality of imaging units, a multi-lens machine is known in which if a rotation axis is rotated, the plurality of imaging units move or rotate simultaneously in a group. In this multi-lens machine, each imaging unit is arranged, for example, on a spherical surface and has a distorted positional relationship with respect to the rotation axis. Therefore, when the rotation axis rotates, the image has a rotational component around the optical axis.

[0003] The distorted positional relationship mentioned here means that the direction of the vertical line passing through the center point of the rotating sphere on the rotation axis is not parallel to the optical axis direction of the imaging unit at the point on the spherical surface, but is inclined with respect to the optical axis direction.

[0004] The rotational component of the image of the imaging unit rotating around the optical axis varies depending on the relative positional relationship between the position where the imaging unit is attached and the optical axis and the rotation axis of the imaging unit. Considering the structure of the rotation mechanism, this rotational component decreases as the position moves away from the end of the rotation axis and increases as the position approaches the end of the rotation axis. Therefore, the images obtained from the respective imaging units have rotations with different rotation angles.

[0005] Japanese Patent Application Laid-Open No. 2008-204384 discloses a method for solving the problem of providing a tilting pan / tilt axis when the imaging device has a tilted mounting position.

[0006] However, the prior art described in Japanese Patent Application Laid-Open No. 2008-204384 is for correcting the tilt in the image caused by the tilt of the mounting position, rather than for correcting the image obtained by the imaging unit arranged in a distorted positional relationship with respect to the rotation axis.

[0007] In addition, when using the technique described in Japanese Patent Application Laid-Open No. 2008-204384 to correct the tilt of the pan / tilt axis, it is necessary to calculate the parameters caused by the installation environment and the parameters based on the pan / tilt angles. Therefore, the processing becomes complicated.

[0008] Therefore, an object of the present invention is to provide an imaging device in which unnecessary rotation of an image caused by rotation of a movable unit can be simply corrected. Summary of the Invention

[0009] To achieve this object, an imaging device according to an aspect of the present invention includes: an imaging unit configured to acquire an image; a movable unit configured to change the direction of the imaging unit by rotation of the movable unit about a predetermined axis; and at least one processor or circuit configured to serve as a correction processing unit, the correction processing unit being configured to perform correction processing for correcting rotation of an image caused by rotation of the movable unit based on information related to a relationship between a rotation angle about the axis and a rotation angle of the image acquired by the imaging unit.

[0010] A control method for an imaging device, the imaging device having: an imaging unit configured to acquire an image; a movable unit configured to change the direction of the imaging unit by rotation of the movable unit about a predetermined axis, wherein the control method includes: correcting rotation of an image caused by rotation of the movable unit based on information related to a relationship between a rotation angle about the axis and a rotation angle of the image acquired by the imaging unit.

[0011] A non-transitory computer-readable storage medium configured to store a computer program for controlling an imaging device, the imaging device having: an imaging unit configured to acquire an image; a movable unit configured to change the direction of the imaging unit by rotation of the movable unit about a predetermined axis, and wherein the computer program includes instructions for performing the following processing: correcting rotation of an image caused by rotation of the movable unit based on information related to a relationship between a rotation angle about the axis and a rotation angle of the image acquired by the imaging unit.

[0012] Other features of the present invention will become clear from the following description of embodiments with reference to the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a diagram showing the configuration of the imaging device according to Embodiment 1.

[0014] Figure 2A is a front view of an example of the imaging device in the present invention, and Figure 2B is a diagram showing an example in which the imaging device in the present invention is rotated downward by 90 degrees.

[0015] Figure 3 is a diagram showing an image of a captured image.

[0016] Figure 4A is a flowchart for describing the operation flow in Embodiment 1, and Figure 4B is a diagram for explaining the result of the process.

[0017] Figure 5AIt is an explanatory diagram of the rotation correction angle, and Figure 5B is a diagram showing an example of a correction angle table.

[0018] Figure 6 is a diagram showing the configuration of the imaging device in Embodiment 2.

[0019] Figure 7 is a flowchart for describing the operation flow in Embodiment 2.

[0020] Figure 8 is a flowchart for describing the operation flow in Embodiment 3.

[0021] Figure 9 is a diagram for explaining a display example in Embodiment 4.

[0022] Figure 10 is a diagram showing the configuration of the imaging device in Embodiment 5. Detailed Description of the Invention

[0023] Hereinafter, preferred modes of the present invention will be described using embodiments with reference to the accompanying drawings. In each figure, the same components or constituent elements are denoted by the same reference numerals, and repeated descriptions will be omitted or simplified.

[0024] In addition, in the examples, an example in which a network camera is applied as an imaging device will be described. Here, the imaging device includes a digital still camera, a digital video camera, a smartphone having a camera, a tablet computer having a camera, an electronic instrument having an imaging function (such as a vehicle-mounted camera), and the like.

[0025] [Embodiment 1]

[0026] Hereinafter, Figure 1 Embodiment 1 of the present invention will be described. Figure 1 is a diagram showing the configuration of the imaging device in Embodiment 1.

[0027] The imaging device 101 includes a movable unit 102, an imaging unit 103, a processing unit 104, etc. The display unit 105 serves as a display unit for displaying an image. Here, the display unit 105 can be an external monitor or the like, or can be a display unit of a terminal of a PC, and is connected to the imaging device via a network.

[0028] The processing unit 104 can also be included in a terminal such as an external PC connected to the imaging device via a network. That is, the imaging device in this example is not limited to an integrated system, but includes a system configured as a separate main body. The display unit 105 can be formed integrally with the imaging device.

[0029] When the display unit 105 is connected to a terminal such as a PC, the shooting control and pan / tilt control, etc. of the imaging device can be remotely controlled based on a control signal from the PC. As Figure 2A shown, the movable unit 102 is configured to rotatably hold the imaging unit 103 in the tilt direction about a predetermined axis 202. When the imaging device 101 is installed, the movable unit 102 can be manually rotated, for example, in the pan direction about the axis 204 (as Figure 2A shown).

[0030] The imaging unit 103 includes a lens barrel that includes a lens and an image sensor to acquire a video signal (image). The imaging unit 103 is supported by a spherical housing.

[0031] The processing unit 104 has a built-in CPU as a computer and serves as a control unit that executes various operations of the entire imaging device based on a computer program stored in a program memory (not shown).

[0032] The processing unit 104 includes an angle storage unit 106, an angle correction value storage unit 107, an image acquisition unit 108, an image rotation correction processing unit 109, and a detection function processing unit 110.

[0033] The angle storage unit 106 stores the angle value (pan angle) at the initial setting of the movable unit 102. Here, the angle storage unit 106 is constituted by, for example, SRAM, DRAM, PROM, or EEPROM, etc., but is not limited thereto. The angle information stored in the angle storage unit 106 is sent to the angle correction value storage unit 107. On the other hand, the image acquired in the imaging unit 103 is transmitted to the image acquisition unit 108.

[0034] The image acquired in the image acquisition unit 108 is output to the display unit 105. The interface to the display unit 105 can be wired or wireless and is used to transmit the image using, for example, IP allocation, HDMI (registered trademark), or SDI, etc. via a communication unit (not shown). The image sent to the display unit is also sent to the image rotation correction processing unit 109.

[0035] In addition, in the image rotation correction processing unit 109, the image is rotationally corrected using the angle correction information transmitted by the angle correction value storage unit 107. Using the detection function processing unit 110, various detection processes are performed on the image that has been rotationally corrected.

[0036] The detection process has various types of detection functions such as processing based on images and sounds, such as person recognition, person counting, intrusion detection, removal detection, and scream detection, etc. In addition, the detection result is superimposed and displayed on the display unit 105.

[0037] Hereinafter, the structure of the imaging device 101 will be described with reference to Figure 2A and Figure 2B the imaging device 101 will be described with reference to FIGS. Figure 2A and Figure 2B FIGS. are diagrams showing examples of the imaging device in the present invention. In addition, Figure 2A is a front view of an example of the imaging device of the present invention, and Figure 2B is a diagram showing an example in which the imaging device of the present invention is rotated downward by 90 degrees.

[0038] Although Figure 2A and Figure 2B show an example in which the upper part of the imaging device 101 is fixedly mounted on the XY plane of the ceiling, the upper part of the imaging device 101 may be mounted facing downward or may be mounted on the XZ plane or the YZ plane.

[0039] In addition, the imaging unit 103 can rotate in the pitch direction with the movable axis 202 as the pitch rotation axis, and can rotate in the pan direction (as shown by the arrow 203) with the movable axis 204 as the pan rotation axis.

[0040] The imaging units 103a to 103d are accommodated in a spherical housing, and the movable unit 102 pivotally supports the spherical housing rotatably at both ends in the radial direction of the spherical housing, and can perform rotation in the pitch direction with the movable axis 202 as the rotation axis. In addition, by rotating the movable unit 102 around the movable axis 204, the imaging units 103a to 103d can be rotated in the pan direction.

[0041] First, an example of manually performing these rotations will be described, but as will be described later, these rotations can be rotated electrically by using a motor or the like. The movable axes 202 and 204 themselves may not rotate. That is, any imaging unit can be used as long as the imaging unit rotates around the axes of the movable axes 202 and 204 and the movable axes also include such axes.

[0042] In this example, the optical axes of the imaging units 103a to 103d are arranged in a twisted direction (tilt direction) that is not parallel to the vertical line drawn from the center point (not shown) on the axis of the movable unit 102. The optical axes of the respective imaging units point outward from the center of the spherical housing.

[0043] Although four imaging units are shown as the imaging units 103a to 103d in the figure, the number of imaging units is not limited to this, and one or more of the imaging units can be used to obtain the effect of this example as long as a twisted positional relationship with respect to the movable axis is provided.

[0044] Here, if the optical axis of the imaging unit 103 points in a direction parallel to the direction perpendicular to the rotation axis of the movable unit 102, it is defined as non-distorted. On the other hand, it is defined as having an increasing distortion as the optical axis of the imaging unit 103 becomes closer to being parallel to the rotation axis of the movable unit 102. That is, in the case of a greater distortion, when the movable unit 102 rotates, the image acquired by the imaging unit 103 rotates more.

[0045] Figure 2B The following state is shown, in which, using the movable unit 102 of the imaging device 101, from Figure 2A this state, the spherical housing is rotated about the movable axis 202 in the direction of arrow 201 by approximately 90 degrees. Here, if the imaging unit 103a is concerned, then in Figure 2B it, Figure 2A the positive Z-axis direction (the upward direction of the image acquired by the imaging unit 103a) in this state is rotated by a predetermined angle due to the rotation of the center of the movable axis 202 and is inclined in the positive direction of the Y-axis direction.

[0046] Thus, the rotation of the imaging unit 103a about its optical axis occurs along with the movement of the movable unit 102 in the direction of arrow 201. This is because the optical axis of the imaging unit 103a is not parallel to the line perpendicular to the movable axis 202, and they have a distorted positional relationship. As a result, if the image acquired by the imaging unit 103a is displayed on the display unit, then in Figure 2B this case, in Figure 2A the upright screen includes an image rotated, for example, by approximately 45 degrees to 90 degrees to the right.

[0047] Figure 3 Examples of the display images of the imaging unit 103a before and after the rotation of the movable unit 102 are shown. Figure 3 It is an image diagram of a captured image.

[0048] Figure 3 A in Figure 2A shows the captured image of the imaging unit 103a before the movable unit 102 rotates about the movable axis 202 in the direction of arrow 201 ( Figure 3 this state). In addition, Figure 2B B in

[0049] shows the captured image of the imaging unit 103a after the movable unit 102 rotates about the movable axis 202 in the direction of arrow 201 ( Figure 2B this state). As the movable axis 202 rotates, the viewing angle (rotation) of the person image that was upright with respect to the XY plane before the rotation changes.

[0049] That is, a change in perspective that is a mixture of rotation about the optical axis centered on the Y axis (in the direction of arrow 301) and displacement in the X direction (in the direction of arrow 302) and the Z direction (in the direction of arrow 303).

[0050] As a result of the rotation about the Y axis, a displayed image of a person's tilt (such as the image in A to the image in B) is obtained. Figure 3 to Figure 3 in B) is obtained.

[0051] On the other hand, when using a detection function such as person detection using image recognition or the like, a method for recognizing a human body by detecting the contour shape of a person can be used. In this case, a person photographed at a certain angle may not be recognized as a person, and there is a possibility of false detection. In this example, when unnecessary rotation occurs in the image of the imaging unit in this way, for example, due to the pitching operation of the movable unit 102, before performing the detection process using image recognition or the like, a correction process for correcting the rotation component by internal processing is performed. Therefore, the false detection rate is reduced.

[0052] Figure 4A and Figure 4B are diagrams for explaining the control in this example, and the control flow in this example will be described with reference to the flowchart of Figure 4A .

[0053] In step S401, the process starts. In addition, in step S402, when the imaging device is installed, the user makes an initial setting of the pan / tilt angle of the lens barrel disposed in the imaging unit. The angle at the time of the initial setting is detected when the imaging device is started up.

[0054] In addition, in step S403, based on the set angle and the distance from the rotation axis, referring to the rotation correction table (rotation correction information), the images captured by the respective imaging units are rotated and corrected by internal processing. Regarding this, instead of using a table, the rotation correction information can be obtained by mathematical calculation or the like.

[0055] The rotation correction table is stored in the angle correction value storage unit 107, and in the rotation correction table, the relationship between the rotation angle about the drive axis and the rotation angle of the image of the imaging unit based on the layout relationship between the imaging unit and the drive axis is shown.

[0056] Here, step S403 and the image rotation correction processing unit 109 function as a correction processing unit configured to perform a correction process to correct the rotation of the image caused by the movable unit based on the table.

[0057] Figure 4B shows an example of the result of the correction process in step S403.

[0058] Figure 4B The image shown in B of Figure 3 (corresponding to the image of B in

[0059] ) has an unnecessary rotation caused by the pitch operation and the person is not in an upright position. The correction process in step S403 corrects such an image by performing a rotation correction so that the person in the image has the upright direction of the person. Figure 4B In step S403, the rotated image shown in B of

[0060] is corrected to an upright image as shown in C. After that, in step S404, the detection function processing unit 110 performs image recognition processing such as face recognition and person detection using the image that has been rotation-corrected. Here, step S404 serves as a detection unit for performing a predetermined detection operation based on the image. Figure 5A and Figure 5B to describe a method for calculating the rotation correction angle.

[0061] Figure 5A is an explanatory diagram of the rotation correction angle, and Figure 5B is a diagram showing an example of a correction angle table.

[0062] Here, for simplicity, it is assumed that the lens barrel of the imaging device 101 is regarded as a sphere 501, the position where the imaging unit is attached is point 502, and the rotation axis for performing pitch rotation is axis 503.

[0063] In addition, a in the figure is the radius of the sphere, and r is the distance from the intersection point 505 of the perpendicular line drawn from point 502 to axis 503 to the center 506 of the sphere.

[0064] In addition, point 502 represents the position of the imaging unit where the imaging unit faces a direction parallel to the XY plane in FIG. 2, the axis 503 from the pitch horizontal position ( Figure 2A in the state) is used as the rotation axis, and the rotation angle is θ. In addition, it is assumed that the optical axis of the imaging unit points in a direction, for example, from the center 506 outward through point 502.

[0065] At this time, the trajectory along which the imaging unit can move corresponds to the solid line part of the circle 504, and the circle 504 corresponds to the intersection line between the plane passing through point 505 and perpendicular to axis 503 and the sphere 501. When performing a pitch rotation by an angle θ from the pitch horizontal position ( Figure 2A in the state), the component in the rotation direction around the optical axis of the imaging unit can be expressed by the following Expression 1:

[0066] [Expression 1]

[0067]

[0068] Based on this, it can be seen that when the rotation angle θ of the pitch axis increases, and when the position of the attached imaging unit becomes closer to the pitch rotation axis 503 (when r increases), the rotational component gradually becomes dominant.

[0069] Figure 5B An example of the angle correction table 504 included in the angle correction value storage unit 107 is shown. Here, the angle correction table 504 is a table showing the relationship between the rotation angle around the movable axis and the rotation angle of the image of the imaging unit based on the arrangement relationship between the imaging unit and the movable axis.

[0070] In this example, the imaging device has four imaging units, but the number of imaging units is not limited to this. In addition, it is assumed that the angle correction table includes correction tables with details corresponding to multiple imaging units. Here, a table is shown when the imaging units are arranged symmetrically left and right as shown in FIG. 2.

[0071] That is, the imaging units A to D are arranged on the intersection line between the predetermined plane including the axis 503 and the sphere 501, and in this predetermined plane, the imaging units A and B and the imaging units D and C have a line-symmetrical positional relationship with respect to the line perpendicular to the axis 503 passing through the center 506. However, the arrangement of the imaging units is not limited to this, and an asymmetric case is also possible.

[0072] In the table, the positions of the respective imaging units arranged above the imaging device are uniquely determined, as well as the correction angles corresponding to the pitch angle θ manually initially set by the installer, for example. The position of the imaging unit mentioned here is the position of the attached Figure 2A and Figure 2B imaging units 103a to 103d shown, and it varies depending on the imaging device. Here, for simplicity, although the pitch angle setting value is shown in one-degree increments, it is assumed that a finer resolution can also be set.

[0073] Optionally, if high image recognition ability (detection ability) is provided, the pitch angle setting value can be set at a coarser angle. In addition, when the characteristics of the image recognition ability (detection ability) change non-linearly with respect to the rotation angle of the image, the pitch angle spacing can be unevenly set according to the non-linear characteristic curve. That is, the setting values in this table are set according to the characteristics of the image recognition ability (detection ability).

[0074] The rotation correction is performed on each acquired image using the angle of the correction angle shown in this table, the detection function processing is performed using image recognition, etc., and then it is displayed on the display unit 105.

[0075] In as Figure 2A andFigure 2B In the imaging device shown above, as described above, the imaging units are arranged in a left-right symmetric manner. In such a left-right symmetric case, rotation correction can be performed by providing only a correction table for the imaging units on one side with respect to the axis of symmetry.

[0076] That is, in Figure 2A and Figure 2B a pair of imaging units 103a and imaging unit 103d have a symmetric positional relationship, and a pair of imaging units 103b and imaging unit 103c have a symmetric positional relationship. Therefore, the angle correction table included in the angle correction value storage unit 107 can be merely a table for the imaging units on one side with respect to the axis of symmetry. That is, a table for one imaging unit in a pair of imaging units can be provided, and the table for the other imaging unit in the pair of imaging units can be omitted.

[0077] In the image obtained by the imaging unit on the opposite side of the axis of symmetry, correction can be performed if correction is made by rotating in the reverse direction by the angle of the rotation correction value of the imaging unit at the symmetric position. When the imaging units are arranged in a left-right symmetric manner in this way, rotation correction can be effectively performed using the capacity of the angle correction value storage unit 107.

[0078] [Embodiment 2]

[0079] Embodiment 2 of the present invention will be described below.

[0080] Below, reference will be made to Figure 6 to describe Embodiment 2. Figure 6 is a diagram showing the configuration of the imaging device in Embodiment 2. Although the angle storage unit 106 and the angle correction value storage unit 107 in the imaging device shown in Figure 1 are used to determine the correction angle, in this example, movement is performed electrically. Therefore, the method for determining the correction angle is different from that of Embodiment 1.

[0081] In Figure 6 the imaging unit 103 is driven for panning / tilting, etc. by the drive of the drive unit 601. Here, the drive unit 601 serves as a drive unit for rotating the movable unit around the movable axis. Although the drive unit includes motors such as a DC motor, a stepping motor, and an ultrasonic motor, and uses a gear mechanism having gears or belts to rotate the drive shaft, the drive method is not limited thereto. The drive unit 601 performs driving according to a drive instruction sent from the drive angle calculation unit 603 via the drive instruction unit 602.

[0082] The drive instruction unit 602 includes devices such as a motor driver, and the drive angle calculation unit 603 calculates the drive angle using a microcomputer, an IC capable of performing calculation processing (such as an FPGA and an ASIC, etc.). In addition, the drive angle calculation unit 603 calculates the drive angle according to the operation amount of the operation unit 604 using the imaging device 101.

[0083] The drive angle calculated in the drive angle calculation unit 603 is transmitted to the angle correction value storage unit 107, the rotation correction angle corresponding to the drive angle is extracted from the table, and correction is performed using the image rotation correction processing unit 109.

[0084] In Figure 6 , the drive angle is directly output from the drive angle calculation unit 603 to the angle correction value storage unit 107. However, when it is difficult to control the drive angle, for example, when driving using a DC motor, etc., it is desirable to separately use a sensing sensor or an encoder (such as an acceleration sensor or a gyro sensor, etc.) to obtain the rotation angle.

[0085] This example also includes the control when using a sensor for such angle sensing. In this case, the angle value sensed by the sensor for angle sensing installed in the drive unit 601 is transmitted to the angle correction value storage unit 107. In Figure 6 , the other components shown are the same as the components described in the first embodiment, and their description will be omitted.

[0086] Next, Figure 7 the control sequence of this example will be described. Figure 7 is a flowchart for describing the operation process in the second embodiment.

[0087] In step S701, the operation process starts, and in step S702, if a command for rotating the rotation axis is sent from the drive instruction unit 602 to the drive unit 601 during shooting, the drive shaft rotates.

[0088] In addition, after moving to a predetermined position, if the rotation stops in step S703, the movement amount (rotation angle) from before the drive is calculated in step S704. The process of step S704 is an example of open control, and the control sequence is different when using a separate sensor for control and when performing feedback control.

[0089] In step S705, the distance r from the position of the drive shaft at the attached imaging unit and the correction angle corresponding to the angle θ of the drive shaft driven to rotate are extracted from the table. In step S706, using the extracted correction angle, rotation correction is performed through internal processing. That is, correction processing is performed by obtaining the rotation angle of the image from the storage unit according to the rotation angle of the drive unit. Further, in step S707, image processing such as image recognition using each processing function is performed using the image that has been rotation-corrected.

[0090] When no drive instruction is provided, the process proceeds from step S702 to step S707, and image processing such as detection processing is performed at any time.

[0091] Although in the second embodiment, a sequence in which rotation correction is not performed during rotation is provided, when it is also necessary to sense the image being driven, for example, when driving at a low speed, etc., calculation for rotation correction can also be performed during driving.

[0092] In this example, it is necessary to consider that the image may rotate not only due to the pitch angle but also due to a change in the pan angle. When the pitch points to the horizontal direction, the image is laterally shifted as the pan rotates. However, when the pitch angle with respect to the horizontal axis increases, the image includes a motion that includes a rotation component due to the pan rotation performed. Therefore, when the drive shaft is moved electrically, a table considering the arrangement mode of the drive shaft (a combination of the two angles of the pan angle and the pitch angle) is held in the angle correction value storage unit.

[0093] [Embodiment 3]

[0094] Embodiment 3 of the present invention will be described below. Figure 8 is a flowchart for describing the operation process of Embodiment 3.

[0095] Embodiment 3 shows the control in an imaging device having various types of detection / sensing functions. Examples of the sensing functions include various functions such as a function for detecting a human body, a function for detecting an intruder, and a function for detecting the removal of an object, etc.

[0096] Although there are various detection methods for these functions, when the object image includes movement in the rotation direction, in some cases, it may cause false detection or may not cause false detection. For example, in intrusion detection, etc., where an alarm is issued based on a judgment on whether an intrusion into a specific area has occurred, in some cases, when entering the area and when exiting the area, it may be recognized conversely due to the rotation of the image, and intrusion detection, etc., may cause false detection.

[0097] On the other hand, for carry-out detection that sounds an alarm when the placed item is carried out, etc., a method that uses the time difference of images for detection can be used. In this type of detection where the direction of the image does not affect recognition, the influence of rotation is small. Depending on the type of detection function used in this way, there are cases where rotation correction is required and cases where rotation correction is not required. Based on this, the control sequence will be described.

[0098] In step S801, the operation process starts, and in step S802, the pan / tilt angle is set to the default at the time of installation. In addition, in step S803, the detection function to be used is selected. In step S804, a determination is made as to whether the selected detection function is of a type that requires rotation correction. If the determination result is yes (it is determined that correction is required), then as in Embodiment 1, control is performed in the order of step S805 and step S806. Rotation correction is performed.

[0099] In step S804, when it is determined that rotation correction is not required in the selected detection function, the process proceeds to the process of step S806 without performing correction, and the process of the detection function is performed. That is, step S804 and step S805 serve as a correction processing unit that is configured to switch whether to perform correction in the correction processing unit according to the type of detection function in the detection unit.

[0100] [Embodiment 4]

[0101] Embodiment 4 of the present invention will be described below. Figure 9 It is a diagram for explaining a display example on the display unit 105 in Embodiment 4.

[0102] The displayed images A to D are images captured by the imaging units 103a to 103d. Although the case where the images of the four imaging units are shown as an example is shown, the number of images on the displayed video is not limited to this.

[0103] The images obtained from the imaging units 103a to 103d, the rotation correction function operation window 901, and the rotation display function operation window 902 are provided on the display unit 105.

[0104] The rotation correction function operation window 901 is an operation window for switching whether to independently use the rotation correction function in each imaging unit. When switching this operation window, a switch is made as to whether to perform image rotation correction on the images obtained by each imaging unit using internal processing before processing the detection function.

[0105] This enables the display of whether to perform correction processing on the display screen of the display unit.

[0106] In addition, the rotation display function operation window 902 is an operation window for switching whether to perform rotation correction on the image displayed on the display unit 105. Similar to the rotation correction function operation window 901, this operation window can also be independently set and operated for each image from the imaging unit.

[0107] Therefore, the image rotation correction processing unit 109 can switch whether to correct each image obtained from each imaging unit.

[0108] When using this rotation display function operation window 902 to switch the ON (on) / OFF (off) of rotation correction, it is possible to switch between the image before rotation correction and the image after rotation correction displayed on the display unit 105.

[0109] In addition, although the rotation correction function operation window 901 and the rotation display function operation window 902 in the figure show a method for switching between ON and OFF in a pull-down manner, the operation method is not limited to this. Additionally, the rotation correction function operation window 901 and the rotation display function operation window 902 act independently and do not necessarily need to be displayed in a group.

[0110] [Embodiment 5]

[0111] Embodiment 5 of the present invention will be described below. Embodiments 1 to 4 include captured images that have undergone rotation in the processing unit 104 (which is described as the rotation correction function) of the imaging device 101. The difference between Embodiments 1 to 4 and Embodiment 5 is that in Embodiment 5, instead of rotating the image, the detection direction is adjusted in the detection function processing to match the rotation angle of the imaging unit.

[0112] The following will refer to Figure 10 Provide a description. Figure 10 is a diagram showing the configuration of the imaging device in Embodiment 5. Here, the constituent elements different from those in Embodiment 1 in Embodiment 5 will be described. The rotation correction angle extracted in the angle correction value storage unit 107 is transmitted to the detection function correction processing unit 1001.

[0113] The detection function correction processing unit 1001 performs a detection function that takes into account that the image received from the image acquisition unit 108 has an angle correction from the angle correction value storage unit 107. That is, the detection function correction processing unit 1001 serves as a correction processing unit configured to change the detection direction of image recognition according to the angle correction when performing detection processing such as image recognition.

[0114] Although the image rotation correction processing unit 109 is provided in the first embodiment, the fifth embodiment is characterized in that the image does not undergo the process of rotating the image, but undergoes a process that takes rotation into account during the detection function processing stage. In this example, when such a process is performed, it is possible to prevent the loss of the peripheral portion of the image by rotating and processing the image, and the captured image can be effectively utilized.

[0115] Therefore, even when a suspicious person appears in the peripheral portion of the image, an effect capable of reliable sensing can be obtained.

[0116] In the above description, each unit may include discrete electronic circuits, or a part or the whole thereof may be constituted by an FPGA or a CPU or the like.

[0117] [Embodiment 6]

[0118] In the above embodiments, the images obtained by the imaging units 103a to 103d are respectively processed, and these images can be synthesized to form, for example, a panoramic image. In Embodiment 6, the process for synthesizing the panoramic image will be described.

[0119] Since each of the images obtained by the respective imaging units among the imaging units 103a to 103d is rotated at its own angle due to distortion, if these images are synthesized without correcting each rotation, the synthesized image will be distorted.

[0120] In Embodiment 6, the rotation of each of the images obtained by the respective imaging units is corrected by the image rotation correction processing unit 109 and then synthesized. More specifically, the image rotation correction processing unit 109 corrects each of the images obtained by the respective imaging units based on the rotation correction table. Since each of the imaging units has a different amount of distortion, the tables for each of the imaging units are different, and the correction amounts for each of the images are different.

[0121] After correcting each of the images obtained from the imaging units 103a to 103d, these images are synthesized to generate a panoramic image to be displayed on the display unit 105.

[0122] Although the present invention has been described with reference to the exemplary embodiments, it should be understood that the present invention is not limited to the disclosed exemplary embodiments. The scope of the appended claims should be given the broadest interpretation so as to cover all such variations and equivalent structures and functions.

[0123] A computer program configured to implement the functions of the above-described embodiments by using part or all of the controls in this example can be supplied to the image processing apparatus via a network or various storage media. Further, a computer (or a CPU, an MPU, etc.) in the image processing apparatus can read and execute the program. In this case, the program and the storage medium configured to store the program constitute the present invention.

[0124] This application claims priority to Japanese Patent Application No. 2020-023732, filed on Feb. 14, 2020, and Japanese Patent Application No. 2021-020926, filed on Feb. 12, 2021, the entire contents of which are incorporated herein by reference.

Claims

1. An imaging device, which comprises: a first imaging unit configured to acquire a first image; a second imaging unit configured to acquire a second image; a movable mount configured to rotatably hold the first imaging unit and the second imaging unit about a predetermined axis, wherein optical axes of the first imaging unit and the second imaging unit are not parallel to a vertical line perpendicular to the predetermined axis, and wherein both the first imaging unit and the second imaging unit rotate according to rotation of the movable mount when the movable mount rotates about the predetermined axis; and a correction processing unit configured to perform a first correction processing and a second correction processing, the first correction processing being for correcting rotation of the first image caused by rotation of the movable mount based on information related to a relationship between a rotation angle of the movable mount about the predetermined axis and a rotation angle of the first image acquired by the first imaging unit, the second correction processing being for correcting rotation of the second image caused by rotation of the movable mount based on information related to a relationship between a rotation angle of the movable mount about the predetermined axis and a rotation angle of the second image acquired by the second imaging unit, wherein, in the information, at least for a given rotation angle of the movable mount, the rotation angle of the first image is different from the rotation angle of the second image.

2. The imaging device according to claim 1, wherein, the imaging device further comprises a third imaging unit configured to acquire a third image, wherein the first imaging unit and the third imaging unit are arranged line-symmetrically with respect to the vertical line, and wherein, in the information, the rotation angle of the first image acquired by the first imaging unit is the same as the rotation angle of the third image acquired by the third imaging unit.

3. The imaging device according to claim 1, further comprises: a detection unit configured to perform a predetermined detection operation on an image.

4. The imaging device according to claim 3, wherein, before the detection unit performs the predetermined detection operation on the image, the correction processing unit performs the correction processing on the image.

5. The imaging device according to claim 3, wherein, the correction processing unit performs a correction processing for correcting a detection direction in the detection unit.

6. The imaging device according to claim 3, wherein, the correction processing unit determines whether to perform correction in the correction processing unit according to a type of a detection function in the detection unit.

7. The imaging device according to claim 1, comprises: a plurality of imaging units including the first imaging unit and the second imaging unit, wherein the correction processing unit is capable of performing switching on whether to correct each image acquired from each imaging unit among the plurality of imaging units.

8. The imaging device according to claim 1, wherein, the correction processing unit synthesizes a plurality of corrected images to generate a synthesized image.

9. The imaging device according to claim 1, further comprises: A drive unit including a motor that rotates the first imaging unit and the second imaging unit about the predetermined axis. The correction processing unit obtains the rotation angles of the first image and the second image based on the information according to the rotation angle of the drive unit about the predetermined axis, and performs the first correction processing and the second correction processing.

10. The imaging device according to claim 1, wherein the relationship between the rotation angle of the movable mount about the predetermined axis and the rotation angle of the first image obtained by the first imaging unit is determined based on the position of the first imaging unit, and the relationship between the rotation angle of the movable mount about the predetermined axis and the rotation angle of the second image obtained by the second imaging unit is determined based on the position of the second imaging unit.

11. A control method for an imaging device, the imaging device having: a first imaging unit configured to obtain a first image; a second imaging unit configured to obtain a second image; and a movable mount configured to rotatably hold the first imaging unit and the second imaging unit about a predetermined axis, wherein the optical axes of the first imaging unit and the second imaging unit are not parallel to a vertical line perpendicular to the predetermined axis, and wherein, when the movable mount rotates about the predetermined axis, both the first imaging unit and the second imaging unit rotate according to the rotation of the movable mount. The control method includes: correcting the rotation of the first image caused by the rotation of the movable mount based on information related to the relationship between the rotation angle of the movable mount about the predetermined axis and the rotation angle of the first image obtained by the first imaging unit; and correcting the rotation of the second image caused by the rotation of the movable mount based on information related to the relationship between the rotation angle of the movable mount about the predetermined axis and the rotation angle of the second image obtained by the second imaging unit. wherein, in the information, at least for a given rotation angle of the movable mount, the rotation angle of the first image is different from the rotation angle of the second image.

12. A non-transitory computer-readable storage medium storing a computer program for controlling an imaging device, the imaging device having: a first imaging unit configured to obtain a first image; a second imaging unit configured to obtain a second image; and a movable mount configured to rotatably hold the first imaging unit and the second imaging unit about a predetermined axis. wherein the optical axes of the first imaging unit and the second imaging unit are not parallel to a vertical line perpendicular to the predetermined axis, and wherein, when the movable mount rotates about the predetermined axis, both the first imaging unit and the second imaging unit rotate according to the rotation of the movable mount. The computer program, when executed by a processor, implements the following processing: correcting rotation of the first image caused by rotation of the movable mount based on information related to a relationship between a rotation angle of the movable mount about the predetermined axis and a rotation angle of the first image acquired by the first imaging unit; and correcting rotation of the second image caused by rotation of the movable mount based on information related to a relationship between a rotation angle of the movable mount about the predetermined axis and a rotation angle of the second image acquired by the second imaging unit, wherein, in the information, for at least a given rotation angle of the movable mount, the rotation angle of the first image is different from the rotation angle of the second image.

13. A computer program product comprising a computer program for controlling an imaging device having: a first imaging unit configured to acquire a first image; a second imaging unit configured to acquire a second image; and a movable mount configured to rotatably hold the first imaging unit and the second imaging unit about a predetermined axis, wherein optical axes of the first imaging unit and the second imaging unit are not parallel to a vertical line perpendicular to the predetermined axis, and wherein, when the movable mount rotates about the predetermined axis, both the first imaging unit and the second imaging unit rotate according to the rotation of the movable mount, wherein the computer program, when executed by a processor, implements the following processes: correcting rotation of the first image caused by rotation of the movable mount based on information related to a relationship between a rotation angle of the movable mount about the predetermined axis and a rotation angle of the first image acquired by the first imaging unit; and correcting rotation of the second image caused by rotation of the movable mount based on information related to a relationship between a rotation angle of the movable mount about the predetermined axis and a rotation angle of the second image acquired by the second imaging unit, wherein, in the information, for at least a given rotation angle of the movable mount, the rotation angle of the first image is different from the rotation angle of the second image.

Images