Image pickup system, operation method of image pickup system, and program product

Abstract

The invention relates to an imaging system, an operation method of the imaging system, and a program product. An imaging system includes a processor, a first imaging device having a first optical system, and a second imaging device having a second optical system. The focal length of the first optical system is longer than the focal length of the second optical system. The processor performs: a process for aligning a first captured image captured by the first imaging device and a second captured image captured by the second imaging device at at least two locations including an end portion of a field angle used in imaging by the imaging system, the first captured image being captured by the first imaging device, and the second captured image being captured by the second imaging device; and acquiring field angle related information related to the field angle according to the alignment result.

Description

[0001] This invention is a divisional application of the following application, the original application information of which is as follows: Application date: January 17, 2023 Application Number: 202180060291.1 Invention Title: Camera System, Operating Method of Camera System, and Storage Medium Technical Field

[0002] The present invention relates to a camera system, a method for operating the camera system, and a program product. Background Technology

[0003] Japanese Patent No. 5610867 discloses a camera device comprising a first camera optical system and a second camera optical system having a different field of view than the first camera optical system. Both the first and second camera optical systems have focusing lenses for focusing. The camera device described in Japanese Patent No. 5610867 includes: a subject distance calculation member that calculates the distance to the main subject determined as the object of the image by means of the focusing action of the focusing lenses of the first and second camera optical systems; and a control member that drives the focusing lenses to control focusing based on the distance to the main subject calculated by the subject distance calculation member.

[0004] In the imaging device described in Japanese Patent Application No. 5610867, when the main subject is within the field of view of the first imaging optical system, the control member drives the focusing lenses of both the first and second imaging optical systems to focus on the main subject based on the distance to the main subject calculated by the subject distance calculation member through the focusing action of the first imaging optical system. Furthermore, when the main subject is outside the field of view of the first imaging optical system but within a determination area corresponding to the field of view of the first imaging optical system within the field of view of the second imaging optical system, the control member drives the focusing lenses of both the first and second imaging optical systems to focus on the main subject based on the distance to the main subject calculated by the subject distance calculation member through the focusing action of the second imaging optical system. Additionally, when the main subject is outside the field of view of the first imaging optical system and outside the determination area, the control member provides notification control regarding this situation.

[0005] Japanese Patent Publication No. 2007-081473 discloses a camera device having multiple optical systems. These optical systems include a first optical system with a relatively wide field of view, a second optical system with a relatively narrow field of view arranged separately from the first optical system, and a display member for displaying an image obtained by the first optical system. The camera device with multiple optical systems described in Japanese Patent Publication No. 2007-081473 includes: a first operation member for inputting a desired field of view; a display control member for cutting out a photographic area from an image displayed on the display member, calculated based on the desired field of view input from the first operation member and the distance between the optical axes of the first and second optical systems, taking into account parallax; a second operation member for inputting a photographic instruction; and a storage member for storing the image obtained by the second optical system corresponding to the photographic area as a photographic image, based on the photographic instruction input from the second operation member. Summary of the Invention

[0006] One embodiment of the present invention provides a camera system, a method for operating the camera system, and a program product that can obtain information related to the desired field of view when using a first camera device and a second camera device with different focal lengths in conjunction with the optical systems.

[0007] means for solving technical problems The first aspect of the present invention is a camera system comprising: a processor; a first camera device having a first optical system; and a second camera device having a second optical system, wherein the focal length of the first optical system is longer than the focal length of the second optical system. The processor performs the following processing: aligning a first camera image obtained by the first camera device with a second camera image obtained by the second camera device at at least two locations including the end of the field of view used in the shooting based on the camera system; and obtaining field of view related information based on the alignment result.

[0008] The second aspect of the present invention is the camera system involved in the first aspect, wherein the processor obtains field of view related information based on the end alignment result. The end alignment result is obtained by aligning the first end of the field of view of the first camera device or the center of the field of view of the first camera device with the second end of the end side of the field of view of the second camera device.

[0009] The third aspect of the present invention is the camera system involved in the second aspect, wherein the processor obtains field-of-view related information based on the field-of-view center alignment result, and the field-of-view center alignment result is obtained by aligning the center of the field of view of the first camera device with the center of the field of view of the second camera device.

[0010] The fourth aspect of the present invention is the camera system involved in the third aspect, wherein the processor derives the field of view center alignment result and the end alignment result based on the position-specific information that determines the positional relationship between the first camera device and the second camera device.

[0011] The fifth aspect of the present invention is the camera system involved in the fourth aspect, wherein the position-specific information includes information on the offset between the first camera device and the second camera device.

[0012] The sixth aspect of the present invention is the camera system involved in the fourth or fifth aspect, wherein the position-specific information includes information related to the height difference between the first camera device and the second camera device.

[0013] The seventh aspect of the present invention is a camera system involved in any one of the fourth to sixth aspects, wherein, when the distance to the first subject is less than a first threshold, the processor performs field-angle center alignment processing to align the center of the field of view of the first camera device with the center of the field of view of the second camera device using position-specific information; and when the distance to the second subject is less than a second threshold, the processor performs end alignment processing to align the center of the field of view of the first end or the first camera device with the second end using position-specific information.

[0014] The eighth method of the present invention is the camera system involved in the seventh method, wherein the processor derives at least one of the first subject distance and the second subject distance.

[0015] The ninth aspect of the present invention is the camera system of the eighth aspect, wherein the first camera device has an autofocus function and is rotatably mounted on a turntable, and the processor performs the following processing: when the distance of the subject is derived from at least one of the distance of the first subject and the distance of the second subject, the subject distance is derived using the autofocus function; when the deriving of the subject distance using the autofocus function fails, the turntable is controlled according to the first camera image, and after aligning the center of the field of view of the first camera device with a high-contrast subject near the center of the field of view of the first camera device, the subject distance is derived using the autofocus function.

[0016] The tenth aspect of the present invention is the camera system involved in the eighth or ninth aspect, wherein the processor, in a state of entering the region of the longest focal length side of the first optical system among multiple regions in which the focal length of the first optical system is divided in stages, causes the first camera device to focus, and derives at least one of the distance of the first subject and the distance of the second subject based on the focusing result.

[0017] The eleventh aspect of the present invention is the camera system involved in the tenth aspect, wherein the processor enables the first camera device to focus when the focal length of the first optical system is at its longest, and derives at least one of the distance of the first subject and the distance of the second subject based on the focusing result.

[0018] The 12th aspect of the present invention is a camera system involved in any of the 7th to 11th aspects, wherein the processor performs end alignment processing after performing field-of-view center alignment processing.

[0019] The 13th aspect of the present invention is a camera system involved in any of the 7th to 12th aspects, wherein the processor performs at least one of field-of-view center alignment processing and end alignment processing by image analysis based on the first camera image and the second camera image.

[0020] The 14th aspect of the present invention is the camera system involved in the 13th aspect, wherein, in the end alignment processing, the processor performs alignment of the center of the field of view of the first end or the first camera device with the second end based on the scale ratio of the first camera image and the second camera image used in the image analysis in the field of view center alignment processing.

[0021] The 15th aspect of the present invention is a camera system according to any one of the 7th to 14th aspects, wherein, regarding the processor, an opposite end alignment process is further performed, which aligns the first opposite end located on the opposite side of the first end or the center of the field of view of the first camera device with the second opposite end located on the opposite side of the second end of the field of view of the second camera device; and obtains the field of view of the second camera device as field of view related information based on at least one of a first result in which the position of the first end or the center of the field of view of the first camera device is aligned with the position of the second end in the end alignment process, and a second result in which the position of the first opposite end or the center of the field of view of the first camera device is aligned with the position of the second opposite end in the opposite end alignment process.

[0022] The 16th aspect of the present invention is the camera system involved in the 15th aspect, wherein, when the distance to the third subject is greater than or equal to the third threshold, and when the position of the center of the field of view of the first opposite end or the first camera device is aligned with the position of the second opposite end, the processor obtains the field of view of the second camera device as field of view related information based on the second result.

[0023] The 17th aspect of the present invention is a camera system involved in any of the 1st to 16th aspects, wherein the optical information related to the first optical system is known, and the optical information related to the second optical system is unknown.

[0024] The 18th aspect of the present invention is a camera system involved in any one of the 1st to 17th aspects, wherein the focal length of the first optical system is more than twice the focal length of the second optical system.

[0025] The 19th aspect of the present invention is a camera system comprising: a processor; and a first camera device, which is used in conjunction with a second camera device having a second optical system and has the first optical system, wherein the focal length of the first optical system is longer than the focal length of the second optical system. The processor performs the following processing: aligning a first camera image obtained by the first camera device with a second camera image obtained by the second camera device at at least two locations including the end of the field of view used in the shooting based on the camera system; and obtaining field of view related information based on the alignment result.

[0026] The 20th aspect of the present invention is an operation method of a camera system. The camera system includes: a processor; a first camera device having a first optical system; and a second camera device having a second optical system, wherein the focal length of the first optical system is longer than the focal length of the second optical system. The operation method of the camera system includes the following steps: aligning a first camera image obtained by the first camera device with a second camera image obtained by the second camera device at at least two locations including the end of the field of view used in the shooting of the camera system; and obtaining field of view related information based on the alignment result.

[0027] The 21st aspect of the present invention is a program for causing a computer to perform processing. The computer includes a processor, a first imaging device having a first optical system, and a second imaging device having a second optical system. The program is applicable to an imaging system in which the focal length of the first optical system is longer than that of the second optical system. The processing includes the following steps: aligning a first image obtained by the first imaging device with a second image obtained by the second imaging device at at least two locations including the end of the field of view used in the shooting based on the imaging system; and obtaining field of view related information based on the alignment result. Attached Figure Description

[0028] Figure 1 This is a schematic perspective view showing an example of the external structure of a camera system.

[0029] Figure 2This is a conceptual diagram illustrating an example of the structure of a camera system, including a telephoto camera, a wide-angle camera, and a dual-axis rotary table.

[0030] Figure 3 This is a conceptual diagram illustrating an example of the hardware structure of the electrical system of a camera system, which includes a management device, a telephoto camera, a wide-angle camera, and a dual-axis rotary table.

[0031] Figure 4 This is a conceptual diagram illustrating an example of the hardware structure of the electrical systems of a camera system, including a telephoto camera and a wide-angle camera.

[0032] Figure 5 This is a block diagram illustrating an example of how the CPU performs bit-alignment processing according to a bit-alignment procedure in a management device.

[0033] Figure 6 This is a function block diagram representing an example of the functions of the CPU in a management device.

[0034] Figure 7 This is a conceptual diagram illustrating an example of processing content where images obtained from telephoto and wide-angle cameras, capturing the area of ​​the subject, are displayed on a monitor.

[0035] Figure 8 This is a conceptual diagram illustrating an example of the processing performed by the field-of-view center alignment unit on the wide-angle camera and the processing performed by the control unit.

[0036] Figure 9 This is a conceptual diagram illustrating an example of the processing performed by the field-of-view center alignment unit on the telephoto camera and the processing performed by the control unit.

[0037] Figure 10 This is a conceptual diagram used to illustrate the method for deriving the distance to the first subject.

[0038] Figure 11 This is a conceptual diagram used to illustrate the first rotation angle.

[0039] Figure 12 This is a conceptual diagram used to illustrate the calculation method for the first rotation angle.

[0040] Figure 13 This is a conceptual diagram illustrating an example of the processing performed by the end-center alignment unit on the wide-angle camera and the processing performed by the control unit.

[0041] Figure 14 This is a conceptual diagram illustrating an example of the processing performed by the end-center alignment unit on the telephoto camera and the processing performed by the control unit.

[0042] Figure 15This is a conceptual diagram used to illustrate the method for deriving the distance to the second subject.

[0043] Figure 16 It is a conceptual diagram used to illustrate the first rotation angle and half angle of view.

[0044] Figure 17 This is a conceptual diagram used to illustrate the calculation method for half-angle perspective.

[0045] Figure 18 This is a conceptual diagram illustrating an example of the processing content of the acquisition and control departments.

[0046] Figure 19 This is a flowchart illustrating an example of a bitwise operation process.

[0047] Figure 20 This is a flowchart illustrating an example of the process for aligning the center of the field of view.

[0048] Figure 21 This is a flowchart illustrating an example of the end-alignment process.

[0049] Figure 22A This is a flowchart of the first variation of the end alignment process.

[0050] Figure 22B yes Figure 22A Continuing from the flowchart shown.

[0051] Figure 23 This is a conceptual diagram representing a variation of the processing content of the acquisition department.

[0052] Figure 24 This is a flowchart of the first variation of the process for aligning the center of the field of view.

[0053] Figure 25 This is a flowchart of the second variation of the end alignment process.

[0054] Figure 26 This is a flowchart of the second variation of the process for aligning the center of the field of view.

[0055] Figure 27 This is a flowchart of the third variation of the process for aligning the center of the field of view.

[0056] Figure 28 This is a conceptual diagram illustrating an example of a method for aligning a telephoto camera and a wide-angle camera, taking into account their elevation differences.

[0057] Figure 29 This is a conceptual diagram illustrating a variation of the method for calculating half-angle.

[0058] Figure 30This is a conceptual diagram illustrating an example of how a bit-handling program is installed from a storage medium containing the bit-handling program into a computer that manages the device. Detailed Implementation

[0059] Hereinafter, an example of an implementation of the camera system, camera system operation method and program product involved in the present invention will be described with reference to the accompanying drawings.

[0060] First, let me explain the vocabulary used in the following instructions.

[0061] CPU stands for Central Processing Unit. RAM stands for Random Access Memory. DRAM stands for Dynamic Random Access Memory. SRAM stands for Static Random Access Memory. NVM stands for Non-Volatile Memory. SSD stands for Solid State Drive. HDD stands for Hard Disk Drive. EEPROM stands for Electrically Erasable and Programmable Read Only Memory. USB stands for Universal Serial Bus. ASIC stands for Application Specific Integrated Circuit. PLD stands for Programmable Logic Device. FPGA stands for Field-Programmable Gate Array. SoC stands for "System-on-a-chip". CMOS stands for "Complementary Metal Oxide Semiconductor". CCD stands for "Charge Coupled Device". EL stands for "Electro-Luminescence". I / F stands for "Interface". fps stands for "frames per second". TOF stands for "Time-of-Flight". AF stands for "AutoFocus". Hereinafter, for ease of explanation, a CPU is illustrated as an example of a "processor" according to the present invention, but the "processor" according to the present invention can also be a combination of multiple processing devices such as a CPU and a GPU. As an example of a "processor" according to the present invention, in the case of a combination of a CPU and a GPU, the GPU operates under the control of the CPU and undertakes the execution of image processing.

[0062] Furthermore, in this specification, "orthogonal" means, in addition to being perfectly orthogonal, orthogonal in the sense of errors generally permissible in the technical field to which this invention pertains and permissible without departing from the scope of this invention. Also, in this specification, "perpendicular" means, in addition to being perfectly perpendicular, perpendicular in the sense of errors generally permissible in the technical field to which this invention pertains and permissible without departing from the scope of this invention. Furthermore, in this specification, "consistent" means, in addition to being perfectly consistent, consistent in the sense of errors generally permissible in the technical field to which this invention pertains and permissible without departing from the scope of this invention. And, in the following description, the numerical range indicated by "~" refers to the range including the values ​​described before and after "~" as both the lower and upper limits.

[0063] As an example, such as Figure 1 As shown, the camera system 10 includes a telephoto camera 12, a wide-angle camera 14, a dual-axis rotary table 16, a dual-axis rotary table 18, and a management device 20. Here, the telephoto camera 12 is an example of the "first camera device" according to the present invention, the wide-angle camera 14 is an example of the "second camera device" according to the present invention, and the dual-axis rotary table 16 is an example of the "rotary table" according to the present invention.

[0064] The management device 20 manages the telephoto camera 12, the wide-angle camera 14, the dual-axis rotary table 16, and the dual-axis rotary table 18. Hereinafter, for ease of explanation, the telephoto camera 12, the wide-angle camera 14, the dual-axis rotary table 16, and the dual-axis rotary table 18 will not be marked with symbols and will be referred to as "managed object devices" without distinction.

[0065] The management device 20 includes a management device main body 22, a receiving device 24, and a display 26. The management device main body 22 includes a computer 104 (reference). Figure 3 The management unit 22 controls the entire camera system 10. A receiver 24 and a display 26 are connected to the management unit 22. The receiver 24 receives various instructions from the user of the camera system 10. Examples of receiver 24 include a keyboard, mouse, and / or touch panel. The various instructions received by the receiver 24 are controlled by the management unit 22. The display 26 displays various information (e.g., images and text) under the control of the management unit 22. Examples of display 26 include a liquid crystal display (LCD) or an EL display.

[0066] The managed object device can be communicatively connected to the management device main body 22 and operates under the control of the management device main body 22. The connection between the managed object device and the management device main body 22 can be either wired or wireless.

[0067] The telephoto camera 12, also known as a long-range camera, is a digital camera with a longer focal length than the wide-angle camera 14. The telephoto camera 12 is rotatably mounted on a dual-axis rotary table 16. The management device 20 controls the panning or tilting of the telephoto camera 12 by operating the dual-axis rotary table 16. The dual-axis rotary table 16 has a panning axis PA1 and a tilting axis TA1. Under the control of the management device 20, panning of the telephoto camera 12 is achieved by rotating it around the panning axis PA1, and tilting of the telephoto camera 12 is achieved by rotating it around the tilting axis TA1.

[0068] Wide-angle camera 14 can capture a wider field of view than telephoto camera 12 (see reference). Figure 7 A digital camera. A wide-angle camera 14 is rotatably mounted on a dual-axis rotary table 18. The management device 20 operates the dual-axis rotary table 18 to pan or tilt the wide-angle camera 14. That is, the dual-axis rotary table 18 has a pan axis PA2 and a tilt axis TA2. Under the control of the management device 20, panning of the wide-angle camera 14 is achieved by rotating it around the pan axis PA2, and tilting of the wide-angle camera 14 is achieved by rotating it around the tilt axis TA2.

[0069] The main body 22 of the management device acquires image data obtained by the telephoto camera 12, i.e., telephoto camera image data, and acquires image data obtained by the wide-angle camera 14, i.e., wide-angle camera image data.

[0070] As an example, such as Figure 2 As shown, the telephoto camera 12 includes a lens barrel 28 and a telephoto camera body 30. A lens barrel side mount 32 is provided on the lens barrel 28, and a body side mount 34 is provided on the telephoto camera body 30. The lens barrel 28 is mounted to the telephoto camera body 30 via the lens barrel side mount 32 and the body side mount 34. A camera optical system 38 is built into the lens barrel 28, and an image sensor 36 is built into the telephoto camera body 30.

[0071] The imaging optical system 38 is an example of the "first optical system" involved in the present invention. The imaging optical system 38 images light representing a subject onto an image sensor 36. The imaging optical system 38 has various lenses. Specifically, the imaging optical system 38 has an objective lens 40, a focusing lens 42, a zoom lens 44, and a main lens 46. Although in Figure 2 The example is simplified, but each of the objective lens 40, focusing lens 42, zoom lens 44 and main lens 46 is actually a lens group that combines multiple lenses.

[0072] Objective lens 40, focusing lens 42, zoom lens 44, and main lens 46 are arranged sequentially from the object side (subject side) to the image side (image sensor 36 side). The imaging optical system 38 also has an aperture 48, which is positioned between the zoom lens 44 and the main lens 46.

[0073] The lens barrel 28 is equipped with a focusing lens drive mechanism 50, a zoom lens drive mechanism 52, and an aperture drive mechanism 54. The focusing lens drive mechanism 50 includes a focusing cam ring (not shown) with a cam groove formed on its outer periphery and a focusing motor (not shown). The focusing lens drive mechanism 50 holds the focusing lens 42. The focusing cam ring rotates around the optical axis OA1 of the imaging optical system 38 by receiving power from the focusing motor. The focusing lens 42 moves along the optical axis OA1 by receiving a propulsive force in the direction of the optical axis OA1 generated by the rotation of the focusing cam ring around the optical axis OA1.

[0074] The zoom lens drive mechanism 52 includes a zoom cam ring (not shown) with cam grooves formed on its outer periphery and a zoom motor (not shown). The zoom lens drive mechanism 52 holds the zoom lens 44. The zoom cam ring rotates around the optical axis OA1 by receiving power from the zoom motor. The zoom lens 44 moves along the optical axis OA1 by receiving a propulsive force in the direction of the optical axis OA1 generated by the rotation of the zoom cam ring around the optical axis OA1.

[0075] The aperture drive mechanism 54 includes an aperture motor (not shown). The aperture 48 has multiple blades that can be opened and closed. The multiple blades are opened and closed by receiving power from the aperture motor.

[0076] A lens barrel-side contact 56 is provided on the lens barrel-side bayonet 32. The aforementioned stepper motor and other electrical components are assembled on each of the focusing lens drive mechanism 50, the zoom lens drive mechanism 52, and the aperture drive mechanism 54. Each electrical component of the focusing lens drive mechanism 50, the zoom lens drive mechanism 52, and the aperture drive mechanism 54 is connected to the lens barrel-side contact 56.

[0077] A main body side contact 58 is provided on the main body side bayonet 34. In the main body side bayonet 34, the main body side contact 58 is located at a position corresponding to the position of the lens barrel side contact 56 on the lens barrel side bayonet 32.

[0078] A control device 60 is built into the telephoto camera body 30 and is connected to the body-side contact 58. When the lens barrel 28 is mounted to the telephoto camera body 30 via the lens barrel-side bayonet 32 ​​and the body-side bayonet 34, the lens barrel-side contact 56 contacts the body-side contact 58. Thus, the electrical components of the focusing lens drive mechanism 50, the zoom lens drive mechanism 52, and the aperture drive mechanism 54 are electrically connected to the control device 60.

[0079] The control device 60 controls the electrical components of the focusing lens drive mechanism 50, the zoom lens drive mechanism 52, and the aperture drive mechanism 54.

[0080] The control device 60 monitors the drive amount of the focusing motor and the zoom motor. The control device 60 derives the position of the focusing lens 42 on the optical axis OA1 from the drive amount of the focusing motor, and derives the position of the zoom lens 44 on the optical axis OA1 from the drive amount of the zoom motor.

[0081] In the telephoto camera 12, the focus position is adjusted by the focusing lens 42. Therefore, on the light-receiving surface 36A of the image sensor 36, the subject light is imaged at the focus position corresponding to the distance to the subject by moving the focusing lens 42 along the optical axis OA1. Here, "focus position" refers to the position of the focusing lens 42 on the optical axis OA1 in the focusing state. The focus position is derived from the drive quantity of the focusing motor by the control device 60. The control device 60 controls the focusing lens 42 to be aligned to the focus position by operating the focusing lens drive mechanism 50. Furthermore, for ease of explanation, the control of aligning the focusing lens 42 to the focus position will also be referred to as "AF control" below.

[0082] The image sensor 36 is configured such that the center of the light-receiving surface 36A coincides with the optical axis OA1, and the light-receiving surface 36A is orthogonal to the optical axis OA1. Furthermore, as an example of the image sensor 36, a CMOS image sensor or a CCD image sensor can be cited.

[0083] Image sensor 36 is driven under the control of control device 60. Image sensor 36 captures images of the subject at a specified frame rate (e.g., 30 fps) according to a timing signal supplied from control device 60. Control device 60 acquires image data obtained by image sensor 36 of the subject captured by image sensor 36, i.e., telephoto camera image data.

[0084] In addition, Figure 2The example also shows an example of the structure of a wide-angle camera 14. The wide-angle camera 14 includes a lens barrel 62 and a wide-angle camera body 64. The lens barrel 62 corresponds to the lens barrel 28 of the telephoto camera 12, and the wide-angle camera body 64 corresponds to the telephoto camera body 30 of the telephoto camera 12. A lens barrel-side bayonet 66 is provided on the lens barrel 62, and a body-side bayonet 68 is provided on the wide-angle camera body 64. In the wide-angle camera 14, the lens barrel-side bayonet 66 corresponds to the lens barrel-side bayonet 32 ​​of the telephoto camera 12, and the body-side bayonet 68 corresponds to the body-side bayonet 34 of the telephoto camera 12. A camera optical system 72 is built into the lens barrel 62, and an image sensor 70 is built into the wide-angle camera body 64. In the wide-angle camera 14, the image sensor 70 corresponds to the image sensor 36 of the telephoto camera 12. A light-receiving surface 70A is formed on the image sensor 70. In the wide-angle camera 14, the light-receiving surface 70A corresponds to the light-receiving surface 36A of the telephoto camera 12.

[0085] Aside from the difference in focal length, the basic structure of the wide-angle camera 14 is the same as that of the telephoto camera 12. Here, "different focal length" means that the focal length of the camera optical system 38 of the telephoto camera 12 is longer than the focal length of the camera optical system 72 of the wide-angle camera 14. The longer focal length of the camera optical system 38 of the telephoto camera 12 compared to the focal length of the camera optical system 72 of the wide-angle camera 14 means that, between the telephoto camera 12 and the wide-angle camera 14, the relationship regarding focal length is: "Focal length of the camera optical system 38 of the telephoto camera 12 ≥ focal length of the camera optical system 72 of the wide-angle camera 14 × 2".

[0086] Furthermore, provided that the focal length of the camera optical system 38 of the telephoto camera 12 is set to be more than twice the focal length of the camera optical system 72 of the wide-angle camera 14, the actual focal length of the camera optical system 72 of the wide-angle camera 14 is preferably 6mm to 20mm, and the actual focal length of the camera optical system 38 of the telephoto camera 12 is preferably 12.5mm to 2000mm. Moreover, the focal length ratio (= (focal length of the camera optical system 38 of the telephoto camera 12) / (focal length of the camera optical system 72 of the wide-angle camera 14)) is preferably 2 to 333 times.

[0087] The camera optical system 72 is an example of the "second optical system" involved in this invention, and includes an optical axis OA2, an objective lens 74, a focusing lens 76, a main lens 80, and an aperture 82. In the wide-angle camera 14, the optical axis OA2 corresponds to the optical axis OA1 of the telephoto camera 12. In the wide-angle camera 14, the objective lens 74 corresponds to the objective lens 40 of the telephoto camera 12. In the wide-angle camera 14, the focusing lens 76 corresponds to the focusing lens 42 of the telephoto camera 12. In the wide-angle camera 14, the main lens 80 corresponds to the main lens 46 of the telephoto camera 12. In the wide-angle camera 14, the aperture 82 corresponds to the aperture 48 of the telephoto camera 12.

[0088] The lens barrel 62 is equipped with a focusing lens drive mechanism 84, a zoom lens drive mechanism 86, and an aperture drive mechanism 88. In the wide-angle camera 14, the focusing lens drive mechanism 84 corresponds to the focusing lens drive mechanism 50 of the telephoto camera 12. In the wide-angle camera 14, the zoom lens drive mechanism 86 corresponds to the zoom lens drive mechanism 52 of the telephoto camera 12. In the wide-angle camera 14, the aperture drive mechanism 88 corresponds to the aperture drive mechanism 54 of the telephoto camera 12.

[0089] A lens barrel-side contact 90 is provided on the lens barrel-side bayonet 66. A body-side contact 92 is provided on the body-side bayonet 68. In the wide-angle camera 14, the lens barrel-side contact 90 corresponds to the lens barrel-side contact 56 of the telephoto camera 12. In the wide-angle camera 14, the body-side contact 92 corresponds to the body-side contact 58 of the telephoto camera 12.

[0090] A control device 94 is built into the wide-angle camera body 64. In the wide-angle camera 14, the control device 94 corresponds to the control device 60 of the telephoto camera body 30. The control device 94 acquires image data obtained by the image sensor 70 through the image sensor 70 capturing the subject, i.e., wide-angle camera image data.

[0091] The dual-axis rotary table 16 includes a panning mechanism 96 and a tilting mechanism 98. The panning mechanism 96 includes a telephoto camera-side panning motor 96A. The telephoto camera-side panning motor 96A is located in the management device 20 (reference). Figure 1 Driven under the control of ), when a telephoto camera 12 is mounted on the dual-axis rotary table 16, the panning mechanism 96 uses the driving force of the motor 96A to pan the telephoto camera sideways around the panning axis PA1 (see reference). Figure 1 The power of the telephoto lens is transmitted to the telephoto camera 12, causing the telephoto camera 12 to pan. The tilt mechanism 98 has a telephoto camera side tilt motor 96B. The telephoto camera side tilt motor 96B is in the management device 20 (reference). Figure 1Driven under the control of ), when a telephoto camera 12 is mounted on the dual-axis rotary table 16, the pitch mechanism 98 uses the driving force of the telephoto camera's side pitch motor 96B as the force around the pitch axis TA1 (reference). Figure 1 The power of the telephoto lens is transmitted to the telephoto camera 12, causing the telephoto camera 12 to tilt.

[0092] The dual-axis rotary table 18 includes a panning mechanism 100 and a tilting mechanism 102. The panning mechanism 100 includes a wide-angle camera-side panning motor 100A. The wide-angle camera-side panning motor 100A is located in the management device 20 (reference). Figure 1 Driven under the control of ), when a wide-angle camera 14 is installed on the dual-axis rotary table 18, the panning mechanism 100 uses the driving force of the motor 100A to pan the wide-angle camera sideways around the panning axis PA2 (see reference). Figure 1 The power of the [unclear] is transmitted to the wide-angle camera 14, causing the wide-angle camera 14 to pan. The pitch mechanism 102 has a wide-angle camera side pitch motor 100B. The wide-angle camera side pitch motor 100B is in the management device 20 (reference) Figure 1 Driven under the control of ), when a wide-angle camera 14 is installed on the dual-axis rotary table 16, the pitch mechanism 102 uses the driving force of the wide-angle camera's side pitch motor 100B as the force around the pitch axis TA2 (reference). Figure 1 The power of the camera is transmitted to the wide-angle camera 14, causing the wide-angle camera 14 to tilt.

[0093] As an example, such as Figure 3 As shown, the main body 22 of the management device includes a computer 104. Furthermore, the main body 22 of the management device includes communication I / Fs 106, 108, 110A, 110B, 112A, and 112B.

[0094] Computer 104, CPU 116, NVM 118, and RAM 120. Computer 104 is an example of a "computer suitable for a camera system" according to the present invention, and CPU 116 is an example of a "processor" according to the present invention.

[0095] CPU116, NVM118, and RAM120 are connected to bus 114. Figure 3 In the example shown, for ease of illustration, a single bus 114 is depicted, but multiple buses are also possible. Bus 114 can be a serial bus or a parallel bus that includes a data bus, an address bus, and a control bus.

[0096] The NVM118 stores various parameters and programs. Examples of NVM118 include various non-volatile storage devices such as EEPROM, SSD, and / or HDD. The RAM120 temporarily stores various information and serves as working memory. Examples of RAM120 include DRAM or SRAM.

[0097] Various programs are stored in the NVM118. The CPU116 reads the required program from the NVM118 and executes the read program on the RAM120. The CPU116 controls the entire camera system 10, including the management device 20, according to the program executed on the RAM120.

[0098] A receiver 24 and a display 26 are connected to the bus 114. The CPU 116 receives instructions from the receiver 24 and displays various information on the display 26.

[0099] The telephoto camera 12 has a communication I / F 122, which is connected to the control device 60. The wide-angle camera 14 also has a communication I / F 124, which is connected to the control device 94.

[0100] The dual-axis rotary table 16 has communication I / F 126 and 128. Communication I / F 126 is connected to the panning mechanism 96, and communication I / F 128 is connected to the pitch mechanism 98. Furthermore, the dual-axis rotary table 18 also has communication I / F 130 and 132. Communication I / F 130 is connected to the panning mechanism 100, and communication I / F 132 is connected to the pitch mechanism 102.

[0101] Communication I / Fs 106, 108, 110A, 110B, 112A, and 112B are connected to bus 114. Communication I / F 106 is connected to communication I / F 122 of telephoto camera 12, and CPU 116 sends and receives information with control device 60 via communication I / Fs 106 and 122. Communication I / F 108 is connected to communication I / F 124 of wide-angle camera 14, and CPU 116 sends and receives information with control device 94 via communication I / Fs 108 and 124.

[0102] Communication I / F110A is connected to communication I / F126 of the dual-axis rotary table 16. CPU 116 controls the electrical system components of the pan / tilt mechanism 96 (e.g., a pan / tilt motor 96A for a telephoto camera) via communication I / F110A and 126. Communication I / F112A is connected to communication I / F128 of the dual-axis rotary table 16. CPU 116 controls the electrical system components of the pitch mechanism 98 (e.g., a pitch motor 96B for a telephoto camera) via communication I / F112A and 128.

[0103] Communication I / F110B is connected to communication I / F130 of the dual-axis rotary table 18. CPU 116 controls the electrical system components of the pan / tilt mechanism 100 (e.g., the pan / tilt motor 100A for a wide-angle camera) via communication I / F110B and 130. Communication I / F112B is connected to communication I / F132 of the dual-axis rotary table 18. CPU 116 controls the electrical system components of the pitch mechanism 102 (e.g., the pitch motor 100B for a wide-angle camera) via communication I / F112B and 132.

[0104] As an example, such as Figure 4 As shown, in the telephoto camera 12, the control device 60 includes a computer 134 and an image memory 136. The computer 134 includes a CPU 138, an NVM 140, and a RAM 142.

[0105] CPU138, NVM140, and RAM142 are connected to bus 144. Figure 4 In the example shown, for ease of illustration, a single bus 114 is depicted, but multiple buses are also possible. Bus 114 can be a serial bus or a parallel bus that includes a data bus, an address bus, and a control bus.

[0106] The NVM140 stores various parameters and programs. Examples of NVM140 include various non-volatile storage devices such as EEPROM, SSD, and / or HDD. The RAM142 temporarily stores various information and serves as working memory. Examples of RAM142 include DRAM or SRAM.

[0107] Various programs are stored in the NVM140. The CPU138 reads the required programs from the NVM140 and executes the read programs on the RAM142. The CPU138 controls the entire telephoto camera 12 according to the programs executed on the RAM142.

[0108] An image sensor 36 and an image memory 136 are connected to bus 144. The image memory 136 temporarily stores the telephoto camera image data. That is, the CPU 138 acquires the telephoto camera image data from the image sensor 36 and temporarily stores the acquired telephoto camera image data in the image memory 136. Then, the CPU 138 retrieves the telephoto camera image data from the image memory 136 as needed.

[0109] The electrical system components of the focusing lens drive mechanism 50, the zoom lens drive mechanism 52, and the aperture drive mechanism 54 are connected to the bus 144. The CPU 138 controls the electrical system components of the focusing lens drive mechanism 50, the zoom lens drive mechanism 52, and the aperture drive mechanism 54.

[0110] A communication I / F 122 is connected to the bus 144, and the CPU 138 sends and receives information with the computer 140 of the management device 20 via the communication I / F 122 and 106.

[0111] In the wide-angle camera 14, the control device 94 includes a computer 146 and an image memory 148. The computer 140 includes a CPU 150, an NVM 152, and a RAM 154. In the wide-angle camera 14, the computer 140 corresponds to the computer 146 of the telephoto camera 12. That is, in the wide-angle camera 14, the CPU 150, NVM 152, and RAM 154 correspond to the CPU 138, NVM 140, and RAM 154 of the telephoto camera 12.

[0112] The wide-angle camera 14 has a bus 156. In the wide-angle camera 14, the bus 156 corresponds to the bus 144 of the telephoto camera 12.

[0113] An image sensor 70 and an image memory 148 are connected to bus 156. The image memory 148 temporarily stores wide-angle camera image data. That is, the CPU 150 acquires wide-angle camera image data from the image sensor 70 and stores the acquired wide-angle camera image data in the image memory 148. Then, the CPU 150 retrieves wide-angle camera image data from the image memory 148 as needed.

[0114] The electrical system components of the focusing lens drive mechanism 84, the zoom lens drive mechanism 86, and the aperture drive mechanism 88 are connected to the bus 156. The CPU 150 controls the electrical system components of the focusing lens drive mechanism 84, the zoom lens drive mechanism 86, and the aperture drive mechanism 88.

[0115] A communication I / F 124 is connected to bus 156. CPU 150 sends and receives information with computer 104 of management device 20 via communication I / F 124 and 108. In recent years, as a method for monitoring a target area while using a surveillance camera, a method is being researched that allows a telephoto camera 12 and a wide-angle camera 14 to work together to monitor the target area (hereinafter also referred to as the "telephoto / wide-angle camera collaborative monitoring method"). In the telephoto / wide-angle camera collaborative monitoring method, the monitor can monitor a large area using the wide-angle camera image represented by the wide-angle camera image data, and can also monitor distant subjects using the telephoto camera image represented by the telephoto camera image data. Furthermore, as a method to improve the convenience for the monitor, a method has been researched in which the monitor specifies the part to be magnified using the wide-angle camera image, thereby controlling the zoom level and rotation of the telephoto camera 12.

[0116] However, for the telephoto camera 12 and the wide-angle camera 14 to cooperate, detailed optical information of both the telephoto camera 12 and the wide-angle camera 14 is required. For the telephoto camera 12 to properly capture the real-space area specified by the wide-angle camera image, it needs at least information related to the field of view of the wide-angle camera 14 as detailed optical information of the wide-angle camera 14. Furthermore, it is envisioned that when the telephoto camera 12 and the wide-angle camera 14 are capturing a close-range subject, due to the parallax generated between the telephoto camera 12 and the wide-angle camera 14, the telephoto camera 12 may be unable to properly capture the real-space area specified by the observer through the wide-angle camera image.

[0117] Therefore, given this situation, alignment processing is performed in the camera system 10 (see reference). Figure 5 and Figure 19 As an example, such as Figure 5 As shown, the bit alignment process is implemented by the CPU 116 executing the bit alignment process 158. The bit alignment process 158 is an example of a "program" involved in this invention. Figure 5 In the example, a bit alignment process 158 is stored in NVM 118, and CPU 116 reads the bit alignment process 158 from NVM 118 and executes it on RAM 120. CPU 116 performs bit alignment processing according to the bit alignment process 158 executed on RAM 120.

[0118] The CPU 116 performs alignment processing. First, it aligns a telephoto camera image captured by the telephoto camera 12 with a wide-angle camera image captured by the wide-angle camera 14 at at least two locations, including the end of the field of view used in the shooting based on the camera system 10. Then, the CPU 116 obtains field-of-view related information based on the alignment result, relating to the field of view used in the shooting based on the camera system 10. The alignment processing will be explained in more detail below.

[0119] As an example, such as Figure 6 As shown, the CPU 116 operates as an alignment unit 116A, a control unit 116B, and an acquisition unit 116C by executing the alignment processing program 158. Furthermore, the alignment unit 116A has a field-of-view center alignment unit 116A1 and an end alignment unit 116A2.

[0120] As an example, such as Figure 7As shown, in this embodiment, the optical information related to the imaging optical system 38 of the telephoto camera 12 is known, while the optical information related to the imaging optical system 72 of the wide-angle camera 14 is unknown. For example, the field of view of the telephoto camera 12 (hereinafter also referred to as the "telephoto camera field of view") is a known field of view, while the field of view of the wide-angle camera 14 (hereinafter also referred to as the "wide-angle camera field of view") is an unknown field of view. Although details will be described later, in this embodiment, the wide-angle camera field of view is derived by alignment processing performed by the CPU 116, and the control unit 116B and the acquisition unit 116C (see reference) are used to obtain the field of view. Figure 6 and Figure 18 ) Obtain information including the exported wide-angle camera field of view as field of view related information.

[0121] The control unit 116B acquires telephoto camera image data from the telephoto camera 12 and displays the telephoto camera image represented by the acquired telephoto camera image data as a real-time preview image on the display 26. Furthermore, the control unit 116B acquires wide-angle camera image data from the wide-angle camera 14 and displays the wide-angle camera image represented by the acquired wide-angle camera image data as a real-time preview image on the display 26. The display 26 displays the telephoto camera image and the wide-angle camera image on different screens. Figure 7 In the example, in the two horizontally arranged front view images on the display 26, a telephoto camera image is displayed on the left side of the front view and a wide-angle camera image is displayed on the right side of the front view.

[0122] Furthermore, examples of displaying telephoto and wide-angle camera images as real-time preview images on display 26 can be given here, but the present invention is not limited to this, and images other than real-time preview images can also be displayed. As an example of images other than real-time preview images, images that can be viewed later can be given.

[0123] As an example, such as Figure 8 As shown, the control unit 116B controls the center-overlapping display of the crosshair 160 of the wide-angle camera image on the display 26. The field-of-view center alignment unit 116A1 performs AF control on the wide-angle camera 14. That is, the wide-angle camera 14 has an AF function, and the field-of-view center alignment unit 116A1 focuses on the center of the wide-angle camera's field of view by activating the AF function of the wide-angle camera 14. Here, the center of the wide-angle camera's field of view corresponds to the center of the wide-angle camera image. Thus, the wide-angle camera image obtained by the wide-angle camera 14 is displayed on the display 26 while the image is focused on the center of the wide-angle camera's field of view.

[0124] Furthermore, the field-of-view center alignment unit 116A1 stores the wide-angle camera image data obtained by the wide-angle camera 14 in the RAM 120 while focusing on the center of the wide-angle camera's field of view. In addition, the wide-angle camera image data stored in the RAM 120 is updated each time a frame is captured.

[0125] As an example, such as Figure 9 As shown, the control unit 116B controls the center-overlapping display of the crosshair 162 of the telephoto camera image on the display 26. The field-of-view center alignment unit 116A1 performs AF control on the telephoto camera 12. That is, the telephoto camera 12 has an AF function, and the field-of-view center alignment unit 116A1 focuses on the center of the telephoto camera's field of view by activating the AF function of the telephoto camera 12. Here, the center of the telephoto camera's field of view corresponds to the center of the telephoto camera image. Thus, the telephoto camera image obtained by the telephoto camera 12 is displayed on the display 26 while it is focused on the center of the telephoto camera's field of view.

[0126] The field-of-view center alignment unit 116A1 acquires telephoto camera image data captured by the telephoto camera 12 while it is focused on the center of the telephoto camera's field of view. Furthermore, the field-of-view center alignment unit 116A1 acquires wide-angle camera image data from the RAM 120. Then, the field-of-view center alignment unit 116A1 performs image analysis based on the telephoto camera image data acquired from the telephoto camera 12 and the wide-angle camera image data acquired from the RAM 120 to align the center of the telephoto camera's field of view with the center of the wide-angle camera's field of view.

[0127] In this case, firstly, the field-of-view center alignment unit 116A1 calculates the rotation direction and rotation amount of the telephoto camera 12 by performing image analysis on the telephoto camera image data acquired from the telephoto camera 12 and the wide-angle camera image data acquired from the RAM 120. Here, the rotation direction and rotation amount of the telephoto camera 12 refer to the rotation direction and rotation amount required to align the centers of the wide-angle camera image and the telephoto camera image. Specifically, the field-of-view center alignment unit 116A1 calculates how to align the center of the telephoto camera image with the center of the wide-angle camera image (e.g., a point image). Figure 9 The center of the crosshair 162 shown is Figure 8 The required rotation direction and rotation amount when the center of the crosshairs 160 shown are aligned are used as the rotation direction and rotation amount of the telephoto camera 12. Furthermore, for ease of explanation, unless otherwise specified, "rotation" will be explained as panning. That is, rotation direction refers to panning direction, and rotation amount refers to panning amount.

[0128] Next, the field-of-view center alignment unit 116A1 controls the rotation of the dual-axis rotary table 16 according to the calculated rotation direction and rotation amount. That is, the field-of-view center alignment unit 116A1 causes the telephoto camera 12 to pan by operating the dual-axis rotary table 16 according to the calculated rotation direction and rotation amount.

[0129] The field-of-view center alignment unit 116A1 uses the AF function of the telephoto camera 12 to derive the subject distance. Therefore, the field-of-view center alignment unit 116A1 obtains the focus position and telephoto camera characteristic information from the telephoto camera 12. The focus position is obtained by the field-of-view center alignment unit 116A1 each time AF control is performed. The telephoto camera characteristic information is information representing the characteristics of the telephoto camera 12, referring to the information required to derive the subject distance. Examples of the information required to derive the subject distance include, for example, information related to the specifications of the telephoto camera's imaging optical system 38 and information related to the specifications of the image sensor 36.

[0130] As an example, such as Figure 10 As shown, a subject distance derivation table 164 is stored in the NVM 118. The subject distance derivation table 164 is set for each of a plurality of telephoto camera characteristic information that are different from each other. The subject distance derivation table 164 for each of the plurality of telephoto camera characteristic information establishes a corresponding association between the focus position and the subject distance. The field-of-view center alignment unit 116A1 obtains the subject distance derivation table 164 corresponding to the telephoto camera characteristic information obtained from the telephoto camera 12 from the NVM 118. Then, the field-of-view center alignment unit 116A1 derives the subject distance corresponding to the focus position obtained from the telephoto camera 12 from the subject distance derivation table 164 obtained from the NVM 118. The subject distance thus derived by the field-of-view center alignment unit 116A1 is an example of the "first subject distance" involved in the present invention. Furthermore, for ease of explanation, the subject distance derived from the field-of-view center alignment unit 116A1 will also be referred to as the "first subject distance". Additionally, here, an example of deriving the first subject distance from the subject distance derivation table 164 has been provided, but the technology of the present invention is not limited thereto. The first subject distance can also be calculated using a formula that sets the telephoto camera characteristic information and focus position as independent variables and the first subject distance as a dependent variable.

[0131] As an example, such as Figure 11As shown, the NVM118 includes a predetermined value storage area 118A and a field-of-view center alignment storage area 118B. The predetermined value storage area 118A stores predetermined values, an example of "position-specific information" as described in this invention. These predetermined values ​​determine the positional relationship between the telephoto camera 12 and the wide-angle camera 14. Examples of predetermined values ​​include horizontal offset and depth offset. Here, the horizontal offset refers to the horizontal offset between the telephoto camera 12 and the wide-angle camera 14. Furthermore, the depth offset refers to the forward / backward offset between the telephoto camera 12 and the wide-angle camera 14, i.e., the depth offset between them. Additionally, the horizontal offset and depth offset are examples of the "offset between the first imaging device and the second imaging device" as described in this invention.

[0132] The field-of-view center alignment unit 116A1 determines whether the distance to the first subject is greater than or equal to a first threshold. Here, the first threshold is a fixed value obtained in advance through testing based on actual equipment and / or computer simulation, representing the distance to the subject in the real-space area specified by the user, which is unsuitable for the telephoto camera 12 to capture properly using the wide-angle camera image due to parallax caused by horizontal and depth offsets. Furthermore, while a fixed value is exemplified here as the first threshold, the technology of the present invention is not limited to this; the first threshold can also be a variable value that changes according to external instructions and / or imaging conditions.

[0133] When the distance to the first subject is greater than or equal to the first threshold, the field-of-view center alignment unit 116A1 overwrites and stores the current rotation angle in the field-of-view center alignment storage area 118B. Here, the current rotation angle refers to the angle measured by the field-of-view center alignment unit 116A1 on the dual-axis rotary table 16. Figure 9 The rotation angle of the telephoto camera 12, which is controlled by rotation, is shown. Figure 11 In the example, the rotation angle of the telephoto camera 12 is represented as "θ". c That is, the angle between the parallel line PL (parallel to the optical axis OA2) that is consistent with the optical axis OA1 when the telephoto camera 12 is in its original position (the initial state before the telephoto camera 12 is rotated) and the optical axis OA1 after the telephoto camera 12 is rotated is the rotation angle θ. c .

[0134] When the distance to the first subject is less than a first threshold, the field-of-view center alignment unit 116A1 performs field-of-view center alignment processing. Here, field-of-view center alignment processing refers to aligning the center of the telephoto camera's field of view with the center of the wide-angle camera's field of view using a predetermined value. Specifically, aligning the center of the telephoto camera's field of view with the center of the wide-angle camera's field of view involves calculating, using a predetermined value, the required rotation angle θ to make the center of the telephoto camera's field of view coincide with the center of the wide-angle camera's field of view. c And the calculated rotation angle θ c The field of view center alignment storage area 118B is overwritten and stored. That is, the field of view center alignment unit 116A1 calculates the rotation angle θ by performing field of view center alignment processing. c And the calculated rotation angle θ c It is covered and stored in the center alignment storage area 118B of the field of view.

[0135] In addition, for ease of explanation, the rotation angle covered by the field of view center alignment part 116A1 and stored in the field of view center alignment storage area 118B will also be referred to as the "first rotation angle".

[0136] As an example, such as Figure 12 As shown, when the distance to the first subject is set to "d"... target1 Set the horizontal offset to "d". x "At that time, the rotation angle θ c The field-of-view center alignment unit 116A1 calculates the following formula (1). Then, the rotation angle θ calculated by the field-of-view center alignment unit 116A1 is... c The first rotation angle is covered and stored in the field of view center alignment storage area 118B. In addition, the first rotation angle is an example of the "field of view center alignment result obtained by aligning the center of the field of view of the first camera device with the center of the field of view of the second camera device" involved in the present invention.

[0137] [Formula 1] Thus, if the field of view center alignment process is performed by the field of view center alignment unit 116A1, then the end alignment process is performed by the end alignment unit 116A2. A detailed explanation will follow.

[0138] As an example, such as Figure 13 As shown, the control unit 116B displays one end of the wide-angle camera image on the display 26 (in... Figure 13In the example shown, the control of the crosshairs 160 is superimposed on the center of the left end of the front view of the wide-angle camera image on the display 26. The end alignment unit 116A2 performs AF control on the wide-angle camera 14. That is, the end alignment unit 116A2 focuses on one end of the field of view of the wide-angle camera by activating the AF function of the wide-angle camera 14. Here, one end of the field of view of the wide-angle camera corresponds to one end of the wide-angle camera image. Thus, the wide-angle camera image obtained by the wide-angle camera 14 is displayed on the display 26 while it is focused on one end of the field of view of the wide-angle camera.

[0139] Furthermore, the end alignment unit 116A2 stores the wide-angle camera image data obtained by the wide-angle camera 14 in the RAM 120 while focusing on one end of the wide-angle camera's field of view. In addition, the wide-angle camera image data stored in the RAM 120 is updated each time a frame is captured.

[0140] As an example, such as Figure 14 As shown, the control unit 116B displays one end of the telephoto camera image on the display 26 (in... Figure 14 In the example shown, the control of the crosshairs 162 is superimposed on the center of the left end of the front view of the telephoto camera image on the display 26. The end alignment unit 116A2 performs AF control on the telephoto camera 12. That is, the end alignment unit 116A2 focuses on one end of the telephoto camera's field of view by activating the AF function of the telephoto camera 12. Here, one end of the telephoto camera's field of view corresponds to one end of the telephoto camera image. Thus, the telephoto camera image obtained by the telephoto camera 12 is displayed on the display 26 while it is focused on one end of the telephoto camera's field of view.

[0141] The end alignment unit 116A2 acquires telephoto camera image data captured by the telephoto camera 12 while it is focused on one end of the telephoto camera's field of view. The end alignment unit 116A2 also acquires wide-angle camera image data from the RAM 120. Then, the end alignment unit 116A2 performs image analysis based on the telephoto camera image data acquired from the telephoto camera 12 and the wide-angle camera image data acquired from the RAM 120 to align one end of the telephoto camera's field of view with one end of the wide-angle camera's field of view. Here, one end of the telephoto camera's field of view is an example of the "first end of the field of view of the first imaging device" according to the present invention, and one end of the wide-angle camera's field of view is an example of the "second end of the field of view of the first imaging device (one of the two ends of the field of view of the second imaging device) according to the present invention."

[0142] When aligning one end of the field of view of a telephoto camera with one end of the field of view of a wide-angle camera, the end alignment unit 116A2 first calculates the rotation direction and rotation amount of the telephoto camera 12 by performing image analysis on the telephoto camera image data acquired from the telephoto camera 12 and the wide-angle camera image data acquired from the RAM 120. Here, the rotation direction and rotation amount of the telephoto camera 12 refer to the rotation direction and rotation amount required to align the ends of the wide-angle camera image and the telephoto camera image. Specifically, the end alignment unit 116A2 calculates the center of one end of the telephoto camera image with (for example, making...) Figure 14 The center of the crosshair 162 shown is Figure 13 The rotation direction and amount required for aligning the subject (e.g., a point image) at the center of one end of the wide-angle camera image (the center alignment of the crosshairs 160 shown) are used as the rotation direction and amount for the telephoto camera 12.

[0143] Next, the end alignment unit 116A2 performs rotation control on the dual-axis rotary table 16 according to the calculated rotation direction and rotation amount. That is, the end alignment unit 116A2 causes the telephoto camera 12 to pan by operating the dual-axis rotary table 16 according to the calculated rotation direction and rotation amount.

[0144] The end-alignment unit 116A2 uses the AF function of the telephoto camera 12 to derive the subject distance. Therefore, the focus position and telephoto camera characteristic information are obtained from the end-alignment unit 116A2 and the telephoto camera 12. The focus position is obtained by the end-alignment unit 116A2 each time AF control is performed.

[0145] As an example, such as Figure 15 As shown, the end alignment unit 116A2 obtains a subject distance derivation table 164 from the NVM 118 corresponding to the telephoto camera characteristic information obtained from the telephoto camera 12. Then, the end alignment unit 116A2 derives the subject distance corresponding to the focus position obtained from the telephoto camera 12 from the subject distance derivation table 164 obtained from the NVM 118. Thus, the subject distance derived by the end alignment unit 116A2 is an example of the "second subject distance" involved in the present invention. Furthermore, for ease of explanation, the subject distance derived by the end alignment unit 116A2 will also be referred to as the "second subject distance" below. In addition, here, a formal example of deriving the second subject distance from the subject distance derivation table 164 has been given as an example, but the technology of the present invention is not limited to this, and the second subject distance can also be calculated according to a calculation formula in which the telephoto camera characteristic information and the focus position are set as independent variables and the second subject distance is set as a dependent variable.

[0146] As an example, such as Figure 16As shown, an end alignment storage area 118C is provided in the NVM118. In the end alignment storage area 118C, each of the second rotation angle (described later) and the half-angle view (hereinafter also referred to as "half-angle view") which is half the field of view of the wide-angle camera is individually overwritten and stored.

[0147] The end alignment unit 116A2 determines whether the distance to the second subject is greater than or equal to a second threshold. Here, the second threshold is a fixed value obtained in advance through testing and / or computer simulation based on actual equipment. It represents the distance to the subject in the real space area specified by the user, which is unsuitable for the telephoto camera 12 to capture properly using the wide-angle camera image due to parallax caused by horizontal and depth offsets. Furthermore, while a fixed value is exemplified here as the second threshold, the technology of the present invention is not limited to this; the second threshold can also be a variable value that changes according to external instructions and / or imaging conditions.

[0148] When the distance to the second subject is greater than or equal to the second threshold, the end alignment unit 116A2 overwrites and stores the current rotation angle in the end alignment storage area 118C. Here, the current rotation angle refers to the angle at which the dual-axis rotary table 16 will be rotated by the end alignment unit 116A2. Figure 14 The rotation angle of the telephoto camera 12, which is rotated by the rotation control shown, is the same as the first rotation angle (reference). Figure 11 The angles added together. Figure 16 In the example, the rotation angle of the telephoto camera 12 is represented as "θ". e1 ".

[0149] Furthermore, for ease of explanation, the rotation angle θ, which is covered by the end alignment part 116A2 and stored in the end alignment storage area 118C, will be described below. e1 Also known as the "second rotation angle".

[0150] When the distance to the second subject is less than the second threshold, the end alignment unit 116A2 performs end alignment processing. Here, end alignment processing refers to the process of aligning one end of the telephoto camera's field of view with one end of the wide-angle camera's field of view using a predetermined value.

[0151] Here, aligning one end of the telephoto camera's field of view with one end of the wide-angle camera's field of view means, based on the current rotation angle ( Figure 16 The example shown is the rotation angle θ. e1 The process involves calculating the half-angle of the end alignment unit 116A2 based on the specified value and overwriting and storing the calculated half-angle in the end alignment storage area 118C. That is, the end alignment unit 116A2 calculates the half-angle by performing end alignment processing and overwriting and storing the calculated half-angle in the end alignment storage area 118C.

[0152] As an example, such as Figure 17 As shown, when the distance to the second subject is set to "d"... target2 Set the horizontal offset to "d". x Set the depth direction offset to "d". y "At that time, half angle θ e2 The end alignment unit 116A2 calculates the angle θ according to the following formula (2). Then, the half angle θ calculated by the end alignment unit 116A2 is... e2 It is overwritten and stored in the end alignment storage area 118C. Additionally, the half-view angle θ... e2 and the second rotation angle (rotation angle θ) e1 This is an example of "end alignment result obtained by aligning the first end of the field of view of the first camera device with the second end of the field of view of the second camera device, which is one of the two ends of the field of view of the first camera device" involved in the present invention.

[0153] [Formula 2] As an example, such as Figure 18 As shown, the acquisition unit 116A3 acquires a half-angle view from the end alignment unit 116A2. The half-angle view is an angle derived from the result of aligning the center of the wide-angle camera's field of view with the center of the telephoto camera's field of view, and from aligning the ends of the wide-angle camera's field of view with the ends of the telephoto camera's field of view. The acquisition unit 116A3 acquires the wide-angle camera's field of view based on the half-angle view acquired from the end alignment unit 116A2. That is, the acquisition unit 116A3 calculates the wide-angle camera's field of view by setting the half-angle view acquired from the end alignment unit 116A2 to twice its original value. Furthermore, the wide-angle camera's field of view is an example of "field of view related information" as described in this invention.

[0154] The control unit 116B causes the display 26 to display a telephoto camera image represented by telephoto camera image data and a wide-angle camera image represented by wide-angle camera image data. Furthermore, while displaying both the telephoto camera image and the wide-angle camera image, the control unit 116B also causes the display 26 to display field-of-view related information. The control unit 116B obtains telephoto camera characteristic information from the telephoto camera 12. This telephoto camera characteristic information includes information about the telephoto camera's field of view. The control unit 116B causes the display 26 to display information including the wide-angle camera's field of view, the telephoto camera's field of view, and the difference between the wide-angle camera's field of view and the telephoto camera's field of view (e.g., a value obtained by subtracting the telephoto camera's field of view from the wide-angle camera's field of view) as field-of-view related information. Furthermore, the control unit 116B performs marking to indicate which area within the wide-angle camera image corresponds to the telephoto camera image (…). Figure 18 The control shown in the example (a dashed rectangle) is overlaid on the wide-angle camera image.

[0155] Next, refer to Figures 19-21 The function of the parts of the camera system 10 that are related to the present invention will be explained. Figure 19 The diagram illustrates an example of a bit alignment process executed by CPU 116 at a predetermined timing (e.g., upon reaching a pre-determined time, or upon receiving an external instruction to begin bit alignment). Additionally, Figure 19 The alignment process shown is an example of the "method of operation of a camera system" involved in the present invention.

[0156] exist Figure 19 In the alignment process shown, firstly, in step ST100, the field-of-view center alignment part 116A1 is performed. Figure 20 The field-of-view center alignment shown is taken as an example.

[0157] exist Figure 20 In the field of view center alignment process shown, firstly, in step ST100A, the field of view center alignment unit 116A1 acquires telephoto camera image data from telephoto camera 12 and wide-angle camera image data from wide-angle camera 14.

[0158] In the next step ST100B, the field of view center alignment unit 116A1 performs image analysis on the telephoto camera image data and wide-angle camera image data acquired in step ST100A, causing the telephoto camera 12 to rotate so that the center of the telephoto camera's field of view is aligned with the center of the wide-angle camera's field of view.

[0159] In the next step ST100C, the field-of-view center alignment unit 116A1 focuses on the center of the field of view of the telephoto camera 12 by performing AF control on the telephoto camera 12.

[0160] In the next step ST100D, the field of view center alignment unit 116A1 obtains the focus position and telephoto camera characteristic information from the telephoto camera 12.

[0161] In the next step ST100E, the field-of-view center alignment unit 116A1 obtains the subject distance derivation table 164 from the NVM 118, which corresponds to the telephoto camera characteristic information obtained in step ST100D. Then, the field-of-view center alignment unit 116A1 derives the subject distance corresponding to the focus position obtained in step ST100D from the subject distance derivation table 164 obtained from the NVM 118 as the first subject distance.

[0162] In the next step ST100F, the field-of-view center alignment unit 116A1 determines whether the first subject distance derived in step ST100E is less than a first threshold. If, in step ST100F, the first subject distance is greater than or equal to the first threshold, the determination is rejected, and the field-of-view center alignment process proceeds to step ST100G. If, in step ST100F, the first subject distance is less than the first threshold, the determination is affirmative, and the field-of-view center alignment process proceeds to step ST100H.

[0163] In step ST100G, the field of view center alignment unit 116A1 covers the current rotation angle as the first rotation angle and saves it in the field of view center alignment storage area 118B. Then, the field of view center alignment process ends.

[0164] In step ST100H, the field-of-view center alignment unit 116A1 calculates the first rotation angle according to formula (1) based on the predetermined value stored in the predetermined value storage area 118A and the first subject distance derived in step ST100E. Then, the field-of-view center alignment unit 116A1 covers and saves the calculated first rotation angle in the field-of-view center alignment storage area 118B, and then the field-of-view center alignment process ends.

[0165] exist Figure 19 In step ST102 shown, the end alignment part 116A2 is executed. Figure 21 The end alignment process shown is an example.

[0166] exist Figure 21 In the end alignment process shown, firstly, in step ST102A, the end alignment unit 116A2 acquires telephoto camera image data from telephoto camera 12 and wide-angle camera image data from wide-angle camera 14.

[0167] In the next step ST102B, the end alignment unit 116A2 performs image analysis on the telephoto camera image data and wide-angle camera image data acquired in step ST102A, and rotates the telephoto camera 12 to align the end of the telephoto camera's field of view (e.g., one end of the telephoto camera's field of view) with the end of the wide-angle camera's field of view (e.g., one end of the wide-angle camera's field of view).

[0168] In the next step ST102C, the end alignment unit 116A2 focuses on the end of the telephoto camera's field of view by performing AF control on the telephoto camera 12.

[0169] In the next step ST102D, the end alignment unit 116A2 obtains the focus position and telephoto camera characteristic information from the telephoto camera 12.

[0170] In the next step ST102E, the end alignment unit 116A2 obtains a subject distance derivation table 164 from the NVM 118, corresponding to the telephoto camera characteristic information obtained in step ST102D. Then, the end alignment unit 116A2 derives the subject distance corresponding to the focus position obtained in step ST102D from the subject distance derivation table 164 obtained from the NVM 118 as the second subject distance.

[0171] In the next step ST102F, the end alignment unit 116A2 determines whether the second subject distance derived in step ST102E is less than a second threshold. If, in step ST102F, the second subject distance is greater than or equal to the second threshold, the determination is rejected, and the end alignment process proceeds to step ST102G. If, in step ST102F, the second subject distance is less than the second threshold, the determination is affirmed, and the end alignment process proceeds to step ST102H.

[0172] In step ST102G, the end alignment unit 116A2 covers the current rotation angle as the second rotation angle and saves it in the end alignment storage area 118C, and then the end alignment process ends.

[0173] In step ST102H, the end alignment unit 116A2 calculates the half-angle of view according to formula (2) based on the predetermined value stored in the predetermined value storage area 118A, the first rotation angle stored in the field of view center alignment storage area 118B, and the first subject distance derived in step ST100E. Then, the end alignment unit 116A2 covers and saves the calculated half-angle of view in the end alignment storage area 118C, and then the end alignment process ends.

[0174] exist Figure 19 In step ST104, the acquisition unit 116C determines whether the half-angle has been calculated by performing end alignment processing. If, in step ST104, the half-angle has not been calculated by performing end alignment processing, the determination is negative, and the alignment processing ends. If, in step ST104, the half-angle has been calculated by performing end alignment processing, the determination is positive, and the end alignment processing proceeds to step ST106.

[0175] In step ST106, the acquisition unit 116C calculates the field of view of the wide-angle camera based on the half-angle angle calculated by performing end alignment processing. Then, the control unit 116B acquires field of view related information including the field of view of the wide-angle camera calculated by the acquisition unit 116C, and displays the acquired field of view related information on the display 26. Then, the alignment processing ends.

[0176] As explained above, in the camera system 10, at two locations—the center and the ends of the field of view used in the shooting based on the camera system 10—the telephoto camera image obtained by the telephoto camera 12 and the wide-angle camera image obtained by the wide-angle camera 14 are aligned. Then, based on the alignment result, field-of-view related information is obtained as information relating to the field of view used in the shooting based on the camera system 10. Thus, according to this structure, information relating to the field of view required when using the telephoto camera 12 and the wide-angle camera 14 in conjunction (e.g., the wide-angle camera field of view, etc.) can be obtained.

[0177] Furthermore, in the camera system 10, the wide-angle camera's field of view is obtained based on the field-of-view center alignment result obtained by aligning the center of the telephoto camera's field of view with the center of the wide-angle camera's field of view, and the end alignment result obtained by aligning one end of the telephoto camera's field of view with one end of the wide-angle camera's field of view. Therefore, according to this structure, even if the wide-angle camera's field of view is unknown, it can still be determined.

[0178] Furthermore, in the camera system 10, a first rotation angle is calculated based on the horizontal offset, and a half-angle is calculated based on both the horizontal and depth offsets. Therefore, according to this structure, compared to the case where neither the horizontal nor depth offset is used, high-precision alignment of the center of the telephoto camera's field of view with the center of the wide-angle camera's field of view, as well as high-precision alignment of the ends of the telephoto camera's field of view with the ends of the wide-angle camera's field of view, can be achieved. Moreover, even if the positions of the telephoto camera 12 and the wide-angle camera 14 are offset, information related to the field of view required when using the telephoto camera 12 and the wide-angle camera 14 in conjunction can be obtained.

[0179] Furthermore, in the camera system 10, when the distance to the first subject is less than a first threshold, field-of-view center alignment is performed using a horizontal offset; when the distance to the second subject is less than a second threshold, end alignment is performed using both horizontal and depth offsets. Therefore, according to this structure, even if parallax effects become more pronounced as the subject distance decreases, information related to the field of view required when using the telephoto camera 12 and the wide-angle camera 14 in conjunction can be obtained.

[0180] Furthermore, in the camera system 10, a first subject distance is derived from the field-of-view center alignment part 116A1, and a second subject distance is derived from the end alignment part 116A2. Thus, according to this structure, compared to the case where the user assigns the subject distance to the camera system 10, the subject distance for alignment can be obtained quickly.

[0181] Furthermore, in the camera system 10, the center alignment processing of the field of view is performed ( Figure 19 After the processing of step ST100 shown, end alignment processing is performed. Figure 19 (The processing in step ST102 shown). Thus, according to this structure, compared with the case where end alignment processing is performed without field-of-view center alignment processing, high-precision information can be obtained as information related to the field of view required when using telephoto camera 12 and wide-angle camera 14 in conjunction (refer to steps ST100 and ST102).

[0182] Furthermore, in the camera system 10, field-of-view center alignment and end-angle alignment are performed using image analysis based on images from the telephoto camera and the wide-angle camera. Therefore, according to this structure, accurate alignment can be achieved compared to manually performing the equivalent field-of-view center alignment and end-angle alignment. Alternatively, either field-of-view center alignment or end-angle alignment can be performed using image analysis based on images from the telephoto camera and the wide-angle camera.

[0183] Furthermore, in the camera system 10, the optical information related to the camera optical system 38 of the telephoto camera 12 is known, while the optical information related to the camera optical system 72 of the wide-angle camera 14 is unknown. Therefore, according to this structure, even if the optical information related to the camera optical system 72 of the wide-angle camera 14 is unknown, the field of view of the wide-angle camera can still be determined.

[0184] Furthermore, in the camera system 10, the focal length of the camera optical system 38 of the telephoto camera 12 is more than twice the focal length of the camera optical system 72 of the wide-angle camera 14. Therefore, according to this structure, even though the focal length of the camera optical system 38 of the telephoto camera 12 is more than twice the focal length of the camera optical system 72 of the wide-angle camera 14, information related to the field of view required when using both the telephoto camera 12 and the wide-angle camera 14 can be obtained.

[0185] Furthermore, in the above embodiment, during the end alignment process, the end alignment unit 116A2 aligns one end of the telephoto camera's field of view with one end of the wide-angle camera's field of view. However, the technology of the present invention is not limited to this. In addition to aligning one end of the telephoto camera's field of view with one end of the wide-angle camera's field of view, the end alignment unit 116A2 can also perform opposite-side end alignment processing. Opposite-side end alignment processing refers to the process of aligning the other end of the telephoto camera's field of view with the other end of the wide-angle camera's field of view. Here, the other end of the telephoto camera's field of view is an example of "the first opposite end of the two ends of the field of view of the first imaging device located on the opposite side of the first end" according to the technology of the present invention, and the other end of the wide-angle camera's field of view is an example of "the second opposite end of the two ends of the field of view of the second imaging device located on the opposite side of the second end" according to the technology of the present invention.

[0186] Thus, when the end alignment unit 116A2 performs end alignment processing and the opposite end alignment processing, the acquisition unit 116C only needs to acquire the wide-angle camera field of view based on the first result obtained in the end alignment processing by aligning one end of the telephoto camera field of view with one end of the wide-angle camera field of view (e.g., one half-angle view of the wide-angle camera 14) and the second result obtained in the opposite end alignment processing by aligning the other end of the telephoto camera field of view with the other end of the wide-angle camera field of view (e.g., the other half-angle view of the wide-angle camera 14).

[0187] In this case, for example, Figure 22A and Figure 22B The end alignment process shown is performed by CPU116. Figure 22A and Figure 22B The flowchart shown is Figure 21 The only difference in the flowchart shown is the presence of steps ST102H1 to ST102N. Therefore, the following discussion focuses on... Figure 21 The flowchart shown illustrates the different steps.

[0188] exist Figure 22A In step ST102H1, the end alignment unit 116A2 rotates the telephoto camera 12 by performing image analysis on the telephoto camera image data and wide-angle camera image data obtained in step ST102A, so that the opposite end of the telephoto camera's field of view (e.g., the other end of the telephoto camera's field of view) is aligned with the opposite end of the wide-angle camera's field of view (e.g., the other end of the wide-angle camera's field of view).

[0189] In the next step ST102I, the end alignment unit 116A2 focuses on the opposite end of the telephoto camera's field of view by performing AF control on the telephoto camera 12.

[0190] In the next step ST102J, the end alignment unit 116A2 obtains the focus position and telephoto camera characteristic information from the telephoto camera 12.

[0191] In the next step ST102K, the end alignment unit 116A2 obtains a subject distance derivation table 164 from the NVM 118 corresponding to the telephoto camera characteristic information obtained in step ST102J. Then, the end alignment unit 116A2 derives the subject distance corresponding to the focus position obtained in step ST102J from the subject distance derivation table 164 obtained from the NVM 118 as the third subject distance. Furthermore, while the example of deriving the third subject distance from the subject distance derivation table 164 has been provided here, the technology of the present invention is not limited thereto. The third subject distance can also be calculated using a calculation formula that sets the telephoto camera characteristic information and the focus position as independent variables and the third subject distance as a dependent variable.

[0192] In the next step ST102L, the end alignment unit 116A2 determines whether the third subject distance derived in step ST102K is less than a third threshold. The third threshold can be, for example, the same value as the second threshold described above. If, in step ST102L, the third subject distance is greater than or equal to the third threshold, the determination is rejected, and the end alignment process proceeds to step ST102M. If, in step ST102L, the third subject distance is less than the third threshold, the determination is affirmed, and the end alignment process proceeds to step ST102N.

[0193] In step ST102M, the end alignment unit 116A2 covers the current rotation angle as the third rotation angle and saves it in the end alignment storage area 118C, and then the end alignment process ends.

[0194] In step ST102N, the end alignment unit 116A2 calculates the half-angle of view according to formula (2) based on the predetermined value stored in the predetermined value storage area 118A, the first rotation angle stored in the field of view center alignment storage area 118B, and the third subject distance derived in step ST100K. Here, the half-angle of view refers to the half-angle of view that is paired with the half-angle of view calculated in step ST102H. That is, the half-angle of view calculated in step ST102H is one half-angle of view of the wide-angle camera 14, and in contrast, the half-angle of view calculated in step ST102N is the other half-angle of view of the wide-angle camera 14. In addition, the half-angle of view calculated in step ST102H is an example of the "first result" involved in the present invention, and the half-angle of view calculated in step ST102N is an example of the "second result" involved in the present invention.

[0195] Then, in step ST102N, the end alignment unit 116A2 covers and saves the calculated half-angle view in the end alignment storage area 118C, and then the end alignment process ends.

[0196] Thus, by conducting Figure 22A and Figure 22B The end alignment process shown calculates one and a half angles of view and the other half angle of view of the wide-angle camera 14. Therefore, the acquisition unit 116C can obtain the field of view of the wide-angle camera simply by calculating the sum of one and a half angles of view and the other half angle of view of the wide-angle camera 14. Thus, even if the field of view of the wide-angle camera is unknown, the field of view of the wide-angle camera can be determined.

[0197] Furthermore, here, the acquisition unit 116C calculates the sum of one half-angle view and the other half-angle view of the wide-angle camera 14, but the technology of the present invention is not limited to this. For example, when the distance to the third subject is greater than or equal to the third threshold, and the position at the other end of the telephoto camera's field of view is aligned with the position at the other end of the wide-angle camera's field of view, the end alignment unit 116A2 calculates the other half-angle view of the wide-angle camera 14. As an example, such as... Figure 23 As shown, the acquisition unit 116C can acquire the field of view of the wide-angle camera based solely on the other half of the viewing angle of the wide-angle camera 14. Here, "acquiring the field of view of the wide-angle camera based solely on the other half of the viewing angle of the wide-angle camera 14" means, for example, acquiring a value that doubles the other half of the viewing angle of the wide-angle camera 14 as the field of view of the wide-angle camera.

[0198] Thus, the reason why the acquisition unit 116C acquires the field of view of the wide-angle camera 14 based solely on the other half of the field of view is that, under the premise that "the second threshold = the third threshold," compared to the one half of the field of view of the wide-angle camera 14 when the position of one end of the telephoto camera's field of view is aligned with the position of one end of the wide-angle camera's field of view when the distance to the second subject is less than the second threshold, the other half of the field of view of the wide-angle camera 14 when the position of the other end of the telephoto camera's field of view is aligned with the position of the other end of the wide-angle camera's field of view is greater than the third threshold. That is, because the distance to the third subject is longer than the distance to the second subject, the other half of the field of view of the wide-angle camera 14 is less susceptible to the parallax caused by the positional offset between the telephoto camera 12 and the wide-angle camera 14 than the one half of the field of view of the wide-angle camera 14. Therefore, based on this structure, compared to the case where the wide-angle camera's field of view is calculated based on only one and a half angles of the wide-angle camera 14 when the position of the other end of the telephoto camera's field of view is aligned with the position of the other end of the wide-angle camera's field of view under the condition that the distance to the third subject is greater than or equal to the third threshold, a high-precision field of view can be determined as the wide-angle camera's field of view.

[0199] In the above embodiments, examples of end alignment processing without considering the scale ratio of telephoto camera images to wide-angle camera images are given. However, the technology of the present invention is not limited to this. In the end alignment processing, the end alignment unit 116A2 can align one end of the telephoto camera's field of view with one end of the wide-angle camera's field of view based on the scale ratio of the telephoto camera image to the wide-angle camera image used in the image analysis in the field of view center alignment processing.

[0200] In this case, Figure 24 The field-of-view center alignment process shown is executed by CPU116, and Figure 25 The end alignment process shown is performed by CPU116. Figure 24 The flowchart shown is Figure 20 The difference in the flowchart shown is that it includes step ST200. Figure 25 The flowchart shown is Figure 21 The difference in the flowchart shown is the presence of step ST202, therefore, in this context, it is related to... Figure 20 and Figure 21 The flowchart shown illustrates the different steps.

[0201] exist Figure 24 In step ST200, the field-of-view center alignment unit 116A1 calculates the scale ratio between the telephoto camera image and the wide-angle camera image based on the telephoto camera image data and the wide-angle camera image data acquired in step ST100A. Here, scale ratio refers, for example, to the ratio of the area of ​​the camera subject (refer to...). Figure 7 The ratio of the area of ​​the image region within a wide-angle camera image of a specific subject to the area of ​​the image region within a telephoto camera image.

[0202] exist Figure 25 In step ST202 shown, the end alignment part 116A2 is based on... Figure 24 The scale ratio calculated in step ST200 is used to align one end of the telephoto camera's field of view with one end of the wide-angle camera's field of view. In this case, for example, the end alignment unit 116A2 calculates the rotation angle of the telephoto camera 12 according to a rotation angle calculation formula that sets the scale ratio as an independent variable and the rotation angle of the telephoto camera 12 as a dependent variable, and makes the telephoto camera 12 rotate only by the calculated rotation angle. As a result, in the pre-processing stage of step ST102B, one end of the telephoto camera's field of view can be brought close to or aligned with one end of the wide-angle camera's field of view, and the time required for end alignment processing can be shortened compared to the case where the scale ratio is not used.

[0203] In the above embodiments, the description is based on the premise that the first subject distance using the AF function is successfully derived in the field-of-view center alignment process, but the technology of the present invention is not limited thereto. For example, if the deriving of the first subject distance using the AF function fails, the field-of-view center alignment unit 116A1 controls the dual-axis rotary table 16 according to the telephoto camera image, and after aligning the center of the telephoto camera's field of view with a high-contrast subject (hereinafter also referred to as "high-contrast subject") near the center of the telephoto camera's field of view, the first subject distance can be derived using the AF function.

[0204] In this case, for example, Figure 26 The field-of-view center alignment process shown is executed by CPU116. Figure 20 Compared to the flowchart shown, Figure 26 The difference in the flowchart shown is that steps ST300 to ST312 replace step ST100C. Therefore, here, regarding... Figure 20 The flowchart shown illustrates the different steps.

[0205] exist Figure 26 In step ST300 shown, the field-of-view center alignment unit 116A1 performs contrast AF on the center of the telephoto camera's field of view. Here, contrast AF refers to focusing in a way that maximizes the contrast of a specific part of the image obtained by shooting (in this case, the part corresponding to the center of the telephoto camera's field of view in the telephoto camera image).

[0206] In step ST302, the field-of-view center alignment unit 116A1 determines whether the contrast AF performed in step ST300 has failed. Contrast AF failure means, for example, that the camera cannot focus because it cannot detect high-frequency components above a specified value from a specific part of the image obtained by shooting (here, the part corresponding to the center of the telephoto camera's field of view in the telephoto camera image).

[0207] In step ST302, if the contrast AF does not fail, the determination is rejected, and the field of view center alignment process is transferred to step ST100D. In step ST302, if the contrast AF fails, the determination is affirmed, and the field of view center alignment process is transferred to step ST304.

[0208] In step ST304, the field-of-view center alignment unit 116A1 analyzes the telephoto camera image data acquired in step ST100A and searches for a high-contrast subject near the center of the telephoto camera's field of view. Here, a high-contrast subject near the center of the telephoto camera's field of view refers to, for example, the high-contrast subject closest to the center of the telephoto camera's field of view. A high-contrast subject is defined as a subject with a contrast level sufficient for contrast autofocus (AF).

[0209] In step ST306, the field-of-view center alignment unit 116A1 determines whether a high-contrast subject exists near the center of the telephoto camera's field of view. If, in step ST306, a high-contrast subject does not exist near the center of the telephoto camera's field of view, the determination is denied, and the field-of-view center alignment process proceeds to step ST308. If, in step ST306, a high-contrast subject exists near the center of the telephoto camera's field of view, the determination is affirmed, and the field-of-view center alignment process proceeds to step ST310.

[0210] In step ST308, the field-of-view center alignment unit 116A1 performs error processing, and then the alignment process ends. Here, error processing refers to, for example, causing the display 26 to display messages such as "Autofocus failed," "Unable to align," and / or "Please change the subject located at the center of the field of view," or outputting the same message verbally from a speaker (not shown).

[0211] In step ST310, the field-of-view center alignment unit 116A1 rotates the telephoto camera 12 so that the center of the telephoto camera's field of view is aligned with the high-contrast subject searched by performing the process in step ST304.

[0212] In step ST312, the field-of-view center alignment unit 116A1 performs contrast AF to focus on a high-contrast subject located at the center of the telephoto camera's field of view.

[0213] Thus, through execution Figure 26 The field-of-view center alignment process shown can achieve successful contrast AF even when there is no high-contrast subject at the center of the telephoto camera's field of view. If contrast AF is successful, the field-of-view center alignment unit 116A1 can obtain the focus position from the telephoto camera 12, and therefore the first subject distance can be derived from the focus position.

[0214] Therefore, according to this structure, compared with the case where the distance of the subject is derived by using the AF function after aligning the center of the telephoto camera's field of view with a high-contrast subject located at a position farther than the center of the telephoto camera's field of view, the distance of the first subject can be derived with high accuracy even when there is no high-contrast subject at the center of the telephoto camera's field of view.

[0215] Here, an example of performing contrast AF again after rotating the telephoto camera 12 has been described, but the technology of the present invention is not limited to this. For example, when performing contrast AF on the closest side, contrast AF can also be performed again while narrowing the focus range to the wide-angle side.

[0216] Furthermore, examples are given here to illustrate the... Figure 20 The field-of-view center alignment process shown applies steps ST300 to ST312 (see reference). Figure 26 This is a variation of the processing described above, but the technology of the present invention is not limited thereto. Processing equivalent to steps ST300 to ST312 can also be applied to the end alignment process. In this case, it is possible to facilitate the high-precision derivation of the distance to the second subject.

[0217] exist Figure 26 In the field-of-view center alignment process shown, an example is illustrated of aligning the center of the telephoto camera's field of view with the center of the wide-angle camera's field of view regardless of the focal length of the telephoto camera 12's imaging optical system 38; however, the technology of the present invention is not limited to this. For example, the field-of-view center alignment unit 116A1 can enable the telephoto camera 12 to focus when the focal length of the telephoto camera 12's imaging optical system 38 is at its longest, and derive a first subject distance based on the focusing result (e.g., focusing position).

[0218] In this case, for example, it is executed by CPU116. Figure 27 The field of view center alignment shown is used instead of... Figure 26 The field of view center alignment is shown. Because... Figure 27 The flowchart shown is Figure 26 The flowchart shown differs in that it includes step ST400, therefore it is similar to... Figure 26 The flowchart shown illustrates the different steps.

[0219] exist Figure 27In step ST400, the field-of-view center alignment unit 116A1 controls the imaging optical system 38 of the telephoto camera 12 (specifically, for example, the zoom lens 44) to maximize the focal length of the imaging optical system 38. This allows the field-of-view center alignment unit 116A1 to focus on the telephoto camera 12 even when the depth of field is shallower than the minimum focal length of the imaging optical system 38. Furthermore, if there is a possibility that the distance to the subject being aligned is shorter than the shortest distance at the telephoto end, focusing can be performed while gradually changing the focal length towards the wide-angle side. Therefore, according to this structure, compared to focusing the telephoto camera 12 when the focal length of the imaging optical system 38 is at its shortest, the first subject distance can be derived with high precision based on the focusing result. Furthermore, the technology of the present invention is not limited thereto; a process equivalent to step ST400 can also be applied to the end alignment process. In this case, it can help to derive the distance of the second subject with high accuracy.

[0220] The above examples illustrate controlling the imaging optical system 38 of the telephoto camera 12 to maximize the focal length of the imaging optical system 38 of the telephoto camera 12, but the technology of the present invention is not limited thereto. For example, such as Figure 27 As shown in step ST400, the field-of-view center alignment unit 116A1 can control the imaging optical system 38 of the telephoto camera 12 (specifically, for example, the zoom lens 44) so ​​that the focal length entering the imaging optical system 38 of the telephoto camera 12 is in the region with the longest focal length of the imaging optical system 38 of the telephoto camera 12, among multiple regions that are defined in stages. In this case, compared to focusing the telephoto camera 12 when the focal length of the imaging optical system 38 of the telephoto camera 12 is the shortest, the first subject distance can be derived with high precision based on the focusing result.

[0221] Furthermore, the technology of the present invention is not limited thereto, and the end alignment process can also be applied to a process equivalent to step ST400 (the process of controlling the imaging optical system 38 of the telephoto camera 12 to enter the region of the longest focal length side of the imaging optical system 38 of the telephoto camera 12). In this case, it is possible to help derive the distance of the second subject with high accuracy.

[0222] In the above embodiment, the horizontal offset and the depth offset are exemplified as predetermined values ​​stored in the predetermined value storage area 118A. However, the technology of the present invention is not limited to this. As predetermined values, information related to the height difference between the telephoto camera 12 and the wide-angle camera 14 (e.g., the height difference itself) may also be applied.

[0223] For example, such as Figure 28 As shown, when the telephoto camera 12 and the wide-angle camera 14 have a height difference, the alignment process of the panning direction (horizontal field of view) of the telephoto camera 12 is performed. When the pitch angle of the telephoto camera 12 is set to "θ" so that the center of the telephoto camera's field of view is aligned with a specific subject, the distance from the telephoto camera 12 to the specific subject is set to "Sd1", and the distance from the wide-angle camera 14 to the specific subject is set to "Sd2", the alignment unit 116A derives "Sd1" according to the formula "Sd1=Sd2 / cosθ". The alignment process of the panning direction (horizontal field of view) of the telephoto camera 12 is performed by using the derived "Sd1". Thus, by taking into account the height difference between the telephoto camera 12 and the wide-angle camera 14, the panning direction (horizontal field of view) of the telephoto camera 12 is aligned. Even if there is a height difference between the telephoto camera 12 and the wide-angle camera 14, information related to the field of view required when using the telephoto camera 12 and the wide-angle camera 14 together can be obtained.

[0224] Furthermore, in the technology of the present invention, considering that the alignment processing of the height difference between the telephoto camera 12 and the wide-angle camera 14 is not necessary, the alignment processing can be performed by ignoring the height difference between the telephoto camera 12 and the wide-angle camera 14.

[0225] In the above embodiments, an example is given of aligning one end of the telephoto camera's field of view with one end of the wide-angle camera's field of view by performing end alignment processing by the CPU 116; however, the technology of the present invention is not limited thereto. For example, one end of the telephoto camera's field of view may also be aligned with one end of the wide-angle camera's field of view.

[0226] Here, one end of the field of view refers to the portion outside the center of the field of view. In this case, for example, as... Figure 29 As shown, the half-angle view of the wide-angle camera 14 is calculated using coordinates (hereinafter also referred to as "pixel coordinates"), which determine the pixel position within the pixel row in the pixel group included in the image sensor 70 of the wide-angle camera 14, in the direction corresponding to the panning direction of the telephoto camera 12. Specifically, when the half-angle view of the wide-angle camera 14 is set to "θ e2 “d” represents the distance to a specific subject located outside the center of the field of view (the distance to the second subject). target2 The rotation angle of the telephoto camera 12 from its original position is set to "θ". e3The pixel coordinates corresponding to one end of the wide-angle camera's field of view are set to "0 (origin)", the pixel coordinates corresponding to the position of the specific subject in real space are set to "x1", the pixel coordinates corresponding to the center of the wide-angle camera's field of view are set to "x2", and the horizontal offset is set to "d". x Set the depth direction offset to "d". y When “”, the end alignment part 116A2 calculates the half angle θ according to the following formula (3). e2 .

[0227] [Formula 3] Thus, the location for aligning the telephoto camera image with the wide-angle camera image is not limited to the center and end of the field of view; it can also be a location outside the center of the wide-angle camera's field of view. If the telephoto camera image and the wide-angle camera image are aligned at least two locations, one outside the center of the wide-angle camera's field of view and the other inside the wide-angle camera's field of view, then the half-angle field of view of the wide-angle camera 14 can be determined.

[0228] In the above embodiments, examples are given of the form in which the distance to the first subject and the distance to the second subject are calculated by the CPU 116 based on the focus position, but the technology of the present invention is not limited thereto. For example, either the distance to the first subject or the distance to the second subject can be received by the receiving device 24 or the like, or it can be obtained by ranging by a ranging device using phase difference AF or TOF ranging.

[0229] In the above embodiments, "rotation" is exemplified as panning, but the technology of the present invention is not limited to this, and the technology of the present invention can also be applied to pitch. In this case, for the field of view (vertical field of view) in the pitch direction, it is sufficient for the CPU 116 to perform processing equivalent to the alignment processing described in the above embodiments.

[0230] In the above embodiments, an example of CPU 116 performing alignment processing independent of changes in the scale ratio between the wide-angle camera image and the telephoto camera image has been described. However, it is also possible to consider performing alignment processing by CPU 116 when the scale ratio changes. In this case, for example, if optical information related to the wide-angle camera 14 (e.g., the specifications and focal length of the image sensor 70) is known, CPU 116 can apply the optical information related to the wide-angle camera 14 to limit changes in the scale ratio to a certain extent.

[0231] In the above embodiments, when performing end alignment processing, one end of the telephoto camera's field of view is aligned with one end of the wide-angle camera's field of view. However, the technology of the present invention is not limited to this. When performing end alignment processing, the center of the telephoto camera's field of view can also be aligned with one end of the wide-angle camera's field of view. In this case, it is sufficient to perform alignment image analysis (e.g., matching) using half of the wide-angle camera image and half of the telephoto camera image.

[0232] Furthermore, in Figure 22A In step ST102H1 shown, the opposite end of the telephoto camera's field of view is aligned with the opposite end of the wide-angle camera's field of view. However, the technology of this invention is not limited to this. Figure 22A In step ST102H1 shown, the center of the telephoto camera's field of view can also be aligned with the opposite end of the wide-angle camera's field of view. In this case, it is also possible to obtain... Figure 22A The example shown has the same effect.

[0233] In the above embodiments, an example of alignment processing being performed by the computer 104 of the management device 20 is given, but the technology of the present invention is not limited thereto. Alignment processing can be performed by the control device 60 of the telephoto camera 12, or by the control device 94 of the wide-angle camera 14, or by at least two of the computer 104 of the management device 20, the control device 60 of the telephoto camera 12, and the control device 94 of the wide-angle camera 14.

[0234] Furthermore, the alignment process can be performed by devices including ASICs, FPGAs, and / or PLDs instead of computer 104, or it can be achieved through a combination of hardware and software structures.

[0235] Furthermore, in the above embodiment, in NVM118 (reference) Figure 5 The device stores a bit alignment processing program 158, but the technology of the present invention is not limited thereto. As an example, such as Figure 30 As shown, the alignment processing program 158 can also be stored in any portable storage medium 200, such as an SSD or USB memory, which is a non-temporary storage medium. In this case, the alignment processing program 158 stored in the storage medium 200 is installed in the computer 104, and the CPU 116 executes the alignment processing according to the alignment processing program 158.

[0236] Furthermore, a bit alignment processing program 158 is stored in a storage device such as another computer or server connected to computer 104 via a communication network (not shown). Upon request from management device 20, the bit alignment processing program 158 can be downloaded and installed into computer 104. In this case, the bit alignment processing is performed by computer 104 according to the installed bit alignment processing program 158.

[0237] As the hardware resource for performing the aforementioned bit alignment process, various processors as shown below can be used. For example, a general-purpose processor, i.e., a CPU, can be cited as an example. As described above, this processor functions as a hardware resource for performing bit alignment processing by executing software, i.e., the bit alignment processing program 158. Furthermore, as a processor, for example, a dedicated circuit with a circuit structure specifically designed for performing a particular process, such as an FPGA, PLD, or ASIC, can be cited. Memory is also built into or connected to any processor, and any processor performs bit alignment processing using memory.

[0238] The hardware resources for performing bit alignment can consist of one of these various processors, or a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Furthermore, the hardware resources for performing bit alignment can be a single processor.

[0239] As an example of a single processor, firstly, there exists a configuration where, as exemplified by client and server computers, a combination of one or more CPUs and software constitutes a single processor, which functions as the hardware resource for performing bit-alignment processing. Secondly, there exists a configuration where, as exemplified by a System-on-a-Chip (SoC), a processor is used to implement the functions of an entire system, including multiple hardware resources for performing bit-alignment processing, through a single IC chip. In this way, bit-alignment processing is implemented using one or more of the aforementioned processors as hardware resources.

[0240] Furthermore, as the hardware structure of these various processors, more specifically, they can use circuits composed of circuit elements such as semiconductor elements.

[0241] Furthermore, the above alignment process is just one example. Therefore, it is certainly possible to delete unnecessary steps, add new steps, or switch the processing order without deviating from the main point.

[0242] The above description and illustrations are detailed descriptions of the parts involved in the technology of this invention, and are merely one example of the technology of this invention. For example, the descriptions related to the above structure, function, effect, and effect are examples of the structure, function, effect, and effect of the parts involved in the technology of this invention. Therefore, without departing from the spirit of the technology of this invention, unnecessary parts may be deleted, new elements may be added, or substitutions may be made to the above description and illustrations. Furthermore, in order to avoid complexity and to facilitate understanding of the parts involved in the technology of this invention, descriptions related to common technical knowledge that do not require special explanation in aspects where the technology of this invention can be implemented have been omitted from the above description and illustrations.

[0243] In this specification, "A and / or B" has the same meaning as "at least one of A and B". That is, "A and / or B" means that it can be only A, only B, or a combination of A and B. Furthermore, in this specification, when "and / or" is added to represent more than three items, the same concept as "A and / or B" may also be applied.

[0244] All documents, patent applications and technical standards described in this specification are referenced in this specification to the same extent that each document, patent application and technical standard is specifically and separately described and referenced by reference.

Claims

1. A camera system comprising: processor; A first camera device, having a first optical system; and The second camera device has a second optical system. The focal length of the first optical system is longer than that of the second optical system. The processor performs the following processing: Alignment is performed between the first image captured by the first camera device and the second image captured by the second camera device; Obtain information related to the alignment; and The first camera image, the second camera image, and the information are output to the display.

2. The camera system according to claim 1, wherein, The information is obtained based on the alignment result and is related to the field of view used in the camera.

3. The camera system according to claim 2, wherein, The field of view related information includes: The field of view of the first camera device is the first field of view. The field of view of the second camera device is the second field of view; and The difference information between the first field of view and the second field of view.

4. The camera system according to any one of claims 1 to 3, wherein, The processor displays a marker in the second camera image that overlaps with the area corresponding to the first camera image.

5. The camera system according to any one of claims 1 to 3, wherein, The processor obtains the field of view related information based on the end alignment result as the alignment. The end alignment result is obtained by aligning the first end of the field of view of the first camera device or the center of the field of view of the first camera device with the second end of the field of view of the second camera device.

6. The camera system according to claim 5, wherein, The processor obtains the field of view related information based on the field of view center alignment result, which is obtained by aligning the center of the field of view of the first camera device with the center of the field of view of the second camera device.

7. The camera system according to claim 6, wherein, The processor derives the field-of-view center alignment result and the end alignment result based on position-specific information that determines the positional relationship between the first camera device and the second camera device.

8. The camera system according to claim 7, wherein, The location-specific information includes information about the offset between the first camera device and the second camera device.

9. The camera system according to claim 7 or 8, wherein, The location-specific information includes information relating to the elevation difference between the first camera device and the second camera device.

10. The camera system according to claim 7 or 8, wherein, When the distance to the first subject is less than a first threshold, the processor performs field-of-view center alignment processing to align the center of the field of view of the first camera device with the center of the field of view of the second camera device using the position-specific information. When the distance to the second subject is less than a second threshold, the processor performs end alignment processing to align the center of the field of view of the first end or the first camera device with the second end using the position-specific information.

11. The camera system according to claim 10, wherein, The processor derives at least one of the distance to the first subject and the distance to the second subject.

12. The camera system according to claim 11, wherein, The first camera device has an autofocus function and can be rotatably mounted on a rotary table. The processor performs the following processing: When exporting the subject distance of at least one of the first subject distance and the second subject distance, the subject distance is exported using the autofocus function. If the subject distance cannot be derived using the autofocus function, the subject distance is derived using the autofocus function after the turntable is controlled based on the first camera image and the center of the field of view of the first camera device is aligned with a high-contrast subject near the center of the field of view of the first camera device.

13. The camera system according to claim 11 or 12, wherein, When the processor enters the region with the longest focal length of the first optical system, which is one of the multiple regions defined by the phased division of the focal length of the first optical system, it causes the first imaging device to focus, and derives at least one of the distance to the first subject and the distance to the second subject based on the focusing result.

14. The camera system according to claim 13, wherein, The processor enables the first camera device to focus when the focal length of the first optical system is at its longest, and derives at least one of the distance to the first subject and the distance to the second subject based on the focusing result.

15. The camera system according to claim 10, wherein, The processor performs the end alignment process after performing the field-of-view center alignment process.

16. The camera system according to claim 10, wherein, The processor performs at least one of the field-of-view center alignment processing and the end alignment processing using image analysis based on the first camera image and the second camera image.

17. The camera system according to claim 16, wherein, In the end alignment process, the processor aligns the center of the field of view of the first end or the first camera device with the second end based on the scale ratio of the first camera image and the second camera image used in the image analysis in the field of view center alignment process.

18. The camera system according to claim 10, wherein, Regarding the processor... Furthermore, an opposite-side end alignment process is performed, which involves aligning the first opposite-side end located on the opposite side of the first end among the two ends of the field of view of the first camera device, or the center of the field of view of the first camera device with the second opposite-side end located on the opposite side of the second end among the two ends of the field of view of the second camera device. and Based on at least one of the following: a first result in which the position of the center of the field of view of the first end or the first camera device is aligned with the position of the second end in the end alignment process; and a second result in which the position of the center of the field of view of the first opposite end or the first camera device is aligned with the position of the second opposite end in the opposite end alignment process; the field of view of the second camera device is obtained as the field of view related information.

19. The camera system according to claim 18, wherein, When the distance to the third subject is greater than or equal to the third threshold, and the position of the center of the field of view of the first camera device or the first camera device is aligned with the position of the second camera device, the processor obtains the field of view of the second camera device as the field of view related information based on the second result.

20. The camera system according to any one of claims 1 to 3, wherein, The optical information related to the first optical system is known. The optical information related to the second optical system is unknown.

21. The camera system according to any one of claims 1 to 3, wherein, The focal length of the first optical system is more than twice the focal length of the second optical system.

22. A camera system comprising: a processor; and a first camera device, which is used in conjunction with a second camera device having a second optical system and also having the first optical system. The focal length of the first optical system is longer than that of the second optical system. The processor performs the following processing: at at least two locations including the end of the field of view used in the shooting based on the camera system, the first camera image obtained by the first camera device and the second camera image obtained by the second camera device are aligned; Obtain information related to the alignment; and The first camera image, the second camera image, and the information are output to the display.

23. A method of operating a camera system, the camera system comprising: a processor; a first camera device having a first optical system; and a second camera device having a second optical system, wherein the focal length of the first optical system is longer than the focal length of the second optical system, the method of operating the camera system comprising the following steps: Alignment is performed at at least two locations, including the end of the field of view used in the shooting based on the camera system, between a first camera image obtained by the first camera device and a second camera image obtained by the second camera device; Obtain information related to the alignment; and The first camera image, the second camera image, and the information are output to the display.

24. A program product for causing a computer to perform processing, the computer comprising a processor, a first imaging device having a first optical system and a second imaging device having a second optical system, and adapted to an imaging system in which the focal length of the first optical system is longer than the focal length of the second optical system, the processing comprising the following steps: Alignment is performed at at least two locations, including the end of the field of view used in the shooting based on the camera system, between a first camera image obtained by the first camera device and a second camera image obtained by the second camera device; Obtain information related to the alignment; and The first camera image, the second camera image, and the information are output to the display.

Images