Imaging system, method of operation of an imaging system, and storage medium

Abstract

The present application relates to an imaging system, an operation method of the imaging system, and a storage medium. The imaging system includes a processor, a first imaging device having a first optical system, and a second imaging device having a second optical system. The focal length of the first optical system is longer than the focal length of the second optical system. The processor performs the following processing: at at least two positions including end portions of a field angle used in photographing by the imaging system, performing alignment of a first imaging image obtained by photographing by the first imaging device and a second imaging image obtained by photographing by the second imaging device; and acquiring field angle-related information related to the field angle based on the alignment result.

Description

TECHNICAL FIELD

[0001] The technology of the present application relates to an imaging system, a method of operation of an imaging system, and a storage medium BACKGROUND

[0002] Japanese Patent No. 5610867 discloses an imaging device that has a first imaging optical system and a second imaging optical system having a different angle of view from the first imaging optical system, the first imaging optical system and the second imaging optical system each having a focus lens for focusing. The imaging device described in Japanese Patent No. 5610867 has a subject distance calculation member that calculates a distance to a main subject determined as an imaging target by focusing operation of the focus lenses of the first imaging optical system and the second imaging optical system, and a control member that controls focusing of the focus lenses to control focusing on the main subject based on the distance to the main subject calculated by the subject distance calculation member.

[0003] In the imaging device described in Japanese Patent No. 5610867, in a case where the main subject exists within the angle of view of the first imaging optical system, the control member controls focusing on the main subject by driving the focus lenses of the first imaging optical system and the second imaging optical system based on the distance to the main subject calculated by the subject distance calculation member by focusing operation of the first imaging optical system. Also, in a case where the main subject exists outside the angle of view of the first imaging optical system and within a determination region of the second imaging optical system including a range corresponding to the angle of view of the first imaging optical system, the control member controls focusing on the main subject by driving the focus lenses of the first imaging optical system and the second imaging optical system based on the distance to the main subject calculated by the subject distance calculation member by focusing operation of the second imaging optical system. Further, in a case where the main subject exists outside the angle of view of the first imaging optical system and outside the determination region, the control member performs notification control of this content.

[0004] Japanese Patent No. 2007-081473 discloses an imaging device having a plurality of optical systems including a first optical system having a relatively wide angle of view, a second optical system having a relatively narrow angle of view, which is arranged separately from the first optical system, and a display member that displays an image obtained by the first optical system. The imaging device having a plurality of optical systems described in Japanese Patent No. 2007-081473 has a first operation member that inputs a desired angle of view, a display control member that cuts out a display of a photographing region calculated in consideration of parallax between the desired angle of view input from the first operation member and a distance between optical axes of the first and second optical systems from an image displayed on the display member, a second operation member that inputs a photographing instruction, and a storage member that stores an image obtained by the second optical system corresponding to the photographing region as a photographed image in accordance with the photographing instruction input from the second operation member. SUMMARY

[0005] One embodiment of the present application provides an imaging system capable of obtaining information about a desired angle of view when a first imaging device and a second imaging device having different focal lengths of optical systems are used in cooperation, a method for operating the imaging system, and a storage medium.

[0006] Means for solving technical problems

[0007] A first aspect of the present technology is an imaging system including a processor, a first imaging device having a first optical system, and a second imaging device having a second optical system, the focal length of the first optical system being longer than the focal length of the second optical system, the processor performing: alignment of a first captured image obtained by capturing by the first imaging device and a second captured image obtained by capturing by the second imaging device at at least two positions including end portions of an angle of view used in photographing by the imaging system; and acquisition of angle-of-view-related information about the angle of view based on a result of the alignment.

[0008] A second aspect of the present technology is the imaging system according to the first aspect, in which the processor acquires the angle-of-view-related information based on an end portion alignment result obtained by alignment of a first end portion of the angle of view of the first imaging device or a second end portion of the angle of view of the first imaging device on an end portion side of the angle of view of the first imaging device and both end portions of the angle of view of the second imaging device.

[0009] A third aspect of the present technology is the imaging system according to the second aspect, in which the processor acquires the angle-of-view-related information based on an angle-of-view-center alignment result obtained by alignment of a center of the angle of view of the first imaging device and a center of the angle of view of the second imaging device.

[0010] The present technology is directed to a camera system according to any one of the fourth to sixth aspects, wherein the processor derives the center-to-center alignment result and the end alignment result based on position specification information that specifies a positional relationship between the first imaging device and the second imaging device.

[0011] The present technology is directed to a camera system according to the fourth aspect, wherein the position specification information is information that includes an offset amount of the first imaging device from the second imaging device.

[0012] The present technology is directed to a camera system according to any one of the fourth to sixth aspects, wherein the position specification information includes information related to a height difference between the first imaging device and the second imaging device.

[0013] The present technology is directed to a camera system according to any one of the fourth to sixth aspects, wherein the processor performs center-to-center alignment processing of the field angles of the first imaging device and the second imaging device using the position specification information when the first subject distance is less than a first threshold value, and performs end alignment processing of the first end or the center of the field angle of the first imaging device and the second end using the position specification information when the second subject distance is less than a second threshold value.

[0014] The present technology is directed to a camera system according to the seventh aspect, wherein the processor derives at least one of the first subject distance and the second subject distance.

[0015] The present technology is directed to a camera system according to the eighth aspect, wherein the first imaging device has an auto-focusing function and is rotatably provided on a turntable, and the processor performs processing of deriving the subject distance using the auto-focusing function when at least one of the first subject distance and the second subject distance is derived, and deriving the subject distance using the auto-focusing function after aligning the center of the field angle of the first imaging device with a subject having a high contrast in the vicinity of the center of the field angle of the first imaging device by controlling the turntable based on the first imaging image when the derivation of the subject distance using the auto-focusing function fails.

[0016] The present technology is directed to a camera system according to any one of the eighth to ninth aspects, wherein the processor causes the first imaging device to perform focusing in a state where the first imaging device is in a region where a focal length of the first optical system is the longest among regions determined by the focal length of the first optical system being divided in stages, and derives at least one of the first subject distance and the second subject distance based on a result of the focusing.

[0017] The eleventh aspect of the present technology is the imaging system according to the tenth aspect, wherein the processor performs focusing of the first imaging device in a state where the focal length of the first optical system is the longest, and derives at least one of the first subject distance and the second subject distance based on a result of the focusing.

[0018] The twelfth aspect of the present technology is the imaging system according to any one of the seventh to eleventh aspects, wherein the processor performs the end portion alignment processing after the field angle center alignment processing.

[0019] The thirteenth aspect of the present technology is the imaging system according to any one of the seventh to twelfth aspects, wherein the processor performs at least one of the field angle center alignment processing and the end portion alignment processing using image analysis based on the first imaging image and the second imaging image.

[0020] The fourteenth aspect of the present technology is the imaging system according to the thirteenth aspect, wherein in the end portion alignment processing, the processor performs alignment of the center of the field angle of the first end portion or the first imaging device with the second end portion based on a scale ratio of the first imaging image and the second imaging image used in the image analysis in the field angle center alignment processing.

[0021] The fifteenth aspect of the present technology is the imaging system according to any one of the seventh to fourteenth aspects, wherein with respect to the processor, opposite end portion alignment processing is further performed, the opposite end portion alignment processing performs alignment of a first opposite end portion of the two end portions of the field angle of the first imaging device located on an opposite side of the first end portion or the center of the field angle of the first imaging device with a second opposite end portion of the two end portions of the field angle of the second imaging device located on an opposite side of the second end portion, and the field angle of the second imaging device is acquired as the field angle related information based on at least one of a first result in which the position of the center of the field angle of the first end portion or the first imaging device is aligned with the position of the second end portion in the end portion alignment processing and a second result in which the position of the first opposite end portion or the center of the field angle of the first imaging device is aligned with the position of the second opposite end portion in the opposite end portion alignment processing.

[0022] The sixteenth aspect of the present technology is the imaging system according to the fifteenth aspect, wherein in a case where the position of the center of the field angle of the first opposite end portion or the first imaging device is aligned with the position of the second opposite end portion in a condition where the third subject distance is equal to or greater than the third threshold value, the processor acquires the field angle of the second imaging device as the field angle related information based on the second result.

[0023] The present technology relates to a camera system according to any one of the first to sixteenth aspects, wherein the optical information relating to the first optical system is known and the optical information relating to the second optical system is unknown.

[0024] The present technology relates to a camera system according to any one of the first to seventeenth aspects, wherein the focal length of the first optical system is more than twice the focal length of the second optical system.

[0025] The present technology relates to a camera system according to the nineteenth aspect, wherein the processor performs: alignment of a first captured image obtained by capturing by the first camera and a second captured image obtained by capturing by the second camera at at least two positions including an end portion of a field angle used in capturing by the camera system; and acquisition of field angle-related information relating to the field angle based on a result of the alignment.

[0026] The present technology relates to a method of operating a camera system according to the twentieth aspect, wherein the camera system includes: a processor; a first camera including a first optical system; and a second camera including a second optical system, the focal length of the first optical system being longer than the focal length of the second optical system, the method of operating the camera system including: alignment of a first captured image obtained by capturing by the first camera and a second captured image obtained by capturing by the second camera at at least two positions including an end portion of a field angle used in capturing by the camera system; and acquisition of field angle-related information relating to the field angle based on a result of the alignment.

[0027] The present technology relates to a program for causing a computer to execute a process, the computer including a processor, a first camera including a first optical system, and a second camera including a second optical system, and being applicable to a camera system in which the focal length of the first optical system is longer than the focal length of the second optical system, the program causing the computer to execute a process including: alignment of a first captured image obtained by capturing by the first camera and a second captured image obtained by capturing by the second camera at at least two positions including an end portion of a field angle used in capturing by the camera system; and acquisition of field angle-related information relating to the field angle based on a result of the alignment. BRIEF DESCRIPTION OF DRAWINGS

[0028] Figure 1 is a schematic perspective view showing an example of an external structure of a camera system.

[0029] Figure 2is a conceptual diagram showing an example of a structure of a long-focus camera, a wide-angle camera, and a two-axis turntable included in an imaging system.

[0030] Figure 3 is a conceptual diagram showing an example of a hardware structure of an electrical system of a management device, a long-focus camera, a wide-angle camera, and a two-axis turntable included in an imaging system.

[0031] Figure 4 is a conceptual diagram showing an example of a hardware structure of an electrical system of a long-focus camera and a wide-angle camera included in an imaging system.

[0032] Figure 5 is a block diagram showing an example of a manner in which a CPU in a management device performs a registration process according to a registration process program.

[0033] Figure 6 is a functional block diagram showing an example of functions of a CPU of a management device.

[0034] Figure 7 is a conceptual diagram showing an example of processing content in which a long-focus camera image and a wide-angle camera image obtained by imaging an imaging target region by the long-focus camera and the wide-angle camera are displayed on a display.

[0035] Figure 8 is a conceptual diagram showing an example of processing content by a field-of-view angle center registration section with respect to a wide-angle camera, and processing content by a control section.

[0036] Figure 9 is a conceptual diagram showing an example of processing content by a field-of-view angle center registration section with respect to a long-focus camera, and processing content by a control section.

[0037] Figure 10 is a conceptual diagram for explaining a method of deriving a first subject distance.

[0038] Figure 11 is a conceptual diagram for explaining a first rotation angle.

[0039] Figure 12 is a conceptual diagram for explaining a method of calculating a first rotation angle.

[0040] Figure 13 is a conceptual diagram showing an example of processing content by an end portion center registration section with respect to a wide-angle camera, and processing content by a control section.

[0041] Figure 14 is a conceptual diagram showing an example of processing content by an end portion center registration section with respect to a long-focus camera, and processing content by a control section.

[0042] Figure 15is a conceptual diagram for explaining a derivation method of the 2nd subject distance.

[0043] Figure 16 is a conceptual diagram for explaining the 1st rotation angle and the half view angle.

[0044] Figure 17 is a conceptual diagram for explaining a calculation method of the half view angle.

[0045] Figure 18 is a conceptual diagram showing an example of the processing contents of the acquisition section and the control section.

[0046] Figure 19 is a flowchart showing an example of the flow of the alignment processing.

[0047] Figure 20 is a flowchart showing an example of the flow of the field angle center alignment processing.

[0048] Figure 21 is a flowchart showing an example of the flow of the end alignment processing.

[0049] Figure 22A is a flowchart showing a 1st modification example of the flow of the end alignment processing.

[0050] Figure 22B is Figure 22A a continuation of the flowchart shown in FIG.

[0051] Figure 23 is a conceptual diagram showing a modification example of the processing contents of the acquisition section.

[0052] Figure 24 is a flowchart showing a 1st modification example of the flow of the field angle center alignment processing.

[0053] Figure 25 is a flowchart showing a 2nd modification example of the flow of the end alignment processing.

[0054] Figure 26 is a flowchart showing a 2nd modification example of the flow of the field angle center alignment processing.

[0055] Figure 27 is a flowchart showing a 3rd modification example of the flow of the field angle center alignment processing.

[0056] Figure 28 is a conceptual diagram showing an example of the method when the alignment is performed taking into account the height difference between the tele camera and the wide camera.

[0057] Figure 29 is a conceptual diagram for explaining a modification example of the calculation method of the half view angle.

[0058] Figure 30is a conceptual diagram showing an example of a manner in which a registration program is installed in a computer of a management device from a storage medium in which the registration program is stored. DETAILED DESCRIPTION

[0059] Hereinafter, an example of an embodiment of a camera system, a method of operation of the camera system, and a storage medium to which the present technology is applied will be described with reference to the drawings.

[0060] First, the words used in the following explanation will be explained.

[0061] CPU is an abbreviation for "Central Processing Unit". RAM is an abbreviation for "Random Access Memory". DRAM is an abbreviation for "Dynamic Random Access Memory". SRAM is an abbreviation for "Static Random Access Memory". NVM is an abbreviation for "Non-Volatile Memory". SSD is an abbreviation for "Solid State Drive". HDD is an abbreviation for "Hard Disk Drive". EEPROM is an abbreviation for "Electrically Erasable and Programmable Read Only Memory". USB is an abbreviation for "Universal Serial Bus". ASIC is an abbreviation for "Application Specific Integrated Circuit". PLD is an abbreviation for "Programmable Logic Device". FPGA is an abbreviation for "Field-Programmable Gate Array". SoC is an abbreviation for "System-on-a-chip". CMOS is an abbreviation for "Complementary Metal Oxide Semiconductor". CCD is an abbreviation for "Charge Coupled Device". EL is an abbreviation for "Electro-Luminescence". I / F is an abbreviation for "Interface". fps is an abbreviation for "frames per second". TOF is an abbreviation for "Time-of-Flight". AF is an abbreviation for "Auto Focus". Hereinafter, for the convenience of explanation, a CPU is exemplified as an example of the "processor" to which the present technology pertains, but the "processor" to which the present technology pertains can also be a combination of a plurality of processing devices like a CPU and a GPU. As an example of the "processor" to which the present technology pertains, in the case where a combination of a CPU and a GPU is applied, the GPU acts under the control of the CPU and undertakes execution of image processing.

[0062] Furthermore, in this specification, "orthogonal" means, in addition to being perfectly orthogonal, orthogonal in the sense of errors generally permissible in the technical field to which this invention pertains and permissible without departing from the scope of this invention. Also, in this specification, "perpendicular" means, in addition to being perfectly perpendicular, perpendicular in the sense of errors generally permissible in the technical field to which this invention pertains and permissible without departing from the scope of this invention. Furthermore, in this specification, "consistent" means, in addition to being perfectly consistent, consistent in the sense of errors generally permissible in the technical field to which this invention pertains and permissible without departing from the scope of this invention. And, in the following description, the numerical range indicated by "~" refers to the range including the values ​​described before and after "~" as both the lower and upper limits.

[0063] As an example, such as Figure 1 As shown, the camera system 10 includes a telephoto camera 12, a wide-angle camera 14, a dual-axis rotary table 16, a dual-axis rotary table 18, and a management device 20. Here, the telephoto camera 12 is an example of the "first camera device" according to the present invention, the wide-angle camera 14 is an example of the "second camera device" according to the present invention, and the dual-axis rotary table 16 is an example of the "rotary table" according to the present invention.

[0064] The management device 20 manages the telephoto camera 12, the wide-angle camera 14, the dual-axis rotary table 16, and the dual-axis rotary table 18. Hereinafter, for ease of explanation, the telephoto camera 12, the wide-angle camera 14, the dual-axis rotary table 16, and the dual-axis rotary table 18 will not be marked with symbols and will be referred to as "managed object devices" without distinction.

[0065] The management device 20 includes a management device main body 22, a receiving device 24, and a display 26. The management device main body 22 includes a computer 104 (reference). Figure 3 The management unit 22 controls the entire camera system 10. A receiver 24 and a display 26 are connected to the management unit 22. The receiver 24 receives various instructions from the user of the camera system 10. Examples of receiver 24 include a keyboard, mouse, and / or touch panel. The various instructions received by the receiver 24 are controlled by the management unit 22. The display 26 displays various information (e.g., images and text) under the control of the management unit 22. Examples of display 26 include a liquid crystal display (LCD) or an EL display.

[0066] The managed object device can be communicatively connected to the management device main body 22 and operates under the control of the management device main body 22. The connection between the managed object device and the management device main body 22 can be either wired or wireless.

[0067] The telephoto camera 12, also known as a long-range camera, is a digital camera with a longer focal length than the wide-angle camera 14. The telephoto camera 12 is rotatably mounted on a dual-axis rotary table 16. The management device 20 controls the panning or tilting of the telephoto camera 12 by operating the dual-axis rotary table 16. The dual-axis rotary table 16 has a panning axis PA1 and a tilting axis TA1. Under the control of the management device 20, panning of the telephoto camera 12 is achieved by rotating it around the panning axis PA1, and tilting of the telephoto camera 12 is achieved by rotating it around the tilting axis TA1.

[0068] Wide-angle camera 14 can capture a wider field of view than telephoto camera 12 (see reference). Figure 7 A digital camera. A wide-angle camera 14 is rotatably mounted on a dual-axis rotary table 18. The management device 20 operates the dual-axis rotary table 18 to pan or tilt the wide-angle camera 14. That is, the dual-axis rotary table 18 has a pan axis PA2 and a tilt axis TA2. Under the control of the management device 20, panning of the wide-angle camera 14 is achieved by rotating it around the pan axis PA2, and tilting of the wide-angle camera 14 is achieved by rotating it around the tilt axis TA2.

[0069] The main body 22 of the management device acquires image data obtained by the telephoto camera 12, i.e., telephoto camera image data, and acquires image data obtained by the wide-angle camera 14, i.e., wide-angle camera image data.

[0070] As an example, such as Figure 2 As shown, the telephoto camera 12 includes a lens barrel 28 and a telephoto camera body 30. A lens barrel side mount 32 is provided on the lens barrel 28, and a body side mount 34 is provided on the telephoto camera body 30. The lens barrel 28 is mounted to the telephoto camera body 30 via the lens barrel side mount 32 and the body side mount 34. A camera optical system 38 is built into the lens barrel 28, and an image sensor 36 is built into the telephoto camera body 30.

[0071] The imaging optical system 38 is an example of the "first optical system" involved in the present invention. The imaging optical system 38 images light representing a subject onto an image sensor 36. The imaging optical system 38 has various lenses. Specifically, the imaging optical system 38 has an objective lens 40, a focusing lens 42, a zoom lens 44, and a main lens 46. Although in Figure 2 The example is simplified, but each of the objective lens 40, focusing lens 42, zoom lens 44 and main lens 46 is actually a lens group that combines multiple lenses.

[0072] The objective lens 40, the focus lens 42, the zoom lens 44, and the main lens 46 are arranged in this order from the object side (subject side) to the image side (image sensor 36 side). The imaging optical system 38 also has an aperture 48. The aperture 48 is arranged between the zoom lens 44 and the main lens 46.

[0073] A focus lens driving mechanism 50, a zoom lens driving mechanism 52, and an aperture driving mechanism 54 are provided in the lens barrel 28. The focus lens driving mechanism 50 includes a focus cam ring (omitted from the drawing) formed with a cam groove on the outer periphery, a focus motor (omitted from the drawing), and the like. The focus lens driving mechanism 50 holds the focus lens 42. The focus cam ring rotates around the optical axis OAl of the imaging optical system 38 by receiving power from the focus motor. The focus lens 42 moves along the optical axis OAl by receiving a pushing force in the direction of the optical axis OAl that is generated in conjunction with the rotation of the focus cam ring around the optical axis OAl.

[0074] The zoom lens driving mechanism 52 includes a zoom cam ring (omitted from the drawing) formed with a cam groove on the outer periphery, a zoom motor (omitted from the drawing), and the like. The zoom lens driving mechanism 52 holds the zoom lens 44. The zoom cam ring rotates around the optical axis OAl by receiving power from the zoom motor. The zoom lens 44 moves along the optical axis OAl by receiving a pushing force in the direction of the optical axis OAl that is generated in conjunction with the rotation of the zoom cam ring around the optical axis OAl.

[0075] The aperture driving mechanism 54 includes an aperture motor (omitted from the drawing), and the like. The aperture 48 has a plurality of blades that are openable and closable. The plurality of blades are opened and closed by receiving power from the aperture motor.

[0076] A lens barrel side contact 56 is provided on the lens barrel side mount 32. The above-mentioned stepping motor and the like electrical components are assembled on each of the focus lens driving mechanism 50, the zoom lens driving mechanism 52, and the aperture driving mechanism 54. The electrical components of the focus lens driving mechanism 50, the zoom lens driving mechanism 52, and the aperture driving mechanism 54 are connected to the lens barrel side contact 56.

[0077] A body side contact 58 is provided on the body side mount 34. In the body side mount 34, the body side contact 58 is located at a position corresponding to the position of the lens barrel side contact 56 on the lens barrel side mount 32.

[0078] A control device 60 is built in the tele camera body 30, and the control device 60 is connected to the body-side contact 58. In the case where the lens barrel 28 is attached to the tele camera body 30 via the lens barrel-side mount 32 and the body-side mount 34, the lens barrel-side contact 56 comes into contact with the body-side contact 58. Thus, the respective electric components of the focus lens driving mechanism 50, the zoom lens driving mechanism 52, and the aperture driving mechanism 54 are electrically connected to the control device 60.

[0079] The control device 60 controls the respective electric components of the focus lens driving mechanism 50, the zoom lens driving mechanism 52, and the aperture driving mechanism 54.

[0080] The control device 60 monitors the driving amount of the focus motor and the driving amount of the zoom motor. The control device 60 derives the position of the focus lens 42 on the optical axis OA1 from the driving amount of the focus motor, and derives the position of the zoom lens 44 on the optical axis OA1 from the driving amount of the zoom motor.

[0081] In the tele camera 12, the adjustment of the focus position is performed by the focus lens 42. Thus, on the light-receiving surface 36A of the image sensor 36, the subject light is imaged on the focus position corresponding to the distance to the subject by the focus lens 42 moving in the direction of the optical axis OA1. The "focus position" as referred to herein means the position of the focus lens 42 on the optical axis OA1 in the state of focus. The focus position is derived by the control device 60 from the driving amount of the focus motor. The control device 60 performs the control of aligning the focus lens 42 to the focus position by operating the focus lens driving mechanism 50. Hereinafter, the control of aligning the focus lens 42 to the focus position will also be referred to as "AF control" for convenience of explanation.

[0082] The image sensor 36 is configured such that the center of the light-receiving surface 36A coincides with the optical axis OA1 and the light-receiving surface 36A is orthogonal to the optical axis OA1. In addition, as an example of the image sensor 36, a CMOS image sensor or a CCD image sensor, or the like can be given.

[0083] The image sensor 36 is driven under the control of the control device 60. The image sensor 36 photographs the subject at a prescribed frame rate (for example, 30 fps) in accordance with the timing signal supplied from the control device 60. The control device 60 acquires the image data, that is, the tele camera image data, obtained by the image sensor 36 photographing the subject from the image sensor 36.

[0084] In addition, in the case where the tele camera 12 is used as a still camera, the control device 60 performs the control of the focus lens driving mechanism 50, the zoom lens driving mechanism 52, and the aperture driving mechanism 54 in accordance with the operation of the shutter button 20. Figure 2In the example shown, an example of the structure of the wide-angle camera 14 is also shown. The wide-angle camera 14 has a lens barrel 62 and a wide-angle camera body 64. The lens barrel 62 corresponds to the lens barrel 28 of the telephoto camera 12, and the wide-angle camera body 64 corresponds to the telephoto camera body 30 of the telephoto camera 12. A lens barrel-side mount 66 is provided on the lens barrel 62, and a body-side mount 68 is provided on the wide-angle camera body 64. In the wide-angle camera 14, the lens barrel-side mount 66 corresponds to the lens barrel-side mount 32 of the telephoto camera 12, and the body-side mount 68 corresponds to the body-side mount 34 of the telephoto camera 12. A camera optical system 72 is built into the lens barrel 62, and an image sensor 70 is built into the wide-angle camera body 64. In the wide-angle camera 14, the image sensor 70 corresponds to the image sensor 36 of the telephoto camera 12. A light-receiving surface 70A is formed on the image sensor 70. In the wide-angle camera 14, the light-receiving surface 70A corresponds to the light-receiving surface 36A of the telephoto camera 12.

[0085] The basic structure of the wide-angle camera 14 is the same as that of the telephoto camera 12 except for the focal lengths. Here, "different focal lengths" means that the focal length of the camera optical system 38 of the telephoto camera 12 is longer than the focal length of the camera optical system 72 of the wide-angle camera 14. That the focal length of the camera optical system 38 of the telephoto camera 12 is longer than the focal length of the camera optical system 72 of the wide-angle camera 14 means that, in terms of the relationship of the focal lengths between the telephoto camera 12 and the wide-angle camera 14, the size relationship "focal length of camera optical system 38 of telephoto camera 12 ≥ focal length of camera optical system 72 of wide-angle camera 14 x 2" holds.

[0086] In addition, on the premise that the focal length of the camera optical system 38 of the telephoto camera 12 is set to be more than twice the focal length of the camera optical system 72 of the wide-angle camera 14, the actual focal length of the camera optical system 72 of the wide-angle camera 14 is preferably 6 mm to 20 mm, and the actual focal length of the camera optical system 38 of the telephoto camera 12 is preferably 12.5 mm to 2000 mm. Also, the ratio of the focal lengths (= (focal length of camera optical system 38 of telephoto camera 12) / (focal length of camera optical system 72 of wide-angle camera 14)) is preferably 2 times to 333 times.

[0087] The imaging optical system 72 is an example of the "second optical system" according to the present technology, and has an optical axis OA2, an objective lens 74, a focus lens 76, a main lens 80, and an aperture 82. In the wide-angle camera 14, the optical axis OA2 corresponds to the optical axis OA1 of the telephoto camera 12. In the wide-angle camera 14, the objective lens 74 corresponds to the objective lens 40 of the telephoto camera 12. In the wide-angle camera 14, the focus lens 76 corresponds to the focus lens 42 of the telephoto camera 12. In the wide-angle camera 14, the main lens 80 corresponds to the main lens 46 of the telephoto camera 12. In the wide-angle camera 14, the aperture 82 corresponds to the aperture 48 of the telephoto camera 12.

[0088] A focus lens driving mechanism 84, a zoom lens driving mechanism 86, and an aperture driving mechanism 88 are provided in the lens barrel 62. In the wide-angle camera 14, the focus lens driving mechanism 84 corresponds to the focus lens driving mechanism 50 of the telephoto camera 12. In the wide-angle camera 14, the zoom lens driving mechanism 86 corresponds to the zoom lens driving mechanism 52 of the telephoto camera 12. In the wide-angle camera 14, the aperture driving mechanism 88 corresponds to the aperture driving mechanism 54 of the telephoto camera 12.

[0089] A lens barrel side contact 90 is provided on the lens barrel side mount 66. A main body side contact 92 is provided on the main body side mount 68. In the wide-angle camera 14, the lens barrel side contact 90 corresponds to the lens barrel side contact 56 of the telephoto camera 12. In the wide-angle camera 14, the main body side contact 92 corresponds to the main body side contact 58 of the telephoto camera 12.

[0090] A control device 94 is built in the wide-angle camera main body 64. In the wide-angle camera 14, the control device 94 corresponds to the control device 60 of the telephoto camera main body 30. The control device 94 acquires image data, i.e., wide-angle camera image data, obtained by imaging an object by the image sensor 70.

[0091] The two-axis turntable 16 is provided with a pan mechanism 96 and a tilt mechanism 98. The pan mechanism 96 has a telephoto camera side pan motor 96A. The telephoto camera side pan motor 96A is driven under the control of the management device 20 (refer to FIG. 1). In the case where the telephoto camera 12 is provided on the two-axis turntable 16, the pan mechanism 96 causes the telephoto camera 12 to pan by transmitting the driving force of the telephoto camera side pan motor 96A to the telephoto camera 12 as a power around a pan axis PA1 (refer to FIG. 1). Figure 1 ) as a power around a pan axis PA1 (refer to Figure 1 ) as a power around a pan axis PA1 (refer to Figure 1driven under the control of the management device 20 (refer to FIG. 1). In the case where the long-focus camera 12 is provided on the two-axis turntable 16, the tilt mechanism 98 causes the long-focus camera 12 to tilt by transmitting the driving force of the long-focus camera-side tilt motor 96B as a power around the tilt axis TA1 (refer to FIG. 1) to the long-focus camera 12. Figure 1

[0092] The two-axis turntable 18 is provided with a pan mechanism 100 and a tilt mechanism 102. The pan mechanism 100 has a wide-angle camera-side pan motor 100A. The wide-angle camera-side pan motor 100A is driven under the control of the management device 20 (refer to FIG. 1). In the case where the wide-angle camera 14 is provided on the two-axis turntable 18, the pan mechanism 100 causes the wide-angle camera 14 to pan by transmitting the driving force of the wide-angle camera-side pan motor 100A as a power around the pan axis PA2 (refer to FIG. 1) to the wide-angle camera 14. Figure 1 Figure 1 Figure 1 Figure 3

[0093] As an example, as shown in FIG. 1, the management device main body 22 is provided with a computer 104. Also, the management device main body 22 is provided with communication I / F's 106, 108, 110A, 110B, 112A, and 112B. Figure 3

[0094] The computer 104, the CPU 116, the NVM 118, and the RAM 120. The computer 104 is an example of the "computer suitable for an image pickup system" according to the present technology, and the CPU 116 is an example of the "processor" according to the present technology.

[0095] The CPU 116, the NVM 118, and the RAM 120 are connected to the bus 114. In the example shown in FIG. 1, one bus is illustrated as the bus 114 for the sake of convenience of illustration, but a plurality of buses can also be used. The bus 114 can be a serial bus, or a parallel bus including a data bus, an address bus, and a control bus, and the like. Figure 4

[0096] ​​​​​​​The NVM 118 stores various parameters and various programs. As an example of the NVM 118, various nonvolatile storage devices such as an EEPROM, an SSD, and / or an HDD can be given. The RAM 120 temporarily stores various information and functions as a work memory. As an example of the RAM 120, a DRAM or an SRAM can be given.

[0097] Various programs are stored in the NVM 118. The CPU 116 reads a required program from the NVM 118 and executes the read program on the RAM 120. The CPU 116 controls the entire imaging system 10 including the management device 20 in accordance with the program executed on the RAM 120.

[0098] The receiver 24 and the display 26 are connected on the bus 114, the CPU 116 grasps an instruction accepted by the receiver 24 and causes the display 26 to display various information.

[0099] The tele camera 12 is provided with a communication I / F 122 connected to the control device 60. Also, the wide camera 14 is provided with a communication I / F 124 connected to the control device 94.

[0100] The two-axis turntable 16 is provided with communication I / Fs 126 and 128. The communication I / F 126 is connected to the pan mechanism 96, and the communication I / F 128 is connected to the tilt mechanism 98. Also, the two-axis turntable 18 is provided with communication I / Fs 130 and 132. The communication I / F 130 is connected to the pan mechanism 100, and the communication I / F 132 is connected to the tilt mechanism 102.

[0101] The communication I / Fs 106, 108, 110A, 110B, 112A, and 112B are connected on the bus 114. The communication I / F 106 is connected to the communication I / F 122 of the tele camera 12, and the CPU 116 transmits and receives information between the communication I / Fs 106 and 122. The communication I / F 108 is connected to the communication I / F 124 of the wide camera 14, and the CPU 116 transmits and receives information between the communication I / Fs 108 and 124.

[0102] The communication I / F 110A is connected to the communication I / F 126 of the two-axis turntable 16, and the CPU 116 controls electrical system devices (for example, a motor 96A for pan on the tele camera side and the like) of the pan mechanism 96 via the communication I / Fs 110A and 126. The communication I / F 112A is connected to the communication I / F 128 of the two-axis turntable 16, and the CPU 116 controls electrical system devices (for example, a motor 96B for tilt on the tele camera side and the like) of the tilt mechanism 98 via the communication I / Fs 112A and 128.

[0103] The communication I / F 110B is connected to the communication I / F 130 of the two-axis turntable 18, and the CPU 116 controls the electrical system devices of the pan mechanism 100 (e.g., the wide-angle camera side pan motor 100A, etc.) via the communication I / F 110B and 130. The communication I / F 112B is connected to the communication I / F 132 of the two-axis turntable 18, and the CPU 116 controls the electrical system devices of the tilt mechanism 102 (e.g., the wide-angle camera side tilt motor 100B, etc.) via the communication I / F 112B and 132.

[0104] As an example, as shown in FIG. 6, in the wide-angle camera 10, the control device 60 is provided with a computer 124 and an image memory 126. The computer 124 has a CPU 128, an NVM 130, and a RAM 132. Figure 4

[0105] The CPU 128, the NVM 130, and the RAM 132 are connected to a bus 134. In the example shown, one bus is illustrated as the bus 134 for convenience of illustration, but a plurality of buses can also be used. The bus 114 can be a serial bus, or a parallel bus including a data bus, an address bus, and a control bus, etc. Figure 5

[0106] The NVM 130 stores various parameters and various programs. As an example of the NVM 130, various nonvolatile storage devices such as an EEPROM, an SSD, and / or an HDD, etc. can be given. The RAM 132 temporarily stores various information and functions as a work memory. As an example of the RAM 132, a DRAM or an SRAM, etc. can be given.

[0107] Various programs are stored in the NVM 130. The CPU 128 reads a desired program from the NVM 130 and executes the read program on the RAM 132. The CPU 128 controls the entire wide-angle camera 10 in accordance with the program executed on the RAM 132.

[0108] The image sensor 36 and the image memory 126 are connected to the bus 134. The image memory 126 temporarily holds wide-angle camera image data. That is, the CPU 128 acquires wide-angle camera image data from the image sensor 36 and temporarily stores the acquired wide-angle camera image data in the image memory 126. Then, the CPU 128 acquires wide-angle camera image data from the image memory 126 as needed.

[0109] The electrical system devices of the focus lens driving mechanism 50, the zoom lens driving mechanism 52, and the aperture driving mechanism 54 are connected to the bus 134. The CPU 128 controls the electrical system devices of the focus lens driving mechanism 50, the zoom lens driving mechanism 52, and the aperture driving mechanism 54. ​​

[0110] A communication I / F 122 is connected to the bus 144, and the CPU 138 transmits and receives information between the computer 140 of the management apparatus 20 via the communication I / F 122 and 106.

[0111] In the wide-angle camera 14, the control apparatus 94 is provided with a computer 146 and an image memory 148. The computer 140 has a CPU 150, an NVM 152, and a RAM 154. In the wide-angle camera 14, the computer 140 corresponds to the computer 146 of the telephoto camera 12. That is, in the wide-angle camera 14, the CPU 150, the NVM 152, and the RAM 154 correspond to the CPU 138, the NVM 140, and the RAM 154 of the telephoto camera 12.

[0112] The wide-angle camera 14 is provided with a bus 156. In the wide-angle camera 14, the bus 156 corresponds to the bus 144 of the telephoto camera 12.

[0113] The image sensor 70 and the image memory 148 are connected to the bus 156. The image memory 148 temporarily holds wide-angle camera image data. That is, the CPU 150 acquires wide-angle camera image data from the image sensor 70, and stores the acquired wide-angle camera image data in the image memory 148. Then, the CPU 150 acquires wide-angle camera image data from the image memory 148 as necessary.

[0114] The respective electric system devices of the focus lens driving mechanism 84, the zoom lens driving mechanism 86, and the aperture driving mechanism 88 are connected to the bus 156. The CPU 150 controls the respective electric system devices of the focus lens driving mechanism 84, the zoom lens driving mechanism 86, and the aperture driving mechanism 88.

[0115] A communication I / F 124 is connected to the bus 156, and the CPU 150 transmits and receives information between the computer 104 of the management apparatus 20 via the communication I / F 124 and 108,

[0116] In recent years, as a method of monitoring a monitoring target region while using a monitoring camera, a method of monitoring a monitoring target region while the telephoto camera 12 and the wide-angle camera 14 cooperate with each other (hereinafter, also referred to as a "telephoto / wide-angle camera cooperative monitoring method") is being studied. In the telephoto / wide-angle camera cooperative monitoring method, a monitor can monitor a wide range through a wide-angle camera image represented by wide-angle camera image data, and can also monitor a distant subject through a telephoto camera image represented by telephoto camera image data. Also, as a method of improving the convenience of the monitor, a method in which the monitor specifies a site to be enlarged through a wide-angle camera image, thereby controlling the magnification of the telephoto camera 12 and the rotation of the telephoto camera 12 is also studied.

[0117] However, in order to perform the cooperation of the tele camera 12 and the wide camera 14, detailed optical information of the wide camera 14 is required in addition to the detailed optical information of the tele camera 12. The tele camera 12 needs at least information about the angle of view of the wide camera 14 as the detailed optical information of the wide camera 14 in order to properly photograph the real space region designated by the wide camera image. Also, it is assumed that in the case where a close-range subject is photographed by the tele camera 12 and the wide camera 14, the tele camera 12 cannot properly photograph the real space region designated by the monitor as a result of the parallax generated between the tele camera 12 and the wide camera 14.

[0118] Therefore, in view of such a situation, the alignment processing is performed in the imaging system 10 (refer to Figure 19 and Figure 5 ). As an example, the alignment processing is implemented by the CPU 116 executing the alignment processing program 158 as shown in Figure 5 . The alignment processing program 158 is an example of the "program" to which the present technology pertains. In the example shown in Figure 6 , the alignment processing program 158 is stored in the NVM 118, and the CPU 116 reads the alignment processing program 158 from the NVM 118 and executes it on the RAM 120. The CPU 116 performs the alignment processing in accordance with the alignment processing program 158 executed on the RAM 120.

[0119] The CPU 116 first performs the alignment of the tele camera image obtained by the photographing by the tele camera 12 and the wide camera image obtained by the photographing by the wide camera 14 at at least two positions including the end portions of the angle of view used in the photographing based on the imaging system 10 by performing the alignment processing. Then, the CPU 116 acquires the angle of view related information about the angle of view used in the photographing based on the imaging system 10 from the alignment result. Hereinafter, the content of the alignment processing is described in more detail.

[0120] As an example, as shown in Figure 7 , the CPU 116 acts as the alignment section 116A, the control section 116B, and the acquisition section 116C by executing the alignment processing program 158. Also, the alignment section 116A has the angle of view center alignment section 116Al and the end portion alignment section 116A2.

[0121] As an example, as shown in Figure 6As shown, in the present embodiment, optical information related to the imaging optical system 38 of the tele camera 12 is known, and optical information related to the imaging optical system 72 of the wide camera 14 is unknown. For example, the angle of view of the tele camera 12 (hereinafter, also referred to as "tele camera angle of view") is a known angle of view, and the angle of view of the wide camera 14 (hereinafter, also referred to as "wide camera angle of view") is an unknown angle of view. Although details are described later, in the present embodiment, the wide camera angle of view is derived by the CPU 116 performing a center alignment process, and the control section 116B and the acquisition section 116C (refer to Figure 18 and Figure 7 ) acquire information including the derived wide camera angle of view as the angle of view related information.

[0122] The control section 116B acquires tele camera image data from the tele camera 12, and causes a tele camera image represented by the acquired tele camera image data to be displayed as a live view image on the display 26. Also, the control section 116B acquires wide camera image data from the wide camera 14, and causes a wide camera image represented by the acquired wide camera image data to be displayed as a live view image on the display 26. The display 26 displays the tele camera image and the wide camera image on different screens. In the example shown, the tele camera image is displayed on a screen on the left side of the front view in the two screens of the front view arranged horizontally within the display 26, and the wide camera image is displayed on a screen on the right side of the front view. Figure 8

[0123] In addition, here, the tele camera image and the wide camera image can be cited as examples of the form in which live view images are displayed on the display 26, but the present technology is not limited to this, and images other than live view images can also be displayed. As an example of an image other than a live view image, a playback image can be cited.

[0124] As an example, as shown in Figure 9 , the control section 116B performs control to display a cross line 160 superimposed at the center of the wide camera image within the display 26. The angle of view center alignment section 116Al performs AF control on the wide camera 14. That is, the wide camera 14 has an AF function, and the angle of view center alignment section 116Al focuses on the center of the wide camera angle of view by activating the AF function of the wide camera 14. Here, the center of the wide camera angle of view corresponds to the center of the wide camera image. Thus, the wide camera image obtained by imaging by the wide camera 14 in a state in which the center of the wide camera angle of view is focused on is displayed on the display 26.

[0125] ​Further, the field-of-view center alignment section 116Al stores the wide-angle camera image data obtained by the wide-angle camera 14 in the state of focusing on the center of the wide-angle camera field-of-view into the RAM 120. In addition, the wide-angle camera image data stored in the RAM 120 is updated every time the amount of one frame is shot.

[0126] As an example, as shown in FIG. 16, the control section 116B performs control to superimpose and display a cross line 162 on the center of the tele camera image in the display 26. The field-of-view center alignment section 116Al performs AF control on the tele camera 12. That is, the tele camera 12 has an AF function, and the field-of-view center alignment section 116Al focuses on the center of the tele camera field-of-view by activating the AF function of the tele camera 12. Here, the center of the tele camera field-of-view corresponds to the center of the tele camera image. Thus, the tele camera image obtained by the tele camera 12 in the state of focusing on the center of the tele camera field-of-view is displayed on the display 26. Figure 9

[0127] The field-of-view center alignment section 116Al acquires the tele camera image data obtained by the tele camera 12 in the state of focusing on the center of the tele camera field-of-view from the tele camera 12. Further, the field-of-view center alignment section 116Al acquires the wide-angle camera image data from the RAM 120. Then, the field-of-view center alignment section 116Al performs alignment of the center of the tele camera field-of-view and the center of the wide-angle camera field-of-view by using image analysis based on the tele camera image data acquired from the tele camera 12 and the wide-angle camera image data acquired from the RAM 120.

[0128] In this case, first, the field-of-view center alignment section 116Al calculates the turning direction and the turning amount of the tele camera 12 by performing image analysis on the tele camera image data acquired from the tele camera 12 and the wide-angle camera image data acquired from the RAM 120. Here, the turning direction and the turning amount of the tele camera 12 refer to the turning direction and the turning amount required to align the centers between the wide-angle camera image and the tele camera image. Specifically, the field-of-view center alignment section 116Al calculates the turning direction and the turning amount required to align the center in the tele camera image and the subject (for example, a point image) located at the center in the wide-angle camera image (for example, to align the center of the cross line 162 shown in FIG. 16 and the center of the cross line 160 shown in FIG. 15) as the turning direction and the turning amount of the tele camera 12. In addition, hereinafter, for convenience of explanation, the "turning" is described on the premise that the panning is taken as an example unless otherwise specified. That is, the turning direction refers to the panning direction, and the turning amount refers to the panning amount. Figure 8 Figure 10 ​​​

[0129] Next, the field-of-view center alignment unit 116A1 controls the rotation of the dual-axis rotary table 16 according to the calculated rotation direction and rotation amount. That is, the field-of-view center alignment unit 116A1 causes the telephoto camera 12 to pan by operating the dual-axis rotary table 16 according to the calculated rotation direction and rotation amount.

[0130] The field-of-view center alignment unit 116A1 uses the AF function of the telephoto camera 12 to derive the subject distance. Therefore, the field-of-view center alignment unit 116A1 obtains the focus position and telephoto camera characteristic information from the telephoto camera 12. The focus position is obtained by the field-of-view center alignment unit 116A1 each time AF control is performed. The telephoto camera characteristic information is information representing the characteristics of the telephoto camera 12, referring to the information required to derive the subject distance. Examples of the information required to derive the subject distance include, for example, information related to the specifications of the telephoto camera's imaging optical system 38 and information related to the specifications of the image sensor 36.

[0131] As an example, such as Figure 11 As shown, a subject distance derivation table 164 is stored in the NVM 118. The subject distance derivation table 164 is set for each of a plurality of telephoto camera characteristic information that are different from each other. The subject distance derivation table 164 for each of the plurality of telephoto camera characteristic information establishes a corresponding association between the focus position and the subject distance. The field-of-view center alignment unit 116A1 obtains the subject distance derivation table 164 corresponding to the telephoto camera characteristic information obtained from the telephoto camera 12 from the NVM 118. Then, the field-of-view center alignment unit 116A1 derives the subject distance corresponding to the focus position obtained from the telephoto camera 12 from the subject distance derivation table 164 obtained from the NVM 118. The subject distance thus derived by the field-of-view center alignment unit 116A1 is an example of the "first subject distance" involved in the present invention. Furthermore, for ease of explanation, the subject distance derived from the field-of-view center alignment unit 116A1 will also be referred to as the "first subject distance". Additionally, here, an example of deriving the first subject distance from the subject distance derivation table 164 has been provided, but the technology of the present invention is not limited thereto. The first subject distance can also be calculated using a formula that sets the telephoto camera characteristic information and focus position as independent variables and the first subject distance as a dependent variable.

[0132] As an example, such as Figure 9As shown, the NVM118 includes a predetermined value storage area 118A and a field-of-view center alignment storage area 118B. The predetermined value storage area 118A stores predetermined values, an example of "position-specific information" as described in this invention. These predetermined values ​​determine the positional relationship between the telephoto camera 12 and the wide-angle camera 14. Examples of predetermined values ​​include horizontal offset and depth offset. Here, the horizontal offset refers to the horizontal offset between the telephoto camera 12 and the wide-angle camera 14. Furthermore, the depth offset refers to the forward / backward offset between the telephoto camera 12 and the wide-angle camera 14, i.e., the depth offset between them. Additionally, the horizontal offset and depth offset are examples of the "offset between the first imaging device and the second imaging device" as described in this invention.

[0133] The field-of-view center alignment unit 116A1 determines whether the distance to the first subject is greater than or equal to a first threshold. Here, the first threshold is a fixed value obtained in advance through testing based on actual equipment and / or computer simulation, representing the distance to the subject in the real-space area specified by the user, which is unsuitable for the telephoto camera 12 to capture properly using the wide-angle camera image due to parallax caused by horizontal and depth offsets. Furthermore, while a fixed value is exemplified here as the first threshold, the technology of the present invention is not limited to this; the first threshold can also be a variable value that changes according to external instructions and / or imaging conditions.

[0134] When the distance to the first subject is greater than or equal to the first threshold, the field-of-view center alignment unit 116A1 overwrites and stores the current rotation angle in the field-of-view center alignment storage area 118B. Here, the current rotation angle refers to the angle measured by the field-of-view center alignment unit 116A1 on the dual-axis rotary table 16. Figure 11 The rotation angle of the telephoto camera 12, which is controlled by rotation, is shown. Figure 12 In the example, the rotation angle of the telephoto camera 12 is represented as "θ". c That is, the angle between the parallel line PL (parallel to the optical axis OA2) that is consistent with the optical axis OA1 when the telephoto camera 12 is in its original position (the initial state before the telephoto camera 12 is rotated) and the optical axis OA1 after the telephoto camera 12 is rotated is the rotation angle θ. c .

[0135] When the distance to the first subject is less than a first threshold, the field-of-view center alignment unit 116A1 performs field-of-view center alignment processing. Here, field-of-view center alignment processing refers to aligning the center of the telephoto camera's field of view with the center of the wide-angle camera's field of view using a predetermined value. Specifically, aligning the center of the telephoto camera's field of view with the center of the wide-angle camera's field of view involves calculating, using a predetermined value, the required rotation angle θ to make the center of the telephoto camera's field of view coincide with the center of the wide-angle camera's field of view. c And the calculated rotation angle θ c The field of view center alignment storage area 118B is overwritten and stored. That is, the field of view center alignment unit 116A1 calculates the rotation angle θ by performing field of view center alignment processing. c And the calculated rotation angle θ c It is covered and stored in the center alignment storage area 118B of the field of view.

[0136] In addition, for ease of explanation, the rotation angle covered by the field of view center alignment part 116A1 and stored in the field of view center alignment storage area 118B will also be referred to as the "first rotation angle".

[0137] As an example, such as Figure 13 As shown, when the distance to the first subject is set to "d"... target1 Set the horizontal offset to "d". x "At that time, the rotation angle θ c The rotation angle θ is calculated by the field-of-view center alignment unit 116A1 according to the following formula (1). Then, the rotation angle θ calculated by the field-of-view center alignment unit 116A1 is... c The first rotation angle is covered and stored in the field of view center alignment storage area 118B. In addition, the first rotation angle is an example of the "field of view center alignment result obtained by aligning the center of the field of view of the first camera device with the center of the field of view of the second camera device" involved in the present invention.

[0138] [Formula 1]

[0139]

[0140] Thus, if the field of view center alignment process is performed by the field of view center alignment unit 116A1, then the end alignment process is performed by the end alignment unit 116A2. A detailed explanation will follow.

[0141] As an example, such as Figure 13 As shown, the control unit 116B displays one end of the wide-angle camera image on the display 26 (in... Figure 14In the example shown, the control of the crosshairs 160 is superimposed on the center of the left end of the front view of the wide-angle camera image on the display 26. The end alignment unit 116A2 performs AF control on the wide-angle camera 14. That is, the end alignment unit 116A2 focuses on one end of the field of view of the wide-angle camera by activating the AF function of the wide-angle camera 14. Here, one end of the field of view of the wide-angle camera corresponds to one end of the wide-angle camera image. Thus, the wide-angle camera image obtained by the wide-angle camera 14 is displayed on the display 26 while it is focused on one end of the field of view of the wide-angle camera.

[0142] Furthermore, the end alignment unit 116A2 stores the wide-angle camera image data obtained by the wide-angle camera 14 in the RAM 120 while focusing on one end of the wide-angle camera's field of view. In addition, the wide-angle camera image data stored in the RAM 120 is updated each time a frame is captured.

[0143] As an example, such as Figure 14 As shown, the control unit 116B displays one end of the telephoto camera image on the display 26 (in... Figure 14 In the example shown, the control of the crosshairs 162 is superimposed on the center of the left end of the front view of the telephoto camera image on the display 26. The end alignment unit 116A2 performs AF control on the telephoto camera 12. That is, the end alignment unit 116A2 focuses on one end of the telephoto camera's field of view by activating the AF function of the telephoto camera 12. Here, one end of the telephoto camera's field of view corresponds to one end of the telephoto camera image. Thus, the telephoto camera image obtained by the telephoto camera 12 is displayed on the display 26 while it is focused on one end of the telephoto camera's field of view.

[0144] The end alignment unit 116A2 acquires telephoto camera image data captured by the telephoto camera 12 while it is focused on one end of the telephoto camera's field of view. The end alignment unit 116A2 also acquires wide-angle camera image data from the RAM 120. Then, the end alignment unit 116A2 performs image analysis based on the telephoto camera image data acquired from the telephoto camera 12 and the wide-angle camera image data acquired from the RAM 120 to align one end of the telephoto camera's field of view with one end of the wide-angle camera's field of view. Here, one end of the telephoto camera's field of view is an example of the "first end of the field of view of the first imaging device" according to the present invention, and one end of the wide-angle camera's field of view is an example of the "second end of the field of view of the first imaging device (one of the two ends of the field of view of the second imaging device) according to the present invention."

[0145] In the case where the alignment of the end of the field of view of the tele camera with the end of the field of view of the wide camera is performed, first, the end alignment section 116A2 calculates the rotation direction and the rotation amount of the tele camera 12 by performing image analysis on the tele camera image data acquired from the tele camera 12 and the wide camera image data acquired from the RAM 120. Here, the rotation direction and the rotation amount of the tele camera 12 refer to the rotation direction and the rotation amount required to align the ends of the wide camera image and the tele camera image. Specifically, the end alignment section 116A2 calculates the rotation direction and the rotation amount required to align a subject (e.g., a point image) located at the center of the end of the wide camera image with the center of the cross line 162 shown in FIG. 16 (e.g., to align the center of the cross line 160 shown in FIG. 15) as the rotation direction and the rotation amount of the tele camera 12. Figure 13 Figure 15 Next, the end alignment section 116A2 performs rotation control of the two-axis turntable 16 in accordance with the calculated rotation direction and the rotation amount. That is, the end alignment section 116A2 causes the tele camera 12 to pan by operating the two-axis turntable 16 in the calculated rotation direction and the rotation amount.

[0146] Next, the end alignment section 116A2 performs rotation control of the two-axis turntable 16 in accordance with the calculated rotation direction and the rotation amount. That is, the end alignment section 116A2 causes the tele camera 12 to pan by operating the two-axis turntable 16 in the calculated rotation direction and the rotation amount.

[0147] The end alignment section 116A2 derives the subject distance using the AF function of the tele camera 12. Therefore, the focus position and the tele camera characteristic information are acquired from the end alignment section 116A2 and the tele camera 12. The focus position is acquired by the end alignment section 116A2 each time the AF control is performed.

[0148] As an example, as shown in FIG. 16, the end alignment section 116A2 acquires the subject distance derivation table 164 corresponding to the tele camera characteristic information acquired from the tele camera 12 from the NVM 118. Then, the end alignment section 116A2 derives the subject distance corresponding to the focus position acquired from the tele camera 12 from the subject distance derivation table 164 acquired from the NVM 118. Thus, the subject distance derived by the end alignment section 116A2 is an example of the "second subject distance" according to the present technology. Hereinafter, for convenience of explanation, the subject distance derived by the end alignment section 116A2 will also be referred to as the "second subject distance". Note that, here, the form example in which the second subject distance is derived from the subject distance derivation table 164 is described as an example, but the technology of the present application is not limited thereto, and the second subject distance can be calculated according to an operation formula in which the tele camera characteristic information and the focus position are set as independent variables and the second subject distance is set as a dependent variable. Figure 16 As an example, as shown in FIG. 16, the end alignment section 116A2 acquires the subject distance derivation table 164 corresponding to the tele camera characteristic information acquired from the tele camera 12 from the NVM 118. Then, the end alignment section 116A2 derives the subject distance corresponding to the focus position acquired from the tele camera 12 from the subject distance derivation table 164 acquired from the NVM 118. Thus, the subject distance derived by the end alignment section 116A2 is an example of the "second subject distance" according to the present technology. Hereinafter, for convenience of explanation, the subject distance derived by the end alignment section 116A2 will also be referred to as the "second subject distance". Note that, here, the form example in which the second subject distance is derived from the subject distance derivation table 164 is described as an example, but the technology of the present application is not limited thereto, and the second subject distance can be calculated according to an operation formula in which the tele camera characteristic information and the focus position are set as independent variables and the second subject distance is set as a dependent variable.

[0149] Figure 14 ​​As shown, an end alignment storage area 118C is provided in the NVM118. In the end alignment storage area 118C, each of the second rotation angle (described later) and the half-angle view (hereinafter also referred to as "half-angle view") which is half the field of view of the wide-angle camera is individually overwritten and stored.

[0150] The end alignment unit 116A2 determines whether the distance to the second subject is greater than or equal to a second threshold. Here, the second threshold is a fixed value obtained in advance through testing and / or computer simulation based on actual equipment. It represents the distance to the subject in the real space area specified by the user, which is unsuitable for the telephoto camera 12 to capture properly using the wide-angle camera image due to parallax caused by horizontal and depth offsets. Furthermore, while a fixed value is exemplified here as the second threshold, the technology of the present invention is not limited to this; the second threshold can also be a variable value that changes according to external instructions and / or imaging conditions.

[0151] When the distance to the second subject is greater than or equal to the second threshold, the end alignment unit 116A2 overwrites and stores the current rotation angle in the end alignment storage area 118C. Here, the current rotation angle refers to the angle at which the dual-axis rotary table 16 will be rotated by the end alignment unit 116A2. Figure 11 The rotation angle of the telephoto camera 12, which is rotated by the rotation control shown, is the same as the first rotation angle (reference). Figure 16 The angles added together. Figure 16 In the example, the rotation angle of the telephoto camera 12 is represented as "θ". e1 ".

[0152] Furthermore, for ease of explanation, the rotation angle θ, which is covered by the end alignment part 116A2 and stored in the end alignment storage area 118C, will be described below. e1 Also known as the "second rotation angle".

[0153] When the distance to the second subject is less than the second threshold, the end alignment unit 116A2 performs end alignment processing. Here, end alignment processing refers to the process of aligning one end of the telephoto camera's field of view with one end of the wide-angle camera's field of view using a predetermined value.

[0154] Here, aligning one end of the telephoto camera's field of view with one end of the wide-angle camera's field of view means, based on the current rotation angle ( Figure 17 The example shown is the rotation angle θ. e1 The process involves calculating the half-angle of the end alignment and the specified value, and then overwriting and storing the calculated half-angle in the end alignment storage area 118C. That is, the end alignment unit 116A2 calculates the half-angle by performing end alignment processing, and then overwrites and stores the calculated half-angle in the end alignment storage area 118C.

[0155] As an example, as shown in Figure 18 Fig. 6, when the second subject distance is set to "d target2 ", the horizontal direction offset is set to "d x ", and the depth direction offset is set to "d y ", the half view angle θ e2 is calculated by the end alignment section 116A2 according to the following equation (2). Then, the half view angle θ e2 calculated by the end alignment section 116A2 is covered and stored in the end alignment storage area 118C. In addition, the half view angle θ e2 and the second rotation angle (rotation angle θ e1 ) are examples of the "end alignment result obtained by performing alignment of the end of the field of view of the first imaging device with the second end of the field of view of the second imaging device" according to the present technology.

[0156] [Equation 2]

[0157]

[0158] As an example, as shown in Figure 18 Fig. 7, the acquisition section 116A3 acquires the half view angle from the end alignment section 116A2. The half view angle is an angle derived from a result obtained by performing alignment of the center of the wide-angle camera field of view with the center of the telephoto camera field of view and a result obtained by performing alignment of the end of the wide-angle camera field of view with the end of the telephoto camera field of view. The acquisition section 116A3 acquires the wide-angle camera field of view according to the half view angle acquired from the end alignment section 116A2. That is, the acquisition section 116A3 calculates the wide-angle camera field of view by doubling the half view angle acquired from the end alignment section 116A2. In addition, the wide-angle camera field of view is an example of the "field of view related information" according to the present technology.

[0159] The control section 116B causes the display 26 to display the tele camera image represented by the tele camera image data and the wide camera image represented by the wide camera image data. Also, the control section 116B causes the display 26 to further display the field of view angle related information in a state where the tele camera image and the wide camera image are displayed. The control section 116B acquires the tele camera characteristic information from the tele camera 12. The tele camera characteristic information is information including the tele camera field of view angle. The control section 116B causes the display 26 to display information including the wide camera field of view angle, the tele camera field of view angle, the difference between the wide camera field of view angle and the tele camera field of view angle (for example, a value obtained by subtracting the tele camera field of view angle from the wide camera field of view angle) as the field of view angle related information. Also, the control section 116B performs control to superimpose and display a mark (a dotted rectangular frame in the illustrated example) representing which region in the wide camera image corresponds to the tele camera image on the wide camera image. Figures 19-21

[0160] Next, the operation of the part of the present technology relating to the imaging system 10 will be described with reference to Figure 19 FIG. 10. In Figure 19 FIG. 10, an example of a flow of the alignment processing performed by the CPU 116 at a prescribed timing (for example, at a predetermined time, or when a start instruction of the alignment processing is externally given) is shown. Note that Figure 19 the flow of the alignment processing shown in FIG. 10 is an example of the "imaging system operation method" of the present technology.

[0161] In Figure 20 the alignment processing shown in FIG. 10, first, in step ST100, the field of view angle center alignment section 116A1 performs Figure 20 the field of view angle center alignment processing shown in FIG. 10 as an example.

[0162] In Figure 19 the field of view angle center alignment processing shown in FIG. 10, first, in step ST100A, the field of view angle center alignment section 116A1 acquires the tele camera image data from the tele camera 12 and the wide camera image data from the wide camera 14.

[0163] In the next step ST100B, the field of view angle center alignment section 116A1 causes the tele camera 12 to swivel so that the center of the tele camera field of view angle is aligned with the center of the wide camera field of view angle by performing image analysis on the tele camera image data and the wide camera image data acquired in step ST100A.

[0164] In the next step ST100C, the field of view angle center alignment section 116A1 focuses on the center of the tele camera field of view angle by performing AF control on the tele camera 12. ​

[0165] In next step ST100D, the field angle center alignment section 116A1 acquires the focus position and the tele camera characteristic information from the tele camera 12.

[0166] In next step ST100E, the field angle center alignment section 116A1 acquires the object distance derivation table 164 corresponding to the tele camera characteristic information acquired in step ST100D from the NVM 118. Then, the field angle center alignment section 116A1 derives the object distance corresponding to the focus position acquired in step ST100D as the 1st object distance from the object distance derivation table 164 acquired from the NVM 118.

[0167] In next step ST100F, the field angle center alignment section 116A1 determines whether the 1st object distance derived in step ST100E is smaller than the 1st threshold value. In step ST100F, in the case where the 1st object distance is the 1st threshold value or more, the determination is negated, and the field angle center alignment processing is transferred to step ST100G. In step ST100F, in the case where the 1st object distance is smaller than the 1st threshold value, the determination is affirmed, and the field angle center alignment processing is transferred to step ST100H.

[0168] In step ST100G, the field angle center alignment section 116A1 overwrites and stores the current rotation angle as the 1st rotation angle in the field angle center alignment storage area 118B, and then the field angle center alignment processing is ended.

[0169] In step ST100H, the field angle center alignment section 116A1 calculates the 1st rotation angle in accordance with equation (1) from the prescribed value stored in the prescribed value storage area 118A and the 1st object distance derived in step ST100E. Then, the field angle center alignment section 116A1 overwrites and stores the calculated 1st rotation angle in the field angle center alignment storage area 118B, and then the field angle center alignment processing is ended.

[0170] In Figure 21 In step ST102 shown in FIG. 10, the end alignment section 116A2 executes Figure 21 the end alignment processing shown in FIG. 11 as an example.

[0171] In Figure 19 In the end alignment processing shown in FIG. 11, first, in step ST102A, the end alignment section 116A2 acquires the tele camera image data from the tele camera 12 and the wide camera image data from the wide camera 14.

[0172] In the next step ST102B, the end alignment section 116A2 rotates the tele camera 12 by image analysis of the tele camera image data and the wide camera image data acquired in the step ST102A so that the end of the tele camera angle of view (for example, one end of the tele camera angle of view) is aligned with the end of the wide camera angle of view (for example, one end of the wide camera angle of view).

[0173] In the next step ST102C, the end alignment section 116A2 focuses on the end of the tele camera angle of view by AF control of the tele camera 12.

[0174] In the next step ST102D, the end alignment section 116A2 acquires the focus position and the tele camera characteristic information from the tele camera 12.

[0175] In the next step ST102E, the end alignment section 116A2 acquires the object distance derivation table 164 corresponding to the tele camera characteristic information acquired in the step ST102D from the NVM 118. Then, the end alignment section 116A2 derives the object distance corresponding to the focus position acquired in the step ST102D as the 2nd object distance from the object distance derivation table 164 acquired from the NVM 118.

[0176] In the next step ST102F, the end alignment section 116A2 determines whether the 2nd object distance derived in the step ST102E is smaller than the 2nd threshold value. In the step ST102F, in the case where the 2nd object distance is equal to or larger than the 2nd threshold value, the determination is negated, and the end alignment processing is transferred to the step ST102G. In the step ST102F, in the case where the 2nd object distance is smaller than the 2nd threshold value, the determination is affirmed, and the end alignment processing is transferred to the step ST102H.

[0177] In the step ST102G, the end alignment section 116A2 overwrites and stores the current rotation angle as the 2nd rotation angle in the end alignment storage area 118C, and then the end alignment processing is ended.

[0178] In the step ST102H, the end alignment section 116A2 calculates the half angle of view in accordance with the equation (2) from the prescribed value stored in the prescribed value storage area 118A, the 1st rotation angle stored in the angle of view center alignment storage area 118B, and the 1st object distance derived in the step ST100E. Then, the end alignment section 116A2 overwrites and stores the calculated half angle of view in the end alignment storage area 118C, and then the end alignment processing is ended.

[0179] In Figure 19In the step ST104 shown, the acquisition section 116C determines whether or not the half view angle is calculated by performing the end portion alignment processing. In the step ST104, in a case where the half view angle is not calculated by performing the end portion alignment processing, the determination is negated, and the alignment processing ends. In the step ST104, in a case where the half view angle is calculated by performing the end portion alignment processing, the determination is affirmed, and the end portion alignment processing proceeds to the step ST106.

[0180] In the step ST106, the acquisition section 116C calculates the wide-angle camera view angle from the half view angle calculated by performing the end portion alignment processing. Then, the control section 116B acquires the view angle related information including the wide-angle camera view angle calculated by the acquisition section 116C, and causes the display 26 to display the acquired view angle related information, and then, the alignment processing ends.

[0181] As explained above, in the imaging system 10, the alignment of the tele camera image obtained by the imaging by the tele camera 12 and the wide-angle camera image obtained by the imaging by the wide-angle camera 14 is performed at two portions of the center of the view angle used in the imaging based on the imaging system 10 and the end portions of the view angle. Then, the view angle related information is acquired as the information related to the view angle used in the imaging based on the imaging system 10 according to the alignment result. Thus, according to this structure, it is possible to obtain the information (for example, the wide-angle camera view angle and the like) related to the view angle required when the tele camera 12 and the wide-angle camera 14 are used in cooperation.

[0182] Also, in the imaging system 10, the wide-angle camera view angle is acquired from the view angle center alignment result obtained by performing the alignment of the center of the tele camera view angle and the center of the wide-angle camera view angle, and the end portion alignment result obtained by performing the alignment of one end of the tele camera view angle and one end of the wide-angle camera view angle. Thus, according to this structure, even if the wide-angle camera view angle is an unknown view angle, it is possible to determine the wide-angle camera view angle.

[0183] Also, in the imaging system 10, the first rotation angle is calculated from the horizontal direction offset amount, and the half view angle is calculated from the horizontal direction offset amount and the depth direction offset amount. Thus, according to this structure, compared with a case where either one of the horizontal direction offset amount and the depth direction offset amount is not used, it is possible to achieve high-precision alignment of the center of the tele camera view angle and the center of the wide-angle camera view angle, and high-precision alignment of the end portions of the tele camera view angle and the end portions of the wide-angle camera view angle. Also, even if the positions of the tele camera 12 and the wide-angle camera 14 are offset, it is possible to obtain the information related to the view angle required when the tele camera 12 and the wide-angle camera 14 are used in cooperation.

[0184] Furthermore, in the camera system 10, when the distance to the first subject is less than a first threshold, field-of-view center alignment is performed using a horizontal offset; when the distance to the second subject is less than a second threshold, end alignment is performed using both horizontal and depth offsets. Therefore, according to this structure, even if parallax effects become more pronounced as the subject distance decreases, information related to the field of view required when using the telephoto camera 12 and the wide-angle camera 14 in conjunction can be obtained.

[0185] Furthermore, in the camera system 10, a first subject distance is derived from the field-of-view center alignment part 116A1, and a second subject distance is derived from the end alignment part 116A2. Thus, according to this structure, compared to the case where the user assigns the subject distance to the camera system 10, the subject distance for alignment can be obtained quickly.

[0186] Furthermore, in the camera system 10, the center alignment processing of the field of view is performed ( Figure 19 After the processing of step ST 100 shown, end alignment processing is performed. Figure 22A (The processing in step ST102 shown). Thus, according to this structure, compared with the case where end alignment processing is performed without field-of-view center alignment processing, high-precision information can be obtained as information related to the field of view required when using the telephoto camera 12 and the wide-angle camera 14 in conjunction (refer to steps ST100 and ST102).

[0187] Furthermore, in the camera system 10, field-of-view center alignment and end-angle alignment are performed using image analysis based on images from the telephoto camera and the wide-angle camera. Therefore, according to this structure, accurate alignment can be achieved compared to manually performing the equivalent field-of-view center alignment and end-angle alignment. Alternatively, either field-of-view center alignment or end-angle alignment can be performed using image analysis based on images from the telephoto camera and the wide-angle camera.

[0188] Furthermore, in the camera system 10, the optical information related to the camera optical system 38 of the telephoto camera 12 is known, while the optical information related to the camera optical system 72 of the wide-angle camera 14 is unknown. Therefore, according to this structure, even if the optical information related to the camera optical system 72 of the wide-angle camera 14 is unknown, the field of view of the wide-angle camera can still be determined.

[0189] Further, in the imaging system 10, the focal length of the imaging optical system 38 of the tele camera 12 is twice or more the focal length of the imaging optical system 72 of the wide camera 14. Thus, according to this configuration, even if the focal length of the imaging optical system 38 of the tele camera 12 is twice or more the focal length of the imaging optical system 72 of the wide camera 14, it is possible to obtain information relating to the field of view angles required when the tele camera 12 and the wide camera 14 are used in cooperation.

[0190] Further, in the above-described embodiment, in the end portion alignment processing, the end portion alignment unit 116A2 performs alignment of one end of the field of view angle of the tele camera and one end of the field of view angle of the wide camera, but the technology of the present application is not limited to this, and the end portion alignment unit 116A2 can perform opposite end portion alignment processing in addition to the alignment of one end of the field of view angle of the tele camera and one end of the field of view angle of the wide camera. The opposite end portion alignment processing refers to processing in which the other end of the field of view angle of the tele camera and the other end of the field of view angle of the wide camera are aligned. Here, the other end of the field of view angle of the tele camera is an example of the "first opposite end portion of the two end portions of the field of view angle of the first imaging device, which is on the opposite side of the first end portion", and the other end of the field of view angle of the wide camera is an example of the "second opposite end portion of the two end portions of the field of view angle of the second imaging device, which is on the opposite side of the second end portion".

[0191] Thus, in the case where the end portion alignment processing and the opposite end portion alignment processing are performed by the end portion alignment unit 116A2, the acquisition unit 116C can acquire the field of view angle of the wide camera based on the first result (for example, one half of the field of view angle of the wide camera 14) obtained by performing alignment of one end of the field of view angle of the tele camera and one end of the field of view angle of the wide camera in the end portion alignment processing, and the second result (for example, the other half of the field of view angle of the wide camera 14) obtained by performing alignment of the other end of the field of view angle of the tele camera and the other end of the field of view angle of the wide camera in the opposite end portion alignment processing.

[0192] In this case, for example, Figure 22B and Figure 22A the end portion alignment processing illustrated in FIG. 17 is executed by the CPU 116. Figure 22B and Figure 21 the flowchart illustrated in FIG. 18 differs from the flowchart illustrated in FIG. 16 only in that it has the steps ST102H to ST102N, and thus the steps different from the flowchart illustrated in FIG. 16 will be described below. Figure 21 Figure 22A In the case where the end portion alignment processing illustrated in FIG. 17 is executed by the CPU 116, the end portion alignment unit 116A2 performs the end portion alignment processing illustrated in FIG. 17.

[0193] In the case where the end portion alignment processing illustrated in FIG. 17 is executed by the CPU 116, the end portion alignment unit 116A2 performs the end portion alignment processing illustrated in FIG. 17. Figure 22A ​In the step ST102H1, the end alignment section 116A2 rotates the tele camera 12 by performing image analysis on the tele camera image data and the wide camera image data acquired in the step ST102A, so that the opposite end portion of the tele camera angle of view (for example, the other end of the tele camera angle of view) is aligned with the opposite end portion of the wide camera angle of view (for example, the other end of the wide camera angle of view).

[0194] In the next step ST102I, the end alignment section 116A2 focuses on the opposite end portion of the tele camera angle of view by performing AF control on the tele camera 12.

[0195] In the next step ST102J, the end alignment section 116A2 acquires the focus position and the tele camera characteristic information from the tele camera 12.

[0196] In the next step ST102K, the end alignment section 116A2 acquires the object distance derivation table 164 corresponding to the tele camera characteristic information acquired in the step ST102J from the NVM 118. Then, the end alignment section 116A2 derives the object distance corresponding to the focus position acquired in the step ST102J from the object distance derivation table 164 acquired from the NVM 118 as the third object distance. Note that, in this embodiment, the form of deriving the third object distance from the object distance derivation table 164 is exemplified, but the technology of the present application is not limited to this, and the third object distance can be calculated according to an operation formula in which the tele camera characteristic information and the focus position are set as independent variables and the third object distance is set as a dependent variable.

[0197] In the next step ST102L, the end alignment section 116A2 determines whether the third object distance derived in the step ST102K is smaller than a third threshold value. The third threshold value is, for example, the same value as the above-described second threshold value. In the step ST102L, in the case where the third object distance is equal to or larger than the third threshold value, the determination is negated, and the end alignment processing is shifted to the step ST102M. In the step ST102L, in the case where the third object distance is smaller than the third threshold value, the determination is affirmed, and the end alignment processing is shifted to the step ST102N.

[0198] In the step ST102M, the end alignment section 116A2 overwrites and stores the current rotation angle as the third rotation angle in the end alignment storage region 118C, and then the end alignment processing is ended.

[0199] In step ST102N, the end alignment unit 116A2 calculates the half-angle of view according to formula (2) based on the predetermined value stored in the predetermined value storage area 118A, the first rotation angle stored in the field of view center alignment storage area 118B, and the third subject distance derived in step ST100K. Here, the half-angle of view refers to the half-angle of view that is paired with the half-angle of view calculated in step ST102H. That is, the half-angle of view calculated in step ST102H is one half-angle of view of the wide-angle camera 14, and in contrast, the half-angle of view calculated in step ST102N is the other half-angle of view of the wide-angle camera 14. In addition, the half-angle of view calculated in step ST102H is an example of the "first result" involved in the present invention, and the half-angle of view calculated in step ST102N is an example of the "second result" involved in the present invention.

[0200] Then, in step ST102N, the end alignment unit 116A2 covers and saves the calculated half-angle view in the end alignment storage area 118C, and then the end alignment process ends.

[0201] Thus, by conducting Figure 22B and Figure 23 The end alignment process shown calculates one and a half angles of view and the other half angle of view of the wide-angle camera 14. Therefore, the acquisition unit 116C can obtain the field of view of the wide-angle camera simply by calculating the sum of one and a half angles of view and the other half angle of view of the wide-angle camera 14. Thus, even if the field of view of the wide-angle camera is unknown, the field of view of the wide-angle camera can be determined.

[0202] Furthermore, here, the acquisition unit 116C calculates the sum of one half-angle view and the other half-angle view of the wide-angle camera 14, but the technology of the present invention is not limited to this. For example, when the distance to the third subject is greater than or equal to the third threshold, and the position at the other end of the telephoto camera's field of view is aligned with the position at the other end of the wide-angle camera's field of view, the end alignment unit 116A2 calculates the other half-angle view of the wide-angle camera 14. As an example, such as... Figure 24 As shown, the acquisition unit 116C can acquire the field of view of the wide-angle camera based solely on the other half of the viewing angle of the wide-angle camera 14. Here, "acquiring the field of view of the wide-angle camera based solely on the other half of the viewing angle of the wide-angle camera 14" means, for example, acquiring a value that doubles the other half of the viewing angle of the wide-angle camera 14 as the field of view of the wide-angle camera.

[0203] Thus, the reason why the acquisition unit 116C acquires the field of view of the wide-angle camera 14 based solely on the other half of the field of view is that, under the premise that "the second threshold = the third threshold," compared to the one half of the field of view of the wide-angle camera 14 when the position of one end of the telephoto camera's field of view is aligned with the position of one end of the wide-angle camera's field of view when the distance to the second subject is less than the second threshold, the other half of the field of view of the wide-angle camera 14 when the position of the other end of the telephoto camera's field of view is aligned with the position of the other end of the wide-angle camera's field of view when the distance to the third subject is greater than the third threshold, has better accuracy. That is, because the distance to the third subject is longer than the distance to the second subject, the other half of the field of view of the wide-angle camera 14 is less susceptible to the parallax caused by the positional offset between the telephoto camera 12 and the wide-angle camera 14 than the one half of the field of view of the wide-angle camera 14. Therefore, based on this structure, compared to the case where the wide-angle camera's field of view is calculated based on only one and a half angles of the wide-angle camera 14 when the position of the other end of the telephoto camera's field of view is aligned with the position of the other end of the wide-angle camera's field of view under the condition that the distance to the third subject is greater than or equal to the third threshold, a high-precision field of view can be determined as the wide-angle camera's field of view.

[0204] In the above embodiments, examples of end alignment processing without considering the scale ratio of telephoto camera images to wide-angle camera images are given. However, the technology of the present invention is not limited to this. In the end alignment processing, the end alignment unit 116A2 can align one end of the telephoto camera's field of view with one end of the wide-angle camera's field of view based on the scale ratio of the telephoto camera image to the wide-angle camera image used in the image analysis in the field of view center alignment processing.

[0205] In this case, Figure 25 The field-of-view center alignment process shown is executed by CPU116, and Figure 24 The end alignment process shown is performed by CPU116. Figure 20 The flowchart shown is Figure 25 The difference in the flowchart shown is that it includes step ST200. Figure 21 The flowchart shown is Figure 20 The difference in the flowchart shown is the presence of step ST202, therefore, in this context, it is related to... Figure 21 and Figure 24 The flowchart shown illustrates the different steps.

[0206] exist Figure 7 In step ST200, the field-of-view center alignment unit 116A1 calculates the scale ratio between the telephoto camera image and the wide-angle camera image based on the telephoto camera image data and the wide-angle camera image data acquired in step ST100A. Here, scale ratio refers, for example, with respect to the camera's object area (reference...). Figure 25The ratio of the area of ​​the image region within a wide-angle camera image of a specific subject to the area of ​​the image region within a telephoto camera image.

[0207] exist Figure 24 In step ST202 shown, the end alignment part 116A2 is based on... Figure 26 The scale ratio calculated in step ST200 is used to align one end of the telephoto camera's field of view with one end of the wide-angle camera's field of view. In this case, for example, the end alignment unit 116A2 calculates the rotation angle of the telephoto camera 12 according to a rotation angle calculation formula that sets the scale ratio as an independent variable and the rotation angle of the telephoto camera 12 as a dependent variable, and makes the telephoto camera 12 rotate only by the calculated rotation angle. As a result, in the pre-processing stage of step ST102B, one end of the telephoto camera's field of view can be brought close to or aligned with one end of the wide-angle camera's field of view, and the time required for end alignment processing can be shortened compared to the case where the scale ratio is not used.

[0208] In the above embodiments, the description is based on the premise that the first subject distance using the AF function is successfully derived in the field-of-view center alignment process, but the technology of the present invention is not limited thereto. For example, if the deriving of the first subject distance using the AF function fails, the field-of-view center alignment unit 116A1 controls the dual-axis rotary table 16 according to the telephoto camera image, and after aligning the center of the telephoto camera's field of view with a high-contrast subject (hereinafter also referred to as "high-contrast subject") near the center of the telephoto camera's field of view, the first subject distance can be derived using the AF function.

[0209] In this case, for example, Figure 20 The field-of-view center alignment process shown is executed by CPU116. Figure 26 Compared to the flowchart shown, Figure 20 The difference in the flowchart shown is that steps ST300 to ST312 replace step ST100C. Therefore, here, regarding... Figure 26 The flowchart shown illustrates the different steps.

[0210] exist Figure 26 In step ST300 shown, the field-of-view center alignment unit 116A1 performs contrast AF on the center of the telephoto camera's field of view. Here, contrast AF refers to focusing in a way that maximizes the contrast of a specific part of the image obtained by shooting (in this case, the part corresponding to the center of the telephoto camera's field of view in the telephoto camera image).

[0211] In step ST302, the field angle center alignment section 116A1 determines whether the contrast AF performed in step ST300 has failed. The contrast AF failure means, for example, a case where focusing is not possible because a high frequency component of a prescribed value or more cannot be detected from a specific portion (here, a portion corresponding to the center of the tele camera field angle in the tele camera image) within the image obtained by the photographing.

[0212] In step ST302, in the case where the contrast AF has not failed, the determination is negated, and the field angle center alignment processing moves to step ST100D. In step ST302, in the case where the contrast AF has failed, the determination is affirmed, and the field angle center alignment processing moves to step ST304.

[0213] In step ST304, the field angle center alignment section 116A1 searches for a high-contrast subject near the center of the tele camera field angle by analyzing the tele camera image data acquired in step ST100A. Here, the high-contrast subject near the center of the tele camera field angle means, for example, a high-contrast subject closest to the center of the tele camera field angle. Here, the high-contrast subject means a subject having a contrast to an extent that can be used for the contrast AF.

[0214] In step ST306, the field angle center alignment section 116A1 determines whether a high-contrast subject exists near the center of the tele camera field angle. In step ST306, in the case where a high-contrast subject does not exist near the center of the tele camera field angle, the determination is negated, and the field angle center alignment processing moves to step ST308. In step ST306, in the case where a high-contrast subject exists near the center of the tele camera field angle, the determination is affirmed, and the field angle center alignment processing moves to step ST310.

[0215] In step ST308, the field angle center alignment section 116A1 performs error processing, and then the alignment processing ends. Here, the error processing means, for example, processing of causing the display 26 to display a message such as "autofocus failure.", "alignment impossible.", and / or "please change the subject located at the center of the field angle.", or outputting the same message in voice from a speaker (omitted from the drawing).

[0216] In step ST310, the field angle center alignment section 116A1 rotates the tele camera 12 so as to align the center of the tele camera field angle with the high-contrast subject searched by performing the processing of step ST304.

[0217] In step ST312, the field angle center alignment section 116A1 performs the contrast AF so as to focus on the high-contrast subject located at the center of the tele camera field angle.

[0218] Thus, through execution Figure 20 The field-of-view center alignment process shown can achieve successful contrast AF even when there is no high-contrast subject at the center of the telephoto camera's field of view. If contrast AF is successful, the field-of-view center alignment unit 116A1 can obtain the focus position from the telephoto camera 12, and therefore the first subject distance can be derived from the focus position.

[0219] Therefore, according to this structure, compared with the case where the distance of the subject is derived by using the AF function after aligning the center of the telephoto camera's field of view with a high-contrast subject located at a position farther than the center of the telephoto camera's field of view, the distance of the first subject can be derived with high accuracy even when there is no high-contrast subject at the center of the telephoto camera's field of view.

[0220] Here, an example of performing contrast AF again after rotating the telephoto camera 12 has been described, but the technology of the present invention is not limited to this. For example, when performing contrast AF on the closest side, contrast AF can also be performed again while narrowing the focus range to the wide-angle side.

[0221] Furthermore, examples are given here to illustrate the... Figure 26 The field-of-view center alignment process shown applies steps ST300 to ST312 (see reference). Figure 26 This is a variation of the processing described above, but the technology of the present invention is not limited thereto. Processing equivalent to steps ST300 to ST312 can also be applied to the end alignment process. In this case, it is possible to facilitate the high-precision derivation of the distance to the second subject.

[0222] exist Figure 27 In the field-of-view center alignment process shown, an example is illustrated of aligning the center of the telephoto camera's field of view with the center of the wide-angle camera's field of view regardless of the focal length of the telephoto camera 12's imaging optical system 38; however, the technology of the present invention is not limited to this. For example, the field-of-view center alignment unit 116A1 can enable the telephoto camera 12 to focus when the focal length of the telephoto camera 12's imaging optical system 38 is at its longest, and derive a first subject distance based on the focusing result (e.g., focus position).

[0223] In this case, for example, it is executed by CPU116. Figure 26 The field of view center alignment shown is used instead of... Figure 27 The field of view center alignment is shown. Because... Figure 26 The flowchart shown is Figure 26 The flowchart shown differs in that it includes step ST400, therefore it is similar to...Figure 27 The steps of the flowchart shown will be described.

[0224] In Figure 27 In the step ST400 shown, the field angle center alignment section 116A1 controls the imaging optical system 38 (specifically, for example, the zoom lens 44) of the tele camera 12 so that the focal length of the imaging optical system 38 of the tele camera 12 becomes the longest. Thereby, the field angle center alignment section 116A1 can focus the tele camera 12 in a state where the depth of field is shallower than in a state where the focal length of the imaging optical system 38 of the tele camera 12 is the shortest. Also, in a case where there is a possibility that the subject distance to the subject set as the alignment target is shorter than the shortest subject distance at the tele end, the focusing can be performed while gradually changing the focal length to the wide angle side. Thus, according to this configuration, the derivation of the first subject distance can be performed with high accuracy according to the focusing result, compared to a case where the tele camera 12 is focused in a state where the focal length of the imaging optical system 38 of the tele camera 12 is the shortest. In addition, the technology of the present application is not limited to this, and a process equivalent to the process of the step ST400 can be applied to the end alignment process. In this case, the derivation of the second subject distance with high accuracy can be facilitated.

[0225] The above-described example illustrates a form example in which the imaging optical system 38 of the tele camera 12 is controlled so that the focal length of the imaging optical system 38 of the tele camera 12 becomes the longest, but the technology of the present application is not limited to this. For example, as shown in the step ST400 of Figure 28 In the step ST400 shown, the field angle center alignment section 116A1 can control the imaging optical system 38 (specifically, for example, the zoom lens 44) of the tele camera 12 so that the focal length of the imaging optical system 38 of the tele camera 12 enters a region on the longest side of the focal length of the imaging optical system 38 of the tele camera 12 among a plurality of regions determined in stages. In this case, the derivation of the first subject distance can be performed with high accuracy according to the focusing result, compared to a case where the tele camera 12 is focused in a state where the focal length of the imaging optical system 38 of the tele camera 12 is the shortest.

[0226] In addition, the technology of the present application is not limited to this, and a process equivalent to the process of the step ST400 (a process of controlling the imaging optical system 38 of the tele camera 12 so that the focal length of the imaging optical system 38 of the tele camera 12 enters a region on the longest side) can be applied to the end alignment process. In this case, the derivation of the second subject distance with high accuracy can be facilitated.

[0227] In the above embodiment, the horizontal offset and the depth offset are exemplified as predetermined values ​​stored in the predetermined value storage area 118A. However, the technology of the present invention is not limited to this. As predetermined values, information related to the height difference between the telephoto camera 12 and the wide-angle camera 14 (e.g., the height difference itself) may also be applied.

[0228] For example, such as Figure 29 As shown, when the telephoto camera 12 and the wide-angle camera 14 have a height difference, the alignment process of the panning direction (horizontal field of view) of the telephoto camera 12 is performed. When the pitch angle of the telephoto camera 12 is tilted so that the center of the telephoto camera's field of view is aligned with a specific subject is set as "θ", the distance from the telephoto camera 12 to the specific subject is set as "Sd1", and the distance from the wide-angle camera 14 to the specific subject is set as "Sd2", the alignment unit 116A derives "Sd1" according to the formula "Sd1 = Sd2 / cosθ". The alignment process of the panning direction (horizontal field of view) of the telephoto camera 12 is performed by using the derived "Sd1". Thus, by taking into account the height difference between the telephoto camera 12 and the wide-angle camera 14, the panning direction (horizontal field of view) of the telephoto camera 12 is aligned. Even if there is a height difference between the telephoto camera 12 and the wide-angle camera 14, information related to the field of view required when using the telephoto camera 12 and the wide-angle camera 14 together can be obtained.

[0229] Furthermore, in the technology of the present invention, considering that the alignment processing of the height difference between the telephoto camera 12 and the wide-angle camera 14 is not necessary, the alignment processing can be performed by ignoring the height difference between the telephoto camera 12 and the wide-angle camera 14.

[0230] In the above embodiments, an example is given of aligning one end of the telephoto camera's field of view with one end of the wide-angle camera's field of view by performing end alignment processing by the CPU 116; however, the technology of the present invention is not limited thereto. For example, one end of the telephoto camera's field of view may also be aligned with one end of the wide-angle camera's field of view.

[0231] Here, one end of the field of view refers to the portion outside the center of the field of view. In this case, for example, as... Figure 22A As shown, the half-angle view of the wide-angle camera 14 is calculated using coordinates (hereinafter also referred to as "pixel coordinates"), which determine the pixel position within the pixel row in the pixel group included in the image sensor 70 of the wide-angle camera 14, in the direction corresponding to the panning direction of the telephoto camera 12. Specifically, when the half-angle view of the wide-angle camera 14 is set to "θ e2" d2", the distance to the specific subject located at a position other than the center of the field angle (2nd subject distance) is set to "d target2 ", the rotation angle of the tele camera 12 from the original position is set to "θ e3 ", the pixel coordinate corresponding to one end of the field angle of the wide camera is set to "0 (origin)", the pixel coordinate corresponding to the position in the real space of the specific subject is set to "xl", the pixel coordinate corresponding to the center of the field angle of the wide camera is set to "x2", the horizontal direction offset amount is set to "d x ", the depth direction offset amount is set to "d y ", the end alignment portion 116A2 calculates the half angle of view θ e2 .

[0232] [Equation 3]

[0233]

[0234] Thus, the position at which the alignment of the tele camera image and the wide camera image is performed is not limited to the center of the field angle and the end of the field angle, but can also be a position other than the center of the field angle of the wide camera. If the alignment of the tele camera image and the wide camera image is performed at at least two positions, one position other than the center of the field angle of the wide camera and another position other than the center within the field angle of the wide camera, the half angle of view of the wide camera 14 is obtained.

[0235] In the above-described embodiment, the form example in which the 1st subject distance and the 2nd subject distance are calculated by the CPU 116 according to the focus position is exemplified, but the technology of the present application is not limited thereto. For example, either one of the 1st subject distance and the 2nd subject distance can be received by the receiver 24 or the like, or can be obtained by distance measurement in the phase difference AF method or the TOF method by a distance measurement device.

[0236] In the above-described embodiment, "pan" is exemplified as the tilt, but the technology of the present application is not limited thereto, and can also be applied to the tilt. In this case, for the field angle in the tilt direction (vertical field angle), it is only necessary that the processing equivalent to the alignment processing described in the above-described embodiment be executed by the CPU 116.

[0237] In the above embodiments, an example of CPU 116 performing alignment processing independent of changes in the scale ratio between the wide-angle camera image and the telephoto camera image has been described. However, it is also possible for CPU 116 to perform alignment processing when the scale ratio changes. In this case, for example, if optical information related to the wide-angle camera 14 (e.g., the specifications and focal length of the image sensor 70) is known, CPU 116 can apply the optical information related to the wide-angle camera 14 to limit changes in the scale ratio to a certain extent.

[0238] In the above embodiments, when performing end alignment processing, one end of the telephoto camera's field of view is aligned with one end of the wide-angle camera's field of view. However, the technology of the present invention is not limited to this. When performing end alignment processing, the center of the telephoto camera's field of view can also be aligned with one end of the wide-angle camera's field of view. In this case, it is sufficient to perform alignment image analysis (e.g., matching) using half of the wide-angle camera image and half of the telephoto camera image.

[0239] Furthermore, in Figure 22A In step ST102H1 shown, the opposite end of the telephoto camera's field of view is aligned with the opposite end of the wide-angle camera's field of view. However, the technology of this invention is not limited to this. Figure 22A In step ST102H1 shown, the center of the telephoto camera's field of view can also be aligned with the opposite end of the wide-angle camera's field of view. In this case, it is also possible to obtain... Figure 5 The example shown has the same effect.

[0240] In the above embodiments, an example of alignment processing being performed by the computer 104 of the management device 20 is given, but the technology of the present invention is not limited thereto. Alignment processing can be performed by the control device 60 of the telephoto camera 12, or by the control device 94 of the wide-angle camera 14, or by at least two of the computer 104 of the management device 20, the control device 60 of the telephoto camera 12, and the control device 94 of the wide-angle camera 14.

[0241] Furthermore, the alignment process can be performed by devices including ASICs, FPGAs, and / or PLDs instead of computer 104, or it can be achieved through a combination of hardware and software structures.

[0242] Furthermore, in the above embodiment, in NVM118 (reference) Figure 30 The device stores a bit alignment processing program 158, but the technology of the present invention is not limited thereto. As an example, such as ​As shown, the alignment processing program 158 can also be stored in a non-temporary storage medium, such as an SSD or a USB memory, or any portable storage medium 200. In this case, the alignment processing program 158 stored in the storage medium 200 is installed in the computer 104, and the CPU 116 executes the above-described alignment processing in accordance with the alignment processing program 158.

[0243] Also, the alignment processing program 158 is stored in a storage device, such as another computer or a server device connected to the computer 104 via a communication network (not shown), and the alignment processing program 158 can be downloaded and installed in the computer 104 in accordance with a request from the management device 20. In this case, the alignment processing is executed by the computer 104 in accordance with the installed alignment processing program 158.

[0244] As the hardware resource that executes the above-described alignment processing, various processors shown below can be used. As the processor, for example, a general-purpose processor, such as a CPU, can be cited, which functions as the hardware resource that executes the alignment processing by executing the software, that is, the alignment processing program 158, as described above. Also, as the processor, for example, a dedicated circuit that is a processor having a circuit structure, such as an FPGA, a PLD, or an ASIC, which is designed specifically to execute a specific process can be cited. In any of the processors, a memory is built in or connected, and any of the processors executes the alignment processing by using the memory.

[0245] The hardware resource that executes the alignment processing can be constituted by one of these various processors, or can be constituted by a combination of two or more processors of the same kind or different kinds (for example, a combination of a plurality of FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the alignment processing can be one processor.

[0246] As an example in which one processor is constituted, there is a first mode in which, as typified by a computer, such as a client and a server, one processor that functions as the hardware resource that executes the alignment processing is constituted by a combination of one or more CPUs and software. There is a second mode in which, as typified by an SoC or the like, a processor that functions as the hardware resource that executes the alignment processing is used. In this way, the alignment processing is implemented by using one or more of the above-described various processors as the hardware resource.

[0247] Further, as the hardware structure of these various processors, more specifically, a circuit in which circuit elements, such as semiconductor elements, are combined can be used.

[0248] Also, the above-described alignment processing is only an example. Therefore, needless steps can be deleted, new steps can be added, or the processing order can be switched, without departing from the scope of the gist of the present technology.

[0249] The above-described description and drawings are a detailed description of the part to which the present technology relates, and are only an example of the present technology. For example, the description regarding the above-described structure, function, action, and effect is a description regarding an example of the structure, function, action, and effect of the part to which the present technology relates. Therefore, needless parts can be deleted, new elements can be added, or substitutions can be made, to the above-described description and drawings, without departing from the scope of the gist of the present technology. Also, in order to avoid complication and in order to easily understand the part to which the present technology relates, the description and drawings shown above omit descriptions regarding common technical knowledge and the like, which are not particularly required in terms of implementing the present technology.

[0250] In the present specification, "A and / or B" has the same meaning as "at least one of A and B". That is, "A and / or B" means only A, only B, or a combination of A and B. Also, in the present specification, when a plurality of elements is connected in series, the phrase "and / or" can be used in the case of connecting three or more elements. In this case, the same concept as "A and / or B" can be applied.

[0251] All literature, patent applications, and technical standards cited in the present specification are cited by reference to the same extent as if each individual document, patent application, or technical standard were specifically and individually indicated to be cited by reference.

Claims

1. A camera system comprising: processor; A first camera device, having a first optical system; and The second camera device has a second optical system. The focal length of the first optical system is longer than that of the second optical system. The processor performs the following processing: Alignment is performed at at least two locations, including the end of the field of view used in the shooting based on the camera system, between a first camera image obtained by the first camera device and a second camera image obtained by the second camera device; Based on the alignment results, obtain field-of-view related information; and The field of view information is obtained based on the end alignment result. The end alignment result is obtained by aligning the first end of the field of view of the first camera device or the center of the field of view of the first camera device with the second end of the field of view of the second camera device.

2. The camera system according to claim 1, wherein, The processor obtains the field of view related information based on the field of view center alignment result, which is obtained by aligning the center of the field of view of the first camera device with the center of the field of view of the second camera device.

3. The camera system according to claim 2, wherein, The processor derives the field-of-view center alignment result and the end alignment result based on position-specific information that determines the positional relationship between the first camera device and the second camera device.

4. The camera system according to claim 3, wherein, The location-specific information includes information about the offset between the first camera device and the second camera device.

5. The camera system according to claim 3 or 4, wherein, The location-specific information includes information relating to the elevation difference between the first camera device and the second camera device.

6. The camera system according to claim 3 or 4, wherein, When the distance to the first subject is less than a first threshold, the processor performs field-of-view center alignment processing to align the center of the field of view of the first camera device with the center of the field of view of the second camera device using the position-specific information. When the distance to the second subject is less than a second threshold, the processor performs end alignment processing to align the center of the field of view of the first end or the first camera device with the second end using the position-specific information.

7. The camera system according to claim 6, wherein, The processor derives at least one of the distance to the first subject and the distance to the second subject.

8. The camera system according to claim 7, wherein, The first camera device has an autofocus function and can be rotatably mounted on a rotary table. The processor performs the following processing: When exporting the subject distance of at least one of the first subject distance and the second subject distance, the subject distance is exported using the autofocus function. If the subject distance cannot be derived using the autofocus function, the subject distance is derived using the autofocus function after the turntable is controlled based on the first camera image and the center of the field of view of the first camera device is aligned with a high-contrast subject near the center of the field of view of the first camera device.

9. The camera system according to claim 7 or 8, wherein, When the processor enters the region with the longest focal length of the first optical system, which is one of the multiple regions defined by the phased division of the focal length of the first optical system, it causes the first imaging device to focus, and derives at least one of the distance to the first subject and the distance to the second subject based on the focusing result.

10. The camera system according to claim 9, wherein, The processor enables the first camera device to focus when the focal length of the first optical system is at its longest, and derives at least one of the distance to the first subject and the distance to the second subject based on the focusing result.

11. The camera system according to claim 6, wherein, The processor performs the end alignment process after performing the field-of-view center alignment process.

12. The camera system according to claim 6, wherein, The processor performs at least one of the field-of-view center alignment processing and the end alignment processing using image analysis based on the first camera image and the second camera image.

13. The camera system according to claim 12, wherein, In the end alignment process, the processor aligns the center of the field of view of the first end or the first camera device with the second end based on the scale ratio of the first camera image and the second camera image used in the image analysis in the field of view center alignment process.

14. The camera system according to claim 6, wherein, Regarding the processor... Furthermore, an opposite-side end alignment process is performed, which involves aligning the first opposite-side end located on the opposite side of the first end among the two ends of the field of view of the first camera device, or the center of the field of view of the first camera device with the second opposite-side end located on the opposite side of the second end among the two ends of the field of view of the second camera device. and Based on at least one of the following: a first result in which the position of the center of the field of view of the first end or the first camera device is aligned with the position of the second end in the end alignment process; and a second result in which the position of the center of the field of view of the first opposite end or the first camera device is aligned with the position of the second opposite end in the opposite end alignment process; the field of view of the second camera device is obtained as the field of view related information.

15. The camera system according to claim 14, wherein, When the distance to the third subject is greater than or equal to the third threshold, and the position of the center of the field of view of the first camera device or the first camera device is aligned with the position of the second camera device, the processor obtains the field of view of the second camera device as the field of view related information based on the second result.

16. The camera system according to any one of claims 1 to 4, wherein, The optical information related to the first optical system is known. The optical information related to the second optical system is unknown.

17. The camera system according to any one of claims 1 to 4, wherein, The focal length of the first optical system is more than twice the focal length of the second optical system.

18. A camera system comprising: a processor; and a first camera device, which is used in conjunction with a second camera device having a second optical system and also having the first optical system. The focal length of the first optical system is longer than that of the second optical system. The processor performs the following processing: aligning a first image captured by the first camera device with a second image captured by the second camera device at at least two locations including the end of the field of view used in the shooting based on the camera system; obtaining field of view related information based on the alignment result; and obtaining the field of view related information based on the end alignment result, wherein the end alignment result is obtained by aligning the first end of the field of view of the first camera device or the center of the field of view of the first camera device with the second end of the field of view of the second camera device, which is located at the end of the field of view of the first camera device.

19. A method of operating a camera system, the camera system comprising: a processor; a first camera device having a first optical system; and a second camera device having a second optical system, wherein the focal length of the first optical system is longer than the focal length of the second optical system, the method of operating the camera system comprising the following steps: Alignment is performed at at least two locations, including the end of the field of view used in the shooting based on the camera system, between a first camera image obtained by the first camera device and a second camera image obtained by the second camera device; Based on the alignment results, obtain field-of-view related information; and The field of view information is obtained based on the end alignment result. The end alignment result is obtained by aligning the first end of the field of view of the first camera device or the center of the field of view of the first camera device with the second end of the field of view of the second camera device.

20. A storage medium storing a program for causing a computer to perform processing, the computer being adapted for a camera system, the camera system comprising a processor, a first camera device having a first optical system, and a second camera device having a second optical system, wherein the focal length of the first optical system is longer than the focal length of the second optical system, the processing comprising the following steps: Alignment is performed at at least two locations, including the end of the field of view used in the shooting based on the camera system, between a first camera image obtained by the first camera device and a second camera image obtained by the second camera device; Based on the alignment results, obtain field-of-view related information; and The field of view information is obtained based on the end alignment result. The end alignment result is obtained by aligning the first end of the field of view of the first camera device or the center of the field of view of the first camera device with the second end of the field of view of the second camera device.

Images