Imaging apparatus, control method, and program

Abstract

To achieve an optimal imaging assistance function in accordance with an imaging purpose and achieve an imaging apparatus that provides imaging assistance information to an imaging person.SOLUTION: The imaging apparatus includes an acquisition unit (CPU 121) that acquires difference information from a captured image of a first camera 100, a providing unit (CPU 121) that provides the acquired difference information or imaging assistance information for assisting one or more captured images generated on the basis of the difference information, and a switching control unit (CPU 121) that controls the display switching of imaging assistance information displayed on a display unit 131, where the captured image is displayed, according to the provided imaging assistance information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an imaging apparatus, a control method and a program therefor, and more particularly to an imaging apparatus having an imaging assist function for assisting an image-taker in imaging. [Background technology]

[0002] Conventionally, imaging devices are known that provide an imaging assist function that assists imaging with the main camera by notifying the photographer of the position of the subject outside the angle of view of the main camera using imaging information acquired by a sub camera with a wider imaging angle than the main camera. For example, when the angle of view of the main camera is included within the angle of view of the sub camera, the positional correspondence between the angle of view of the main camera and the angle of view of the sub camera may be known. In this case, if a subject that was once lost by the main camera with a narrow angle of view is within the angle of view of the sub camera with a wide angle of view, the photographer can easily recognize whether or not the subject can be captured again within the angle of view of the main camera by moving either camera. This makes it possible to provide an imaging assist function.

[0003] Nowadays, there are various methods for acquiring information about a subject. For example, there are "360° hemispherical cameras" and "omnispherical cameras" that have a hemispherical or omnispherical angle of view centered on the photographer. There are also cameras that capture images at wavelengths different from visible light, such as near-infrared cameras. There are also cameras that can acquire the absolute distance to the subject, such as LIDAR (Light Detection And Ranging). By combining and using devices that acquire such different types of subject information as a main camera and a sub camera, it is possible to provide advanced imaging assistance according to various uses and imaging purposes.

[0004] In the framing assist function, even if the subject is outside the angle of view of the main camera, subject information such as the position and speed of the subject is displayed on the display unit of the camera such as the viewfinder or live view (LV) screen. Thus, one of the representative functions is to provide imaging assistance information to facilitate the photographer's imaging. For example, Patent Document 1 discloses imaging with an imaging assist function that combines two types of visible light cameras with different angles of view. By displaying an image captured by a telephoto zoom main camera superimposed on an image captured by a sub camera with a wider angle of view than the main camera, the main subject may be framed out of the angle of view of the main camera while being captured by the main camera. However, even in such a case, if the main subject is within the angle of view of the sub camera with a wider angle of view, it is easy to capture the main subject again within the angle of view of the main camera.

[0005] Furthermore, Patent Document 2 discloses an imaging device that, as an example of displaying imaging assist information, superimposes on the display unit of the main camera the position of a subject outside the field of view of the main camera (the relative angle with respect to the optical axis of the main camera), the subject's velocity vector, etc. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] JP 2010-273046 A [Patent Document 2] JP 2007-74143 A Summary of the Invention [Problem to be solved by the invention]

[0007] However, the imaging device disclosed in Patent Document 1 does not disclose changing the type of the main and sub cameras, and there is a risk that the photographer cannot be provided with the optimal imaging assist function when the type of the main and sub cameras is changed depending on the imaging purpose. Also, the imaging device disclosed in Patent Document 2 discloses an example of providing the photographer with imaging assist information that displays the position and speed of a subject outside the angle of view of the main camera on the display unit of the main camera, but does not disclose the following. In other words, as in Patent Document 1, there is no disclosure of what to do when the type of the main and sub cameras is changed, and there is a risk that the photographer cannot be provided with the optimal imaging assist function when the type of the main and sub cameras is changed depending on the imaging purpose.

[0008] An object of the present invention is to provide an imaging apparatus that provides display content according to imaging assist information and can realize an optimal imaging assist function according to an imaging purpose, and a control method and program thereof. [Means for solving the problem]

[0009] In order to achieve the above-mentioned object, the present invention is characterized in comprising an acquisition unit that acquires difference information from an image captured by a first camera, a provision unit that provides the acquired difference information or imaging assist information that assists in one or more images generated based on the difference information, and a switching control unit that controls display switching of the imaging assist information displayed on a display unit on which the captured image is displayed, in accordance with the provided imaging assist information. Effect of the Invention

[0010] According to the present invention, it is possible to obtain an effect that it is possible to provide display contents according to the imaging assist information and realize an imaging assist function that is optimal for the imaging purpose. [Brief description of the drawings]

[0011] [Figure 1] 1 is a configuration diagram of an imaging apparatus according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a schematic explanatory diagram showing an example of connection between the first and second cameras. [Diagram 3] 13 is an explanatory diagram of a display example of imaging assist information when a semi-spherical camera is connected. FIG. [Figure 4] 13 is an explanatory diagram of a display example of imaging assist information when a wide-angle IR camera is connected. FIG. [Diagram 5] FIG. 11 is an explanatory diagram of an example of connection in which a plurality of types of second cameras are connected. [Figure 6] 11A to 11C are diagrams illustrating an example of switching the display of imaging assist information. [Figure 7] 13 is a flowchart showing a process for switching the display of imaging assist information. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. However, the configurations described in the following embodiments are merely examples, and the scope of the present invention is not limited to the configurations described in the embodiments. First, a first embodiment of the present invention will be described. In the following, "imaging assist information" refers to information for assisting an image-capturing person in imaging.

[0013] <<First embodiment>> In this embodiment, an example will be described in which the first camera is a general digital camera (hereinafter referred to as a "normal camera"), and the second camera is one of a hemispherical camera, LIDAR, IR camera, etc., and is combined with the normal camera to provide "imaging assist information." Also, an example will be described in which multiple second cameras are used in combination with the first camera.

[0014] <Overall composition> 1 shows an example of a configuration in which a first camera 100 and a second camera 200 are connected to provide an imaging assist function. The first camera 100 (imaging device) is a digital camera system capable of imaging with visible light, equipped with a camera body and an interchangeable lens (imaging optical system or imaging optical system) that is detachable from the camera body. However, the present invention is not limited to this, and can also be applied to an imaging device in which the camera body and lens are integrally configured, for example.

[0015] First camera 100 (imaging device) has lens 101, imaging element 107, imaging element drive circuit 124, image processing circuit 125, lens drive circuit 126, CPU 121, and external camera connection and communication unit 135. Display unit 131, operation unit 132, storage medium 133, and storage unit 134 are also connected to CPU 121, and are configured to be able to transmit and receive required information to and from CPU 121.

[0016] (Lens 101) The imaging optical system (image pickup optical system) generates a subject image (optical image) of a subject on a predetermined imaging plane. The lens 101 of the first camera 100 is composed of a group of multiple lenses that constitute the imaging optical system. The lens 101 may be a fixed focal length lens or a zoom lens with a variable focal length zoom function.

[0017] (Image sensor 107) The image sensor 107 is configured with, for example, a CMOS sensor or a CCD sensor and its peripheral circuits, and performs photoelectric conversion according to a subject image formed on the image sensor 107 by an imaging optical system. The image sensor 107 is provided with a two-dimensional single-plate color sensor in which a primary color mosaic filter in a Bayer array is formed on-chip on light receiving pixels arranged in a matrix of, for example, "m" pixels in the horizontal direction and "n" pixels in the vertical direction. The imaging optical system and the image sensor 107 constitute one "imaging section", but are not limited to the single-plate type shown in this embodiment. For example, a three-plate type may be used. Also, a configuration having multiple imaging sections may be used. In other words, the present invention is applicable as long as the imaging optical system is corresponding to the image sensor 107.

[0018] (CPU121) The CPU 121 functions as a control unit that manages various controls of the first camera 100. The CPU 121 has a ROM, a RAM, an A / D converter, a D / A converter, a communication interface circuit, and the like. For example, the CPU 121 reads out a program stored in the ROM, expands it in the RAM, and executes the expanded program. This realizes required functions and processing. For example, the CPU 121 executes a program stored in the ROM to drive various circuits of the first camera (imaging device) 100 and control a series of operations such as focus detection (AF), imaging, image processing, and recording.

[0019] Note that some of the functions of the CPU 121 may be realized as hardware such as ASIC, and some of the circuits may be reconfigurable circuits such as FPGA. For example, some of the calculations for focus detection, which will be described later, may be performed using a dedicated hardware circuit to reduce the calculation time.

[0020] The communication interface circuit of the CPU 121 may be a system for connecting to an external device via a wired cable such as a USB cable or a LAN cable, and may be a system for connecting via wireless communication such as a wireless LAN or a mobile communication line. The system for connecting to a communication partner is not limited to a system for directly connecting to a personal computer, a smartphone, or the like, and may be a system for connecting to a nearby or remote device via an access point or a network.

[0021] (Imaging element driving circuit 124: Image processing circuit 125: Lens driving circuit 126) The imaging element drive circuit 124 controls the imaging operation of the imaging element 107, and also A / D converts the image signal acquired from the imaging element 107 and transmits it to the CPU 121. The image processing circuit 125 performs various processes such as gamma conversion, color interpolation, and JPEG (Joint Photographic Experts Group) compression on the image data output from the imaging element 107. The lens drive circuit 126 drives the lens 101 based on a drive signal provided by the CPU 121 to perform focus adjustment and the like.

[0022] (Display section 131: Operation section 132) The display unit 131 is composed of a display device such as an LCD (liquid crystal display device). The display unit 131 displays various information and images such as information about the imaging mode of the first camera (imaging device) 100, a preview image before imaging, a confirmation image after imaging, and an in-focus state display image at the time of focus detection. The operation unit 132 is composed of a power switch, a release switch, a zoom operation switch, an imaging mode selection switch, etc. The release switch is a two-stage switch with a half-pressed state (SW1 is ON) and a fully-pressed state (SW2 is ON).

[0023] (Storage medium 133: Storage unit 134) Storage medium 133 is, for example, a storage device such as a flash memory or a memory card that is detachable from first camera (imaging device) 100, and stores captured images (image data) etc. in a non-volatile manner. Storage unit 134 stores captured images etc. in a predetermined format in a non-volatile manner.

[0024] (External camera connection and communication unit 135) The external camera connection and communication unit 135 is a connection unit for separately connecting an external imaging device having the same functions as the first camera (imaging device) 100 from the outside. The CPU 121 functions as a communication connection unit for transmitting control signals for driving various actuators, imaging elements, etc. in the external imaging device, and for receiving image signals from an external imaging element. The external camera connection and communication unit 135 may be any unit capable of connecting one or more external cameras.

[0025] Also, a part of the functions of the operation unit 132 may be provided in the form of a touch panel on the display unit 131. By operating the touch panel while a preview image is being displayed on the display unit 131, it becomes possible to perform focus detection for any position in the image.

[0026] FIG. 1 shows an example in which a second camera (imaging device) 200 having the same function as the first camera (imaging device) 100 described above is connected to the first camera (imaging device) 100. In this embodiment, the imaging device is the first camera 100, and an imaging device with a wider angle than the first camera 100 is the second camera 200. Like the first camera (imaging device) 100, the second camera (imaging device) 200 has a lens 201, an imaging element 207, an imaging element driving circuit 224, an image processing circuit 225, a lens driving circuit 226, a CPU 221, and an external camera connection communication unit 235. Similarly, a display unit 231, an operation unit 232, a storage medium 233, and a storage unit 234 are connected to the CPU 221 of the second camera (imaging device) 200, and required information can be transmitted and received from the CPU 221. Therefore, the detailed configuration of the second camera (imaging device) 200 is the same as that of the first camera 100, so a duplicated description will be omitted.

[0027] The second camera (imaging device) 200 may be sensitive to visible light, or may be sensitive to near-infrared wavelengths such as an IR sensor. The second camera (imaging device) 200 does not necessarily have to include the display unit 231, the operation unit 232, the storage medium 233, and the storage unit 234, and may be an imaging device that can acquire difference information with the first camera 100 (imaging device).

[0028] <FIG. 2: Example of changing the second camera 200 of a different type> FIG. 2 is a schematic explanatory diagram showing an example of a connection between the first camera 100 and the second camera 200. In FIG. 2, the first camera 100 is a general digital camera, and the second camera 200 is a 360° semi-spherical camera or a wide-angle IR camera. This is an example of a configuration in which framing assist display is performed by both of them. The second camera 200 is attached to an accessory shoe on the upper surface of the first camera 100. The accessory shoe is provided with an electrical terminal for communication, and the first camera 100 and the second camera 200 are electrically connected by their respective electrical terminals. In addition to connecting using a dedicated connection terminal in this way, the connection may also be performed using a general-purpose interface such as a USB cable.

[0029] <FIG. 2(a): When the second camera 200 is a 360° hemispherical camera> First, an example will be described in which a "360° semi-spherical camera" is used as the second camera 200. The main purpose of using a 360° semi-spherical camera as the second camera 200 is to capture many flying subjects coming from all directions without missing them. The lens 201 of the second camera 200 shown in Fig. 2(a) is a lens with an angle of view covering the upper hemisphere, and the second camera 200 can capture images in all directions of the upper hemisphere.

[0030] When the first camera 100 and the second camera 200 are connected, the following process is executed first. That is, a "signal" indicating that the second camera 200 is a 360° semi-spherical camera is transmitted from the CPU 221 of the second camera 200 to the CPU 121 of the first camera 100 via the external camera connection communication unit 235 and the external camera connection communication unit 135. The "signal" is then stored in the storage unit 134. As a result, the first camera 100 recognizes and stores that the type of the connected second camera 200 is a 360° semi-spherical camera. Next, a calibration is executed to establish correspondence between the positions of the angles of view of the two first and second cameras 100 and 200.

[0031] (calibration) In calibration, first, the photographer uses operation unit 132 to instruct the start of calibration. In response to this, display unit 131 of first camera 100 and display unit 231 of second camera display captured images. The photographer frames the cameras so that at least three or more common subjects are captured on both display unit 131 and display unit 231. Furthermore, the photographer operates operation unit 132 of first camera 100 and operation unit 232 of second camera 200 to instruct the cameras as to the positions of the aforementioned common subjects.

[0032] The coordinates of the angle of view of the second camera 200 instructed by the image capturer are transmitted from the CPU 221 of the second camera 200 to the CPU 121 of the first camera 100 via the external camera connection communication unit 235 and the external camera connection communication unit 135. The CPU 121 of the first camera 100 executes the following process using the received data of the coordinates of the second camera 200 in the angle of view of the first camera 100 instructed by the image capturer. That is, a calculation based on known epipolar geometry is performed to establish correspondence between the coordinates of the angles of view of the first camera 100 and the second camera 200. The CPU 121 stores the result in the storage medium 133.

[0033] A case will be described in which the focal length of lens 101 of first camera 100 is about "600 (mm)" and telephoto captures a flying "wild bird" as a subject. In this case, the angle of view of first camera 100 is about "4°", and it may be difficult to capture the subject within the angle of view.

[0034] <Figure 3(a):Figure 3(b):Example of imaging assist information displayed when a semi-spherical camera is connected> 3(a) and 3(b) are display examples when display unit 131 of first camera 100 is displaying a live view screen. An image captured by first camera 100 is displayed across the entire screen of display unit 131. Lens 101 of first camera 100 is directed toward the sky to capture images of wild birds. First camera 100 stores data for a plurality of imaging assist display patterns, which are displayed according to the type of second camera 200 connected to it, in storage unit 134 or storage medium 133 as an "imaging assist data table."

[0035] CPU 121 of first camera 100 extracts imaging assist information to be displayed from the "imaging assist data table" in response to a signal indicating the type of second camera 200, and displays it on display unit 131. When CPU 121 determines that second camera 200 is a 360° semi-spherical camera as in this embodiment, a radar chart such as that shown in Fig. 3(a) is displayed on display unit 131 (see the lower left of the drawing).

[0036] Reference numeral 301 denotes a radar chart in which the angle of view range of second camera 200 is projected two-dimensionally, and is displayed superimposed on the image captured by first camera 100. Reference numeral 302 denotes the angle of view range of first camera 100 as a circular arc. In FIG. 3(a), the subject "wild bird" is not included in the angle of view of first camera 100. Furthermore, second camera 200 detects the subject from the captured image, and when a subject is detected, the coordinates of the subject are transmitted from second camera 200 to first camera 100, and the position of the subject is displayed as a symbol, as in 303.

[0037] <Figure 3(a); Example of imaging assist information display> In this embodiment, Fig. 3(a) shows a display when a subject is detected in the sky above and to the left rear by the second camera 200. The photographer can recognize that the subject can be captured by the first camera 100 by framing the camera to the left rear by observing the display of the live view image. Thus, Fig. 3(b) shows a display when the photographer frames and captures the subject 304 with the first camera 100. This framing is performed so that the photographer captures a "wild bird" as the subject with the first camera 100. By providing the photographer with imaging assist information as in this embodiment, it becomes easier for the photographer to capture the subject by telephoto imaging.

[0038] <Figure 3(a): Radar chart display> As shown in FIG. 3(b), in the radar chart displayed on display unit 131 of first camera 100, it can be seen that the subject "wild bird" is displayed as a symbol 303 within the field of view of first camera 100.

[0039] <FIG. 2(b): When the second camera 200 is a wide-angle IR camera> Next, an example in which the second camera 200 is replaced with a wide-angle IR camera will be described. The second camera 200 in FIG. 2(b) is a wide-angle IR camera (hereinafter referred to as "IR camera"). In FIG. 2(b), 201 is an infrared condensing lens that captures and condenses infrared rays. The IR camera (200) can capture images with a wider angle of view than the first camera 100. As with the 360° hemispherical camera described above, the IR camera (200) may connect the electrical circuits of both cameras via a dedicated connection terminal provided on the accessory shoe on the top surface of the first camera 100, or may be connected via a general-purpose interface such as a USB cable.

[0040] The IR camera 200 can detect near-infrared wavelengths emitted by living organisms, making it possible to distinguish whether the subject is a living organism or not. IR stands for infrared, and the IR camera 200 is capable of detecting living organisms in particular by utilizing the fact that objects with temperatures higher than the absolute temperature "0 (K)" emit infrared rays. For example, subjects flying in the sky include wild birds, airplanes, and other flying objects (balloons, etc.), and by using the IR camera (200) as the second camera 200, it is a useful imaging device (information acquisition unit) in cases where you want to capture images of only wild birds, for example.

[0041] (Calibration, etc.) Even when an IR camera (200) is connected to the first camera 100, the CPU 221 of the second camera 200 transmits a signal to the first camera 100 indicating that the second camera 200 is an IR camera. The CPU 121 of the first camera 100 recognizes that the second camera 200 is an IR camera (200). Then, similar to the 360° semi-spherical camera described above, a calibration is performed to establish correspondence between the coordinates of the angles of view of the two cameras, the first camera 100 and the second camera 200.

[0042] <Figure 4(a): Example of imaging assist information displayed when a wide-angle IR camera is connected> When an IR camera (200) is connected, the preferred imaging assist information is to display information that allows the photographer to distinguish whether the subject is a living body or a non-living body. FIG. 4(a) shows an example of the display in this case. 311 shows an image captured by the first camera 100 displayed on the display unit 131 of the first camera 100. The first camera 100 automatically switches the display so as to superimpose the captured image depending on the type of the second camera 200. 312 shows the imaging angle of view range of the IR camera (200). Also, 313 shows the angle of view range of the first camera 100. In this example, as shown by the relationship between 312 and 313 in FIG. 3, a part (one area) of the angle of view of the IR camera (200) is the angle of view of the first camera 100. The IR camera (200) performs subject detection on the captured infrared image.

[0043] In Fig. 4(a), the IR camera (200) detects two subjects 314, 315, which are displayed in the lower left of the imaging screen of the first camera 100. In this example, 314 is detected as a living subject and 315 is detected as a non-living subject, and 314 (wild bird) and 315 are displayed to be distinguished by symbols or the like, as in this example. In the example of Fig. 4(a), since the living subject (wild bird) 314 is displayed in the upper left of the angle of view range of the IR camera (200), the photographer can recognize that by framing the first camera 100 at the upper left, the wild bird 314 can be captured within the angle of view of the first camera 100.

[0044] <Figure 4(b): Example of imaging assist information display when LIDAR is connected> In this example, a case where a 360° hemispherical camera and an IR camera are used as the second camera 200 has been described, but a special camera such as a LIDAR may also be used as the second camera 200. FIG. 4(b) is a display example when a LIDAR is used as the second camera. The LIDAR can detect the distance to the subject and estimate the absolute speed and real size of the subject. 316 displays the "size (large) of the subject," "speed (V) and distance (D) to the subject" detected and estimated by the LIDAR in text, and displays the "direction of movement" with an arrow. As described above, in the present invention, the display of imaging assist information that is easy to understand and preferable for the photographer is switched and displayed depending on the type of the connected second camera 200.

[0045] <An example of framing assist display when two types of second cameras (a 360° semi-spherical camera and a wide-angle IR camera) are simultaneously connected to a telephoto camera> Next, an example will be described in which two types of second cameras 200a, 200b (a 360° semi-spherical camera and a wide-angle IR camera) are simultaneously connected to a telephoto camera to perform framing assist display.

[0046] <Figure 5: An explanatory diagram of an example of connecting multiple types of second cameras> 5 is an explanatory diagram of an example in which a telephoto camera is connected as the first camera 100, and a wide-angle IR camera and a 360° semi-spherical camera are connected simultaneously as the second camera 200. The first camera 100 and the second camera 200 used in this embodiment are the same as those described in the first embodiment.

[0047] 200a, 201a are 360° semi-spherical cameras and lenses, and 200b, 201b are wide-angle IR cameras and lenses. The wide-angle IR camera 200b is connected to an accessory shoe on the top of the telephoto camera (100), and the 360° semi-spherical camera 200a is connected to an accessory shoe on the top of the wide-angle IR camera 200b. The three cameras are connected to each other through a dedicated connection terminal provided on the accessory shoe. In this embodiment, the three cameras are connected in series in the vertical direction, but the first camera 100 and two types of second cameras 200 may be connected in parallel. Also, the connection may be made via a general-purpose interface such as a USB cable without using a dedicated connection terminal.

[0048] When the first camera 100 and the second camera 200 are connected, the following process is executed first. That is, the CPUs 221 and 221 of the second cameras 200a and 200b transmit signals to the CPU 121 of the first camera 100 indicating that the type of the second camera 200 is the 360° semi-spherical camera 200a and the wide-angle IR camera 200b. As a result, the CPU 121 of the first camera 100 recognizes that the 360° semi-spherical camera 200a and the wide-angle IR camera 200b are connected as the second camera 200.

[0049] (calibration) Next, in this embodiment, calibration is performed to establish correspondence between the angles of view of the second cameras 200a, 200b, as in the first embodiment. In this embodiment, it is sufficient that the angles of view of at least the first camera 100 and the second cameras 200a, 200b correspond to each other, and therefore it is not essential that the angles of view of the second cameras 200 correspond to each other.

[0050] <FIG. 7: Flowchart showing the display switching process of the imaging assist information> Hereinafter, similar to the first embodiment, the process of switching imaging assist information when capturing an image of a "wild bird" as a subject will be described with reference to the flowchart in Fig. 7. In Fig. 7, the telephoto camera 100 is referred to as "camera 1", the wide-angle IR camera as "camera 2b", and the 360° semi-spherical camera as "camera 2a". The order of the size of the angle of view range of each camera, from largest to smallest, is the 360° semi-spherical camera, the wide-angle IR camera, and the telephoto camera.

[0051] First, the process starts in step S1. Next, in step S2, the wide-angle IR camera 200b detects the subject. Next, in step S3, the CPU 121 judges the detection result. If the subject is not detected (No), in step S5, the 360° semi-spherical camera 200a detects the subject. Next, the CPU 121 judges the detection result in step S6. If the subject is detected in step S6 (Yes), the CPU 121 proceeds to step S7. Then, the CPU 121 superimposes the subject position of the 360° semi-spherical camera 2a on the image captured by the telephoto camera in a radar chart display as shown in FIG. 3(a) of the first embodiment. After executing step S7, the process returns to step S2. Also, if the subject is not detected in step S6 (No), the process returns to step S2.

[0052] <Figure 6: Example of switching the display of imaging assist information: Figure 6(a): Display of subject position of 360° hemispherical camera 2a: Figure 6(b): Display of subject of wide-angle IR camera 2b> On the other hand, if a subject is detected by the wide-angle IR camera 2b in step S3 (Yes), the CPU 121 proceeds to step S4 and switches the display from a radar chart display to a superimposed display of the angle of view of the wide-angle IR camera 2b, as shown in Fig. 6(a) and Fig. 6(b). Fig. 6(a) shows a radar chart display on the display unit 131 by the CPU 121 of an example of imaging by the 360° semi-spherical camera 2a as described in Fig. 3(a). On the other hand, Fig. 6(b) shows an example of imaging by the wide-angle IR camera 2b, with the imaging angle of view range 312 of the IR camera 200, the angle of view range 313 of the first camera 100, and the subject (wild bird) 314, as described in Fig. 4(a).

[0053] In this way, when a subject is detected by 360° semi-spherical camera 2a in step S6, CPU 121 displays the radar chart shown in Fig. 6(a). On the other hand, when subject 314 is detected by wide-angle IR camera 2b in step S3, CPU 121 switches to the display as shown in Fig. 6(b). In this way, a display according to the imaging result of second camera 200 that detected the subject is displayed at the bottom left of display unit 231.

[0054] This display switching process is performed because when the subject is relatively close to the telephoto camera in the angle of view, the superimposed display of the angle of view of the wide-angle IR camera 2b is more preferable as imaging assist information because it is easier for the photographer to recognize the position of the subject. After step S4 is executed, the process returns to step S2.

[0055] In the first embodiment, an example was shown in which the imaging assist display was automatically switched according to the subject detection result by second camera 200, but a display switching function may be assigned to a button on the camera so that switching can be performed manually. In this manner, a configuration can be adopted in which CPU 121 controls switching of imaging assist information displayed on the imaging screen of first camera 100 according to difference information that can be acquired by second camera 200.

[0056] <<Second embodiment>> The second embodiment shows an example of automatic imaging using an automatic camera platform, AICam using artificial intelligence (AI), etc. In the second embodiment, a first camera 100 and one or more second cameras 200 are connected in the same manner as in the first embodiment, and the first camera 100 is installed on an automatic camera platform for automatically framing in response to a control command from the first camera 100. The automatic camera platform is configured to be rotatable in the pan direction and tilt direction in response to a control command from the CPU 121 of the first camera 100.

[0057] Specifically, CPU 121 controls the driving of an automatic camera platform so that first camera 100 is mounted on the automatic camera platform and captures an image by tracking the position of a specific subject (e.g., a living subject) as imaging assist information. Alternatively, first camera 100 and second camera 200 may be mounted on a mounting device that has a function equivalent to an automatic camera platform and can be controlled by CPU 121. Also, first camera 100 has an imaging mode setting function for setting an imaging mode for performing automatic imaging.

[0058] The imaging mode setting function allows the user to select an imaging mode such as "normal", "biome priority", "subject size selection", "subject speed selection", etc. The imaging modes that the user can select change depending on the type of second camera 200 that is connected. For example, when wide-angle IR camera 2b is connected, it is possible to select "biome priority".

[0059] In addition, when LIDAR is connected, it is possible to select "subject size selection" and "subject speed selection". Furthermore, when "subject size selection" is selected, it is possible to select the subject size to be imaged with priority from among "large (1m or more)", "medium (50cm to 1m)", and "small (up to 50cm)". Furthermore, when "subject speed selection" is selected, it is possible to select the subject speed to be imaged with priority from among "high speed (100km / h or more)", "medium speed (30 to 100km / h)", and "low speed (less than 30km / h)". Here, an example is shown in which "subject size" and "subject speed" are selected in three categories, but it is also possible to select four or more categories according to the resolution of the LIDAR to be used, or two categories.

[0060] If the lens of first camera 100 is a variable focal length zoom lens, the "imaging size" can be selected from "large (80% of the angle of view)", "medium (50% of the angle of view)" and "small (30% of the angle of view)". The imaging size may also be divided into categories other than three. The number of categories for the "subject size", "subject speed" and "imaging size" described above is merely an example and is not limited to the number of categories described above.

[0061] First, when performing automatic imaging, the user sets the imaging mode as described above. A plurality of imaging modes may be set. When a plurality of imaging modes are set, the user may additionally set priorities such as the display order and selection order of the imaging modes. Next, after setting the imaging mode, the user presses the imaging start button of the first camera 100, and in response, the CPU 121 starts automatic imaging. It is assumed that the correspondence between the angles of view of the first camera 100 and the second camera 200 in the first embodiment has been implemented in advance.

[0062] <Operation example of the second embodiment> Next, an example of the operation when each imaging mode is selected will be described. The following operation is mainly executed by CPU 121. When the photographer selects "normal", CPU 121 executes automatic imaging by automatically framing the subjects detected by second camera 200 in order from the subjects closest to the angle of view of first camera 100. CPU 121 tracks the subjects detected by second camera 200 to distinguish subjects that have already been imaged in order to avoid duplicate imaging. For example, CPU 121 may add an identifier for distinguishing subjects that have already been imaged and store them in storage unit 134.

[0063] If the photographer selects “biome priority,” the CPU 121 performs automatic imaging by automatically framing subjects detected as biomemories by the wide-angle IR camera 200b in order from the closest subject to the angle of view of the first camera 100.

[0064] When the imaging of all the detected "biological subjects" is completed, CPU 121 continues by imaging the detected "non-biological subjects". If a new "biological subject" is detected while imaging a "non-biological subject", CPU 121 temporarily suspends imaging of the "non-biological subject" and images the newly detected "biological subject". In the present embodiment, an example has been shown in which imaging of a "non-biological subject" is also performed, but it is also possible to set an imaging mode of "biological subjects only" in which only "biological subjects" are imaged.

[0065] When the photographer selects "subject size selection" and "subject speed selection", the CPU 121 performs imaging with priority from the subject that corresponds to the subject size and subject speed set as the priority imaging. When "imaging size" is selected, the CPU 121 detects the imaging size of the subject once when the subject is captured within the angle of view of the first camera 100, and if it differs from the selected "subject size", the following process is performed. In other words, the CPU 121 drives the zoom lens until the angle of view becomes one that can be captured with the set "subject size" and performs automatic imaging. If it is outside the movable limit of the zoom lens, the angle of view of the zoom lens is changed to the movable limit and automatic imaging is performed.

[0066] The CPU 121 generates the imaging assist information described below. That is, the imaging assist information is one or more of the acquired difference information or the imaging assist information generated based on the difference information, the subject position, the subject size, the distance from the subject, the subject speed, information indicating that the subject is a living subject, and information indicating the moving direction of the vehicle body. However, the assist information is not limited to these examples. In addition, when a plurality of imaging assist information are provided, the display unit 131 can be configured to perform display switching control (CPU 121) of the imaging assist information to be displayed.

[0067] As described above, according to the embodiment of the present invention, CPU 121 acquires difference information from an image captured by first camera 100 (acquisition section), and provides the acquired difference information or one or more pieces of imaging assist information generated based on the difference information (provision section). Then, CPU 121 controls display switching of the imaging assist information displayed on display section 131 on which the captured image is displayed, according to the provided imaging assist information (switching control section). As a result, for example, telephoto imaging of a distant moving subject according to the imaging purpose becomes easy, and optimal imaging can be performed, improving user satisfaction.

[0068] <Variation 1> When multiple types of second cameras are connected to one first camera 100, the display control described below can also be performed. As described above, Fig. 6(a) and Fig. 6(b) are explanatory diagrams of the display of the first camera 100 for each of the cases where two types of second cameras 200 are connected to the first camera 100. Fig. 6(a) is a display example for the 360° semi-spherical camera 2a, and Fig. 6(b) is a display example for the wide-angle IR camera 2b. In this way, in the above-mentioned embodiment, image information showing difference information with each second camera 200 is displayed separately.

[0069] However, there are cases where the photographer wants to display a list of image information (hereinafter referred to as "difference image information") showing difference information with the second camera 200 each having a special ability, and to recognize the presence of the living subject 314 from various angles and perform framing, etc. Therefore, when the CPU 121 detects a specific operation of the operation unit 132, it can also be configured to display a list of difference image information between the captured image of the first camera 100 and each of the second cameras 200 in thumbnail format. In this case, similar processing can be realized even if three or more types of second cameras 200 are connected to the first camera 100. Even in this case, it is not necessary to use the entire screen of the display unit 131 of the first camera 100, and it is preferable to display the list in a small vertical direction on the left side of the screen, for example. When the display area in the vertical direction on the left side of the screen is insufficient, all the difference image information can be made visible by performing a scroll operation to scroll upward on the display unit 131 of the touch panel.

[0070] Furthermore, when a plurality of pieces of differential image information with the second camera 200 become unnecessary, the photographer can, for example, perform a double tap operation on the differential image information to erase it from the display unit 131. Furthermore, instead of only arranging the differential image information on the left side of the first camera 100, the photographer can also move the differential image information like an icon to a position that is easy for the photographer to observe by touching and moving the differential image information and then releasing the finger.

[0071] <Variation 2> When a living subject is present in the difference image information, the CPU 121 can display a message to that effect on the display unit 131. The CPU 121 can also drive a voice synthesis LSI or the like to emit a specific sound from a small built-in speaker to call attention. Such attention allows the photographer to reliably recognize living subjects, which are particularly common as subjects for image capture, and capture the living subject without missing it by performing a series of image capture operations such as framing and pressing a release button.

[0072] <Additional Note> The disclosure of this embodiment includes the following configuration, method, and program. (Configuration 1) an acquisition unit that acquires difference information from an image captured by the first camera; a providing unit that provides the acquired difference information or imaging assist information that is generated based on the difference information and that assists in one or more imaging operations; an acquisition unit that acquires difference information from an image captured by the first camera; a switching control unit that controls switching of display of imaging assist information displayed on a display unit that displays the captured image in accordance with the provided imaging assist information. (Configuration 2) The acquisition unit is 2. The imaging device according to configuration 1, wherein difference information is obtained between an image captured by the first camera and an image captured by one or more second cameras having a wider angle than the first camera. (Configuration 3) The providing unit is The imaging device according to configuration 1 or 2, characterized in that it generates the imaging assist information from the acquired difference information or information generated based on the difference information, the imaging assist information being one or more of information indicating the subject position, the subject size, the distance to the subject, the subject speed, and information indicating the subject being a living subject and the moving direction of the subject's body. (Configuration 4) 3. The imaging device according to configuration 2, wherein the second camera is a camera having a viewing angle of 180° or more, or a special camera capable of acquiring information of a wavelength different from that of the first camera. (Configuration 5) 3. The imaging device according to configuration 1 or 2, further comprising a display unit that displays a live view image, an image in the viewfinder, and imaging information acquired by the first camera. (Configuration 6) 3. The imaging device according to configuration 2, further comprising a correspondence obtaining unit that performs positional correspondence between the angle of view of the first camera and the angle of view of the second camera. (Configuration 7) The display unit further includes: 3. The imaging device according to configuration 2, wherein an image captured by the second camera is displayed in a superimposed manner on an image capture screen of the first camera. (Configuration 8) 3. The imaging device according to configuration 2, further comprising a switching unit that automatically or manually switches between different types of second cameras. (Configuration 9) The display unit is 3. The imaging device according to claim 1, wherein, when the imaging assist information provided is a subject position, a position indicating the direction of the subject is displayed as a symbol on a radar chart. (Configuration 10) The display unit is The imaging device according to configuration 1 or 2, characterized in that the imaging assist information provided in the image captured by the first camera and displayed on the display unit displays the size of the subject, the distance from the subject, and the subject speed in text, and also displays the direction of movement of the subject with an arrow. (Configuration 11) The imaging device according to configuration 1 or 2, characterized in that the first camera is mounted on an automatic tripod head, and further includes a control unit that drives and controls the automatic tripod head so as to track and capture a position of a specific subject as imaging assist information. (method) A control method for an imaging device, comprising: an acquisition step of acquiring difference information from an image captured by the first camera; providing the acquired difference information or imaging assist information generated based on the difference information for assisting one or more imaging operations; a switching control step of controlling switching of display of imaging assist information displayed on a display unit on which the captured image is displayed, in accordance with the provided imaging assist information. (program) A program for causing a computer to execute a control method for an imaging device, The control method includes: an acquisition step of acquiring difference information from an image captured by the first camera; providing the acquired difference information or imaging assist information generated based on the difference information for assisting one or more imaging operations; and a switching control step of switching and controlling display of imaging assist information displayed on a display unit on which the captured image is displayed, in accordance with the provided imaging assist information.

[0073] Although the preferred embodiment of the present invention has been described above, the present invention is not limited to the above-mentioned embodiment, and various modifications and changes are possible within the scope of the gist of the present invention. For example, the present invention can be realized by supplying a program that realizes one or more functions of the above-mentioned embodiment to a system or device via a network or a recording medium, and having a computer processor of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0074] 100 First camera (imaging device) 101 Lens 107 Image sensor 121 CPU 124 Image sensor driver circuit 125 Image processing circuit 126 Lens driver circuit 131 Display section 132 Operation section 133 Storage medium 134 Storage section 200 Second camera (imaging device) 200a 360° Hemisphere Camera 200b Wide-angle IR camera 201 Lens 207 Image sensor 221 CPU 224 Image sensor driver circuit 225 Image Processing Circuit 226 Lens driver circuit 231 Display section 232 Operation section 233 Storage medium 234 Storage section

Claims

1. an acquisition unit that acquires difference information from an image captured by the first camera; a providing unit that provides the acquired difference information or imaging assist information that is generated based on the difference information and that assists in one or more imaging operations; a switching control unit that controls switching of display of imaging assist information displayed on a display unit that displays the captured image in accordance with the provided imaging assist information.

2. The acquisition unit is 2. The imaging device according to claim 1, further comprising: a sensor that detects an image captured by the first camera and an image captured by one or more second cameras having a wider angle than the first camera;

3. The providing unit is 3. The imaging device according to claim 1, further comprising: a first imaging assist information generating unit configured to generate imaging assist information for the first and second imaging devices, the first imaging assist information generating unit generating information for the first and second imaging devices, the second imaging assist information generating unit generating information for the second and third imaging devices, the third imaging assist information generating unit generating information for the third and fourth imaging devices, the fourth imaging assist information generating unit generating information for the fourth and fourth imaging devices, the fourth imaging assist information generating unit generating information for the fourth and fourth imaging devices, the fifth imaging assist information generating unit generating information for the fourth and fourth imaging devices, the fifth imaging assist information generating unit generating information for the fourth and fourth imaging devices, the fifth imaging assist information generating unit generating information for the fourth and fourth imaging devices, the sixth ...

4. 3. The imaging device according to claim 2, wherein the second camera is a camera having a viewing angle of 180 degrees or more, or a special camera capable of acquiring information of a wavelength different from that of the first camera.

5. The display unit further includes:

3. The imaging apparatus according to claim 1, further comprising a display unit that displays a live view image, an image in a viewfinder, and imaging information acquired by the first camera.

6. 3. The imaging device according to claim 2, further comprising a correspondence obtaining unit that performs positional correspondence between the angle of view of the first camera and the angle of view of the second camera.

7. The display unit further includes:

3. The imaging device according to claim 2, wherein an image captured by the second camera is displayed in a superimposed manner on an image capture screen of the first camera.

8. 3. The imaging device according to claim 2, further comprising a switching unit that automatically or manually switches between different types of second cameras.

9. The display unit further includes:

3. The imaging apparatus according to claim 1, wherein, when the imaging assist information provided is a subject position, a position indicating the direction of the subject is displayed as a symbol on a radar chart.

10. The display unit further includes: The imaging device according to claim 1 or 2, characterized in that the imaging assist information provided in the image captured by the first camera and displayed on the display unit displays the size of the subject, the distance from the subject, and the subject speed in text, and also displays the direction of movement of the subject with an arrow.

11. 3. The imaging device according to claim 1, further comprising a control unit that drives and controls the automatic tripod head so as to track and capture a position of a specific subject as imaging assist information.

12. A control method for an imaging device, comprising: an acquisition step of acquiring difference information from an image captured by the first camera; providing the acquired difference information or imaging assist information generated based on the difference information for assisting one or more imaging operations; a switching control step of controlling switching of display of imaging assist information displayed on a display unit on which the captured image is displayed, in accordance with the provided imaging assist information.

13. A program for causing a computer to execute a control method for an imaging device, The control method includes: an acquisition step of acquiring difference information from an image captured by the first camera; providing the acquired difference information or imaging assist information generated based on the difference information for assisting one or more imaging operations; and a switching control step of switching and controlling display of imaging assist information displayed on a display unit on which the captured image is displayed, in accordance with the provided imaging assist information.

Images