Control apparatus, image capturing apparatus, image capturing system, method, and storage medium

Abstract

An advantage of some aspects of the invention is to easily switch display of a plurality of images.SOLUTION: A display device is provided with: a display means which displays a first image or a second image outputted by a first imaging unit provided with a first visible light imaging unit and a first non-visible light imaging unit, and a third image or a fourth image outputted by a second imaging unit provided with a second visible light imaging unit and a second non-visible light imaging unit; and a display control means which, when the first image and the third image are being displayed, switches from the first image to the second image on the basis of a first instruction to switch from the first image to the second image different from the first image, and switches from the third image to the fourth image different from the third image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a control device, an imaging device, an imaging system, a method, and a program. [Background technology]

[0002] Conventionally, mobile objects such as drones that can move wirelessly are equipped with an imaging device. The imaging device captures images for the drone operator to operate and images of the drone's surroundings. In order for the operator to remotely control the drone, a client device is used that displays images captured by the drone's imaging device. The operator can remotely control the drone while viewing the images displayed on the client device. A drone can also be equipped with multiple imaging units to capture images of different imaging areas. Furthermore, the multiple imaging units include a visible light imaging unit and an invisible light imaging unit (such as a thermal camera) so that they can capture images even in environments with poor visibility, such as fog or haze.

[0003] For example, Patent Document 1 proposes an in-vehicle imaging device that selects the output of either a visible image from a visible light camera or an infrared image from a thermal camera mounted on the vehicle based on distance information between the vehicle and a subject. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2019-1325 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the in-vehicle imaging device of Patent Document 1, the output (display of a visible image or an infrared image) is switched based on information about the distance between the vehicle and the subject. For example, there are user needs such as when a user wants to display a visible image captured by a visible light camera regardless of the distance to the subject in an environment with good visibility, and when a user wants to display an infrared image captured by a non-visible light camera regardless of the distance to the subject in an environment with poor visibility. However, because the in-vehicle imaging device of Patent Document 1 has the above output characteristics, it may not be able to display the image the user wants to see (i.e., the visible image or infrared image based on the above user need). As a result, the user must switch the output (image display) of each imaging unit, which creates a problem of cumbersome user operations.

[0006] SUMMARY OF THE INVENTION It is therefore an object of the present invention to easily switch between displaying a plurality of images. [Means for solving the problem]

[0007] In order to achieve the object of the present invention, a control device according to one embodiment of the present invention has the following configuration: display means for displaying a first image or a second image output by a first imaging unit including a first visible light imaging unit and a first invisible light imaging unit, and a third image or a fourth image output by a second imaging unit including a second visible light imaging unit and a second invisible light imaging unit, and display control means for switching from the first image to the second image and from the third image to the fourth image different from the third image, when the first image and the third image are displayed, based on a first instruction to switch from the first image to the second image different from the first image. [Effects of the Invention]

[0008] According to the present invention, display switching between a plurality of images can be easily performed. [Brief explanation of the drawings]

[0009] [Figure 1]FIG. 1 is a functional block diagram of an imaging system according to a first embodiment. [Figure 2] FIG. 1 is a hardware configuration diagram of an imaging apparatus according to a first embodiment. [Figure 3] FIG. 2 is a hardware configuration diagram of a client device according to the first embodiment. [Figure 4] FIG. 1 is an external view of a drone equipped with an imaging device according to a first embodiment. [Figure 5] FIG. 2 is an external view of a client device according to the first embodiment. [Figure 6] FIG. 1 is a functional block diagram of a drone equipped with an imaging device according to a first embodiment. [Figure 7] 4 is a state diagram showing the relationship between the output of the first imaging unit and the output of the second imaging unit according to the first embodiment. FIG. [Figure 8] FIG. 4 is a diagram illustrating the appearance of a display 401 in state A according to the first embodiment. [Figure 9] FIG. 4 is a diagram illustrating the appearance of the display 401 in state B according to the first embodiment. [Figure 10] FIG. 4 is a diagram illustrating the appearance of the display 401 in state C according to the first embodiment. [Figure 11] FIG. 4 is a diagram illustrating the appearance of the display 401 in state D according to the first embodiment. [Figure 12] 5 is a flowchart illustrating processing executed by a client device according to the first embodiment. [Figure 13] FIG. 3 is a state transition diagram of the display according to the first embodiment. [Figure 14] 10 is a flowchart illustrating processing executed by a client device according to the second embodiment. [Figure 15] FIG. 10 is a state transition diagram of a display according to the second embodiment. [Figure 16] 10 shows an example in which two types of images are simultaneously displayed in a display area 802 according to the fourth embodiment. [Figure 17] FIG. 13 is a diagram for explaining a user interface for switching images in a display area 802 according to the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the claimed invention. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0011] <<First Embodiment>> <Imaging system> FIG. 1 is a functional block diagram of an imaging system according to the first embodiment.

[0012] The imaging system 10 includes an imaging device 100, a client device 200, and a network 20. The imaging device 100 and the client device 200 are connected via the network 20, which may be wireless or wired.

[0013] <Imaging device> The imaging device 100 includes a control unit 101, a communication unit 102, a storage unit 103, and an imaging unit 130. The imaging unit 130 includes a first imaging unit 110 and a second imaging unit 120.

[0014] The first imaging unit 110 includes a first visible light imaging unit 111 and a first invisible light imaging unit 112 .

[0015] The first image processing unit 113 converts the signals photoelectrically converted by the first visible light imaging unit 111 and the first invisible light imaging unit 112 into video data.

[0016] The second imaging unit 120 includes a second visible light imaging unit 121 and a second non-visible light imaging unit 122 .

[0017] The second image processing unit 123 converts the signals photoelectrically converted by the second visible light imaging unit 121 and the second invisible light imaging unit 122 into video data.

[0018] The imaging device 100 can switch the image displayed on the display unit 203 (described later) by controlling the output of the first image processing unit 113 and the second image processing unit 123. Furthermore, the imaging device 100 can switch the image displayed on the display unit 203 by controlling (outputting / stopping) the operations of the four imaging units (the first visible light imaging unit 111, the first invisible light imaging unit 112, the second visible light imaging unit 121, and the second invisible light imaging unit 122).

[0019] The first visible light imaging unit 111 and the second visible light imaging unit 121 each include an imaging optical system and an imaging element. The imaging optical system has a zoom lens, a focus lens, and an aperture mechanism, and focuses light from a subject onto the light-receiving surface of the imaging element. The zoom lens moves along the optical axis to change the imaging magnification. The focus lens moves along the optical axis to adjust the focus. The aperture mechanism adjusts the amount of light passing through the optical system. Each lens and aperture mechanism also has a driving unit. Each lens and aperture mechanism is controlled by the control unit 101. The imaging element includes semiconductor elements such as a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor. The imaging element photoelectrically converts the subject image based on the light received from the imaging optical system to generate a video signal, which is an analog signal. Pixel data output from the first visible light imaging unit 111 and the second visible light imaging unit 121 is sent to the first image processing unit 113 and the second image processing unit 123, respectively, for signal processing.

[0020] The first non-visible light imaging unit 112 and the second non-visible light imaging unit 122 each include an imaging optical system and an imaging element. The imaging optical system has a zoom lens, a focus lens, and an aperture mechanism, and focuses light from a subject onto the light-receiving surface of the imaging element. The zoom lens moves in the optical axis direction to change the imaging magnification. The focus lens moves in the optical axis direction to adjust the focus. The aperture mechanism adjusts the amount of light passing through the optical system. Each lens and aperture mechanism also has a driving unit. Each lens and aperture mechanism is controlled by the control unit 101. The imaging element has an infrared sensor sensitive to infrared rays such as near-infrared, mid-infrared, and far-infrared. The imaging element photoelectrically converts the subject image based on the light incident from the imaging optical system to generate a video signal, which is an analog signal. Pixel data output from the first non-visible light imaging unit 112 and the second non-visible light imaging unit 122 is sent to the first image processing unit 113 and the second image processing unit 123, respectively, for signal processing. Here, an example has been described in which the imaging element has an infrared sensor, but the imaging element may be an imaging element that is sensitive to wavelengths other than visible light, such as an ultraviolet sensor.

[0021] In the first image processing unit 113 and the second image processing unit 123, pixel data is converted into a digital signal by A / D conversion. The converted digital signal is converted into image data after undergoing correction processes such as black level correction, gamma curve adjustment, temperature correction, scratch correction, noise reduction, and white balance correction, as well as development processes. Data compression processes such as MP4 and JPEG formats are also performed. In addition, the first image processing unit 113 and the second image processing unit 123 perform corrections in accordance with the wavelength characteristics of each image sensor.

[0022] The control unit 101 comprehensively controls each component of the imaging device 100, sets various parameters, and issues instructions for transmitting and receiving data. The control unit 101 is an output control means that controls the operation of each imaging unit. The control unit 101 controls the operation (output / stop) of each imaging unit included in the first imaging unit 110 and the second imaging unit 120, for example, based on an instruction (also referred to as a first instruction) to switch the image in a display area 801 (described later) received via the communication unit 102.

[0023] The communication unit 102 is an output unit that transmits the image data output by the first image processing unit 113 and the second image processing unit 123 to the client device 200 via the network 20. The communication unit 102 also receives operation information input from the operation input unit 205 of the client device 200.

[0024] The storage unit 103 can save and read out the video data output by the first image processing unit 113 and the second image processing unit 123. Furthermore, the storage unit 103 is used as a storage area for programs executed by the control unit 101 (described later), a storage area for various parameters, and a work area during program execution.

[0025] <Client device> The client device 200 includes a control unit 201, a communication unit 202, a display unit 203, a display control unit 204, an operation input unit 205, a storage unit 206, and an I / F unit 207. The client device 200 is a control device that switches the display of multiple images.

[0026] The display unit 203 is controlled by the display control unit 204. The display unit 203 is a display means (for example, a display) that displays images captured by the two imaging units (first imaging unit 110 and second imaging unit 120) and various information. The display unit 203 is, for example, an LCD display, an organic EL display, or a touch panel.

[0027] The display control unit 204 is a display control means that controls the display of images displayed on the display unit 203. The display control unit 204 selects the video and information to be displayed on the display unit 203, controls the display format (superimposed display, split display, etc.), and controls the display area of ​​each video and information. Furthermore, the display control unit 204 may change the video and information, display format, and display area to be displayed on the display unit 203 based on an instruction from the operator. Furthermore, the display control unit 204 can switch the display format (visible image and invisible image) of the image to be displayed on the display unit 203.

[0028] The operation input unit 205 includes a controller (not shown) and a touch panel (not shown). The operation input unit 205 is an input means for inputting user instructions to the image capture device 100 and the client device 200. The user can control the image capture device 100 and the client device 200 by operating the controller and / or the touch panel.

[0029] The storage unit 206 can temporarily store and read out image data output by the first imaging unit 110 and the second imaging unit 120 and received by the communication unit 202, and display data generated by the display control unit 204. Furthermore, the storage unit 206 is used as a storage area for programs executed by the control unit 201, a storage area for various parameters, and a work area during program execution.

[0030] The I / F unit 207 is an interface for connecting the client device 200 to a PC (Personal Computer) (not shown), a storage medium (for example, a hard disk, a memory card, an SD card, a USB memory, etc.) (not shown), and an external display device (not shown).

[0031] The control unit 201 comprehensively controls each component of the client device 200. The control unit 201 is a system control unit that sets various parameters, controls display, and issues instructions for sending and receiving data.

[0032] <Hardware configuration of imaging device> FIG. 2 is a diagram showing the hardware configuration of the imaging device according to the first embodiment.

[0033] The imaging device 100 includes a CPU 301, a RAM 302, a ROM 303, a first imaging unit 110, a second imaging unit 120, and a communication driver 312. The CPU 301 is a central processing unit. The CPU 301 controls the control unit 101, the first image processing unit 113, and the second image processing unit 123 in FIG. 1. The CPU 301 also performs overall control of each function. The RAM 302 is a volatile memory and is a component of the storage unit 103 in FIG. 1. The ROM 303 is a non-volatile memory and is a component of the storage unit 103 in FIG. 1.

[0034] The first imaging unit 110 includes a first visible light imaging unit 111 and a first invisible light imaging unit 112. The first visible light imaging unit 111 includes a visible light lens 304 having a high transmittance of visible light and a visible light imaging element 308 that is sensitive to visible light. The first invisible light imaging unit 112 includes an invisible light lens 305 having a high transmittance of invisible light and an invisible light imaging element 309 that is sensitive to invisible light.

[0035] The second imaging section 120 includes a second visible light imaging section 121 and a second invisible light imaging section 122. The second visible light imaging section 121 includes a visible light lens 306 having a high transmittance of visible light and a visible light imaging element 310 that is sensitive to visible light. The second invisible light imaging section 122 includes an invisible light lens 307 having a high transmittance of invisible light and an invisible light imaging element 311 that is sensitive to invisible light.

[0036] The communication driver 312 is a component of the communication unit 102 in Fig. 1. The communication driver 312 is a driver for the imaging device 100 to communicate with other devices such as the client device 200.

[0037] <Client device hardware configuration> FIG. 3 is a hardware configuration diagram of the client device according to the first embodiment.

[0038] The client device 200 includes a display 401 , a controller 402 , an I / F driver, a communication driver 404 , a CPU 405 , a RAM 406 , and a ROM 407 .

[0039] The display 401 is a component of the display unit 203 in Fig. 1. The display 401 displays images captured by the two imaging units (the first imaging unit 110 and the second imaging unit 120) and various information.

[0040] The controller 402 is a component of the operation input unit 205 in Fig. 1. The controller 402 operates the image capture device 100, changes the display settings of the client device 200, and the like.

[0041] The I / F driver 403 is a component of the I / F unit 207 in Fig. 1. When connecting the client device 200 to an external device via a wired connection, the I / F driver 403 performs format conversion in accordance with the connection standard.

[0042] The communication driver 404 is a driver for wireless communication that constitutes the communication unit 202 in FIG.

[0043] The CPU 405 is a central processing unit. The CPU 405 not only performs the role of the control unit 201 in FIG. 1 but also controls each functional unit. Furthermore, the CPU 405 performs overall control of each functional unit. The RAM 406 is a volatile memory and is a component of the storage unit 206 in FIG. 1. The ROM 407 is a non-volatile memory and is a component of the storage unit 206 in FIG. 1.

[0044] FIG. 4 is an external view of a drone equipped with the imaging device according to the first embodiment.

[0045] The drone 500 includes a first imaging unit 110 and a second imaging unit 120, which are components of the imaging device 100. The drone 500 in Fig. 4 includes two imaging units, but may include three or more imaging units. The first imaging unit 110 and the second imaging unit 120 can be attached to any position on the drone 500.

[0046] The first imaging unit 110 is fixed (mounted) to the front of the drone 500 and is an imaging unit that captures images at a wider angle than the second imaging unit 120. The first imaging unit 110 is attached to the imaging device 100 so as to capture images at a fixed angle of view. The operator of the drone 500 can control the drone 500 while viewing the images captured by the first imaging unit 110. The first visible light imaging unit 111 and the first invisible light imaging unit 112 of the first imaging unit 110 capture images in the same direction, with at least a portion of their imaging areas overlapping. While it is desirable that the focal lengths of the first visible light imaging unit 111 and the first invisible light imaging unit 112 be the same, the focal lengths of the first visible light imaging unit 111 and the first invisible light imaging unit 112 do not necessarily have to be the same. Furthermore, the zoom magnifications of the first visible light imaging unit 111 and the first invisible light imaging unit 112 can be changed independently of each other.

[0047] The second imaging unit 120 is an imaging unit that can be detached from the bottom of the drone 500. The second imaging unit 120 is an imaging unit that captures telephoto images of the environment surrounding the drone 500. The second imaging unit 120 can capture images with a telephoto angle of view greater than that of the first imaging unit 110. The angle of view (zoom magnification) of the second imaging unit 120 can be changed depending on the distance and direction to the subject. Furthermore, the second imaging unit 120 has a drive mechanism for changing the imaging direction, and can therefore be driven to pan and tilt. The pan and tilt drive of the second imaging unit 120 can be controlled by the drone 500 or the imaging device 100. The second visible light imaging unit 121 and the second invisible light imaging unit 122 of the second imaging unit 120 capture images in the same direction, with at least partial imaging areas overlapping. It is desirable that the focal lengths of the second visible light imaging section 121 and the second invisible light imaging section 122 are the same, but they do not necessarily have to be the same. Furthermore, the zoom magnifications of the second visible light imaging section 121 and the second invisible light imaging section 122 can be changed independently of each other.

[0048] Also, an example has been described in which the first imaging unit 110 of the imaging device 100 is attached to the drone 500, and the second imaging unit 120 is attached to a position below and away from the drone 500. However, the first imaging unit 110 and the second imaging unit 120 may be attached to the drone 500 as a single unit.

[0049] Although the example in which the imaging device 100 is mounted on the drone 500 has been described, the present invention is not limited to this. For example, the imaging device 100 may be mounted on any moving object such as an unmanned or manned aircraft, a ship, or a vehicle.

[0050] FIG. 5 is an external view of the client device according to the first embodiment.

[0051] The display 401 of the client device 200 has a main display area 801 that displays an image captured by the first imaging unit 110, and a sub-display area 802 that displays an image (for confirming surrounding information) captured by the second imaging unit 120. The display area 802 is superimposed on the display area 801. Although FIG. 5 shows a configuration in which the controller 402, the display 401, and the client device 200 are integrated, the controller 402, the display 401, and the client device 200 may be configured independently of one another. The touch panel may also have the function of the operation input unit 205. Here, an example is shown in which the display area 802 is superimposed on the display area 801. However, the display area 801 may be provided on the right half of the display 401, and the display area 802 may be provided on the left half of the display 401. The sizes and arrangements of the two display areas (display area 801, display area 802) can be changed as desired.

[0052] <Drone 500> FIG. 6 is a functional block diagram of a drone equipped with an imaging device according to the first embodiment.

[0053] The drone 500 has the functions of the imaging device 100 in Fig. 1, and therefore the description will omit the content that overlaps with Fig. 1. The drone 500 has a driving unit 501, a position acquisition unit 502, a speed acquisition unit 503, an attitude acquisition unit 504, and a distance acquisition unit 505.

[0054] The drive unit 501 is a mechanical unit equipped with a motor and propellers for controlling the navigation and attitude of the drone 500. The drone 500 can move forward and backward, left and right, and up and down. The drive unit 501 is controlled by the control unit 101 based on operation information from the operation input unit 205 received via the communication unit 102, as well as information obtained from the position acquisition unit 502, the speed acquisition unit 503, the attitude acquisition unit 504, and the distance acquisition unit 505. The drive unit 501 also controls the pan-tilt drive of the second imaging unit 120.

[0055] The position acquisition unit 502 receives a signal from a global positioning system (GPS). The position acquisition unit 502 processes the GPS signal to identify the position of the drone 500, and processes a signal from the electronic compass to identify the direction of the drone 500.

[0056] The speed acquisition unit 503 processes digital data output from an acceleration sensor or the like, and calculates the acceleration and speed of the drone 500.

[0057] The attitude acquisition unit 504 processes digital data output from an angular velocity sensor or the like, and calculates the rotation and change in orientation of the drone 500.

[0058] The distance acquisition unit 505 processes data output from distance sensors such as Lidar (Light Detection and Ranging), millimeter wave radar, and ultrasonic sensors, and calculates the distance from the drone 500 to the surrounding environment (for example, an obstacle). In addition, the distance acquisition unit 505 may calculate distance information from the drone 500 to the obstacle based on an evaluation value of focus by phase difference AF (Auto Focus).

[0059] <Image display format> The format of the image displayed on the client device 200 will now be described.

[0060] The images captured by the imaging unit 130 include four types of images. The four types of images include a visible image (also referred to as a "first image") captured by the first visible light imaging unit 111, an invisible image (also referred to as a "second image") captured by the first invisible light imaging unit 112, a visible image (also referred to as a "third image") captured by the second visible light imaging unit 121, and an invisible image (also referred to as a "fourth image") captured by the second invisible light imaging unit 122. However, if all four types of images are displayed on the display 401, the size of each image becomes small. Therefore, the user needs to select a desired image from the four types of images. In this embodiment, an example will be described in which one type of image (a visible image (first image) or an invisible image (second image)) output by the first imaging unit 110 and one type of image (a visible image (third image) or an invisible image (fourth image)) output by the second imaging unit 120 are displayed on the display 401. In other words, two images are displayed on the display 401. Combinations of two images include a first image and a third image (state A described below), a first image and a fourth image (state B described below), a second image and a third image (state C described below), and a second image and a fourth image (state D described below). In addition, in this embodiment, a method of switching images on the display 401 by a user performing an image switching operation will be described.

[0061] FIG. 7 is a state diagram showing the relationship between the output of the first imaging unit and the output of the second imaging unit according to the first embodiment.

[0062] States A to D in FIG. 7 show combinations of the output of the first imaging unit 110 and the output of the second imaging unit 120. The first imaging unit 110 and the second imaging unit 120 each output a visible image (shown as VISIBLE) or an invisible image (shown as THERMAL). Therefore, the display format of the image displayed on the display 401 includes four states. Although the invisible image is described as "THERMAL," the invisible image is not limited to an image captured by an imaging unit sensitive to infrared light. The invisible image may also be an image captured by an imaging unit sensitive to ultraviolet light, which exists outside the wavelength range of visible light.

[0063] The appearance of the display 401 in states A to D will be described with reference to Figs. 8, 9, 10, and 11. Fig. 8 is a diagram illustrating the appearance of the display 401 in state A according to the first embodiment. Fig. 9 is a diagram illustrating the appearance of the display 401 in state B according to the first embodiment. Fig. 10 is a diagram illustrating the appearance of the display 401 in state C according to the first embodiment. Fig. 11 is a diagram illustrating the appearance of the display 401 in state D according to the first embodiment.

[0064] The first visible image 601 (first image) is an image captured by the first visible light imaging section 111. The first invisible image 602 (second image) is an image captured by the first invisible light imaging section 112. In response to an instruction (also referred to as a first instruction) to switch the image in the display area 801, the first visible image 601 or the first invisible image 602 is displayed in the display area 801. Note that in order to clearly distinguish between visible images and invisible images, the images in the drawings are marked with the words VISIBLE (= visible image) or THERMAL (= invisible image). Note that the words VISIBLE and THERMAL in the images can be displayed or hidden at the user's discretion.

[0065] The second visible image 603 (third image) is an image captured by the second visible light imaging section 121. The second invisible image 604 (fourth image) is an image captured by the second invisible light imaging section 122. In response to an instruction (also referred to as a second instruction) to switch the image in the display area 801, the second visible image 603 or the second invisible image 604 is displayed in the display area 801. Note that in order to clearly distinguish between visible images and invisible images, the images in the figures are displayed with the words VISIBLE (visible image) or THERMAL (invisible image). Note that the words VISIBLE and THERMAL in the images can be displayed or hidden at the user's discretion.

[0066] The display switching button 701 is a button for switching the image in the display area 801. The display switching button 701 is a first UI (User Interface) through which the user inputs a first instruction. The display switching button 702 is a button for switching the image in the display area 802. The display switching button 702 is a second UI (User Interface) through which the user inputs a second instruction. The user can arbitrarily operate the display switching button 701 and the display switching button 702. Note that the user may perform the same operation as the display switching button 701 and the display switching button 702, for example, by using the user's voice and line of sight. In this embodiment, an example is shown in which the display control unit 204 switches the image display, but this is not limiting. For example, the imaging device 100 may switch the video data output by the first imaging unit 110 and the second imaging unit 120. For example, based on a first instruction received from the client device 200, the image capturing device 100 transmits the output (video data) from the first visible light imaging section 111 or the first invisible light imaging section 112 and the output (video data) from the second visible light imaging section 121 or the second invisible light imaging section 122 to the client device 200. In this way, when switching the video data output by the image capturing device 100, the image capturing device 100 can control the operation of the first image capturing section 110 (specifically, output / stop of the first visible light imaging section 111 and the first invisible light imaging section 112) and the operation of the second image capturing section 120 (specifically, output / stop of the second visible light imaging section 121 and the second invisible light imaging section 122).

[0067] <Operation description> The user uses the first imaging unit 110 of the drone 500 to capture an image of a wide area. The image captured by the first imaging unit 110 is used by the user to control the drone 500. By checking the image of the area around the drone 500, the user can understand the current position of the drone 500 and whether there are any obstacles near the drone 500.

[0068] In this embodiment, when an image in the display area 801 is not appropriately selected while the drone 500 is flying, a process of switching the image display by the user operating the display switching button 701 will be described. For example, when a first visible image 601 (first image) is displayed in the display area 801 and weather conditions worsen (for example, fog or haze occurs) or there is insufficient illumination, the user operates the display switching button 701 to switch from the first visible image 601 (first image) to the first invisible image 602 (second image). At this time, the visibility of the first imaging unit 110, which performs wide-angle imaging, is reduced due to the fog and haze. Here, by switching from the second visible image 603 (third image) of the second imaging unit 120 to the second invisible image 604 (fourth image), the user's image visibility is further improved.

[0069] When the type of image in the display area 801 (for example, a visible image (first image)) and the type of image in the display area 802 (for example, a visible image (third image)) are the same, the display of the first image and the third image in the display area 801 and the display area 802 is switched by operating the display switching button 701. Conversely, when the type of image in the display area 801 (for example, a visible image (first image)) and the type of image in the display area 802 (for example, a non-visible image (fourth image)) are different, only the first image in the display area 801 is switched by operating the display switching button 701. In other words, when the display switching button 701 is operated, the image in the display area 802 is switched in conjunction with the switching of the image in the display area 801. This allows the user to switch the image in the display area 802 without performing an operation to switch the image in the display area 802. Therefore, the user can easily switch the display of multiple images. In particular, this reduces the workload required for switching the display of images while the user is operating the drone 500, allowing the user to operate the drone 500 safely.

[0070] The advantageous effects of the configuration related to the image display switching described above will be further explained. For example, if the visibility of the first imaging unit 110 deteriorates due to fog or the like while the drone 500 is navigating, the visible distance (visibility) becomes shorter. Therefore, the image output by the second imaging unit 120, which captures images at a longer distance than the first imaging unit 110, shows a situation in which visibility has deteriorated due to fog or the like. Therefore, the image in the display area 802 is switched in conjunction with the switching of the image in the display area 801.

[0071] Although an example of switching from a visible image to an invisible image when the weather worsens while the drone 500 is flying has been described, the image display switching described above can also be applied to switching from an invisible image to a visible image. For example, when the first invisible image 602 (second image) is displayed in the display area 801, if the weather improves (for example, the fog and haze disappear) and the illumination improves, and the imaging environment becomes favorable, the user operates the display switching button 701. Then, the first invisible image 602 (second image) is switched to the first visible image 601 (first image). This allows a color image and a high-resolution image with good image quality to be displayed in the display area 801, thereby improving the user's visibility.

[0072] <Flowchart> Fig. 12 is a flowchart illustrating processing executed by the client device according to the first embodiment. This flowchart is implemented by the CPU 405 executing a program loaded in the RAM 406. The processing of this flowchart will be described with reference to Figs. 8, 9, 10, and 11.

[0073] In S901, the user inputs an instruction (also referred to as a first instruction) to switch the image in the display area 801 (the area that displays the image output by the first imaging unit 110). For example, the user operates the display switching button 701 of the client device 200 to instruct the client device 200 to switch the image in the display area 801. The display control unit 204 receives the first instruction. Note that the first visible image 601 (first image) or the first invisible image 602 (second image) is displayed in the display area 801. The second visible image 603 (third image) or the second invisible image 604 (fourth image) is displayed in the display area 802.

[0074] In S902, the display control unit 204 determines whether one type of image is displayed in each of the display areas 801 and 802. If the display control unit 204 determines that one type of image is not displayed in each of the display areas 801 and 802 (No in S902), the process proceeds to S905. A specific example of when this determination occurs is when two types of images (the second visible image 603 (third image) and the second invisible image 604 (fourth image)) are displayed in the display area 802. On the other hand, if the display control unit 204 determines that one type of image is displayed in each of the display areas 801 and 802 (Yes in S902), the process proceeds to S903.

[0075] In S903, the display control unit 204 determines the state of the display 401. If the display control unit 204 determines that the type of image in the display area 801 is different from the type of image in the display area 802 (if the state of the display 401 is state B or state C, Yes in S903), the process proceeds to S905. Here, state B refers to a state in which the first visible image 601 (first image) is displayed in the display area 801, and the second invisible image 604 (fourth image) is displayed in the display area 802. State C refers to a state in which the first invisible image 602 (second image) is displayed in the display area 801, and the second visible image 603 (third image) is displayed in the display area 802. If the type of image in the display area 801 is not different from the type of image in the display area 802 (if the state of the display 401 is state A or state D, No in S903), the display control unit 204 proceeds to S904. Here, state A refers to a state in which a first visible image 601 (first image) is displayed in the display area 801, and a second visible image 603 (third image) is displayed in the display area 802. State D refers to a state in which a first invisible image 602 (second image) is displayed in the display area 801, and a second invisible image 604 (fourth image) is displayed in the display area 802.

[0076] In S904, the display control unit 204 switches the image in the display area 802 (the second visible image 603 (third image) in state A, and the second invisible image 604 (fourth image) in state D). It is desirable to switch the image simultaneously with the image switching in S905 so as not to deteriorate the user's visibility. For example, when the second visible image 603 (third image) is displayed in the display area 802, the display control unit 204 switches from the second visible image 603 (third image) to the second invisible image 604 (fourth image). Furthermore, when the second invisible image 604 (fourth image) is displayed in the display area 802, the display control unit 204 switches from the second invisible image 604 (fourth image) to the second visible image 603 (third image).

[0077] In S905, the display control unit 204 switches the image in the display area 801 (the first visible image 601 (first image) in state A, and the first invisible image 602 (second image) in state D). For example, when the first visible image 601 (first image) is displayed in the display area 801, the display control unit 204 switches from the first visible image 601 (first image) to the first invisible image 602 (second image). When the first invisible image 602 (second image) is displayed in the display area 801, the display control unit 204 switches from the first invisible image 602 (second image) to the first visible image 601 (first image).

[0078] 12 has described a method for switching only the image in display area 801, and a method for switching both the images in display area 801 and display area 802, depending on the display format (state) of the image displayed on display 401. When display switching button 701 is operated and the same type of image (visible image or invisible image) is displayed in display area 801 and display area 802, display control unit 204 switches not only the image in display area 801, but also the image in display area 802. Conversely, when different types of images are displayed in display area 801 and display area 802, display control unit 204 switches only the image in display area 801.

[0079] <State transition diagram> FIG. 13 is a state transition diagram of the display according to the first embodiment.

[0080] In state A, the same type of image (first visible image 601 and second visible image 603) is displayed in display area 801 and display area 802. Therefore, when display switching button 701 is operated, the first visible image 601 in display area 801 is switched to the first invisible image 602. The second visible image 603 in display area 802 is switched to the second invisible image 604. This completes the transition from state A to state D. (Transition from state A to state D) Display area 801: first visible image 601 (first image) → first invisible image 602 (second image) Display area 802: second visible image 603 (third image) → second invisible image 604 (fourth image)

[0081] In state B, different types of images (first visible image 601 and second invisible image 604) are displayed in display area 801 and display area 802. Therefore, when display switching button 701 is operated, the first visible image 601 in display area 801 is switched to the first invisible image 602. This completes the transition from state B to state D. (Transition from state B to state D) Display area 801: first visible image 601 (first image) → first invisible image 602 (second image) Display area 802: second invisible image 604 (fourth image)

[0082] In state C, different types of images (first invisible image 602 and second visible image 603) are displayed in display area 801 and display area 802. Therefore, when display switching button 701 is operated, the first invisible image 602 in display area 801 is switched to the first visible image 601. This completes the transition from state C to state A. (Transition from state C to state A) Display area 801: first invisible image 602 (second image) → first visible image 601 (first image) Display area 802: second visible image 603 (third image)

[0083] In state D, the same type of image (first invisible image 602, second invisible image 604) is displayed in display area 801 and display area 802. Therefore, when display switching button 701 is operated, the first invisible image 602 in display area 801 is switched to the first visible image 601. The second invisible image 604 in display area 802 is switched to the second visible image 603. This completes the transition from state D to state A. (Transition from state D to state A) Display area 801: first invisible image 602 (second image) → first visible image 601 (first image) Display area 802: second invisible image 604 (fourth image) → second visible image 603 (third image)

[0084] <<Second embodiment>> In the first embodiment, a method for switching the state of the display 401 by a user operating a display switching button 701 (also referred to as a first user interface) has been described. In the second embodiment, a method for switching the state of the display 401 by a user operating a display switching button 702 (also referred to as a second user interface) will be described.

[0085] The user uses the second imaging unit 120 to capture an image of a specific subject and a local area. The second imaging unit 120 can capture images of distant subjects (people, vehicles, animals, etc.). Therefore, the user may switch the image in the display area 802 depending on the environment of the specific subject and local area that the user wants to capture. In this case, if the image in the display area 801 is switched at the same time as the image in the display area 802, the image used by the user to control the drone 500 will no longer be displayed in the display area 801. This will prevent the user from controlling the drone 500 as intended, and there is a risk that the drone 500 will collide with a surrounding object. Therefore, in this embodiment, when the user operates the display switching button 702, only the image in the display area 802 is switched. In other words, even if the image in the display area 802 is switched by operating the display switching button 702, the image in the display area 801 will not be switched.

[0086] In this way, by not switching the image in display area 801 when switching the image in display area 802, it is possible to prevent the user's visibility of the image in display area 801 from deteriorating.

[0087] In the second embodiment, an example has been described in which the image in the display area 801 is not switched when the image in the display area 802 is switched (also referred to as non-linked processing), but a condition for executing the non-linked processing (also referred to as non-linked condition) may be set. Specific examples of non-linked conditions will be described below.

[0088] In the first method, the display control unit 204 determines whether a non-coupling condition is satisfied based on the imaging areas of the first imaging unit 110 and the second imaging unit 120. If the imaging direction and imaging area of ​​the first imaging unit 110 differ from the imaging direction and imaging area of ​​the second imaging unit 120, the subject and environment captured by the first imaging unit 110 differ from the subject and environment captured by the second imaging unit 120. In this case, the display control unit 204 determines that the non-coupling condition is satisfied and executes non-coupling processing. For example, if the first imaging unit 110 captures an image in front of the drone 500 while the second imaging unit 120 captures an image directly below or behind the drone 500, the display control unit 204 executes non-coupling processing.

[0089] In the second method, the display control unit 204 determines whether the non-coupling condition is satisfied based on the respective focal lengths of the first imaging unit 110 and the second imaging unit 120. If the focal length (zoom magnification / imaging range) of the first imaging unit 110 is different from the focal length of the second imaging unit 120, the subject and environment captured by the first imaging unit 110 are different from the subject and environment captured by the second imaging unit 120. In this case, the display control unit 204 determines that the non-coupling condition is satisfied and executes non-coupling processing. For example, if the focal length of the first imaging unit 110 is 16 mm and the focal length of the second imaging unit 120 is 600 mm (i.e., if the difference between the focal lengths of the first imaging unit 110 and the second imaging unit 120 is large), the display control unit 204 executes non-coupling processing. Furthermore, when determining whether the non-linkage condition is satisfied, if the sizes of the image sensors of the first imaging unit 110 and the second imaging unit 120 are different, it is desirable to calculate the focal lengths of the first imaging unit 110 and the second imaging unit 120 using corrections such as full-size (35 mm) conversion.

[0090] In the third method, the display control unit 204 determines whether the non-linkage condition is satisfied based on the evaluation values ​​of the images captured by the first imaging unit 110 and the second imaging unit 120. If there is a large difference between the evaluation values ​​(brightness, contrast) of the image captured by the first imaging unit 110 and the evaluation values ​​(brightness, contrast) of the image captured by the second imaging unit 120, the subject and environment captured by the first imaging unit 110 are different from the subject and environment captured by the second imaging unit 120. In this case, the display control unit 204 determines that the non-linkage condition is satisfied and executes non-linkage processing. For example, if the illuminance of the image captured by the second imaging unit 120 is insufficient and the illuminance of the image captured by the first imaging unit 110 is sufficient, the display control unit 204 executes non-linkage processing.

[0091] In the fourth method, the display control unit 204 determines whether the non-linkage condition is met based on the results of image analysis of the first and second imaging units 110 and 120. The image captured by the first and second imaging units 110 and 120 is subjected to image processing to determine the subject. If there is a large difference between the result of determining the subject in the image captured by the first imaging unit 110 and the result of determining the subject in the image captured by the second imaging unit 120, the subject and environment captured by the first imaging unit 110 are different from the subject and environment captured by the second imaging unit 120. In this case, the display control unit 204 determines that the non-linkage condition is met and executes non-linkage processing. For example, if a person is detected in the image captured by the second imaging unit 120 but not in the image captured by the first imaging unit 110, the display control unit 204 executes non-linkage processing.

[0092] <Flowchart> Fig. 14 is a flowchart illustrating processing executed by a client device according to the second embodiment. This flowchart is implemented by the CPU 405 executing a program loaded in the RAM 406. The processing of this flowchart will be described with reference to Figs. 8, 9, 10, and 11.

[0093] In S911, the user inputs an instruction (also referred to as a second instruction) to switch the image in the display area 802 (the area displaying the image output by the second imaging unit 120). For example, the user operates the display switching button 702 of the client device 200 to instruct the client device 200 to switch the image in the display area 802. The display control unit 204 receives the second instruction.

[0094] In S912, the display control unit 204 determines the state of the display 401. The display control unit 204 determines whether or not an image captured by the second imaging unit 120 is displayed in the display area 802. If the display control unit 204 determines that an image captured by the second imaging unit 120 is not displayed in the display area 802 (No in S912), there is no image to be switched, and the processing ends. If the display control unit 204 determines that an image captured by the second imaging unit 120 is displayed in the display area 802 (Yes in S912), the processing proceeds to S913.

[0095] In S913, the display control unit 204 switches the image in the display area 802. Note that the image in the display area 801 is not switched in the processing of Fig. 12, but this is not limiting. For example, if the non-linking conditions described above are not met, the image in the display area 801 may be switched.

[0096] <State transition diagram> FIG. 15 is a state transition diagram of the display according to the second embodiment.

[0097] In state A, the second visible image 603 is displayed in the display area 802. Therefore, when the display switching button 702 is operated, the second visible image 603 in the display area 802 is switched to the second invisible image 604. This completes the transition from state A to state B. (Transition from state A to state B) Display area 801: First visible image 601 (first image) Display area 802: second visible image 603 (third image) → second invisible image 604 (fourth image)

[0098] In state B, the second invisible image 604 is displayed in the display area 802. Therefore, when the display switching button 702 is operated, the second invisible image 604 in the display area 802 is switched to the second visible image 603. This completes the transition from state B to state A. (Transition from state B to state A) Display area 801: First visible image 601 (first image) Display area 802: second invisible image 604 (fourth image) → second visible image 603 (third image)

[0099] In state C, the second visible image 603 is displayed in the display area 802. Therefore, when the display switching button 702 is operated, the second visible image 603 in the display area 802 is switched to the second invisible image 604. This completes the transition from state C to state D. (Transition from state C to state D) Display area 801: First invisible image 602 (second image) Display area 802: second visible image 603 (third image) → second invisible image 604 (fourth image)

[0100] In state D, the second invisible image 604 is displayed in the display area 802. Therefore, when the display switching button 702 is operated, the second invisible image 604 in the display area 802 is switched to the second visible image 603. This completes the transition from state D to state C. (Transition from state D to state C) Display area 801: First invisible image 602 (second image) Display area 802: second invisible image 604 (fourth image) → second visible image 603 (third image)

[0101] <<Third Embodiment>> In the first embodiment, a method has been described in which a user switches the first image (or the second image) in the display area 801, and switches the third image (or the fourth image) in the display area 802 in response to this switching instruction. In the second embodiment, a method has been described in which a user switches only the third image or the fourth image in the display area 802. However, the imaging system 10 may perform the image switching described in the first and second embodiments. For example, the imaging system 10 may determine whether to switch the first image (or the second image) based on the brightness of the first image (or the second image) captured by the first imaging unit 110. For example, if the brightness of the first visible image 601 (first image) is sufficient, the imaging system 10 displays the first visible image 601 (first image). Conversely, if the brightness of the first visible image 601 (first image) is insufficient, the imaging system 10 displays the first invisible image 602 (second image). The imaging system 10 may also determine whether to switch images based on location information obtained from the location acquisition unit 502 and weather information obtained by communicating with an external device. The imaging system 10 may also determine whether to switch images based on the current time of day. For example, the imaging system 10 displays a first visible image 601 (first image) when the time of day is daytime. The imaging system 10 displays a first invisible image 602 (second image) when the time of day is nighttime.

[0102] When the imaging system 10 switches the image in the display area 801 based on the above-described determination conditions executed by the imaging system 10 without a user's instruction to switch the image (in the case of state A or state D), it switches not only the image in the display area 801 but also the image in the display area 802, as in the first embodiment. Furthermore, when the imaging system 10 switches the image in the display area 801 (in the case of state B or state C), it switches only the image in the display area 801, as in the first embodiment.

[0103] When the imaging system 10 switches the image in the display area 802 based on the above-described determination conditions executed by the imaging system 10 without an instruction from the user to switch the image, the imaging system 10 switches only the image in the display area 802, as in the second embodiment. The above-described determination conditions can also be applied when switching the image in the display area 802.

[0104] <<Fourth Embodiment>> In the first and third embodiments, the third image (or the fourth image) in the display area (for example, the display area 802) may be changed in a way that is not intended by the user. Therefore, with reference to Figs. 16 and 17, a method for notifying the user that the third image (or the fourth image) in the display area 802 will be changed when the third image (or the fourth image) in the display area 802 is changed will be described.

[0105] FIG. 16 shows an example in which two types of images are simultaneously displayed in a display area 802 according to the fourth embodiment.

[0106] FIG. 16 shows a transition from state D to state A after the user operates the display switching button 701. At this time, the user is performing an operation to switch from the second image in the display area 801 to the first image, and therefore the switch to the first image in the display area 801 is an intentional operation by the user. On the other hand, the switch from the fourth image to the third image in the display area 802 is an unintentional operation by the user. Here, the user may not want the switch from the fourth image in the display area 802 to the third image to be performed suddenly. Therefore, as shown in FIG. 16, after the display switching button 701 is operated, two types of images (a second visible image 603 (third image) and a second invisible image 604 (fourth image)) are simultaneously displayed in the display area 802 for a certain period of time (for example, several seconds).

[0107] This prevents the user from losing sight of the subject when switching from the fourth image in the display area 802 to the third image. The following additional explanation is provided regarding the possibility that the user may lose sight of the subject when the fourth image in the display area 802 suddenly switches. Because the user's view of the subject differs between visible and invisible images, a sudden switch in the type of image in the display area 802 changes the way the captured background (scenery) and person appear. By simultaneously displaying the second visible image 603 (third image) and the second invisible image 604 (fourth image) in the display area 802, the user can compare the visible and invisible images, preventing the user from losing sight of the subject. In this embodiment, a case where the display switching button 701 described in the first embodiment is operated will be described. However, the method of image switching performed by the imaging system 10, as described in the third embodiment, may also be applied to this embodiment.

[0108] FIG. 17 is a diagram illustrating a user interface for switching images in the display area 802 according to the fourth embodiment.

[0109] FIG. 17 shows a user interface 1700 for switching the image in the display area 802. The user interface 1700 includes a first option (depicted as "Yes") for deciding to switch the fourth image in the display area 802 and a second option (depicted as "No") for canceling the switching of the fourth image in the display area 802. The user interface 1700 is a third user interface that allows the user to input at least one of the first option and the second option. The third user interface may be displayed, for example, after the user operates the display switching button 701. The user can execute the first option or the second option in the user interface 1700 by operating the operation stick 703 and the enter button 704. This allows the user to switch the image in the display area 802 to the image intended by the user. In FIG. 17, when "Yes" is selected, the state transitions from state D to state A. That is, the fourth image in the display area 802 is switched to the third image. On the other hand, when "No" is selected, the state transitions from state D to state B. In other words, the fourth image in the display area 802 does not change.

[0110] 17 shows an example in which both "Yes" and "No" are displayed on the user interface 1700, but the user interface 1700 may display only "Yes" or only "No." For example, if "Yes" is not selected within a certain period of time after only "Yes" is displayed on the user interface 1700, the state transitions from state D to state B without switching the fourth image in the display area 802. On the other hand, if "No" is not selected within a certain period of time after only "No" is displayed on the user interface 1700, the fourth image in the display area 802 is switched and the state transitions from state D to state A.

[0111] Furthermore, as described in the third embodiment, when the imaging system 10 switches the image in the display area 801, two types of images (i.e., a first image and a second image) may be displayed in the display area 801. Furthermore, a fourth user interface (not shown) may be displayed for selecting whether or not to switch the image in the display area 801 (i.e., the first image or the second image).

[0112] <Supplementary information> Although the display switching button 701 and the display switching button 702 have been described as being buttons configured as hardware, they may be a graphical user interface (GUI). Furthermore, the display unit 203, the control unit 201, and the operation input unit 205 do not have to be configured as an integrated unit. The processing of each of the above-described embodiments may be executed by operating the GUI displayed on a monitor connected to a PC using a mouse, keyboard, or the like.

[0113] As a method for switching between images in display area 801 and display area 802, an example has been shown in which the user selects the image to be displayed on client device 200, but the image output may also be switched by imaging device 100 controlling the output of first imaging unit 110 and second imaging unit 120.

[0114] When switching between images in display area 801 and display area 802, it is desirable to align the angle of view of the images before and after the switch. By aligning the angle of view of the images before and after the switch, the user can view images with the same angle of view even when the images are switched, making it less likely that the subject will be lost. Therefore, when an image switching operation is performed, the focal length of the imaging unit capturing the image after the switch is adjusted by driving the zoom lens of the imaging unit capturing the image before the switch. Alternatively, the angle of view of the images before and after the switch may be aligned by digital zoom.

[0115] Either the visible image or the non-visible image may be a composite image obtained by combining a visible image and a non-visible image, provided that the visible image and the non-visible image are images that are sensitive to different wavelengths.

[0116] (Other embodiments) The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program. It can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0117] The disclosure of this specification includes the following control device, imaging device, imaging system, method, and program. (Item 1) a display means for displaying a first image or a second image output by a first imaging unit including a first visible light imaging unit and a first invisible light imaging unit, and a third image or a fourth image output by a second imaging unit including a second visible light imaging unit and a second invisible light imaging unit; a display control means for switching from the first image to the second image and from the third image to the fourth image, different from the third image, based on a first instruction for switching from the first image to the second image, different from the first image, when the first image and the third image are displayed; Control device. (Item 2) the display control means, when the first image and the third image are being displayed, does not switch from the first image to the second image and switches from the third image to the fourth image based on a second instruction to switch from the third image to the fourth image; Item 1. The control device according to item 1. (Item 3) the display control means determines whether or not to switch from the first image to the second image and to switch from the third image to the fourth image based on at least one of the imaging areas and / or focal lengths of the first imaging unit and the second imaging unit, the evaluation values ​​of the first image and the third image, and / or the results of image analysis. Item 2. The control device according to item 2. (Item 4) a first user interface for inputting the first instruction; Item 1. The control device according to item 1. (Item 5) a second user interface for inputting the second instruction; Item 2. The control device according to item 2. (Item 6) the first imaging unit and the second imaging unit are attached to a moving body; 6. The control device according to any one of items 1 to 5. (Item 7) The moving object is any one of a drone, an aircraft, a ship, and a vehicle. Item 6. The control device according to item 6. (Item 8) the first imaging unit captures an image at a wider angle than the second imaging unit; 8. The control device according to any one of items 1 to 7. (Item 9) the first imaging unit is fixed to a front surface of the moving body; Item 8. The control device according to item 6 or 7. (Item 10) the second imaging unit is driven to pan and tilt; 10. The control device according to any one of items 1 to 9. (Item 11) a third user interface for inputting at least one of an instruction to switch from the third image to the fourth image and an instruction not to switch from the third image to the fourth image; the third user interface is displayed after receiving the first instruction. 11. The control device according to any one of items 1 to 10. (Item 12) the display control means displays the third image and the fourth image based on the first instruction. 12. The control device according to any one of items 1 to 11. (Item 13) the first non-visible light imaging unit and the second non-visible light imaging unit are imaging units sensitive to at least one of near-infrared light, mid-infrared light, far-infrared light, and ultraviolet light; 13. The control device according to any one of items 1 to 12. (Item 14) the display control means switches from the first image to the second image and from the third image to the fourth image based on brightness of the first image; 14. The control device according to any one of items 1 to 13. (Item 15) the display control means does not switch from the first image to the second image and switches from the third image to the fourth image based on brightness of the third image. 15. The control device according to any one of items 1 to 14. (Item 16) the first image and the third image are visible images, and the second image and the fourth image are non-visible images; or the first image and the third image are non-visible images, and the second image and the fourth image are visible images; 16. A control device according to any one of claims 1 to 15. (Item 17) a first imaging unit including a first visible light imaging unit and a first non-visible light imaging unit; a second imaging unit including a second visible light imaging unit and a second non-visible light imaging unit; an output unit that outputs a first image or a second image from the first imaging unit and a third image or a fourth image from the second imaging unit; and an output control means for controlling, when the first image and the third image are displayed, an operation of the first imaging unit so that the first imaging unit outputs the third image, and for controlling an operation of the second imaging unit so that the second imaging unit outputs the fourth image, based on a first instruction to switch from the first image to the second image different from the first image. Imaging device. (Item 18) the first image and the third image are visible images, and the second image and the fourth image are non-visible images; or the first image and the third image are non-visible images, and the second image and the fourth image are visible images; Item 18. The imaging device according to item 17. (Item 19) a first imaging unit including a first visible light imaging unit and a first non-visible light imaging unit; an imaging device including a second imaging unit including a second visible light imaging unit and a second non-visible light imaging unit; and a control device according to any one of items 1 to 16, which communicates with the imaging device. Imaging system. (Item 20) Item 17: The imaging device according to Item 17; a control device communicating with the imaging device, a control device including a display means for displaying the first image or the second image, and the third image or the fourth image output by the imaging device; Imaging system. (Item 21) A method executed by a controller, comprising: a display step of displaying a first image or a second image output by a first imaging unit including a first visible light imaging unit and a first non-visible light imaging unit, and a third image or a fourth image output by a second imaging unit including a second visible light imaging unit and a second non-visible light imaging unit; a display control step of switching from the first image to the second image and from the third image to the fourth image different from the third image, based on a first instruction to switch from the first image to the second image different from the first image, when the first image and the third image are displayed; method. (Item 22) A method performed by an imaging device, comprising: The imaging device is a first imaging unit including a first visible light imaging unit and a first non-visible light imaging unit; a second imaging unit including a second visible light imaging unit and a second non-visible light imaging unit; The method comprises: an output step of outputting a first image or a second image from the first imaging unit and a third image or a fourth image from the second imaging unit; an output control step of controlling, when the first image and the third image are being displayed, an operation of the first imaging unit so that the first imaging unit outputs the third image, and controlling an operation of the second imaging unit so that the second imaging unit outputs the fourth image, based on a first instruction to switch from the first image to the second image different from the first image, method. (Item 23) A program for causing a computer to function as each means of the control device according to any one of items 1 to 16. (Item 24) 18. A program for causing a computer to function as each of the means of the imaging device described in Item 17.

[0118] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention. [Explanation of symbols]

[0119] 10. Imaging System 20 Network 100 Imaging device 200 Client Device

Claims

1. a display means for displaying a first image or a second image output by a first imaging unit including a first visible light imaging unit and a first invisible light imaging unit, and a third image or a fourth image output by a second imaging unit including a second visible light imaging unit and a second invisible light imaging unit; a display control means for switching from the first image to the second image and from the third image to the fourth image, different from the third image, based on a first instruction for switching from the first image to the second image, different from the first image, when the first image and the third image are displayed; Control device.

2. the display control means, when the first image and the third image are being displayed, does not switch from the first image to the second image and switches from the third image to the fourth image based on a second instruction to switch from the third image to the fourth image; The control device according to claim 1 .

3. the display control means determines whether or not to switch from the first image to the second image and to switch from the third image to the fourth image based on at least one of the imaging areas and / or focal lengths of the first imaging unit and the second imaging unit, the evaluation values ​​of the first image and the third image, and / or the results of image analysis. The control device according to claim 2 .

4. a first user interface for inputting the first instruction; The control device according to claim 1 .

5. a second user interface for inputting the second instruction; The control device according to claim 2 .

6. the first imaging unit and the second imaging unit are attached to a moving body; The control device according to claim 1 .

7. The moving object is any one of a drone, an aircraft, a ship, and a vehicle. The control device according to claim 6.

8. the first imaging unit captures an image at a wider angle than the second imaging unit; The control device according to claim 1 .

9. the first imaging unit is fixed to a front surface of the moving body; The control device according to claim 6.

10. the second imaging unit is driven to pan and tilt; The control device according to claim 1 .

11. a third user interface for inputting at least one of an instruction to switch from the third image to the fourth image and an instruction not to switch from the third image to the fourth image; the third user interface is displayed after receiving the first instruction. The control device according to claim 1 .

12. the display control means displays the third image and the fourth image based on the first instruction. The control device according to claim 11.

13. the first non-visible light imaging unit and the second non-visible light imaging unit are imaging units sensitive to at least one of near-infrared light, mid-infrared light, far-infrared light, and ultraviolet light; The control device according to claim 1 .

14. the display control means switches from the first image to the second image and from the third image to the fourth image based on brightness of the first image; The control device according to claim 1 .

15. the display control means does not switch from the first image to the second image and switches from the third image to the fourth image based on brightness of the third image. The control device according to claim 1 .

16. the first image and the third image are visible images, and the second image and the fourth image are non-visible images; or the first image and the third image are non-visible images, and the second image and the fourth image are visible images; The control device according to claim 1 .

17. a first imaging unit including a first visible light imaging unit and a first non-visible light imaging unit; a second imaging unit including a second visible light imaging unit and a second non-visible light imaging unit; an output unit that outputs a first image or a second image from the first imaging unit and a third image or a fourth image from the second imaging unit; and an output control means for controlling, when the first image and the third image are displayed, an operation of the first imaging unit so that the first imaging unit outputs the third image, and for controlling an operation of the second imaging unit so that the second imaging unit outputs the fourth image, based on a first instruction to switch from the first image to the second image different from the first image. Imaging device.

18. the first image and the third image are visible images, and the second image and the fourth image are non-visible images; or the first image and the third image are non-visible images, and the second image and the fourth image are visible images; The imaging device according to claim 17.

19. a first imaging unit including a first visible light imaging unit and a first non-visible light imaging unit; an imaging device including a second imaging unit including a second visible light imaging unit and a second non-visible light imaging unit; and a control device according to any one of claims 1 to 16, in communication with the imaging device. Imaging system.

20. The imaging device according to claim 17; a control device communicating with the imaging device, a control device including a display means for displaying the first image or the second image, and the third image or the fourth image output by the imaging device; Imaging system.

21. A method executed by a controller, comprising: a display step of displaying a first image or a second image output by a first imaging unit including a first visible light imaging unit and a first non-visible light imaging unit, and a third image or a fourth image output by a second imaging unit including a second visible light imaging unit and a second non-visible light imaging unit; a display control step of switching from the first image to the second image and from the third image to the fourth image different from the third image, based on a first instruction to switch from the first image to the second image different from the first image, when the first image and the third image are displayed; method.

22. A method performed by an imaging device, comprising: The imaging device is a first imaging unit including a first visible light imaging unit and a first non-visible light imaging unit; a second imaging unit including a second visible light imaging unit and a second non-visible light imaging unit; The method comprises: an output step of outputting a first image or a second image from the first imaging unit and a third image or a fourth image from the second imaging unit; an output control step of controlling, when the first image and the third image are being displayed, an operation of the first imaging unit so that the first imaging unit outputs the third image, and controlling an operation of the second imaging unit so that the second imaging unit outputs the fourth image, based on a first instruction to switch from the first image to the second image different from the first image, method.

23. A program for causing a computer to function as each of the means of the control device according to any one of claims 1 to 16.

24. A program for causing a computer to function as each of the means of the imaging apparatus according to claim 17.

Images