Display control device, display control method, program, and photographing system

Abstract

Provided are a display control device, a display control method, a program, and a photographing system that assist subject detection using infrared information. This display control device comprises: at least one processor; and at least one memory unit that stores an instruction to be executed by the processor. The processor: acquires a visible light image obtained by being captured in a wavelength range including a visible light range and an infrared image obtained by being captured in a wavelength range including an infrared range; displays the visible light image; acquires the region of a main subject on the basis of temperature information indicated by the infrared image; acquires the brightness distribution of the region of the visible light image corresponding to the region of the main subject; and generates an image for display obtained by adjusting the region of the visible light image under a condition related to sensitivity according to the brightness distribution.

Description

Display control device, display control method and program, and photography system

[0001] The present invention relates to a display control device, a display control method and program, and an imaging system, and more particularly to a technique for assisting subject detection.

[0002] Patent Document 1 describes an imaging device that includes an imaging optical system, an imaging element that has spectral sensitivity in a wavelength region including wavelengths from visible light to infrared light and captures a subject image that is formed by passing through the imaging optical system, an infrared cut filter and a visible light cut filter, either of which is selectively arranged in a retractable manner on the optical path of the subject image from the imaging optical system, a first driving means that positions the selected cut filter on the optical path, a second driving means that moves the position of at least one of the imaging optical system and the imaging element to a position where it focuses on visible light or a position where it focuses on infrared light, and a control means that controls the first driving means and the second driving means in synchronization.

[0003] Patent Document 2 discloses a first imaging device that captures infrared light, a second imaging device that captures visible light, a first object candidate region extraction unit that extracts a first object candidate region where an object is presumed to exist from an infrared image captured by the first imaging device, a second object candidate region extraction unit that extracts a second object candidate region where an object is presumed to exist from a visible light image captured by the second imaging device, a first evaluation value that indicates the degree of object-likeness in the first object candidate region, and a second evaluation value that indicates the degree of object-likeness in the second object candidate region. the weight determination unit that determines a first weight for the first evaluation value and a second weight for the second evaluation value in the weighted addition based on at least one of the ambient temperature and the amount of infrared energy emitted from the imaging area; an object area determination unit that determines an object area where an object exists based on the integrated evaluation value; and an exposure correction unit that determines an exposure correction amount based on pixel signal values ​​in the object area in the visible light image and performs exposure correction on the visible light image.

[0004] Patent document 3 describes a method for modifying at least one parameter used by a video processing algorithm for monitoring a scene, such as a motion detection algorithm, an object detection algorithm, or an object tracking algorithm, the method including: receiving a first video sequence and a second video sequence of a scene, the first video sequence being captured using a thermal camera so as to include thermal information representative of the temperature of the scene; modifying the value of at least one parameter used by the video processing algorithm based on the thermal information included in the first video sequence, wherein at least one parameter of the video processing algorithm is temperature-dependent, and the thermal information is used to determine in which region of the second video sequence the value of the at least one parameter will be modified and by how much the value of the at least one parameter will be modified within the region, while the value of the at least one parameter is modified while remaining unmodified outside the region; and applying the video processing algorithm to the second video sequence.

[0005] JP 2011-97264 A Japanese Patent No. 6356925 A Japanese Patent No. 6546828 A

[0006] One embodiment of the technique of the present disclosure provides a display control device, a display control method and program, and an imaging system that use infrared information to assist in subject detection.

[0007] In order to achieve the above object, a display control device according to a first aspect of the present disclosure includes at least one processor and at least one memory that stores instructions to be executed by the processor, wherein the processor acquires a visible light image obtained by imaging in a wavelength range that includes the visible light range and an infrared image obtained by imaging in a wavelength range that includes the infrared range, displays the visible light image, acquires the area of ​​the main subject based on temperature information indicated by the infrared image, acquires the brightness distribution of the area of ​​the visible light image that corresponds to the area, and generates an image for display in which the area of ​​the visible light image is adjusted according to sensitivity conditions in accordance with the brightness distribution.

[0008] A display control device according to a second aspect of the present disclosure is preferably the display control device according to the first aspect, wherein the processor generates a display image adjusted according to the ratio of the area of ​​the visible light image to the angle of view of the visible light image.

[0009] A display control device according to a third aspect of the present disclosure is preferably a display control device according to the second aspect, wherein the processor generates a display image that is adjusted to include an area wider than the area of ​​the visible light image and that includes the area of ​​the visible light image when the ratio is equal to or greater than a first threshold.

[0010] A display control device according to a fourth aspect of the present disclosure is the display control device according to the third aspect, wherein the range larger than the region of the visible light image is preferably the entire visible light image.

[0011] In a display control device according to a fifth aspect of the present disclosure, in the display control device according to the second aspect, it is preferable that the processor generates a display image that is adjusted to include an area narrower than the area of ​​the visible light image and that includes at least a portion of the area of ​​the visible light image when the ratio is equal to or greater than a second threshold.

[0012] A display control device according to a sixth aspect of the present disclosure is the display control device according to the second aspect, wherein preferably, when the ratio is less than a third threshold, the processor generates the display image without adjustment.

[0013] A display control device according to a seventh aspect of the present disclosure is a display control device according to any of the first to sixth aspects, wherein the processor preferably generates a display image adjusted to include a range wider than the area of ​​the visible light image and including the area of ​​the visible light image when the amount of movement of the area of ​​the visible light image within the angle of view of the visible light image is equal to or greater than a fourth threshold.

[0014] A display control device according to an eighth aspect of the present disclosure is the display control device according to the seventh aspect, wherein the range wider than the region of the visible light image is preferably the entire visible light image.

[0015] A display control device according to a ninth aspect of the present disclosure is a display control device according to any one of the first to eighth aspects, wherein the processor generates a display image that is adjusted to include a range narrower than the area of ​​the visible light image and that includes at least a portion of the area of ​​the visible light image when the amount of movement of the area of ​​the visible light image within the angle of view of the visible light image is equal to or less than a fifth threshold.

[0016] In a display control device according to a tenth aspect of the present disclosure, in a display control device according to any one of the first to ninth aspects, it is preferable that the processor acquires the area of ​​the main subject when the maximum value of the temperature acquired from the temperature information is equal to or greater than a sixth threshold value.

[0017] In a display control device according to an eleventh aspect of the present disclosure, in a display control device according to any one of the first to tenth aspects, it is preferable that the processor adjusts the brightness index of an area of ​​the visible light image when the brightness index is equal to or less than a seventh threshold value.

[0018] In order to achieve the above object, the photography system according to a twelfth aspect of the present disclosure is a photography system including a visible light camera that captures visible light images, an infrared camera that captures infrared images, a monitor that displays the visible light images, and a display control device according to any of the first to eleventh aspects.

[0019] An imaging system according to a thirteenth aspect of the present disclosure is the imaging system according to the twelfth aspect, wherein the visible light camera preferably controls exposure by measuring the photometry of an area of ​​a main subject.

[0020] In order to achieve the above object, a display control method according to a fourteenth aspect of the present disclosure is a display control method in which at least one processor acquires a visible light image obtained by imaging in a wavelength range including the visible light range and an infrared image obtained by imaging in a wavelength range including the infrared range, displays the visible light image, acquires a region of a main subject based on temperature information indicated by the infrared image, acquires a brightness distribution of a region of the visible light image corresponding to the region, and generates an image for display in which the region of the visible light image is adjusted according to sensitivity conditions in accordance with the brightness distribution.

[0021] In order to achieve the above object, a program according to a fifteenth aspect of the present disclosure is a program that causes a computer to execute the display control method according to the fourteenth aspect.

[0022] The present disclosure also includes a non-transitory computer-readable recording medium such as a CD-ROM (Compact Disk-Read Only Memory) that stores the program according to the fifteenth aspect.

[0023] FIG. 1 is a perspective view of an imaging system including an image processing device according to the present disclosure, as seen from an oblique front. FIG. 2 is a rear view of the imaging system shown in FIG. 1. FIG. 3 is a diagram illustrating an example of the relationship between the angle of view of a visible light camera and the angle of view of an infrared camera. FIG. 4 is a block diagram illustrating an example of the internal configuration of the imaging system. FIG. 5 is a flowchart showing steps of a live view image display control method according to a first embodiment. FIG. 6 is a diagram illustrating an example of generation of a display image. FIG. 7 is a diagram illustrating another example of generation of a display image. FIG. 8 is a diagram illustrating the relationship between the ratio of the sensitivity adjustment range to the angle of view of a visible light image and the range over which the sensitivity is adjusted. FIG. 9 is a flowchart showing steps of a live view image display control method according to a second embodiment. FIG. 10 is a diagram illustrating an example of a display image. FIG. 11 is a flowchart showing steps of a live view image display control method according to a modified example of the second embodiment.

[0024] Preferred embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0025] <Photography System> There is a high demand for capturing wild animals in their natural state in photographs. However, it is extremely difficult to capture wild animals in the frame without them noticing, and it is not uncommon for photographers to miss photographic opportunities due to not noticing them. Many animal watchers rely on tools such as binoculars and their own knowledge and experience to find animals. This disclosure therefore supports subject detection by using an infrared camera to detect animal body temperature and visualize their location.

[0026] FIG. 1 is a perspective view of an imaging system including an image processing device according to the present disclosure, as viewed obliquely from the front, and FIG. 2 is a rear view of the imaging system shown in FIG.

[0027] As shown in FIG. 1 , the imaging system 10 includes a visible light camera 100 and an infrared camera 200 .

[0028] The visible light camera 100 captures visible light images by capturing images in a wavelength range that includes the visible light range. The visible light range refers to the wavelength range of electromagnetic waves from 360 nm to 830 nm. Visible light images may be captured in a wavelength range that includes at least a portion of the visible light range. The visible light camera 100 is a mirrorless digital single-lens camera composed of an interchangeable lens 102 and a camera body 104 to which the interchangeable lens 102 is detachable.

[0029] A body mount (not shown) for mounting the interchangeable lens 102 is provided on the front of the camera body 104. On the top surface of the camera body 104, a shutter release button 22, a shutter speed dial 23, an exposure compensation dial 24, a power lever 25, a hot shoe 30, etc. are mainly provided.

[0030] As shown in FIG. 2, the rear surface of the camera body 104 is provided with an EVF (Electronic View Finder) 26, a MENU / OK key 27, a cross key 28, a playback button 29, an LCD (Liquid Crystal Display) 40, and the like.

[0031] The LCD 40 functions as a display that displays a live view image in the shooting mode, plays back and displays a captured visible light image in the playback mode, and also displays various menu screens. Note that in the shooting mode, when the eye is brought close to the EVF 26, an eye sensor (not shown) automatically switches the display to the EVF 26, and when the eye is taken away, the display switches to the LCD 40. The EVF 26 and the LCD 40 correspond to the "monitor" in this disclosure.

[0032] The MENU / OK key 27 is an operation key that functions both as a menu button for issuing a command to display a menu on the screen of the LCD 40 and as an OK button for issuing a command to confirm and execute a selection.

[0033] The cross key 28 is an operation unit for inputting instructions in four directions (up, down, left, and right), and functions as a button for selecting an item from a menu screen or for instructing the selection of various setting items from each menu. The up and down keys of the cross key 28 function as a zoom switch during shooting or a playback zoom switch in playback mode, and the left and right keys function as frame-by-frame (forward and backward) buttons in playback mode. The playback button 29 is a button for switching to a playback mode in which captured and recorded still images or videos are displayed on the LCD 40.

[0034] The hot shoe 30 is a mounting portion for attaching an external accessory to the visible light camera 100. Two-way communication control between the external accessory and the visible light camera 100 can be performed via electrical contacts provided on the hot shoe 30.

[0035] In the photography system 10, an infrared camera 200 is attached to a hot shoe 30. The visible light camera 100 and the infrared camera 200 are connected via a communication cable 12 conforming to the USB (Universal Serial Bus) standard so that data can be transmitted and received.

[0036] The infrared camera 200 captures an infrared image corresponding to the visible light image captured by the visible light camera 100 by capturing an image in a wavelength range that includes at least a portion of the infrared range. The infrared range refers to a wavelength range of electromagnetic waves between approximately 0.8 μm and 1000 μm. The infrared image may be captured in a wavelength range that includes at least a portion of the infrared range. "Corresponding to a visible light image" means that the infrared image is captured at approximately the same time and with approximately the same angle of view as the visible light image.

[0037] It is preferable that the visible light image and the infrared image are captured approximately simultaneously, but they do not have to be captured exactly simultaneously; they may be captured with a time difference within a range that allows for an allowable positional shift of a common subject between the images due to a time difference in the capture timing of the two images. For example, the visible light image and the infrared image may be captured within a time period that can be considered to be approximately simultaneous, including a time difference within a range that can be perceived as substantially simultaneous by human perception.

[0038] The infrared camera 200 includes the concepts of a thermography device and a thermal camera that acquires a heat map image that visualizes the temperature distribution of a subject. A heat map image may also be called a thermography image (thermal image). The infrared image may be an image before heat mapping or may be a heat map image. In this specification, the term "infrared image" includes the concept of a heat map image unless the context indicates otherwise.

[0039] The infrared camera 200 is an infrared sensor configured with an infrared lens 202 and a camera body 204 to which the infrared lens 202 is attached. The infrared camera 200 may include a light source such as an infrared LED (Light Emitting Diode) that projects infrared rays toward a subject.

[0040] The infrared lens 202 is a lens that selectively transmits and collects light in the infrared range. The optical axis of the infrared lens 202 may be parallel to the optical axis of the interchangeable lens 102 of the visible light camera 100.

[0041] Fig. 3 is a diagram showing an example of the relationship between the angle of view AV of the visible light camera 100 and the angle of view AI of the infrared camera 200. F3A in Fig. 3 shows a case where the optical axis of the interchangeable lens 102 of the visible light camera 100 and the optical axis of the infrared lens 202 of the infrared camera 200 are parallel, the focal length of the interchangeable lens 102 and the focal length of the infrared lens 202 are the same, and the size of the image sensor 110 (see Fig. 4) of the visible light camera 100 and the size of the image sensor 210 (see Fig. 4) of the infrared camera 200 are the same. The angles of view AV and AI have the same width, and the positions of the angles of view AV and AI are shifted up and down depending on the amount of vertical shift between the optical axes of the interchangeable lens 102 and the infrared lens 202.

[0042] It is sufficient that the angle of view AV of the visible light camera 100 and the angle of view AI of the infrared camera 200 overlap at least partially. F3B in Fig. 3 shows a case where the angle of view AI of the infrared camera 200 is wider than the angle of view AV of the visible light camera 100, and the respective optical axes are offset up and down and left and right. F3C in Fig. 3 shows a case where the angle of view AI of the infrared camera 200 is narrower than the angle of view AV of the visible light camera 100, and the respective optical axes are offset up and down and left and right. Even in this relationship, the visible light image and the infrared image partially overlap, and have approximately the same angle of view.

[0043] It is preferable that the imaging system 10 acquires in advance the correspondence between the pixels of the visible light image and the infrared image according to the difference in the angle of view between the two images. The correspondence between the positions of pixels of the visible light image and the infrared image can be determined based on conditions such as the configuration of the imaging optical systems of the visible light camera 100 and the infrared camera 200 and the spatial arrangement between the two. Furthermore, the correspondence between pixel positions between the two images can be determined based on information about corresponding points of the subject that are common to both images.

[0044] The photographing system 10 is not limited to a system configuration combining the visible light camera 100 and the infrared camera 200, but may also be configured to be capable of acquiring visible light images and infrared images of the same photographing range at the same angle of view by splitting the optical path of the photographing optical system, such as a single-lens camera equipped with both a visible light sensor and an infrared sensor.

[0045] <Internal Configuration of the Imaging System> Fig. 4 is a block diagram showing an example of the internal configuration of the imaging system 10. As shown in Fig. 4, the visible light camera 100 includes an image sensor 110, a processor 120, a memory 130, a display driver 140, an operation unit 150, an input / output interface 160, a sensor driver 170, and an AFE (Analog Front End) 180.

[0046] The image sensor 110 is configured by a CMOS (Complementary Metal-Oxide Semiconductor) color image sensor. Note that the image sensor 110 is not limited to a CMOS type, and may be a CCD (Charge Coupled Device) type image sensor.

[0047] The image sensor 110 has red (R), green (G), and blue (B) color filters arranged in a periodic color array (e.g., Bayer array, X-Trans (registered trademark), etc.) on multiple pixels composed of photoelectric conversion elements (photodiodes) arranged two-dimensionally in the x direction (horizontal direction) and y direction (vertical direction), and a microlens is arranged on each photodiode.

[0048] An optical image of a subject formed on the light-receiving surface of the image sensor 110 by the imaging optical system of the interchangeable lens 102 is converted into an electrical signal by the image sensor 110. Charges corresponding to the amount of incident light are accumulated in each pixel of the image sensor 110, and an electrical signal corresponding to the amount of charge (signal charge) accumulated in each pixel is read out from the image sensor 110 as an image signal.

[0049] The processor 120 executes instructions stored in the memory 130. The hardware structure of the processor 120 is various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various functional units, a GPU (Graphics Processing Unit), which is a processor specialized for image processing, a PLD (Programmable Logic Device), which is a processor whose circuit configuration can be changed after manufacture such as an FPGA (Field Programmable Gate Array), and a dedicated electrical circuit, which is a processor having a circuit configuration designed specifically for executing specific processing such as an ASIC (Application Specific Integrated Circuit).

[0050] A single processing unit may be configured with one of these various processors, or may be configured with two or more processors of the same or different types (e.g., multiple FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU). Multiple functional units may also be configured with a single processor. Examples of multiple functional units configured with a single processor include: a first configuration, as typified by a client or server computer, in which a single processor is configured with a combination of one or more CPUs and software, and this processor operates as multiple functional units; and a second configuration, as typified by a SoC (System on Chip), in which a processor is used to realize the functions of an entire system including multiple functional units on a single IC (Integrated Circuit) chip. In this way, the various functional units are configured with one or more of the above-mentioned various processors as a hardware structure.

[0051] Furthermore, the hardware structure of these various processors is, more specifically, an electric circuit made up of a combination of circuit elements such as semiconductor elements.

[0052] The processor 120 performs overall control of each unit of the visible light camera 100 and various processes in accordance with user operations using the operation unit 150. The various processes performed by the processor 120 include a process for assisting in capturing a visible light image using an infrared image, which will be described later.

[0053] The memory 130 stores instructions to be executed by the processor 120. The memory 130 includes a flash memory (not shown), a random access memory (RAM) (not shown), and a read only memory (ROM) (not shown). The memory 130 also includes a memory card that is detachable from the camera body 104.

[0054] The flash memory and ROM are non-volatile memories that store firmware, various programs including the display control program according to the present disclosure, and captured images (still images, video images), etc.

[0055] The RAM functions as a working area for processing by the processor 120, and also temporarily stores firmware, display control programs, etc. stored in the non-volatile memory. Note that the processor 120 may have a portion of the memory 130 (RAM) built in.

[0056] The display driver 140 converts the input digital image signals into a signal format for display and outputs them to the EVF 26 or LCD 40 in sequence.

[0057] 1 and 2, the shutter release button 22, the shutter speed dial 23, the exposure compensation dial 24, the power lever 25, the MENU / OK key 27, the cross key 28, the playback button 29, etc. The LCD 40 may be configured as a touch panel display and used as the operation unit 150.

[0058] 1 and 2 and a connector portion (not shown) to which the communication cable 12 is connected. The input / output interface 160 transmits and receives data and signals to and from the infrared camera 200.

[0059] The sensor driver 170 controls the reading of image signals from the image sensor 110 in accordance with instructions from the processor 120. The sensor driver 170 also has an electronic shutter function that, in response to an electronic shutter control signal from the processor 120, discharges (resets) the electric charges accumulated in each pixel of the image sensor 110 and starts exposure.

[0060] The AFE 180 performs various analog signal processing on the analog image signal obtained by capturing an image of a subject with the image sensor 110, and converts the processed image signal into a digital image signal. The analog processing in the AFE 180 includes, for example, color separation processing and AGC (Automatic Gain Control). The AGC functions as a sensitivity adjustment unit that adjusts the sensitivity (ISO sensitivity (ISO: International Organization for Standardization)) during shooting, and adjusts the gain of the amplifier that amplifies the input image signal so that the signal level of the image signal falls within an appropriate range.

[0061] When capturing a still image or a video, image data for each RGB pixel (mosaic image data) output via the image sensor 110 and the AFE 180 is input to and temporarily stored in the memory 130. Note that when the image sensor 110 is a CMOS image sensor, the AFE 180 is often built into the image sensor 110.

[0062] The processor 120 also functions as a digital signal processing unit that performs various types of digital signal processing on image data temporarily stored in the memory 130. That is, the processor 120 performs digital signal processing such as offset processing, gain control processing including sensitivity correction, gamma correction processing, demosaicing processing (also called demosaicing processing or synchronization processing), and RGB / YCrCb conversion processing on the image data input via the AFE 180, and stores the image data after digital signal processing back in the memory 130. Note that, for example, in the case of the image sensor 110 having color filters of three colors, RGB, the demosaicing processing is a process of calculating all RGB color information for each pixel from a mosaic image made up of RGB, and generating synchronized RGB three-plane image data from the mosaic data (dot-sequential RGB data).

[0063] The RGB / YCrCb conversion process is a process for converting the synchronized RGB data into luminance data (Y) and color difference data (Cr, Cb).

[0064] Furthermore, when recording still images or moving images, the processor 120 compresses the uncompressed luminance data Y and color difference data Cb, Cr temporarily stored in the RAM of the memory 130. In the case of still images, the data is compressed in, for example, JPEG (Joint Photographic Coding Experts Group) format, and in the case of moving images, the data is compressed in, for example, H.264 format. The compressed image data is recorded in the flash memory of the memory 130. In addition, in playback mode, the processor 120 reads the compressed image data from the flash memory of the memory 130, decompresses the read image data, generates uncompressed image data, and displays it on the LCD 40 or the like via the display driver 140.

[0065] When displaying a live view image on the EVF 26 or the LCD 40, the processor 120 captures visible light images at a predetermined frame rate (e.g., 30 frames per second (fps) or 60 fps). That is, the sensor driver 170 controls the reading of image signals from the image sensor 110 at the predetermined frame rate. The processor 120 also outputs digital image signals that have been read from the image sensor 110 and digitally processed by the AFE 180 to the display driver 140. The display driver 140 converts the input time-series digital image signals into a signal format for display and outputs them sequentially to the EVF 26 or the LCD 40. As a result, a live view image is displayed in real time on the EVF 26 or the LCD 40.

[0066] The shutter release button 22 is a shooting instruction unit for inputting instructions for shooting still images and videos, and is configured as a two-stage stroke switch consisting of a so-called "half-press" (S1 press) and a "full press" (S2 press).

[0067] An S1_ON signal is output when the shutter release button 22 is "half-pressed," and an S2_ON signal is output when the shutter release button 22 is further pressed from the "half-press" position to the "full press." In still image shooting mode, when the S1_ON signal is output, the processor 120 executes shooting preparation processes such as AF control (Auto Focus control) and AE control (Auto Exposure control), and when the S2_ON signal is output, it executes still image shooting and recording processes.

[0068] When performing AF control, the processor 120 calculates values ​​necessary for AF control based on a digital image signal. In the case of so-called contrast AF, for example, the processor 120 calculates an integrated value (focus evaluation value) of the high-frequency components of the G signal within a predetermined AF area. The processor 120 moves the focus lens included in the lens group of the interchangeable lens 102 to a position where the focus evaluation value is maximized (i.e., the position where the contrast is maximized) during AF control. Note that the AF is not limited to contrast AF. For example, phase difference AF may be performed, in which the amount of defocus is detected based on pixel data from phase difference detection pixels provided in the image sensor, and the focus lens is moved so that this defocus amount becomes zero.

[0069] When performing AE control, the processor 120 detects the brightness of the subject (subject luminance) and calculates a numerical value (exposure value (EV value)) necessary for AE control corresponding to the subject luminance. The processor 120 can determine the F-number, shutter speed, and ISO sensitivity from a predetermined program diagram based on the calculated EV value, and perform AE control.

[0070] It goes without saying that AF control and AE control are performed automatically when the auto mode is set by the operation unit 150, and that AF control and AE control are not performed when the manual mode is set.

[0071] In addition, in the video shooting mode, when the shutter release button 22 is fully pressed and an S2_ON signal is output, the camera body 104 enters a video recording mode in which video recording begins, and performs image processing and recording processing of the video. After that, when the shutter release button 22 is fully pressed again and an S2_ON signal is output, the camera body 104 enters a standby state and temporarily suspends the video recording processing.

[0072] As shown in FIG. 4 , the infrared camera 200 includes an image sensor 210 , an AFE 220 , and an input / output interface 230 .

[0073] The image sensor 210 receives light in a wavelength range including the infrared range and outputs an electrical signal. The image sensor 210 may be a sensor having high sensitivity to any of the wavelength ranges of the near-infrared range (0.8 μm to 2.5 μm), the mid-infrared range (2.5 μm to 4.0 μm), and the far-infrared range (4.0 μm to 1000 μm). Note that, as an infrared sensor corresponding to the far-infrared range, for example, a sensor having sensitivity in the range of 4.0 μm to 14 μm can be used. The image sensor 210 can be a thermal infrared sensor such as a microbolometer or an SOI (Silicon on Insulator) diode type. The resolution of the image sensor 210 and the resolution of the image sensor 110 may be the same or different.

[0074] An optical image of the subject formed on the light receiving surface of the image sensor 210 by the imaging optical system of the infrared lens 202 is converted into an electrical signal by the image sensor 210. Charges corresponding to the amount of incident light are accumulated in each pixel of the image sensor 210, and an electrical signal corresponding to the amount of charge (signal charge) accumulated in each pixel is read out from the image sensor 210 as an image signal.

[0075] The AFE 220 performs various types of analog signal processing on analog image signals obtained by capturing an image of a subject with the image sensor 210, and converts the processed image signals into digital image signals.

[0076] The value of each pixel of the infrared image captured by the infrared camera 200 indicates the intensity of the infrared light, and the intensity of the infrared light is proportional to the temperature of the object.

[0077] 1 and 2 , and a connector portion (not shown) to which the communication cable 12 is connected. The input / output interface 230 transmits and receives data and signals to and from the visible light camera 100. The infrared image captured by the infrared camera 200 is output to the visible light camera 100 via the input / output interface 230.

[0078] When the visible light camera 100 configured in this manner is powered on, the visible light camera 100 and the infrared camera 200 enter a shooting standby state. In the shooting standby state, the visible light camera 100 and the infrared camera 200 start shooting video. The frame rate of the infrared camera 200 may be the same as the frame rate of the visible light camera 100. The frame rate of the infrared camera 200 may be lower than the frame rate of the visible light camera 100. The frame rates of the visible light camera 100 and the infrared camera 200 do not have to be synchronized. Even in this relationship, the visible light image and the infrared image are captured at approximately the same time.

[0079] The video captured by the infrared camera 200 is input to the visible light camera 100 via the communication cable 12. The video captured by the visible light camera 100 is displayed as a live view image on the EVF 26 or the LCD 40 with a specific image, which will be described later, superimposed thereon.

[0080] The user can determine the composition, confirm the subject they want to photograph, and set the photographing conditions by viewing the live view image displayed on the EVF 26 or the LCD 40. A method for controlling the display of a live view image using the photographing system 10 will be described below.

[0081] 5 is a flowchart showing steps of a display control method for live view images according to a first embodiment. The display control method displays live view images that assist a user in detecting a subject such as an animal. The display control method is realized by the processor 120 reading and executing a display control program from the memory 130. The display control program may be provided via the input / output interface 160.

[0082] The processor 120 (an example of a "display control device") starts capturing video for live view images. That is, the visible light camera 100 captures visible light images, and the infrared camera 200 captures infrared images. The captured infrared images are input to the visible light camera 100 via the input / output interface 160. The following processing is performed for each frame.

[0083] In step ST1, the processor 120 determines whether or not an infrared sensor is installed in the visible light camera 100. If the processor 120 determines that an infrared sensor is not installed in the visible light camera 100, the processor 120 ends the processing of this flowchart. On the other hand, if the processor 120 determines that an infrared sensor such as the infrared camera 200 is installed in the visible light camera 100, the processor 120 proceeds to the processing of step ST2.

[0084] In step ST2, the processor 120 acquires an infrared image taken by the infrared camera 200 and obtains a heat map image based on the infrared image. The processor 120 may obtain the heat map image from the infrared camera 200.

[0085] In step ST3, processor 120 compares the temperature within the field of view of the infrared image with a sixth threshold based on the heat map image acquired in step ST2, and determines whether the maximum temperature within the field of view of the infrared image is equal to or greater than the sixth threshold. If processor 120 determines that the maximum temperature is less than the sixth threshold, it proceeds to step ST4. In step ST4, processor 120 determines that a subject (an example of a "main subject") cannot be detected within the field of view of the infrared image, and ends the processing of this flowchart.

[0086] On the other hand, if the processor 120 determines that the maximum temperature value is equal to or greater than the sixth threshold, the process proceeds to step ST5. In step ST5, the processor 120 detects the position of the subject within the angle of view of the infrared image based on the heat map image. For example, the processor 120 determines the position of the maximum temperature value within the angle of view of the infrared image as the position of the subject.

[0087] In step ST6, processor 120 calculates the subject range within the field of view of the infrared image. For example, processor 120 determines, based on the heat map image, the range that includes the subject position detected in step ST5 and has a temperature of 30 degrees or higher as the subject range. By calculating the subject range in this manner, if the main subject is an animal, the subject range will follow the animal's outline. In this way, processor 120 obtains the area of ​​the main subject based on the temperature information indicated by the infrared image.

[0088] In step ST7, the processor 120 acquires a visible light image captured by the visible light camera 100, which corresponds to the infrared image acquired in step ST2.

[0089] In step ST8, processor 120 calculates the average brightness of the subject area. That is, processor 120 identifies an area of ​​the visible light image acquired in step ST7 that corresponds to the area of ​​the main subject in the infrared image acquired in step ST6. Next, processor 120 obtains the brightness distribution of the identified area of ​​the visible light image. Furthermore, processor 120 calculates the average brightness (an example of a "brightness index") of the identified area of ​​the visible light image from the obtained brightness distribution.

[0090] In step ST9, processor 120 compares the average brightness calculated in step ST8 with a seventh threshold and determines whether the average brightness is equal to or less than the seventh threshold. If processor 120 determines that the average brightness is greater than the seventh threshold, it proceeds to step ST10. In step ST10, processor 120 determines that sensitivity adjustment of the visible light image is unnecessary and ends the processing of this flowchart.

[0091] On the other hand, if processor 120 determines that the average brightness is equal to or less than the seventh threshold, it proceeds to step ST11. In step ST11, processor 120 determines whether or not to adjust the brightness of the visible light image. For example, if the maximum sensitivity has already been reached, the sensitivity cannot be increased, and the visible light image cannot be brightened by adjusting the sensitivity. Therefore, if processor 120 determines that the sensitivity cannot be increased and therefore the brightness of the visible light image should not be adjusted, it proceeds to step ST12. In step ST12, processor 120 determines that sensitivity adjustment of the visible light image is unnecessary, and ends the processing of this flowchart.

[0092] On the other hand, if processor 120 determines that the sensitivity can be increased and therefore that the brightness of the visible light image should be adjusted, the process proceeds to step ST13. In step ST13, processor 120 sets the subject position in the visible light image as the photometric point for capturing the visible light image. That is, when capturing a still visible light image, the subject luminance is detected by photometry of the subject position set in step ST13, and the EV value is calculated. Furthermore, processor 120 determines the F-number, shutter speed, and ISO sensitivity from a predetermined program diagram based on the calculated EV value.

[0093] Next, in step ST14, processor 120 calculates a sensitivity adjustment range. The sensitivity adjustment range is a range of the visible light image that includes the region of the visible light image identified in step ST8. The sensitivity adjustment range is, for example, an elliptical range, a rectangular range, or a polygonal range.

[0094] In step ST15, processor 120 determines whether the sensitivity adjustment range is equal to or greater than 30% of the shooting range. That is, processor 120 compares the sensitivity adjustment range calculated in step ST14 with the angle of view of the visible light image, and determines whether the sensitivity adjustment range is equal to or greater than 30% of the visible light image.

[0095] If processor 120 determines that the sensitivity adjustment range is less than 30% of the visible light image, it proceeds to step ST16. In step ST16, processor 120 generates a display image in which the sensitivity of the sensitivity adjustment range of the visible light image has been adjusted. Processor 120 then displays the generated display image on EVF 26 or LCD 40, and ends the processing of this flowchart.

[0096] On the other hand, if processor 120 determines that the sensitivity adjustment range is equal to or greater than 30% of the visible light image, the process proceeds to step ST17. In step ST17, processor 120 generates a display image in which the overall sensitivity of the visible light image has been adjusted. Furthermore, processor 120 displays the generated display image on EVF 26 or LCD 40, and then ends the process of this flowchart.

[0097] The processor 120 adjusts the sensitivity by a gain control process that adjusts the gain (an example of a "sensitivity condition"). The sensitivity is adjusted when the average brightness of the subject range is equal to or lower than a threshold value (step ST9). Therefore, the processor 120 adjusts the sensitivity to increase at least the sensitivity of the sensitivity adjustment range based on the average brightness of the subject range. For example, in the process of step ST16, the sensitivity of pixels within the sensitivity adjustment range is adjusted to be relatively higher than the sensitivity of pixels outside the sensitivity adjustment range. Furthermore, in the process of step ST17, the sensitivity of all pixels in the visible light image is adjusted to be relatively higher. The sensitivity adjustment may be performed by adjusting the ISO sensitivity (an example of a "sensitivity condition") in the AGC of the AFE 180.

[0098] Fig. 6 is a diagram illustrating an example of generation of a display image. F6A in Fig. 6 shows the visible light image IK1 acquired in step ST7. The visible light image IK1 includes a main subject SB1 that corresponds to the subject range within the angle of view of the infrared image. In step ST14, a rectangular sensitivity adjustment range RA1 that includes the main subject SB1 is set in the visible light image IK1.

[0099] The sensitivity adjustment range RA1 is determined to be equal to or greater than 30% of the visible light image IK1 in step ST15, and as a result, the overall sensitivity of the visible light image IK1 is adjusted in step ST17.

[0100] 6 shows display image ID1 in which the sensitivity has been adjusted for the entire visible light image IK1. This display image ID1 is displayed on the EVF 26 or LCD 40. This allows a live view image with improved visibility of the main subject SB1 to be displayed, thereby assisting the user in subject detection.

[0101] 7 is a diagram illustrating another example of generation of a display image. F7A in FIG. 7 shows the visible light image IK2 acquired in step ST7. The visible light image IK2 includes a main subject SB2 that corresponds to the subject range within the angle of view of the infrared image. In step ST14, a rectangular sensitivity adjustment range RA2 that includes the main subject SB2 is set for the visible light image IK2.

[0102] In step ST15, the sensitivity adjustment range RA2 is determined to be less than 30% of the visible light image IK2. As a result, the sensitivity of the visible light image IK2 is adjusted within the sensitivity adjustment range RA2 in step ST16. The sensitivity is not adjusted for ranges outside the sensitivity adjustment range RA2.

[0103] 7 shows display image ID2 in which the sensitivity has been adjusted within sensitivity adjustment range RA2 of visible light image IK2. This display image ID2 is displayed on the EVF 26 or LCD 40. Therefore, a live view image with improved visibility of main subject SB2 can be displayed, thereby assisting the user in detecting the subject.

[0104] In this way, by displaying a live view image in which the visibility of the main subject is improved according to the ratio of the sensitivity adjustment range to the angle of view of the visible light image, the user can find the main subject, such as an animal, even if the visibility of the subject in the visible light image is poor. Furthermore, the user can determine the composition taking into account the position of the main subject and capture a visible light still image of the main subject by fully pressing the shutter release button 22. Note that the sensitivity adjustment of this embodiment is not performed when capturing a visible light still image. A visible light still image is captured using an F-number, shutter speed, and ISO sensitivity based on the photometry results at the photometry point.

[0105] It should be noted that the relationship between the ratio of the sensitivity adjustment range to the angle of view of the visible light image and the range over which the sensitivity is adjusted is not limited to this example. Fig. 8 is a diagram showing the relationship between the ratio of the sensitivity adjustment range to the angle of view of the visible light image and the range over which the sensitivity is adjusted. In Fig. 8, the horizontal axis represents the ratio of the sensitivity adjustment range to the angle of view of the visible light image, with the ratio increasing toward the right in Fig. 8.

[0106] As shown in Fig. 8, when the proportion of the sensitivity adjustment range is equal to or greater than a first threshold, the sensitivity of a relatively wide range of the visible light image may be adjusted to generate an image for display. For example, when the proportion of the sensitivity adjustment range is equal to or greater than the first threshold, a range that includes at least the sensitivity adjustment range of the visible light image but is wider than the sensitivity adjustment range may be adjusted. The range wider than the sensitivity adjustment range may be the entire visible light image. The "30%" in step S15 is an example of the first threshold.

[0107] As shown in FIG. 8 , when the proportion of the sensitivity adjustment range is equal to or greater than a second threshold, the sensitivity of a portion of the visible light image may be adjusted to generate a display image. For example, when the proportion of the sensitivity adjustment range is equal to or greater than the second threshold, the sensitivity may be adjusted to a range that includes at least a portion of the sensitivity adjustment range of the visible light image but is narrower than the sensitivity adjustment range. When the proportion of the sensitivity adjustment range is equal to or greater than a first threshold, the sensitivity of the entire visible light image may be adjusted. When the proportion of the sensitivity adjustment range is less than the first threshold and equal to or greater than a second threshold, the sensitivity of a portion of the visible light image may be adjusted. Furthermore, the first threshold and the second threshold may be the same value. In this case, the user may be able to select in advance whether to adjust the sensitivity of the entire visible light image or the sensitivity of a portion of the visible light image.

[0108] Furthermore, as shown in FIG. 8 , when the proportion of the sensitivity adjustment range is less than the third threshold, the display image may be generated without adjusting the sensitivity of the sensitivity adjustment range. That is, the sensitivity is not adjusted for the entire visible light image. Not adjusting the sensitivity includes adjusting the sensitivity to an extent that the human eye cannot discern whether the sensitivity has been adjusted in the display image, i.e., not adjusting the sensitivity at all. When the proportion of the sensitivity adjustment range is less than the second threshold and equal to or greater than the third threshold, the sensitivity of a portion of the visible light image may be adjusted. In this case, the sensitivity may be adjusted for a portion of a range that is relatively smaller than when the proportion of the sensitivity adjustment range is less than the first threshold and equal to or greater than the second threshold.

[0109] 9 is a flowchart showing the steps of a live view image display control method according to a second embodiment. The processes of steps ST1 to ST14 are the same as those of the first embodiment. After calculating the sensitivity adjustment range in step ST14, processor 120 proceeds to the process of step ST21.

[0110] In step ST21, it is determined whether the amount of movement of the subject position over 0.1 seconds is equal to or greater than 3% of the shooting range of the visible light image. That is, processor 120 calculates the difference between the subject position in the visible light image corresponding to the subject position detected from the infrared image of the frame 0.1 seconds ago and the subject position in the visible light image corresponding to the subject position detected from the infrared image of the current frame as the amount of movement of the subject position over 0.1 seconds. Processor 120 may also determine the difference between the position of the sensitivity adjustment range of the visible light image of the frame 0.1 seconds ago and the position of the sensitivity adjustment range of the visible light image of the current frame as the amount of movement of the subject position over 0.1 seconds. Furthermore, processor 120 compares the calculated amount of movement with the angle of view of the visible light image and determines whether the amount of movement is equal to or greater than 3% of the angle of view of the visible light image.

[0111] If processor 120 determines that the amount of movement of the subject position during 0.1 seconds is less than 3% of the angle of view of the visible light image, it proceeds to step ST22. In step ST22, processor 120 generates a display image in which the sensitivity within the sensitivity adjustment range of the angle of view of the visible light image has been adjusted. Processor 120 then displays the generated display image on EVF 26 or LCD 40, and ends the processing of this flowchart.

[0112] On the other hand, if processor 120 determines that the amount of movement of the subject position during 0.1 seconds is equal to or greater than 3% of the angle of view of the visible light image, it proceeds to step ST23. In step ST23, processor 120 generates a display image in which the overall sensitivity of the angle of view of the visible light image has been adjusted. Furthermore, processor 120 displays the generated display image on EVF 26 or LCD 40, and ends the processing of this flowchart.

[0113] In this way, when the amount of movement of the main subject is relatively small, the visibility of the main subject is improved by adjusting the sensitivity of the sensitivity adjustment range within the angle of view of the visible light image. On the other hand, when the amount of movement of the main subject is relatively large, the subject range in the heat map image based on the infrared image may become blurred and broad. Furthermore, when the amount of movement of the main subject is relatively large, the user may lose sight of the main subject. Therefore, when the amount of movement of the main subject is relatively large, the visibility of the main subject is improved by adjusting the overall sensitivity of the angle of view of the visible light image.

[0114] Fig. 10 shows an example of a display image. F10A in Fig. 10 shows an example in which the amount of movement of the position of main subject SB3 is less than 3%. F10A shows display image ID3 in which the sensitivity of the sensitivity adjustment range including main subject SB3 has been adjusted in step ST22. On the other hand, F10B in Fig. 10 shows an example in which the amount of movement of the position of main subject SB4 is 3% or more. F10B shows display image ID4 in which the sensitivity of the entire visible light image has been adjusted in step ST23.

[0115] By displaying display image ID3 or display image ID4 on the EVF 26 or LCD 40, a live view image can be displayed that improves the visibility of the main subject according to the amount of movement of the main subject, thereby assisting the user in detecting the subject.

[0116] Note that the relationship between the rate of movement of the subject position and the range over which the sensitivity is adjusted is not limited to this example. When the rate of movement of the subject position is equal to or greater than the fourth threshold, the sensitivity of a relatively wide range of the visible light image may be adjusted to generate an image for display. For example, when the rate of movement of the subject position is equal to or greater than the fourth threshold, a range that includes at least the sensitivity adjustment range of the visible light image but is wider than the sensitivity adjustment range may be adjusted. The range wider than the sensitivity adjustment range may be the entire visible light image. The "3%" in step S21 is an example of the fourth threshold.

[0117] If the rate of movement of the subject position is equal to or less than a fifth threshold, the sensitivity of a portion of the visible light image may be adjusted to generate a display image. For example, if the rate of movement of the subject position is equal to or less than the fifth threshold, the sensitivity may be adjusted within a range that includes at least a portion of the sensitivity adjustment range of the visible light image, but is narrower than the sensitivity adjustment range. The "3%" in step S21 is an example of the fifth threshold.

[0118] Furthermore, if the rate of movement of the subject position is equal to or less than the eighth threshold, the display image may be generated without adjusting the sensitivity, i.e., the sensitivity of the entire visible light image is not adjusted.

[0119] 11 is a flowchart showing the steps of a live view image display control method according to a modified example of embodiment 2. The processes in steps ST1 to ST21 are the same as those in embodiment 2.

[0120] If processor 120 determines in step ST21 that the amount of movement of the subject position in 0.1 seconds is less than 3% of the angle of view of the visible light image, it proceeds to step ST32. In step ST32, processor 120 generates a display image in which the sensitivity of a relatively narrow range within the angle of view of the visible light image has been adjusted. Processor 120 then displays the generated display image on EVF 26 or LCD 40, and ends the processing of this flowchart.

[0121] On the other hand, if it is determined in step ST21 that the amount of movement of the subject position during 0.1 seconds is equal to or greater than 3% of the angle of view of the visible light image, the process proceeds to step ST33. In step ST33, processor 120 generates a display image in which the sensitivity of a relatively wide range of the angle of view of the visible light image has been adjusted. Furthermore, processor 120 displays the generated display image on EVF 26 or LCD 40, and the process of this flowchart ends.

[0122] In this way, by generating a display image in which sensitivity conditions are adjusted according to the ratio of the amount of movement of the subject position to the angle of view of the visible light image, it is possible to display a live view image with improved visibility of the main subject, thereby assisting the user in subject detection.

[0123] <Others> So far, an example has been described in which the display control process is completed at each frame rate, but the image for which the sensitivity adjustment range is calculated and the image for which the sensitivity is adjusted may be images of different frames. For example, the sensitivity adjustment range may be calculated from an acquired image, and the sensitivity of an image acquired thereafter may be adjusted based on the calculation result.

[0124] The technical scope of the present invention is not limited to the scope described in the above embodiments. The configurations and the like in each embodiment can be appropriately combined with each other within the scope that does not deviate from the spirit of the present invention.

[0125] DESCRIPTION OF SYMBOLS 10...Photographing system 12...Communication cable 22...Shutter release button 23...Shutter speed dial 24...Exposure compensation dial 25...Power lever 27...MENU / OK key 28...Cross key 29...Playback button 30...Hot shoe 100...Visible light camera 102...Interchangeable lens 104...Camera body 110...Image sensor 120...Processor 130...Memory 140...Display driver 150...Operation unit 160...Input / output interface 170...Sensor driver 200...Infrared camera 202...Infrared lens 204...Camera body 210...Image sensor 230...Input / output interface AI...Field of view of infrared camera AV...Field of view of visible light camera ID1...Image to be displayed ID2...Image to be displayed ID3...Image to be displayed ID4...Image to be displayed IK1...Visible light image IK2...Visible light image RA1...Sensitivity adjustment range RA2...Sensitivity adjustment range SB1...Main subject SB2...Main subject SB3...Main subject SB4...Main subject ST1-ST17, ST21-ST23, ST32-ST33...Process of display control method

Claims

1. at least one processor; at least one memory storing instructions for execution by said processor; Equipped with The processor: A visible light image obtained by imaging in a wavelength range including the visible light range and an infrared image obtained by imaging in a wavelength range including the infrared range are obtained; displaying the visible light image; acquiring a region of a main subject based on temperature information indicated by the infrared image; obtaining a brightness distribution of a region of the visible light image corresponding to the region; generating a display image in which a region of the visible light image is adjusted according to a sensitivity condition in accordance with the brightness distribution; generating the adjusted display image in accordance with a ratio of an area of ​​the visible light image to an angle of view of the visible light image; generating a display image in which a range wider than the area of ​​the visible light image including the area of ​​the visible light image is adjusted when the ratio is equal to or greater than a first threshold value; Display control device.

2. The range larger than the area of ​​the visible light image is the entire visible light image. The display control device according to claim 1 .

3. at least one processor; at least one memory storing instructions for execution by said processor; Equipped with The processor: A visible light image obtained by imaging in a wavelength range including the visible light range and an infrared image obtained by imaging in a wavelength range including the infrared range are obtained; displaying the visible light image; acquiring a region of a main subject based on temperature information indicated by the infrared image; obtaining a brightness distribution of a region of the visible light image corresponding to the region; generating a display image in which a region of the visible light image is adjusted according to a sensitivity condition in accordance with the brightness distribution; generating the adjusted display image in accordance with a ratio of an area of ​​the visible light image to an angle of view of the visible light image; If the ratio is equal to or greater than a second threshold, a display image is generated by adjusting a range narrower than the area of ​​the visible light image, the range including at least a part of the area of ​​the visible light image. Display control device.

4. The processor: generating an image for display without said adjustment if said ratio is less than a third threshold value; The display control device according to claim 3 .

5. at least one processor; at least one memory storing instructions for execution by said processor; Equipped with The processor: A visible light image obtained by imaging in a wavelength range including the visible light range and an infrared image obtained by imaging in a wavelength range including the infrared range are obtained; displaying the visible light image; acquiring a region of a main subject based on temperature information indicated by the infrared image; obtaining a brightness distribution of a region of the visible light image corresponding to the region; generating a display image in which a region of the visible light image is adjusted according to a sensitivity condition in accordance with the brightness distribution; generating a display image in which a range wider than the area of ​​the visible light image and including the area of ​​the visible light image is adjusted when the amount of movement of the area of ​​the visible light image within the angle of view of the visible light image is equal to or greater than a fourth threshold value; Display control device.

6. The range larger than the area of ​​the visible light image is the entire visible light image. The display control device according to claim 5 .

7. at least one processor; at least one memory storing instructions for execution by said processor; Equipped with The processor: A visible light image obtained by imaging in a wavelength range including the visible light range and an infrared image obtained by imaging in a wavelength range including the infrared range are obtained; displaying the visible light image; acquiring a region of a main subject based on temperature information indicated by the infrared image; obtaining a brightness distribution of a region of the visible light image corresponding to the region; generating a display image in which a region of the visible light image is adjusted according to a sensitivity condition in accordance with the brightness distribution; generating a display image in which an adjusted range narrower than the visible light image region is included in the visible light image region and includes at least a part of the visible light image region when the amount of movement of the visible light image region within the angle of view of the visible light image is equal to or less than a fifth threshold value; Display control device.

8. The processor: If the maximum value of the temperature acquired from the temperature information is equal to or greater than a sixth threshold, the area of ​​the main subject is acquired. The display control device according to claim 1 .

9. The processor: When the brightness index of the region of the visible light image is equal to or less than a seventh threshold, the adjustment is performed. The display control device according to claim 1 .

10. a visible light camera that captures the visible light image; an infrared camera that captures the infrared image; a monitor for displaying the visible light image; A display control device according to any one of claims 1 to 7; An imaging system comprising:

11. the visible light camera controls exposure by measuring the area of ​​the main subject; The imaging system according to claim 10.

12. At least one processor A visible light image obtained by imaging in a wavelength range including the visible light range and an infrared image obtained by imaging in a wavelength range including the infrared range are obtained; displaying the visible light image; acquiring a region of a main subject based on temperature information indicated by the infrared image; obtaining a brightness distribution of a region of the visible light image corresponding to the region; generating a display image in which a region of the visible light image is adjusted according to a sensitivity condition in accordance with the brightness distribution; generating the adjusted display image in accordance with a ratio of an area of ​​the visible light image to an angle of view of the visible light image; generating a display image in which a range wider than the area of ​​the visible light image including the area of ​​the visible light image is adjusted when the ratio is equal to or greater than a first threshold value; Display control method.

13. At least one processor A visible light image obtained by imaging in a wavelength range including the visible light range and an infrared image obtained by imaging in a wavelength range including the infrared range are obtained; displaying the visible light image; acquiring a region of a main subject based on temperature information indicated by the infrared image; obtaining a brightness distribution of a region of the visible light image corresponding to the region; generating a display image in which a region of the visible light image is adjusted according to a sensitivity condition in accordance with the brightness distribution; generating the adjusted display image in accordance with a ratio of an area of ​​the visible light image to an angle of view of the visible light image; If the ratio is equal to or greater than a second threshold, a display image is generated by adjusting a range narrower than the area of ​​the visible light image, the range including at least a part of the area of ​​the visible light image. Display control method.

14. At least one processor A visible light image obtained by imaging in a wavelength range including the visible light range and an infrared image obtained by imaging in a wavelength range including the infrared range are obtained; displaying the visible light image; acquiring a region of a main subject based on temperature information indicated by the infrared image; obtaining a brightness distribution of a region of the visible light image corresponding to the region; generating a display image in which a region of the visible light image is adjusted according to a sensitivity condition in accordance with the brightness distribution; generating a display image in which a range wider than the area of ​​the visible light image and including the area of ​​the visible light image is adjusted when the amount of movement of the area of ​​the visible light image within the angle of view of the visible light image is equal to or greater than a fourth threshold value; Display control method.

15. At least one processor A visible light image obtained by imaging in a wavelength range including the visible light range and an infrared image obtained by imaging in a wavelength range including the infrared range are obtained; displaying the visible light image; acquiring a region of a main subject based on temperature information indicated by the infrared image; obtaining a brightness distribution of a region of the visible light image corresponding to the region; generating a display image in which a region of the visible light image is adjusted according to a sensitivity condition in accordance with the brightness distribution; generating a display image in which an adjusted range narrower than the visible light image region is included in the visible light image region and includes at least a part of the visible light image region when the amount of movement of the visible light image region within the angle of view of the visible light image is equal to or less than a fifth threshold value; Display control method.

16. A program that causes a computer to execute the display control method according to any one of claims 12 to 15.

17. A non-transitory computer-readable recording medium having the program according to claim 16 recorded thereon.