Image composition device and image composition method

Abstract

To provide a novel technique for obtaining an appropriate composite image by compositing a plurality of images.SOLUTION: An image composition device acquires picked-up images, which are generated by cameras, and temperature setting values indicating temperature settings of the cameras on a camera basis. The image composition device uses a reference setting value and the temperature setting values of the temperature settings to determine a picked-up image subjected to correction as a correction target image from among a plurality of picked-up images. The image composition device corrects the correction target image on the basis of the reference setting value and the temperature setting value and generates a corrected image. The image composition device composites the picked-up images, which are not subjected to the correction, and the corrected image, thereby generating a composite image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to an image synthesis device and an image synthesis method. [Background technology]

[0002] A technology has been developed that combines images captured by multiple cameras to obtain a single composite image that captures a wider area. For example, Patent Document 1 discloses a technology that combines multiple images of a subject that has few features that can serve as a reference point for image composition. Specifically, the unmanned aerial vehicle in Patent Document 1 includes a camera and a container filled with a liquid or the like. The unmanned aerial vehicle sprays the liquid or the like onto the subject that has few features that can serve as a reference point for image composition, and then captures an image of the subject. The multiple captured images obtained from the camera are combined using the spray pattern created on the subject by the spraying of the liquid or the like as a reference. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-018705 Summary of the Invention [Problem to be solved by the invention]

[0004] Patent Document 1 mentions image alignment in image synthesis. However, image alignment is not the only consideration when synthesizing images. The present disclosure has been made in light of the above-mentioned problems, and one of its purposes is to provide a new technology for synthesizing multiple images to obtain an appropriate synthesized image. [Means for solving the problem]

[0005] The image synthesis device provided by the present disclosure includes an acquisition unit that acquires, for each camera, an image generated by the camera and a temperature setting value indicating the temperature setting of the camera; a determination unit that determines, from among multiple image captures, an image to be corrected as an image to be corrected using a reference setting value and the temperature setting value; a correction unit that corrects the image to be corrected based on the reference setting value and the temperature setting value to generate a corrected image; and a generation unit that generates a composite image by combining an image not selected as an image to be corrected with the corrected image.

[0006] The image compositing method provided by the present disclosure is executed by a computer and includes an acquisition step of acquiring, for each camera, captured images generated by the cameras and a temperature setting value indicating the camera's temperature setting, a determination step of determining, from among multiple captured images, a captured image to be corrected as a correction target image using a reference setting value and the temperature setting value, a correction step of correcting the correction target image based on the reference setting value and the temperature setting value to generate a corrected image, and a generation step of generating a composite image by combining the captured image not selected as the correction target with the corrected image. [Effects of the Invention]

[0007] According to the present disclosure, a new technique is provided for obtaining an appropriate composite image by combining multiple images. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of an overview of an image synthesis device. [Figure 2] FIG. 10 is a diagram illustrating a case where the same object is captured by two cameras. [Figure 3] FIG. 1 is a block diagram illustrating a functional configuration of an image synthesis device. [Figure 4] FIG. 5 is a block diagram illustrating an example of the hardware configuration of a computer 500 that realizes the image synthesis device. [Figure 5]10 is a flowchart illustrating a flow of processing executed by the image synthesis device. [Figure 6] FIG. 10 is a diagram illustrating an example of correction information. [Figure 7] 10A and 10B are diagrams illustrating examples of correction information determined for each type of subject. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are designated by the same reference numerals, and duplicate explanations will be omitted as necessary for clarity. Furthermore, unless otherwise specified, predetermined values ​​such as predetermined values ​​and threshold values ​​are stored in advance in a storage unit accessible from a device that uses the values. Furthermore, unless otherwise specified, the storage unit is composed of one or any number of storage devices.

[0010] <Summary> Fig. 1 is a diagram illustrating an example of an outline of the operation of the image synthesis device 2000. Fig. 1 is a diagram conceptually illustrating an example of the operation of the image synthesis device 2000 to facilitate understanding of the image synthesis device 2000, and the operation of the image synthesis device 2000 is not limited to that shown in Fig. 1.

[0011] The image synthesis device 2000 acquires multiple captured images 20 generated by different cameras 10, and corrects one or more of the captured images 20 as necessary. Here, an image generated by correcting the captured image 20 is called a corrected image 40. The image synthesis device 2000 synthesizes the captured image 20 and the corrected image 40 to generate a synthesized image 50.

[0012] The camera 10 is, for example, an infrared camera configured to sense infrared rays and generate an image. The infrared camera may be any of various infrared cameras, such as a far-infrared camera, a mid-infrared camera, or a near-infrared camera. Note that the camera 10 is not limited to an infrared camera and may be any camera.

[0013] The camera 10 is configured so that the temperature setting can be selected from a plurality of setting values. For example, assume that three temperature setting values, high, medium, and low, are available. In this case, the temperature setting value of the camera 10 can be selected from these three setting values.

[0014] For example, in a camera 10 with a temperature setting of medium, pixel values ​​from 0 to 255 are assigned to a temperature range of 10 to 40 degrees Celsius. In this case, in a camera 10 with a temperature setting of low, pixel values ​​from 0 to 255 are assigned to a temperature range of 0 to 30 degrees Celsius. In addition, in a camera 10 with a temperature setting of high, pixel values ​​from 0 to 255 are assigned to a temperature range of 20 to 50 degrees Celsius. Note that the temperature setting values ​​do not need to be divided in 30-degree increments and may be different. For example, high temperatures may be 30 to 55 degrees Celsius, medium temperatures may be 10 to 45 degrees Celsius, and low temperatures may be minus 10 to plus 20 degrees Celsius.

[0015] The temperature setting value of the camera 10 can be set based on any criteria. For example, the temperature setting value of the camera 10 is set according to the ambient temperature of the camera 10. The ambient temperature of the camera 10 is a temperature related to the imaging environment of the camera 10, such as the temperature around the camera 10 or the temperature around the subject of the camera 10.

[0016] The higher the temperature around the subject of camera 10, the higher the temperature of the subject. The higher the temperature of the subject, the larger the value observed by camera 10 (i.e., the image value of captured image 20). Therefore, for example, the higher the temperature around camera 10, the higher the temperature setting value of camera 10 is set so that a higher temperature can be observed.

[0017] When the temperature setting value of the camera 10 can be set in this way, a situation may arise in which the temperature setting values ​​of the multiple cameras 10 do not match. Therefore, even when the same object is captured by multiple cameras 10, the pixel values ​​in the captured image 20 may tend to differ depending on the temperature setting values ​​of the cameras 10. For example, there may be captured images 20 with relatively large pixel values ​​overall and captured images 20 with relatively small pixel values ​​overall.

[0018] FIG. 2 is a diagram illustrating a case in which the same object is imaged by two cameras 10. In FIG. 2, camera 10-1 images the left side of spherical object 70 to generate captured image 20-1. Meanwhile, camera 10-2 images the right side of object 70 to generate captured image 20-2. The temperature setting value of camera 10-1 is set to a high temperature, while the temperature setting value of camera 10-2 is set to a low temperature. Note that in captured image 20 and composite image 50 in FIG. 2, areas with larger pixel values ​​are represented by arranging smaller black dots. In other words, areas with smaller pixel values ​​are represented by arranging larger black dots.

[0019] Due to the difference in the temperature setting values, the pixel values ​​of captured image 20-1 and captured image 20-2 are generally different. Specifically, the pixel values ​​of captured image 20-1 are generally larger than the pixel values ​​of captured image 20-2. This is because the temperature setting value of camera 10-1 is set to a high temperature, so the observed values ​​of camera 10-1 tend to be relatively large, while the temperature setting value of camera 10-2 is set to a low temperature, so the observed values ​​of camera 10-2 tend to be relatively small.

[0020] In the flow on the left side of FIG. 2, captured image 20-1 and captured image 20-2 are combined without correction to generate composite image 50-1. The pixel value trends are generally different between the left and right sides of composite image 50-1. In this way, if captured images 20 generated by multiple cameras 10 with different temperature settings are combined without correction, the pixel values ​​will be different on the left and right sides of object 70, for example, resulting in an image that looks like a different object. In other words, a composite image that depicts a situation that differs from reality will be generated.

[0021] Therefore, the image synthesis device 2000 corrects the captured image 20, taking into account the difference in the temperature setting value of the camera 10. For example, in the flow on the right side of FIG. 2, the captured image 20-1 is corrected to generate a corrected image 40-1. Then, the corrected image 40-1 and the captured image 20-2 are synthesized to generate a synthesized image 50-2. As a result, for example, the left and right sides of the object 70 have similar pixel values, resulting in an image in which it is clear that they are the same object. In other words, a synthesized image that more closely resembles reality is generated.

[0022] To take into account differences in the temperature settings of the cameras 10, the image composition device 2000 acquires temperature settings 30 indicating the temperature settings of the respective cameras 10 at the time of generating each captured image 20 (see FIG. 1). Then, the image composition device 2000 uses the temperature settings 30 to determine the captured image 20 to be corrected.

[0023] The captured image 20 to be corrected is determined based on, for example, a reference temperature setting value (hereinafter, referred to as a reference setting value). Specifically, the image composition device 2000 determines the captured image 20 whose corresponding temperature setting value 30 is different from the reference setting value as the captured image 20 to be corrected.

[0024] The image synthesis device 2000 generates a corrected image 40 by correcting the value of each pixel of each captured image 20 determined as a correction target based on the temperature setting value 30 and the reference setting value corresponding to that captured image 20. The image synthesis device 2000 generates a synthesized image 50 using the captured image 20 and the corrected image 40.

[0025] <Examples of effects> According to the image composition device 2000, when combining multiple captured images 20 generated by multiple cameras 10 with configurable temperature settings, correction is performed on the captured images 20 taking into account differences in the temperature settings. As a result, even if the temperature settings of the multiple cameras 10 do not match, it is possible to generate a composite image equivalent to a composite image generated using multiple cameras 10 set to the same temperature settings. Therefore, by combining multiple captured images 20 using the image composition device 2000, it is possible to obtain a composite image that represents a more realistic situation than when multiple captured images 20 are combined without correction taking into account differences in the temperature settings.

[0026] The image synthesis device 2000 of this embodiment will be described in more detail below.

[0027] <Example of functional configuration> 3 is a block diagram illustrating an example of the functional configuration of the image composition device 2000. In this example, the image composition device 2000 includes an acquisition unit 2020, a determination unit 2040, a correction unit 2060, and a generation unit 2080. The acquisition unit 2020 acquires, for each of the multiple cameras 10, a captured image 20 and a temperature setting value 30 of the camera 10 at the time the captured image 20 was generated. The determination unit 2040 determines the captured image 20 to be corrected using the temperature setting value 30. The correction unit 2060 corrects each captured image 20 to be corrected based on the temperature setting value 30 and the reference setting value corresponding to that captured image 20. The generation unit 2080 generates a composite image 50 by combining a corrected image 40 generated for each captured image 20 to be corrected with each captured image 20 that is not to be corrected.

[0028] <Example of hardware configuration> Each functional component of the image synthesis device 2000 may be realized by hardware that realizes the functional component (e.g., a hardwired electronic circuit, etc.), or by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it). Below, a case where each functional component of the image synthesis device 2000 is realized by a combination of hardware and software will be further described.

[0029] FIG. 4 is a block diagram illustrating an example of the hardware configuration of a computer 500 that realizes the image synthesis device 2000. For example, the computer 500 is a stationary computer such as a PC (Personal Computer) or a server machine. Alternatively, the computer 500 may be a portable computer such as a smartphone or a tablet terminal. Alternatively, the computer 500 may be realized by a semiconductor chip such as an SoC (System on Chip). The computer 500 may be a dedicated computer designed to realize the image synthesis device 2000, or may be a general-purpose computer.

[0030] For example, by installing a predetermined application on the computer 500, the computer 500 realizes each function of the image synthesis device 2000. The application is configured with a program for realizing each functional component of the image synthesis device 2000. The program can be acquired by any method. For example, the program can be acquired from a storage medium (such as a DVD (Digital Versatile Disk) or a USB (Universal Serial Bus) memory) on which the program is stored. Alternatively, the program can be acquired by downloading the program from a server device that manages the storage device on which the program is stored.

[0031] The computer 500 includes a bus 502, a processor 504, a memory 506, a storage device 508, an input / output interface 510, and a network interface 512. The bus 502 is a data transmission path for the processor 504, the memory 506, the storage device 508, the input / output interface 510, and the network interface 512 to transmit and receive data to and from each other. However, the method for connecting the processor 504 and other components to each other is not limited to a bus connection.

[0032] The processor 504 is one of various processors, such as a central processing unit (CPU), a digital signal processor (DSP), a graphics processing unit (GPU), or a field-programmable gate array (FPGA). The memory 506 is a main storage device realized using a random access memory (RAM) or the like. The storage device 508 is an auxiliary storage device realized using a hard disk, a solid state drive (SSD), a memory card, or a read only memory (ROM) or the like.

[0033] The input / output interface 510 is an interface for connecting the computer 500 with an input / output device. The network interface 512 is an interface for connecting the computer 500 to a network. The network may be a LAN (Local Area Network) or a WAN (Wide Area Network).

[0034] The storage device 508 stores programs (programs that realize the above-mentioned applications) that realize the various functional components of the image synthesis device 2000. The processor 504 reads these programs into the memory 506 and executes them to realize the various functional components of the image synthesis device 2000. The storage device 508 may also function as a memory unit that stores values ​​and information used by the image synthesis device 2000.

[0035] The image synthesis device 2000 may be realized by one computer 500 or by multiple computers 500. In the latter case, the multiple computers 500 may have the same hardware configuration as each other, or may have different hardware configurations from each other.

[0036] <Processing flow> 5 is a flowchart illustrating the flow of processing executed by the image composition device 2000. The acquisition unit 2020 acquires, for each of the multiple cameras 10, the captured image 20 and the temperature setting value 30 of the camera 10 at the time the captured image 20 was generated (S102). The determination unit 2040 determines the captured image 20 to be corrected using the temperature setting value 30 (S104). The correction unit 2060 corrects each captured image 20 to be corrected based on the temperature setting value 30 corresponding to the captured image 20 and the reference setting value, thereby generating a corrected image 40 (S106). The generation unit 2080 composites the corrected image 40 with each captured image 20 that is not to be corrected to generate a composite image 50 (S108).

[0037] <Example of using the image synthesis device 2000> The image synthesis device 2000 can be used in various situations. For example, the image synthesis device 2000 is used to generate a bird's-eye image capturing a wide range. In this case, the ground is captured from a high place (for example, the sky) using a plurality of cameras 10, and the captured images are synthesized to generate a composite image 50, which is a bird's-eye image capturing a wide range.

[0038] For example, the multiple cameras 10 are provided on a flying object such as a drone. By moving the flying object, the image synthesis device 2000 can generate bird's-eye images of each of multiple locations. Here, one camera 10 may be provided on one flying object, or multiple cameras 10 may be provided on one flying object. In the former case, the multiple flying objects fly in formation, for example, by program control or remote control.

[0039] The image synthesis device 2000 may repeatedly generate the synthetic image 50 for a specific location. In this case, for example, the camera 10 is fixed to the ceiling or outer wall of a building so as to capture an image downward.

[0040] The composite image 50 is not limited to a bird's-eye view image. For example, the composite image 50 may be a panoramic image. In this case, for example, the multiple cameras 10 are installed on one or more vehicles traveling on the ground so as to capture images with a horizontal view.

[0041] <Acquisition of captured image 20 and temperature setting value 30: S102> The acquisition unit 2020 acquires the captured image 20 and the temperature setting value 30 for each of the multiple cameras 10 (S102). There are various methods for acquiring the captured image 20 and the temperature setting value 30. For example, a pair of the captured image 20 generated by the camera 10 and the temperature setting value 30 of the camera 10 at the time the captured image 20 was generated is associated with an identifier for each camera 10 and stored in a storage unit accessible from the image synthesis device 2000. In this case, the acquisition unit 2020 accesses the storage unit and acquires the pair of the captured image 20 and the temperature setting value 30 for each camera 10.

[0042] Here, it is preferable that the multiple captured images 20 used to generate one composite image 50 were generated at the same time or at times close enough to be considered identical. Therefore, for example, when a composite image 50 is generated for a specific time t1, the acquisition unit 2020 acquires, for each camera 10, the captured image 20 whose generation time is closest to t1 and the temperature setting value 30 associated with that captured image 20.

[0043] The acquisition unit 2020 may acquire, from each camera 10, a pair of the captured image 20 and the temperature setting value 30 for that camera 10. For example, the acquisition unit 2020 accesses the camera 10 and acquires the pair of the captured image 20 and the temperature setting value 30 for that camera 10. Alternatively, for example, each camera 10 may be configured to transmit the pair of the captured image 20 and the temperature setting value 30 to the image composition device 2000. In this case, the acquisition unit 2020 receives the pair of the captured image 20 and the temperature setting value 30 transmitted from the camera 10, thereby acquiring the pair of the captured image 20 and the temperature setting value 30 for that camera 10.

[0044] The acquisition unit 2020 may first acquire the captured image 20, and then acquire the temperature setting value 30 corresponding to the captured image 20. For example, assume that the history of temperature settings in the camera 10 is stored in the storage unit. In this case, the acquisition unit 2020 acquires the captured image 20 and extracts the temperature setting value at the time the captured image 20 was generated from the history of temperature settings for the camera 10 that generated the captured image 20. This results in a pair of the captured image 20 and the temperature setting value 30 being obtained.

[0045] The temperature setting value 30 may be metadata attached to the captured image 20. For example, the settings and location information of the camera that generated the image data are attached to the image data as metadata. Therefore, the temperature setting value of the camera 10 at the time of generating the captured image 20 may be indicated as one piece of metadata attached to the captured image 20. In this case, the acquisition unit 2020 acquires the captured image 20, and thereby also acquires the temperature setting value 30 corresponding to the captured image 20.

[0046] <Determining the captured image 20 to be corrected: S104> The determination unit 2040 determines the captured image 20 to be corrected (S104). To do so, the determination unit 2040 determines whether the temperature setting value 30 corresponding to each captured image 20 matches the reference setting value for that captured image 20. If the temperature setting value 30 corresponding to a particular captured image 20 matches the reference setting value, the determination unit 2040 determines that the captured image 20 is not to be corrected. On the other hand, if the temperature setting value 30 corresponding to a particular captured image 20 does not match the reference setting value, the determination unit 2040 determines that the captured image 20 is to be corrected.

[0047] There are various ways to determine the reference setting value. For example, the reference setting value is determined in advance. In this case, for example, the reference setting value is stored in advance in a storage unit accessible from the image synthesis device 2000. The determination unit 2040 acquires the reference setting value by accessing this storage unit.

[0048] The reference setting value may be dynamically determined based on the temperature setting value 30. For example, the determination unit 2040 uses the most frequent value of the acquired plurality of temperature setting values ​​30 as the reference setting value.

[0049] For example, suppose that the acquisition unit 2020 acquires ten temperature setting values ​​30. Five of the temperature setting values ​​30 indicate "high temperature," three of the temperature setting values ​​30 indicate "medium temperature," and two of the temperature setting values ​​30 indicate "low temperature." In this case, the determination unit 2040 uses "high temperature" as the reference setting value.

[0050] In this way, by using the most frequent temperature setting value as the reference setting value, the number of captured images 20 to be corrected can be reduced, and therefore the load of the correction process on the image composition device 2000 can be reduced.

[0051] <Correction execution: S106> The correction unit 2060 performs correction processing on each captured image 20 that is the correction target (S106). The correction processing to be performed on the captured image 20 is determined for each image. For example, the correction processing to be performed on a particular captured image 20 is determined based on the temperature setting value 30 corresponding to that captured image 20 and the reference setting value.

[0052] If the temperature setting value 30 corresponding to the captured image 20 is greater than the reference setting value, the correction process performed on the captured image 20 is a process of decreasing the value of each pixel of the captured image 20. On the other hand, if the temperature setting value 30 corresponding to the captured image 20 is less than the reference setting value, the correction process performed on the captured image 20 is a process of increasing the value of each pixel of the captured image 20.

[0053] For example, correction processes are predefined in association with a plurality of pairs of reference setting values ​​and temperature setting values. Information associating correction processes with pairs of reference setting values ​​and temperature setting values ​​is hereinafter referred to as correction information. The determination unit 2040 applies the correction process determined by the correction information to the captured image 20.

[0054] FIG. 6 is a diagram illustrating an example of correction information. Correction information 200 indicates a reference setting value 202, a temperature setting value 204, and a correction process 206. The correction process 206 indicates a correction process corresponding to the pair of reference setting value 202 and temperature setting value 204. For example, the first record in FIG. 6 indicates a correction method in which "-a is applied to the value of each pixel of the captured image 20" as a correction process when the reference setting value is a low temperature and the temperature setting value of the camera 10 that generated the captured image 20 is a medium temperature. Note that in FIG. 6, a, b, c, and d are all assumed to be positive values.

[0055] The correction performed on each pixel of the captured image 20 may be common to all pixels, or may be different for each pixel. In the latter case, for example, the magnitude of the correction for each pixel value is expressed as a function of the pixel's position (x, y). For example, the correction process 206 for the third record in FIG. 6 represents a correction method of "applying +f(x, y) to the pixel at position (x, y)." Note that in FIG. 6, the output values ​​of f(x, y) and g(x, y) are both positive values.

[0056] The correction process corresponding to a pair of reference set value and temperature set value is determined, for example, by a prior test. For example, pixel values ​​are compared between an image captured by capturing an image of the object 70 with the camera 10 set to a low temperature and an image captured by capturing an image of the same object 70 with the camera 10 set to a high temperature. This makes it possible to determine a correction process corresponding to "reference set value 202 = low temperature and temperature set value 204 = high temperature" and a correction method corresponding to "reference set value 202 = high temperature and temperature set value 204 = low temperature."

[0057] The effect of the temperature setting value on the pixel values ​​of the captured image 20 may differ depending on the type of subject captured by the camera 10. Therefore, the correction information 200 may be defined for each type of subject. For example, the type of subject may be a dam, a solar panel, a farm field, etc.

[0058] 7 is a diagram illustrating an example of correction information 200 defined for each type of subject. In FIG. 7, correction information 200 is defined for each of object types T1, T2, and T3.

[0059] When correction information 200 is defined for each type of subject, correction unit 2060 identifies the type of subject captured by camera 10 (in other words, the type of subject included in captured image 20) and acquires correction information 200 corresponding to the identified type. Then, correction unit 2060 corrects captured image 20 using the acquired correction information 200. Here, a method for identifying the type of subject captured by camera 10 will be described later.

[0060] In this way, by using correction information 200 according to the type of subject captured in the captured image 20, corrections to the captured image 20 can be made more accurately, taking into account the effect that the type of subject captured by the camera 10 has on the pixel values ​​of the captured image 20.

[0061] When correction information 200 is defined for each type of subject, the subject may be the same in all captured images 20, or may be different for each captured image 20. In the former case, the correction unit 2060 identifies the type of subject that is common to all captured images 20, and acquires correction information 200 corresponding to the identified type of subject. Then, the correction unit 2060 uses the acquired single correction information 200 to correct all captured images 20 to be corrected.

[0062] When the subject differs for each captured image 20, the correction unit 2060 identifies the type of subject for each captured image 20 to be corrected. Furthermore, the correction unit 2060 acquires correction information 200 corresponding to the identified type of subject. Then, the correction unit 2060 performs correction processing on each captured image 20 to be corrected, using the correction information 200 acquired for each captured image 20.

[0063] Note that if the subject differs for each captured image 20, the determination unit 2040 may determine a reference setting value for each type of subject. For example, assume that, among the captured images 20 acquired by the acquisition unit 2020, N captured images 20 depict an object of type T1, and M captured images 20 depict an object of type T2. In this case, the determination unit 2040 determines the reference setting value S1 for these N captured images 20 using the temperature setting value 30 corresponding to each of the N captured images 20 depicting the object of type T1. For example, the most frequent value of the N temperature setting values ​​30 is used as the reference setting value S1. Then, the determination unit 2040 determines, among these N captured images 20, the captured images 20 whose corresponding temperature setting value 30 does not match the reference setting value S1 as the captured images 20 to be corrected.

[0064] Similarly, the determination unit 2040 uses the temperature setting values ​​30 corresponding to the M captured images 20 in which an object of type T2 is captured to determine the reference setting value S2 for these M captured images 20. Then, the determination unit 2040 determines, among these M captured images 20, the captured images 20 whose corresponding temperature setting values ​​30 do not match the reference setting value S2 as the captured images 20 to be corrected.

[0065] There are various methods for identifying the type of subject in the captured image 20. For example, the acquisition unit 2020 acquires information indicating the type of subject (hereinafter referred to as subject information). The correction unit 2060 identifies the type of subject indicated in the subject information as the type of subject in the captured image 20.

[0066] If the subject is the same in all the captured images 20, the acquisition unit 2020 acquires, for example, subject information for one representative camera 10 and treats the type of subject indicated in the subject information as the type of subject in all the captured images 20. Alternatively, if the type of subjects in 50% or more of all the captured images 20 is the same type of subject, the acquisition unit 2020 treats that type as the type of subject in all the captured images 20. On the other hand, if the subject is different for each captured image 20, for example, if the type of subjects in less than 50% of all the captured images 20 is the same type of subject, the acquisition unit 2020 acquires subject information for each of the captured images 20 to be corrected.

[0067] Alternatively, for example, the correction unit 2060 may identify the type of subject in the captured image 20 by executing an object recognition process on the captured image 20 .

[0068] Alternatively, for example, the correction unit 2060 may use map information and position information indicating the position of the location captured in the captured image 20 (in other words, the position of the location captured by the camera 10) to identify the type of subject in the captured image 20. Specifically, the correction unit 2060 uses the map information to identify the type of location (for example, a dam or a farm field) corresponding to the position indicated in the position information as the type of subject.

[0069] According to this method of using location information and map information, the type of subject in the captured image 20 can be identified in the image synthesis device 2000. This improves the usability of the image synthesis device 2000.

[0070] There are various methods for identifying the position of the place captured in the captured image 20. For example, it is assumed that location information called a geotag or the like is attached to the captured image 20. In this case, the correction unit 2060 uses the location information attached to the captured image 20 as location information indicating the position of the place captured in the captured image 20.

[0071] Alternatively, for example, if a history of the position of camera 10 is recorded, that record may be used. Specifically, the determination unit 2040 extracts the position of camera 10 at the time of generating captured image 20 from the history of the position of camera 10. The extracted position is used as the position of the location captured in captured image 20. Note that the correction unit 2060 may further use the angle of view of camera 10 in addition to the position of camera 10 to identify the location captured by camera 10.

[0072] In the case where the above-described location information and map information are used, the subject in all captured images 20 may be the same. This is particularly effective when capturing images of a wide range of subjects, such as a dam or a farm field. In this case, for example, the correction unit 2060 acquires location information for one representative camera 10, and identifies the location captured by this camera 10 based on the location information and map information. The correction unit 2060 then treats the type of the identified location as the type of subject common to all captured images 20.

[0073] On the other hand, if the subject is different for each captured image 20, the correction unit 2060 acquires the position information of each camera 10 that generated each captured image 20 to be corrected. Then, the correction unit 2060 identifies the location where the captured image 20 was captured for each captured image 20 based on the position information and map information.

[0074] <Image composition: S108> The generation unit 2080 generates a composite image 50 by combining the captured image 20 and the corrected image 40 (S108). In generating the composite image 50, for a captured image 20 that is not a correction target, the captured image 20 is used as is. On the other hand, for a captured image 20 that is a correction target, the corrected image 40 generated from the captured image 20 is used instead of the captured image 20.

[0075] The generation unit 2080 determines the arrangement of each captured image 20 and each corrected image 40 based on the arrangement of each camera 10. For example, if the camera 10 is configured to capture an image of the ground below, the arrangement of the captured image 20 and the corrected image 40 is determined so as to match the arrangement of the cameras 10 when the ground is viewed from above.

[0076] The composite image 50 is created by, for example, stitching together adjacent captured images 20 or corrected images 40 in the determined arrangement. For example, a technique for generating a panoramic image from multiple images can be used to stitch together the images.

[0077] Although the present invention has been described above with reference to the embodiments, the present invention is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present disclosure.

[0078] In the above embodiments, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored in a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable media or tangible storage media include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable media or communication media include electrical, optical, acoustic, or other forms of propagated signals. [Explanation of symbols]

[0079] 10 Camera 20 Captured images 30 Temperature setting value 40 Corrected Images 50 composite images 70 Object 200 Correction Information 202 Standard setting value 204 Temperature setting value 206 Correction Processing 500 computers 502 Bus 504 processor 506 memory 508 Storage Devices 510 Input / Output Interface 512 network interface 2000 Image Synthesis Device 2020 Acquisition Department 2040 Decision Section 2060 Correction Unit 2080 Generation part

Claims

1. an acquisition unit that acquires, for each camera, a captured image generated by the camera and a temperature setting value that indicates a temperature setting of the camera; a determination unit that determines, as a correction target image, the captured image to be corrected from among the plurality of captured images using a reference setting value of the temperature setting and the temperature setting value; a correction unit that corrects the correction target image based on the reference setting value and the temperature setting value to generate a corrected image; and a generating unit that generates a composite image by combining the captured image that was not the correction target with the corrected image.

2. information defining a correction process corresponding to each pair of the reference set value and the temperature set value is set as correction information; The correction unit The image synthesis device according to claim 1 , wherein the correction information is used to identify the correction process corresponding to the reference setting value and the temperature setting value, and the correction process is performed on the correction target image to generate the corrected image.

3. The correction information is further defined for each type of subject, The image composition device according to claim 2 , wherein the correction unit identifies a type of subject of the captured image to be corrected, and corrects the captured image using the correction information corresponding to the type of subject.

4. The image composition device according to claim 3 , wherein the correction unit identifies the type of the subject using map information and position information of the camera that generated the correction target image.

5. an acquisition step of acquiring, for each camera, a captured image generated by the camera and a temperature setting value indicating a temperature setting of the camera; a determination step of determining, as a correction target image, the captured image to be corrected from among the plurality of captured images using a reference setting value of the temperature setting and the temperature setting value; a correction step of correcting the correction target image based on the reference setting value and the temperature setting value to generate a corrected image; and generating a composite image by combining the captured image that was not the correction target with the corrected image.

Images