IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND IMAGE PROCESSING PROGRAM

Abstract

To provide an image processing device for aligning images of different types.SOLUTION: An image processing device is provided, comprising a reception unit configured to receive input of two images of different types, an association unit configured to align corresponding positions in the two received images, and an output unit configured to output information indicative of the corresponding positions in the two images.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an image processing device that specifies a correspondence relationship between two different types of images, and an image processing method and program thereof. [Background technology]

[0002] In recent years, images captured by artificial satellites (satellite images) are sometimes analyzed to analyze conditions on the ground. Although multiple captured images may be necessary for this analysis, the positions of the captured images captured by the artificial satellites do not necessarily match. For example, Patent Document 1 discloses a technique for aligning pixels between captured images. Patent Document 2 discloses a synthesis technique for superimposing details of an object detected in an infrared image on the corresponding position of a visible light image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-48898 A [Patent Document 2] JP 2019-118043 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the above Patent Document 1, the position is aligned using pixels of the captured image, and there is a problem that the positions of places in the captured image do not necessarily match. In addition, in Patent Document 1, there is also a problem that the position cannot be aligned between captured images of different types. In addition, in Patent Document 2, there is also a problem that the positions of places in the captured images cannot be aligned. In addition, when aligning satellite captured images from different satellites, there is a problem that the position cannot be aligned depending on the imaging standard or imaging conditions.

[0005] SUMMARY OF THE PRESENT DISCLOSURE In view of the above, an object of the present invention is to provide an image processing device, an image processing method, and an image processing program capable of performing intra-image registration between images of different types. [Means for solving the problem]

[0006] In order to solve the above problem, an image processing device according to one embodiment of the present invention includes a receiving unit that receives input of two different types of images, a matching unit that aligns corresponding locations in the two received images, and an output unit that outputs information indicating corresponding positions in the two images.

[0007] In order to solve the above problem, an image processing method according to one embodiment of the present invention includes a computer executing a receiving step of receiving input of two different types of images, a matching step of aligning corresponding locations in the two received images, and an output step of outputting information indicating corresponding positions in the two images.

[0008] In order to solve the above problem, an image processing program according to one embodiment of the present invention provides a computer with a reception function for receiving input of two different types of images, a matching function for aligning corresponding locations within the two received images, and an output function for outputting information indicating corresponding positions within the two images.

[0009] In addition, the above image processing device may include a memory unit that stores a learning model that has learned the correspondence between a first image classified as a first type of image and a second image classified as a second type of image different from the first type and including the same location as the first image, wherein the reception unit receives an input of a third image that is a first type of image and a fourth image that is a second type of image, the correspondence unit aligns a position in the third image with a corresponding position in the fourth image based on the third image, the fourth image, and the learning model, and the output unit outputs information indicating the corresponding positions in the third image and the fourth image.

[0010] In the image processing device, the fourth image may be an image captured at the same location as the third image.

[0011] In addition, the above image processing device may be provided with an input unit that accepts input of at least three corresponding positions for each of the third image and the fourth image, and a learning unit that re-learns the learning model using the third image in which the positions of the three points are indicated by the input unit and a fourth image in which the positions of the corresponding three points are indicated.

[0012] In addition, the above image processing device may be provided with an input unit that accepts input of a third image, a fourth image, and a superimposed image in which the third image and the fourth image are superimposed, and a learning unit that re-learns the learning model using the third image, the fourth image, and the superimposed image input by the input unit.

[0013] In the image processing device, the first image, the second image, the third image, and the fourth image may be satellite images captured by a satellite.

[0014] In addition, in the above image processing device, the receiving unit may receive input of a first image classified as a first type of image and a second image classified as a second type of image different from the first type, the second type of image including the same location as the first image, and may also receive input of information that serves as an index for matching for either of the images, and the matching unit may perform rule-based matching with the other image based on the index information.

[0015] In addition, in the above image processing device, the matching unit may separate a foreground, which is a dynamic object in the image, from a background other than the foreground, from two images of different types, and the matching unit may match the backgrounds of the two images of different types. Effect of the Invention

[0016] According to the image processing device of the present invention, by using a learning model that learns the correspondence between a first image classified as a first type of image and a second image classified as a second type of image different from the first type and capturing the same location as the first image, it is possible to align images of different types within the image and output the information, which can be used, for example, to analyze phenomena based on differences between images. [Brief description of the drawings]

[0017] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of an image processing device. [Diagram 2] (a) An example of a first type of image. (b) An example of a second type of image. [Diagram 3] 1 is a flowchart illustrating an example of the operation of an image processing device, and an example of generating a learning model. [Figure 4] 10 is a flowchart illustrating an example of the operation of the image processing device, showing a process of aligning different types of captured images. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0018] An image processing apparatus according to the present invention will now be described with reference to the drawings.

[0019] <Embodiment> <Configuration> 1 is a block diagram showing an example of the configuration of an image processing device 100. The image processing device 100 is a captured image analysis device having a function of receiving a plurality of captured images and performing intra-position alignment between the captured images. The image processing device 100 is a computer system including a processor and a memory. The image processing device 100 can be realized by a so-called PC, a notebook PC, a tablet terminal, a server device, etc., but is not limited to these. In addition, the image processing device 100 does not have to be realized by a single device, and may be realized by a parallel computer system using a plurality of computers.

[0020] As shown in FIG. 1, the image processing device 100 includes a communication unit 110, an input unit 120, a control unit 130, a storage unit 140, and an output unit 150.

[0021] The communication unit 110 is a communication interface that communicates with other image processing devices outside the image processing device 100 and exchanges information. As an example, the communication unit 110 receives an image captured by the artificial satellite 10 via a communication network and transmits the image to the control unit 130. The communication network may be either wired or wireless, and any communication protocol may be used as long as communication can be performed. The captured image may be received directly from the artificial satellite 10 or via another device. The communication unit 110 may receive multiple captured images captured at the same location as the captured image, and the types of the multiple captured images may be different types of captured images. The multiple captured images may be captured at the same time as long as they include the same location, or may be captured at different times. The type of captured image may determine the standard of the captured image, and may be, for example, a visible light image (RGB image), a SAR (Synthetic Aperture Radar) image, an infrared light image, a YUV image, or the like, but is not limited to these. Any type may be used as long as the types of the two types of captured images compared by the control unit 130 are different. Furthermore, the communication unit 110 may transmit information indicating the positional correspondence between the captured images of different types to an external device in accordance with an instruction from the control unit 130.

[0022] The input unit 120 receives input from an operator of the image processing device 100 and transmits the received input to the control unit 130. The input unit 120 may be realized by, for example, a mouse, a touch pad, a keyboard, or the like, but is not limited to these. The input unit 120 may be, for example, a connection port to which a flash memory storing a plurality of different types of satellite captured images is connected and for transmitting the satellite captured images to the control unit 130. The input unit 120 may receive input of two types of captured images and transmit them to the control unit 130.

[0023] The storage unit 140 has a function of storing programs and various data required for the operation of the image processing device 100. The storage unit 140 can be realized by a storage medium such as an HDD, an SSD, or a flash memory. The storage unit 140 stores a learning model 141. The storage unit 140 may also store one or more pairs of at least two captured images of different types captured at the same location.

[0024] The learning model 141 is a model that has learned the relationship between at least two different types of captured images. Here, the two different types may be two of a plurality of types including the above-mentioned RGB image, SAR image, infrared light image, and YUV image. In the present embodiment, as an example, it is assumed that the two different types are RGB image and SAR image. Therefore, the learning model 141 is a model that can accept the input of an RGB image and an SAR image and identify which positions correspond to each other. In addition, the relationship between captured images here may refer to the correspondence relationship between positions in two types of captured images. That is, the learning model 141 is a model that has learned the correspondence relationship between a first captured image classified as a first type of image and a second captured image classified as a second type of image different from the first type, the second captured image being captured at the same location as the first captured image, and as described above, is a model that has learned the correspondence between corresponding positions in both captured images. In this case, capturing an image of the same place means that the captured images have at least a part of the imaging range of the imaging target overlapping each other. The learning model 141 stored in the storage unit 140 may be generated by the image processing device 100 or may be generated by another device. The captured image may be a satellite image.

[0025] The control unit 130 is a processor that controls each unit of the image processing device 100. The control unit 130 executes various programs using various data stored in the storage unit 140, thereby realizing functions that the image processing device 100 should realize.

[0026] The control unit 130 functions as a receiving unit 131, an association unit 132, and an output unit 133. The control unit 130 may also function as a learning unit 134.

[0027] The receiving unit 131 receives captured images classified as a first type of image (the first captured image and the third captured image) and second captured images classified as a second type of image different from the first type (the second captured image and the fourth captured image) captured at the same location as the first type of image. The receiving unit 131 receives these captured images from the communication unit 110, the input unit 120, or the storage unit 140. The receiving unit 131 transmits the received two different types of captured images (the third captured image and the fourth captured image) to the association unit 132.

[0028] The association unit 132 associates corresponding positions in the two different captured images transmitted from the reception unit 131. The association unit 132 performs the association by inputting the two transmitted captured images to a learning model 141. The association unit 132 transmits to the output unit 133 information indicating the correspondence between the positions in the two captured images obtained as a result of the association.

[0029] The output unit 133 outputs information indicating the correspondence between positions in the two captured images associated by the association unit 132. As an example of an output mode, the output unit 133 may display the information indicating the correspondence on the display unit 150, or may transmit the information to an external device via the communication unit 110. Furthermore, the information indicating the correspondence between positions in the captured images may be in any mode as long as the correspondence is known, and as an example, the information may be a superimposed image in which two captured images are superimposed so that positions in the respective captured images correspond to each other, or may be coordinate values ​​indicating corresponding positions in the two captured images, or may be information pointing to corresponding positions in the two captured images.

[0030] The learning unit 134 learns the relationship between the two types of captured images and generates a learning model 141. The learning unit 134 performs learning using two different types of captured images and information indicating corresponding positions between the two different types of captured images as teacher data. The learning unit 134 may perform learning using, for example, a neural network, a decision tree, a random forest, a support vector machine, or the like, but is not limited to these.

[0031] The display unit 150 has a function of outputting information specified by the control unit 130. The display unit 150 can be realized by a monitor provided in the image processing device 100 or connected to the image processing device 100. The output unit 150 outputs, for example, information associating positions within two captured images of different types.

[0032] The above is an example of the configuration of the image processing device 100.

[0033] <Regarding image alignment> The alignment of captured images of different types and teacher data will be described with reference to Fig. 2. Fig. 2(a) and Fig. 2(b) are examples of captured images 201 and 202 captured at the same location. The captured images 201 and 202 are images of different types. As an example, the captured image 201 may be an SAR image, and the captured image 202 may be an RGB image.

[0034] As can be understood by comparing Figures 2(a) and 2(b), the imaging ranges of the captured images do not need to be exactly the same, and at least a portion of them may overlap. Correspondence between positions in the captured images may be to identify which position coordinates of a point in one captured image correspond to which position coordinates in the other captured image. If the correspondence between at least three points (or two points in some cases) can be identified, the correspondence between the entire captured images can also be identified.

[0035] In the examples of Figures 2(a) and 2(b), points P1a, P1b, and P1c in Figure 2(a) correspond to points P2a, P2b, and P2c in Figure 2(b), respectively. When receiving input of a captured image 201 and a captured image 202 of a different type from the captured image 201, the image processing device 100 may output information indicating that, for example, point P1a (coordinate value) corresponds to point P2a (coordinate value), point P1b (coordinate value) corresponds to point P2b (coordinate value), and point P1c (coordinate value) corresponds to point P2c (coordinate value). In this case, the learning model 141 may be a model that has been learned using as one of the teacher data information correlating a pair of captured image 201 (in FIG. 2(a) where points P1a, P1b, and P1c are not present) and captured image 202 (in FIG. 2(b) where points P2a, P2b, and P2c are not present), and a pair of captured image 201 (see FIG. 2(a)) and captured image 202 (see FIG. 2(b)) on which information indicating the respective corresponding positions is plotted.

[0036] Furthermore, the information indicating the positional correspondence between two different captured images output by the image processing device 100 may be, for example, a superimposed image in which the captured image 201 shown in Fig. 2(a) and the captured image 202 shown in Fig. 2(b) are superimposed such that the point P1a is superimposed on the point P2a, the point P1b is superimposed on the point P2b, and the point P1c is superimposed on the point P2c. In this case, the learning model 141 may be a model that has been learned using information that associates the captured image 201 and the captured image 202 with the superimposed image of the captured image 201 and the captured image 202 as one of the teacher data.

[0037] <Operation> 3 and 4 are flowcharts showing an operation example of the image processing device 100. The flowchart in FIG. 3 is a flowchart showing a processing example when generating the learning model 141. Here, an example is described in which the learning model 141 is generated using captured images in which corresponding positions in captured images are plotted as teacher data as described using FIG. 2, but the method for generating the learning model 141 is not limited to this. The learning model 141 may be generated using other methods as long as it is a model that can associate positions in captured images between two different types of captured images.

[0038] 3, the receiving unit 131 of the image processing device 100 receives a first captured image (step S301). The receiving unit 131 transmits the received first captured image to the learning unit .

[0039] The receiving unit 131 also receives a second captured image (step S302). The second captured image is an image captured at the same location as the first captured image, and is an image of a different type from the first captured image. The receiving unit 131 transmits the second captured image to the learning unit 134.

[0040] The input unit 120 of the image processing device 100 receives an input of three positions in the first captured image and three corresponding positions in the second captured image (step S303). The input unit 120 transmits the received input contents to the learning unit 134. Note that this may receive an input of image information plotting corresponding positions in the first captured image and the second captured image. Here, three points are used, but there may be more than three points, and for example, input of six or ten points may be received. The relatively larger the number of specified points, the higher the matching accuracy (the accuracy of alignment between two images) that is obtained.

[0041] The learning unit 134 learns the correspondence between the first captured image and the second captured image (step S304). That is, the learning unit 134 learns the correspondence between the positions in the first captured image and the positions in the second captured image. The learning unit 134 learns the correspondence between the positions in the captured images of the multiple first captured images and the second captured images.

[0042] The learning unit 134 generates a learning model 141 based on the learning, stores it in the storage unit 140 (step S305), and ends the process. This allows the image processing device 100 to generate the learning model 141 used to estimate the positional correspondence between two different types of captured images.

[0043] The flowchart in FIG. 4 is a flowchart showing an example of processing for aligning accepted captured images using the learning model 141.

[0044] 4, the receiving unit 131 of the image processing device 100 receives a third captured image to be processed (step S401). The receiving unit 131 transmits the received third captured image to the association unit 132.

[0045] The receiving unit 131 also receives a fourth captured image, which is an image captured at the same location as the third captured image but is of a different type from the third captured image (step S402). The receiving unit 131 transmits the received fourth captured image to the association unit 132. Here, the type of the third captured image is the same type as the first captured image. Also, the type of the fourth captured image is the same type as the second captured image.

[0046] The association unit 132 estimates a correspondence relationship between a position in the third captured image and a position in the fourth captured image transmitted from the reception unit 131. That is, the association unit 132 inputs the third captured image and the fourth captured image to the learning model 141 (step S403), and estimates information indicating a position in the captured image of the third captured image and a corresponding position in the captured image of the fourth captured image (step S404). The association unit 132 transmits the estimated result to the output unit 133.

[0047] The output unit 133 outputs information indicating corresponding positions between the positions in the third captured image and the positions in the fourth captured image based on the information transmitted from the association unit 132 (step S405), and ends the process.

[0048] <Summary> As described above, the image processing device 100 can receive input of two different types of captured images and output information on corresponding positions in each captured image. Therefore, even if different types of captured images are used, the image processing device 100 can provide information that serves as a basis for analyzing information in the captured images.

[0049] <Modification> Although one embodiment of the present invention has been described above, it goes without saying that the idea of ​​the present invention is not limited to this. Various modifications included in the idea of ​​the present invention will be described below.

[0050] (1) In the above embodiment, the learning model 141 is a model that has learned the correspondence relationship between positions in a captured image between an RGB image and an SAR image, but the type of image may be other types of images, such as an infrared image or a YUV image, as described above. That is, the learning model 141 may be a model that has learned the correspondence relationship between positions of other combinations, for example, a model that has learned the correspondence relationship between positions of an RGB image and an infrared image. In this case, the learning model 141 is a model that receives input of an RGB image and an infrared image and outputs information indicating the correspondence relationship between the respective positions.

[0051] In the above embodiment, the storage unit 140 may store an individual learning model that learns the correspondence relationship between positions in captured images of two types of captured images corresponding to possible combinations. In this way, the image processing device 100 can output information indicating the correspondence relationship between positions of the input captured images no matter what two types of captured images are input.

[0052] The learning model stored in the storage unit 140 of the information processing device 100 may include a learning model that has learned the correspondence between positions within images of the same type. The image processing device 100 basically accepts input of two different types of captured images and outputs information indicating the correspondence between the positions of the images, but may also be configured to handle input of two captured images of the same type.

[0053] Furthermore, the learning model 141 may be a learning model that has learned the correspondence between three or more types of captured images and positions within the captured images. That is, instead of preparing a plurality of learning models to handle various combinations of two types of captured images as described above, a similar process may be realized by one learning model. When a plurality of learning models are prepared, the accuracy of the alignment between the two types of captured images improves, but the effort of preparing the learning models increases and the processing load of the image processing device 100 increases. On the other hand, when one learning model is used, the accuracy of the alignment between the two types of captured images may be lower than the former.

[0054] (2) In the above embodiment, the image processing device 100 may perform pre-processing such as a process of removing information that becomes noise in each captured image for the alignment process. The process of removing noise may be, for example, removal of objects that are not fixed in place in the captured image, and may be, for example, a process of erasing moving objects such as vehicles, people, animals, etc. from the captured image. In addition, the pre-processing may be a process of removing clouds (sharpening the image) when, for example, clouds make it difficult to see in the situation in which the captured image is captured. By performing these pre-processing processes, it is possible to improve the possibility of performing alignment between captured images with higher accuracy.

[0055] (3) In the above embodiment, the learning unit 134 may perform re-learning using two new captured images (the third captured image and the fourth captured image) that are input. That is, the learning unit 134 may perform re-learning by using, as new teacher data, information that associates the two captured images for estimating the corresponding positional relationship input to the image processing device 100 with the estimation result estimated by the image processing device 100. In addition, if the estimation result by the image processing device 100 is not preferable for the user of the image processing device 100, information that associates the corrected estimation result may be used as new teacher data. With this configuration, the learning model 141 is updated, and each time it is updated, it becomes possible to more accurately align the two captured images.

[0056] (4) In the above embodiment, the captured image is a satellite image, but the captured image is not limited to a satellite image. For example, the captured image may be an aerial photograph captured by a drone or the like.

[0057] Furthermore, one of the two types of captured images does not have to be a captured image. That is, one of the two images to be aligned may not be a captured image, but may simply be image information (an image generated by a method other than imaging). One example of the image information may be a map image. In this case, the learning model is preferably a model that has learned the correspondence between the captured image and the map image, and by inputting the captured image and the map image, information indicating the correspondence between the two can be obtained.

[0058] Furthermore, the two images to be associated do not have to be both captured images. The two images may be, for example, a current map image and an old map image. In this case, the images can be said to be of different types in terms of being new and old. Alternatively, the images can be said to be of different types in terms of being a scanned image (or a captured image) of a handwritten map and map information. In this case, the learning model shown in the above embodiment may be a model that learns the correspondence between old images and new images, and in particular, may be a model that learns the correspondence between an old (for example, handwritten) map image and a latest map image (for example, Google map (registered trademark)).

[0059] (5) In the above embodiment, the matching unit 132, which performs the correspondence between two images, performs the correspondence by using the learning model 141. However, the correspondence by the matching unit 132 is not limited to the correspondence by the learning model 141. The matching unit 132 may perform the correspondence by a rule base. The rule base here may be a rule base that performs the correspondence by a predetermined algorithm, or a rule base that performs the correspondence using a feature amount obtained by analyzing an image. For example, the matching unit 132 analyzes two images and extracts a feature amount of each image. As an example of the feature amount, specifically, the shape of an edge detected from an image or a pixel value may be, but is not limited to these. Then, the matching unit 132 may perform the correspondence by determining that a place where the extracted feature amounts are similar to each other by a predetermined amount or more is a place where the two images correspond to each other. Alternatively, the receiving unit 131 may receive, from a user of the image processing device, a designation of a location serving as an index of a location in the other image to be identified, calculate a feature from the image of the designated index location, and search for a location similar to the calculated feature from the feature of the other image to perform the correspondence. For example, the correspondence unit 132 may receive, for one image, a designation of a range including a characteristic character string (e.g., a logo mark) in the image as information serving as an index, and the correspondence unit 132 may perform correspondence with a location in the other image where the characteristic character string is estimated to be present. In this manner, the correspondence unit 132 may perform correspondence without using the learning model 141.

[0060] (6) In the above embodiment, the matching by the matching unit 132 may separate two images to be matched into a foreground and a background, and match the backgrounds after the separation. Here, the foreground may refer to a dynamic object in the image, and the background may refer to the other parts. For example, in the case of a captured image, a dynamic object may refer to an object that is not necessarily captured in the same place when the same place is captured at different times, and a specific example may be a moving object such as a human, an animal, or a vehicle. Therefore, the background may refer to other objects that do not basically move. For example, plants and trees may move due to wind or the like, but their location is basically fixed, so they are classified as backgrounds. By matching background images in which dynamic information, that is, information that may become noise in the matching, is excluded from the image, the accuracy of the matching can be improved.

[0061] As an example, the foreground and background of an image can be separated by the association unit 132 by inputting each of the two images to be associated into a learning model that has learned the correspondence between a predetermined image, a foreground extracted (manually extracted) from the predetermined image, and a background extracted (manually extracted) from the predetermined image. Alternatively, the association unit 132 may be configured to learn in advance what objects are available as foreground objects, perform image analysis, and separate the background image by removing the learned foreground objects.

[0062] (7) In the above embodiment, each functional unit of the image processing device 100 may be realized by a logic circuit (hardware) or a dedicated circuit formed in an integrated circuit (IC (Integrated Circuit) chip, LSI (Large Scale Integration)), or may be realized by software using a CPU (Central Processing Unit) and a memory. Each functional unit may be realized by one or more integrated circuits, and the functions of multiple functional units may be realized by one integrated circuit. LSI may be called VLSI, super LSI, ultra LSI, etc. depending on the degree of integration. Here, the "circuit" may include the meaning of digital processing by a computer, that is, functional processing by software. The circuit may be realized by a reconfigurable circuit (for example, FPGA: Field Programmable Gate Array).

[0063] When each functional unit of the image processing device 100 is realized by software, the image processing device 100 includes a CPU that executes instructions of an image processing program, which is software that realizes each function, a ROM (Read Only Memory) or a storage device (these are referred to as "recording media") in which the image processing program and various data are recorded so as to be readable by a computer (or CPU), and a RAM (Random Access Memory) in which the image processing program is expanded. The object of the present invention is achieved by having the computer (or CPU) read and execute the image processing program from the recording medium. As the recording medium, a "non-transient tangible medium", for example, a tape, a disk, a card, a semiconductor memory, a programmable logic circuit, etc., can be used. The image processing program may be supplied to the computer via any transmission medium (such as a communication network or a broadcast wave) capable of transmitting the image processing program. The present invention may also be realized in the form of a data signal embedded in a carrier wave in which the image processing program is embodied by electronic transmission.

[0064] The image processing program can be implemented using, for example, a scripting language such as ActionScript or JavaScript (registered trademark), an object-oriented programming language such as Objective-C or Java (registered trademark), or a markup language such as HTML5, but is not limited to these.

[0065] (8) The various configurations and aspects shown in the above embodiments may be combined as appropriate. [Explanation of symbols]

[0066] 100 Image processing device 110 Communications Department 120 Input section 130 Control section 131 Reception 132 Mapping section 133 Output section 134 Learning Department 140 Storage section 150 Display section

Claims

1. A reception unit that receives input of two different types of images; a matching unit that matches corresponding locations in the two received images; an output unit that outputs information indicating corresponding positions in the two images; A storage unit stores a learning model that learns a correspondence relationship between corresponding positions of a first image classified as a first type of image and a second image classified as a second type of image different from the first type and including the same location as the first image, the receiving unit receives input of a third image that is the first type of image and a fourth image that is the second type of image; The matching unit aligns a position in the third image with a corresponding position in the fourth image based on the third image, the fourth image, and the learning model; the output unit outputs information indicating corresponding positions in the third image and the fourth image; One of the two different types of images is any one of a visible light image (RGB image), a SAR (Synthetic Aperture Radar) image, an infrared light image, a YUV image, and a map image, and the other of the two different types of images is any one of a visible light image (RGB image), a SAR (Synthetic Aperture Radar) image, an infrared light image, a YUV image, and a map image that is different from the one of the two different types of images. Image processing device.

2. The fourth image is an image including the same location as the third image.

2. The image processing device according to claim 1,

3. an input unit that receives an input of at least three positions corresponding to each other for the third image and the fourth image; A learning unit that re-learns the learning model using the third image in which the positions of the three points are indicated by the input unit and a fourth image in which the positions of the corresponding three points are indicated.

2. The image processing device according to claim 1,

4. an input unit that receives input of the third image, the fourth image, and a superimposed image in which the third image and the fourth image are superimposed such that corresponding positions of the third image and the fourth image are superimposed; a learning unit that re-learns the learning model using the third image and the fourth image input by the input unit and the superimposed image.

2. The image processing device according to claim 1,

5. The first image, the second image, the third image, and the fourth image are satellite images captured by a satellite.

2. The image processing device according to claim 1,

6. The association unit separates a foreground, which is a dynamic object in the two different types of images, from a background other than the foreground and the background, The association unit associates the backgrounds of the two images of different types with each other.

6. The image processing device according to claim 1,

7. A computer having access to a memory unit that stores a learning model that has learned a correspondence relationship between corresponding positions of a first image classified as a first type of image and a second image classified as a second type of image different from the first type, the second image including the same location as the first image, A receiving step of receiving input of two different types of images; A matching step of matching corresponding locations in the two received images; and outputting information indicating corresponding positions in the two images; The receiving step receives input of a third image which is the first type of image and a fourth image which is the second type of image; The matching step includes aligning a position in the third image with a corresponding position in the fourth image based on the third image, the fourth image, and the learning model; the output step outputs information indicating corresponding positions in the third image and the fourth image; One of the two different types of images is any one of a visible light image (RGB image), a SAR (Synthetic Aperture Radar) image, an infrared light image, a YUV image, and a map image, and the other of the two different types of images is any one of a visible light image (RGB image), a SAR (Synthetic Aperture Radar) image, an infrared light image, a YUV image, and a map image that is different from the one of the two different types of images. Image processing methods.

8. A computer having access to a memory unit that stores a learning model that has learned a correspondence relationship between corresponding positions of a first image classified as a first type of image and a second image classified as a second type of image different from the first type, the second image including the same location as the first image, A receiving function for receiving input of two different types of images; A matching function for matching corresponding locations in the two received images; an output function of outputting information indicating corresponding positions in the two images; the receiving function receives input of a third image which is the first type of image and a fourth image which is the second type of image; The matching function aligns a position in the third image with a corresponding position in the fourth image based on the third image, the fourth image, and the learning model; the output function outputs information indicating corresponding positions in the third image and the fourth image; One of the two different types of images is any one of a visible light image (RGB image), a SAR (Synthetic Aperture Radar) image, an infrared light image, a YUV image, and a map image, and the other of the two different types of images is any one of a visible light image (RGB image), a SAR (Synthetic Aperture Radar) image, an infrared light image, a YUV image, and a map image that is different from the one of the two different types of images. Image processing program.

Images