Information processing device, detection device, information processing method, and program

Abstract

In the present invention, a generation unit generates a detector using a first image set. The detector includes an intermediate data generator and a difference detector. The intermediate data generator has a function for receiving input of an image set that includes two or more images and generating an intermediate data set that includes two or more items of intermediate data. The difference detector has a function for detecting differences between two or more images by using an intermediate data set. A first update unit, by inputting a second image set to the intermediate data generator, generates an intermediate data set corresponding to the second image set. Then, the first update unit updates the intermediate data generator by using the generated intermediate data set. The second image set includes a differently angled image that is obtained at an imaging angle that differs from that of any of the two or more images included in the first image set.

Description

Information processing device, detection device, information processing method, and recording medium

[0001] The present invention relates to an information processing device, a detection device, an information processing method, and a program.

[0002] It is useful to detect differences between multiple images. For example, by comparing multiple images of the same location and detecting the differences, it is possible to understand changes in the state of that location.

[0003] Non-Patent Document 1 discloses a fully convolutional neural network for change detection in image pairs.

[0004] Patent Document 1 discloses a device that generates a change map or the like based on an image captured at a reference time, an image captured at an arbitrary time, and metadata of those images. Patent Document 1 also describes that the metadata includes, for example, data indicating the position of the artificial satellite at the time of capture and data indicating the direction of the antenna used for capture.

[0005] Patent Document 2 discloses an image processing device that extracts an image including an object existence region where an object exists from each of observed images, and deforms the object existence region based on the observation angle of each image and the size of the object. Patent Document 2 also describes determining changes in the object using the images obtained by the deformation.

[0006] Non-Patent Document 2 describes adversarial learning of an encoder and a classifier for application to unsupervised domains.

[0007] Patent Literature 3 discloses a learning device that performs machine learning (semi-supervised learning) using supervised data and unsupervised data. Specifically, Patent Literature 3 describes that the learning device performs data conversion equivalent to domain adaptation on data with domain information and data without domain information, and performs machine learning such as class identification.

[0008] Patent Literature 4 describes a method for training a domain adversarial neural network. Specifically, Patent Literature 4 discloses a feature extraction unit that extracts first features from labeled source data and extracts second features from unlabeled target data. Patent Literature 4 also discloses a label prediction unit that predicts a label of source data based on the first features and predicts a label of target data based on the second features. Patent Literature 4 then describes training the label prediction unit and the feature extraction unit.

[0009] International Publication No. 2020 / 070852 International Publication No. 2022 / 018791 International Publication No. 2019 / 102962 Japanese Patent Application Laid-Open No. 2021-144703

[0010] Rodrigo Caye Daudt, 2 others, "Fully Convolutional Siamese Networks for Change Detection", online, 2018, Computing Research Repository, arXiv:1810.08462v1Eric Tzeng, 3 others, "Adversarial Discriminative Domain Adaptation", online, 2017, Computing Research Repository, arXiv:1702.05464v1

[0011] In the technologies disclosed in Non-Patent Document 1 and Patent Document 1, the accuracy of change detection depends on the training data used for learning, and therefore, there are cases where change detection cannot be performed well for images obtained under unlearned conditions. The technologies disclosed in Non-Patent Document 2, Patent Document 2, and Patent Document 4 do not enable detection of differences between multiple images. The technology disclosed in Patent Document 3 requires that the size of the object be known in advance.

[0012] In view of the above-mentioned problems, an example of an object of the present invention is to provide an information processing device, a detection device, an information processing method, and a program that enable difference detection for various image sets.

[0013] According to one aspect of the present invention, there is provided an information processing device comprising: a generation means for generating, using a first image set, a detector including an intermediate data generator that accepts input of an image set including two or more images and generates an intermediate data set including two or more intermediate data; and a difference detector that detects differences between the two or more images using the intermediate data set; and a first update means for inputting a second image set to the intermediate data generator to generate the intermediate data set corresponding to the second image set and updating the intermediate data generator using the intermediate data set, wherein the second image set includes different-angle images obtained at a different imaging angle from any of the two or more images included in the first image set.

[0014] According to one aspect of the present invention, there is provided a detection device comprising: the intermediate data generator updated by the first update means provided in the above-mentioned information processing device; and a detection means for detecting differences between two or more images using the difference detector.

[0015] According to one aspect of the present invention, an information processing method is provided in which one or more computers use a first image set to generate a detector including: an intermediate data generator that accepts an input of an image set including two or more images and generates an intermediate data set including two or more intermediate data; and a difference detector that uses the intermediate data set to detect differences between the two or more images; a second image set is input to the intermediate data generator to generate the intermediate data set corresponding to the second image set, and the intermediate data generator is updated using the intermediate data set; and the second image set includes different-angle images obtained at a different imaging angle from any of the two or more images included in the first image set.

[0016] According to one aspect of the present invention, there is provided a computer-readable recording medium having a program recorded thereon, the program causing a computer to function as: a generating means that generates, using a first image set, a detector including an intermediate data generator that accepts input of an image set including two or more images and generates an intermediate data set including two or more intermediate data; and a difference detector that detects differences between the two or more images using the intermediate data set; and a first updating means that inputs a second image set to the intermediate data generator to generate the intermediate data set corresponding to the second image set and updates the intermediate data generator using the intermediate data set, wherein the second image set includes different-angle images obtained at a different imaging angle from any of the two or more images included in the first image set.

[0017] According to one aspect of the present invention, it is possible to provide an information processing device, a detection device, an information processing method, and a program that enable difference detection for a variety of image sets.

[0018] 1 is a diagram illustrating an overview of an information processing device according to a first embodiment. FIG. 2 is a diagram illustrating an overview of an information processing method according to the first embodiment. FIG. 3 is a diagram for explaining the relationship between a captured image and a difference map. FIG. 4 is a diagram for explaining capturing of a satellite image. FIG. 5 is a diagram illustrating an example of a functional configuration of a detector according to the first embodiment. FIG. 6 is a diagram for explaining functions of an intermediate data generator and a difference detector according to the first embodiment. FIG. 7 is a diagram illustrating an example of a functional configuration of an information processing device according to the first embodiment. FIG. 8 is a diagram illustrating an example of a configuration of data held in a first image set storage unit. FIG. 9 is a diagram illustrating an example of a configuration of data held in a second image set storage unit. FIG. 10 is a flowchart illustrating an example of a processing flow performed by a generation unit according to the first embodiment. FIG. 11 is a flowchart illustrating an example of a processing flow performed by a first update unit according to the first embodiment. FIG. 12 is a diagram for explaining the function of a classifier used by the first update unit. FIG. 13 is a flowchart illustrating a modified example of the flow of an information processing method according to the first embodiment. FIG. 14 is a diagram illustrating an example of a computer for realizing an information processing device. FIG. 15 is a diagram for explaining the function of a classifier used in an information processing device according to a second embodiment. FIG. 16 is a flowchart illustrating an example of a processing flow performed by a first update unit of an information processing device according to the second embodiment. FIG. 17 is a diagram illustrating an example of a functional configuration of an information processing device according to a third embodiment. FIG. 18 is a flowchart illustrating an example of a processing flow of an information processing method according to the third embodiment. 10 is a flowchart illustrating a flow of processing executed by a second update unit according to a third embodiment. FIG. 11 is a diagram for explaining functions of an intermediate data generator and a difference detector according to a fourth embodiment. FIG. 12 is a diagram illustrating an overview of a detection device according to a fifth embodiment. FIG. 13 is a diagram illustrating an overview of a detection method according to the fifth embodiment.

[0019] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, like components are designated by like reference numerals, and the description thereof will be omitted as appropriate.

[0020] First Embodiment FIG. 1 is a diagram illustrating an overview of an information processing device 10 according to a first embodiment. The information processing device 10 includes a generation unit 130 and a first update unit 150. The generation unit 130 generates a detector using a first image set. The detector includes an intermediate data generator and a difference detector. The intermediate data generator has a function of accepting an input of an image set including two or more images and generating an intermediate data set including two or more intermediate data. The difference detector has a function of detecting differences between the two or more images using the intermediate data set. The first update unit 150 inputs a second image set to the intermediate data generator to generate an intermediate data set corresponding to the second image set. The first update unit 150 then updates the intermediate data generator using the generated intermediate data set. The second image set includes different-angle images obtained at imaging angles different from those of any of the two or more images included in the first image set.

[0021] According to this information processing device 10, a detector capable of detecting differences between various image sets can be obtained.

[0022] FIG. 2 is a diagram illustrating an overview of an information processing method according to this embodiment. The information processing method according to this embodiment is executed by one or more computers. The information processing method according to this embodiment includes steps S10 and S11. In step S10, the one or more computers generate a detector using a first image set. The detector includes an intermediate data generator and a difference detector. The intermediate data generator has a function of accepting an input of an image set including two or more images and generating an intermediate data set including two or more intermediate data. The difference detector has a function of detecting differences between the two or more images using the intermediate data set. In step S11, the one or more computers input a second image set to the intermediate data generator to generate an intermediate data set corresponding to the second image set, and update the intermediate data generator using the generated intermediate data set. The second image set includes different-angle images obtained at an imaging angle different from that of any of the two or more images included in the first image set.

[0023] This information processing method results in a detector capable of differential detection for various image sets.

[0024] This information processing method can be executed by the information processing device 10 according to this embodiment.

[0025] FIG. 3 is a diagram for explaining the relationship between captured images and a difference map. As an example, we will explain a case where two images are compared to generate a map showing the differences. The difference map can be used to detect changes in the parking situation, for example, with a car in a parking lot as the target object. The two images are obtained by capturing the same area.

[0026] Here, when two images are captured at the same imaging angle, as shown in the upper part of Figure 3, the appearance and position of the target object are relatively consistent between the two images, so it is relatively easy to obtain a difference map by taking the difference between the images.

[0027] On the other hand, when two images are captured at different angles, as shown in the bottom row of Figure 3, the appearance and position of the same object will differ between the images, making it difficult to obtain a difference map or to match objects between the images.

[0028] 4 is a diagram illustrating satellite image capture. When multiple satellite images of a predetermined target area (e.g., a parking lot) are obtained, the images can be captured on multiple different orbits (orbit a, orbit b, and orbit c). The image capture angle of the image depends on the positional relationship between the satellite and the target area at the time of image capture. Therefore, for example, if the satellite orbits at the time of image capture are different, the captured images will have different imaging angles (imaging angle A, imaging angle B, and imaging angle C).

[0029] As mentioned above, it is difficult to compare images captured at different imaging angles. However, if the multiple images used to generate a difference map are limited to images captured at the same imaging angle, the number of usable images will be extremely limited. As a result, change detection cannot necessarily be performed as frequently as required. Therefore, it is important to be able to detect differences between multiple images captured at different imaging angles.

[0030] Furthermore, a trained model that has undergone machine learning can be used for difference detection. For example, a model that takes an image pair as input and outputs a change map can be considered. In machine learning of this model, for example, a pair of images acquired at different imaging angles and their correct change maps can be used as training data. In this way, the imaging angles of the images used in machine learning can be used well by the model as trained imaging angles. However, the model cannot perform stable change detection for images acquired at imaging angles other than the trained imaging angles, i.e., untrained imaging angles. In order to accommodate various imaging angles, a large amount of training data must be prepared. In particular, preparing a large number of correct change maps requires time and effort.

[0031] According to the information processing device 10 of this embodiment, a detector that can perform good difference detection can be realized regardless of whether the imaging angle of the input image is a learned imaging angle or an unlearned imaging angle.

[0032] Hereinafter, a detailed example of the information processing device 10 and the information processing method according to this embodiment will be described.

[0033] Fig. 5 is a diagram illustrating the functional configuration of the detector 20 according to this embodiment. Fig. 6 is a diagram for explaining the functions of the intermediate data generator 210 and the difference detector 230 according to this embodiment. The detector 20 is configured to include the intermediate data generator 210 and the difference detector 230. The intermediate data generator 210 and the difference detector 230 each include a neural network.

[0034] When an image set including two or more images is input to the detector 20, the image set is input to the intermediate data generator 210. Note that although Fig. 6 shows a case where the image set includes only two images, the number of images included in the image set is not limited to two. The image set may include three or more images.

[0035] The output data of the intermediate data generator 210 includes an intermediate data set. The intermediate data set includes intermediate data corresponding to each image included in the input image set. In the example of FIG. 6 , the image set input to the intermediate data generator 210 consists of image Xa and image Xb. The intermediate data generator 210 then outputs an intermediate data set consisting of intermediate data fa corresponding to image Xa and intermediate data fb corresponding to image Xb.

[0036] The input data to the difference detector 230 includes the intermediate data set. The difference detector 230 outputs estimated difference data. The estimated difference data is data indicating the difference between the image Xa and the image Xb.

[0037] The estimated difference data output by the difference detector 230 is the result of the difference detection by the detector 20. The detector 20 can, for example, detect differences related to a target object between two or more images included in the input image set. Each image may include one or more target objects. The target object is not particularly limited, and may be, for example, a moving object such as a vehicle, an aircraft, a ship, a living thing, or an object, or a non-moving object such as a building, or may be terrain, etc. An example of an aircraft is an aircraft. An example of a ship is a vessel. An example of an object is a cargo or a container.

[0038] Each of the two or more images constituting the image set is, for example, a satellite image. Furthermore, each of the two or more images is, for example, at least one of a SAR (Synthetic Aperture Radar) image and an optical image. The image set may be composed of only images of the same type, or may include images of different types. For example, in an image set consisting of image Xa and image Xb, image Xa may be a SAR image and image Xb may be an optical image. However, multiple image sets used to generate and update one detector 20 are all image sets of the same combination. In other words, a detector 20 to be applied to an image set consisting of SAR images and optical images is generated and updated using only image sets consisting of SAR images and optical images.

[0039] The entire area included in a certain image is called the "image capture area" of that image. In other words, the image is an image obtained by capturing the image capture area of ​​that image. While the image capture target of an image is not particularly limited, it is preferable that two or more images included in an image set all include the target area in their image capture area. This allows the detector 20 to detect changes in the target area over time. For example, an area included in both the image capture area of ​​image Xa and the image capture area of ​​image Xb can be considered the target area.

[0040] The target area is not particularly limited, but may include, for example, a parking lot. In this case, the detector 20 may be used to detect changes in the parking status of vehicles in the parking lot. As another example, the target area may include an airport. In this case, the detector 20 may be used to detect changes in the parking status of aircraft at the airport. The target area may include, for example, a port. In this case, the detector 20 may be used to detect changes in the berthing status of ships at the port. The target area may include, for example, a container yard. In this case, the detector 20 may be used to detect changes in the placement status of containers at the container yard.

[0041] The intermediate data indicates, for example, information about a target object in an image. Specifically, the intermediate data may indicate the position of the target object in the image. The intermediate data may be a vector indicating feature quantities. The intermediate data generator may be a feature extractor. Two or more images are simultaneously input to the intermediate data generator 210 as an image set. In this way, the intermediate data of one image may be influenced by other images included in the same image set. Furthermore, an intermediate data set suitable for detecting differences between two or more images included in the image set is generated.

[0042] The estimated difference data output by the difference detector 230 may be, for example, a difference map indicating the difference between two images. For example, if there is an object at a certain location in one image and there is no object at that location in the other image, it can be said that there is a difference at that location. The difference map may be a map indicating the presence or absence of a difference at each location in the target region using brightness.

[0043] When the image set includes three or more images, the difference detector 230 may output multiple estimated difference data. For example, the difference detector 230 may output estimated difference data for multiple image pairs, each of which may consist of three or more images included in the image set. Furthermore, one of the three or more images included in the image set may be designated as a reference image. In this case, the difference detector 230 may output estimated difference data indicating the differences between the reference image and each of the other images.

[0044] 7 is a diagram illustrating an example of the functional configuration of an information processing device 10 according to this embodiment. The information processing device 10 is a device that performs machine learning on a detector 20. In the example of FIG. 7 , the information processing device 10 further includes a first image set storage unit 301, a second image set storage unit 302, a detector storage unit 303, and a classifier storage unit 304. However, one or more of the first image set storage unit 301, the second image set storage unit 302, the detector storage unit 303, and the classifier storage unit 304 may be provided outside the information processing device 10.

[0045] The first image set storage unit 301 stores a first image set, and the second image set storage unit 302 stores a second image set.

[0046] Fig. 8 is a diagram illustrating an example of the configuration of data stored in the first image set storage unit 301. Fig. 9 is a diagram illustrating an example of the configuration of data stored in the second image set storage unit 302.

[0047] Of the image sets used to generate and update the detector 20, the image set used by the generator 130, i.e., the image set used in step S10, is referred to as the first image set.

[0048] Among the image sets, an image set that includes a different-angle image is called a second image set. The different-angle image is an image obtained at a different imaging angle from any of the multiple images included in the first image set. The mere term "image set" is a general term for the first image set and the second image set.

[0049] The imaging angle indicates the direction from which the imaging area is imaged. The imaging angle may be a combination of multiple angles. The imaging angle is the angle of a line passing through the satellite capturing the image and the center of the imaging area. The imaging angle may be a combination of an azimuth angle and an incidence angle. The azimuth angle is an angle based on the range direction of the satellite, and the incidence angle is an angle based on the zenith direction. When the imaging angle is a combination of multiple angles, if at least one of the multiple angles is different, the imaging angles can be said to be different from each other. In other words, the same imaging angle means that all of the multiple angles are the same.

[0050] In step S11, the first update unit 150 uses at least the second image set to update the intermediate data generator 210. The generation unit 130 does not use the second image set to update the detector 20.

[0051] As will be described in detail later, the generation unit 130 uses the first image set and the ground truth difference data corresponding to the first image set to update the intermediate data generator 210 and the difference detector 230. On the other hand, the ground truth difference data is not used by the first update unit 150 (described later) to update the intermediate data generator 210 using the second image set.

[0052] 8, a plurality of first image sets are stored in the first image set storage unit 301. Each image is associated with a shooting angle. The shooting angle may be included in the metadata of each image.

[0053] Furthermore, each first image set is associated with correct difference data. The correct difference data is data used as correct data in the machine learning performed in step S10. The correct difference data has the same configuration as the estimated difference data.

[0054] In step S10, the detector 20 is trained using a plurality of first image sets. An imaging angle that matches the imaging angle of at least one of the plurality of images used for training in step S10 is called a "trained imaging angle." An imaging angle other than the trained imaging angle is called an "untrained imaging angle."

[0055] In the example of Fig. 9, the second image set storage unit 302 stores a plurality of second image sets. Each image is associated with a capture angle. The capture angle may be included in the metadata of each image. There is no need for correct difference data to be prepared for the second image sets.

[0056] For ease of understanding, among the imaging angles shown in FIG. 9 , the imaging angles that have been learned are underlined. Each second image set includes at least one image obtained at an imaging angle that has not been learned. That is, the second image set may be a combination of images obtained at imaging angles that have not been learned, or may be a combination of an image obtained at an imaging angle that has not been learned and an image obtained at an imaging angle that has been learned. Images obtained at imaging angles that have not been learned are also referred to as "different angle images."

[0057] As long as each second image set includes images captured at different angles, the first image set storage unit 301 and the second image set storage unit 302 do not need to include information indicating the imaging angle of each image.

[0058] 7 , the following describes the processes executed by the functional components of the information processing device 10. In step S10, the generation unit 130 generates the detector 20. Generating the detector 20 includes updating the detector 20 through machine learning.

[0059] The detector storage unit 303 stores the detector 20. That is, the detector storage unit 303 stores the intermediate data generator 210 and the difference detector 230. Specifically, the detector storage unit 303 stores a plurality of parameters for configuring the intermediate data generator 210 and a plurality of parameters for configuring the difference detector 230. The generation unit 130 and the first update unit 150 can each read out and use the plurality of parameters from the detector storage unit 303.

[0060] 10 is a flowchart illustrating the flow of processing performed by the generation unit 130 according to this embodiment. FIG. 10 is also a flowchart specifically illustrating an example of the processing of step S10.

[0061] In step S101, the generation unit 130 acquires a first image set. The generation unit 130 can acquire the first image set by reading it from the first image set storage unit 301. The generation unit 130 may acquire multiple first image sets at once or sequentially. The generation unit 130 may acquire a predetermined number of first image sets. Instead of reading the first image sets from the first image set storage unit 301, the generation unit 130 may acquire first image sets from another device. Furthermore, the generation unit 130 acquires correct difference data associated with each first image set. The generation unit 130 can acquire the correct difference data in a manner similar to that used to acquire the first image sets.

[0062] The generator 130 inputs a first image set to the intermediate data generator 210 of the detector 20. The intermediate data generator 210 outputs an intermediate data set corresponding to the input first image set. In this manner, the generator 130 generates the intermediate data set (step S102).

[0063] Next, the generator 130 inputs the generated intermediate data set to the difference detector 230. The difference detector 230 outputs estimated difference data corresponding to the input intermediate data set. That is, the difference detector 230 outputs estimated difference data corresponding to the first image set input to the detector 20 (step S103). By generating estimated difference data in this manner, the generator 130 can estimate differences between images included in the first image set.

[0064] In step S10, the output data from the intermediate data generator 210 may be automatically input to the difference detector 230.

[0065] Next, the generation unit 130 uses the estimated difference data and the correct difference data to calculate the loss La (step S104). The calculation of the loss La can be performed using an existing method.

[0066] The generation unit 130 then updates the intermediate data generator 210 and the difference detector 230 using the calculated loss La (step S105). Specifically, the generation unit 130 updates the intermediate data generator 210 using the loss La. The generation unit 130 updates a plurality of parameters for the intermediate data generator 210 stored in the detector storage unit 303. As a method by which the generation unit 130 updates the intermediate data generator 210, an existing method such as backpropagation can be used. Furthermore, the generation unit 130 updates the difference detector 230 using the loss La. The generation unit 130 updates a plurality of parameters for the difference detector 230 stored in the detector storage unit 303. As a method by which the generation unit 130 updates the difference detector 230, an existing method such as backpropagation can be used.

[0067] In this way, the generation unit 130 updates the intermediate data generator 210 and the difference detector 230 so as to reduce the loss La. That is, the generation unit 130 updates the detector 20 so as to reduce the loss La. In this way, the generation unit 130 can update the detector 20 so that the estimated difference data output from the detector 20 approaches the correct difference data.

[0068] Furthermore, the generation unit 130 may repeatedly update the detector 20 using multiple first image sets in one step S10. The update of the detector 20 is repeated, for example, until a termination condition is satisfied. The termination condition may be, for example, at least one of the number of updates in step S10 being equal to or greater than a predetermined number and the loss La being equal to or less than a predetermined standard.

[0069] Note that each update of the detector 20 may be performed by batch processing using two or more first image sets. That is, one loss La may be calculated using multiple comparison results between estimated difference data and ground truth difference data.

[0070] According to step S10, the detector 20 is trained so that the detector 20 can perform good difference detection for the image set obtained at the trained imaging angle.

[0071] Fig. 11 is a flowchart illustrating the flow of processing performed by the first updating unit 150 according to this embodiment. Fig. 11 is also a flowchart specifically illustrating the processing of step S11. Fig. 12 is a diagram for explaining the function of the discriminator 250 used by the first updating unit 150.

[0072] The first updating unit 150 inputs the intermediate data set obtained using the first image set and the intermediate data set obtained using the second image set to the discriminator 250. The discriminator 250 has a function of determining whether the intermediate data set is an intermediate data set obtained from an image set including images from different angles (i.e., the second image set). The first updating unit 150 then updates the intermediate data generator 210 so that the discrimination result by the discriminator 250 becomes a predetermined result. The function of the discriminator 250 and the processing performed by the first updating unit 150 will be described in detail below.

[0073] The input data to the classifier 250 includes an intermediate dataset. The classifier 250 outputs a classification result Vp based on the input intermediate dataset. The classification result Vp is, for example, the likelihood that the intermediate dataset input to the classifier 250 is an intermediate dataset obtained from the second image set. Alternatively, the classification result Vp may be the likelihood that the intermediate dataset input to the classifier 250 is not an intermediate dataset obtained from the second image set. The classifier 250 according to this embodiment outputs one classification result Vp for each intermediate dataset.

[0074] The classifier 250 includes a neural network. In this embodiment, the classifier 250 includes a neural network that has been subjected to machine learning in advance. The classifier storage unit 304 holds a plurality of parameters for configuring the classifier 250. The first update unit 150 can read and use the plurality of parameters from the classifier storage unit 304.

[0075] In step S201, the first updating unit 150 acquires an image set. For example, the first updating unit 150 can acquire the second image set by reading it from the second image set storage unit 302. The first updating unit 150 may acquire multiple second image sets at once or sequentially. The first updating unit 150 may acquire a predetermined number of second image sets. Instead of reading the second image sets from the second image set storage unit 302, the first updating unit 150 may acquire the second image sets from another device.

[0076] The first updating unit 150 reads and uses a plurality of parameters for configuring the intermediate data generator 210 from the detector storage unit 303. The plurality of parameters for configuring the intermediate data generator 210 are the parameters updated by the generating unit 130 in step S10.

[0077] The first updating unit 150 inputs the acquired image set to the intermediate data generator 210. An intermediate data set corresponding to the input image set is output from the intermediate data generator 210. In this manner, the first updating unit 150 generates the intermediate data set (step S202).

[0078] Next, the first update unit 150 executes discrimination by inputting the generated intermediate data set to the discriminator 250 (step S203). The discriminator 250 outputs a discrimination result Vp corresponding to the input intermediate data set.

[0079] In step S11, the output data from the intermediate data generator 210 may be automatically input to the discriminator 250.

[0080] Next, the first updating unit 150 calculates the loss Lb using the discrimination result Vp and the dummy target Vd (step S204).

[0081] The dummy target Vd is information indicating a predetermined judgment result. For example, when the judgment result Vp output from the discriminator 250 is a likelihood within a range of 0 to 1, the dummy target Vd is a predetermined value within a range of 0 to 1. The dummy target Vd is not particularly limited, but may be, for example, 0, 0.5, or 1.

[0082] The loss Lb can be obtained by, for example, comparing the discrimination result Vp with the dummy target Vd. Specifically, the loss Lb can be the difference between the discrimination result Vp and the dummy target Vd. However, any existing method can be used to calculate the loss Lb.

[0083] The first updating unit 150 then updates the intermediate data generator 210 using the calculated loss Lb (step S205). Specifically, the first updating unit 150 updates a plurality of parameters for the intermediate data generator 210 that are stored in the detector storage unit 303. The first updating unit 150 can use an existing method, such as an error backpropagation algorithm, to update the intermediate data generator 210. In this way, the first updating unit 150 updates the intermediate data generator 210 so as to reduce the loss Lb.

[0084] The above describes an example in which the first update unit 150 updates the intermediate data generator 210 using the second image set, but the first update unit 150 may also update the intermediate data generator 210 using the first image set.

[0085] When the first updating unit 150 updates the intermediate data generator 210 using the first image set, the first updating unit 150 acquires the first image set instead of acquiring the second image set in step S201. The first updating unit 150 can acquire the first image set by reading it from the first image set storage unit 301. The first updating unit 150 may acquire multiple first image sets at once or sequentially. The first updating unit 150 may acquire a predetermined number of first image sets. The first updating unit 150 may acquire first image sets from another device instead of reading them from the first image set storage unit 301.

[0086] The first image set acquired by the first updating unit 150 may be the same as or different from the first image set acquired by the generating unit 130. In other words, when the first updating unit 150 acquires a plurality of first image sets, at least some of the first image sets may be the same as the first image set acquired by the generating unit 130. However, the first image set does not include images taken at different angles.

[0087] Even when the first update unit 150 acquires the first image set in step S201, the processing content from step S202 to step S205 is the same as when the second image set is acquired. The first update unit 150 performs an update using at least the second image set. It is preferable that the first update unit 150 performs an update using both the first image set and the second image set.

[0088] The dummy target Vd used is the same whether updating using the first image set or the second image set. That is, according to step S11, learning is performed on the intermediate data generator 210 so that the intermediate data generator 210 generates an intermediate data set that makes discrimination by the discriminator 250 more difficult. Then, based on the intermediate data set obtained using the intermediate data generator 210, the difference detector 230 can accurately detect differences regardless of whether the intermediate data set is obtained from the first image set or the second image set. That is, a detector 20 that can accurately detect differences even for images at different angles is obtained.

[0089] The first update unit 150 may use multiple image sets in one step S11 and repeatedly update the classifier 250. The update of the classifier 250 is repeated, for example, until a termination condition is satisfied. The termination condition may be, for example, at least one of the number of updates in step S11 being equal to or greater than a predetermined number and the loss Lb being equal to or less than a predetermined standard.

[0090] Note that each update of the discriminator 250 may be performed by batch processing using two or more image sets. That is, one loss Lb may be calculated using multiple comparison results between the discrimination result Vp and the dummy target Vd.

[0091] Fig. 13 is a flowchart showing a modified example of the flow of the information processing method according to this embodiment. In the example of Fig. 2, the generation unit 130 does not update the intermediate data generator 210 and the difference detector 230 after the first update unit 150 updates the intermediate data generator 210. On the other hand, in the example of Fig. 13, after the first update unit 150 updates the intermediate data generator 210, the generation unit 130 again updates the intermediate data generator 210 and the difference detector 230.

[0092] In the example of Fig. 13, the first update unit 150 performs step S11 multiple times. Then, each time step S11 is performed, the generation unit 130 updates the detector 20. Specifically, after the processing of step S11 is completed, the first update unit 150 determines whether a termination condition is satisfied (step S12). The termination condition may be, for example, that step S11 is executed a predetermined number of times or more. If the termination condition is satisfied (Yes in step S12), the processing of the information processing device 10 is terminated.

[0093] If the termination condition is not satisfied (No in step S12), the generation unit 130 updates the detector 20 (step S13). The method by which the generation unit 130 updates the detector 20 in step S13 is the same as the method by which the generation unit 130 updates the detector 20 in step S10. However, in step S13, the generation unit 130 further updates the detector 20 including the intermediate data generator 210 updated by the first update unit 150.

[0094] Following step S13, the first updating unit 150 performs step S11 again.

[0095] As described above, in step S11, the intermediate data generator 210 may be updated multiple times. Similarly to step S10, in step S13, the detector 20 may be updated multiple times.

[0096] 7 shows an example in which the first image set storage unit 301 and the second image set storage unit 302 are provided separately, but the first image set and the second image set may be stored in a single storage unit. In this case, the first image set and the second image set are stored in the storage unit in a state in which they can be distinguished from each other. For example, each image set may be associated with classification information indicating whether it is the first image set or the second image set. The generation unit 130 can select and acquire the first image set based on the classification information.

[0097] As another example, an image capturing angle may be associated with each image included in the image set. In this case, after reading out the image set from the storage unit, the generation unit 130 and the first update unit 150 may determine whether the image set corresponds to the first image set or the second image set based on the image capturing angle.

[0098] Specifically, the generation unit 130 may first select multiple image sets to be used in step S10 as first image sets, and identify the imaging angles of the images included in those image sets as learned imaging angles. The generation unit 130 may then register the identified learned imaging angles in a learned list that defines imaging angles. For image sets that the generation unit 130 or the first update unit 150 subsequently reads from the storage unit, the type of image set can be identified using the learned list.

[0099] The hardware configuration of the information processing device 10 will be described below. Each functional component of the information processing device 10 (the generation unit 130 and the first update unit 150) may be realized by hardware that realizes the functional component (e.g., a hardwired electronic circuit, etc.), or may be realized by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it, etc.). Below, a case where each functional component of the information processing device 10 is realized by a combination of hardware and software will be further described.

[0100] FIG. 14 is a diagram illustrating a computer 1000 for implementing the information processing device 10. The computer 1000 is any computer. For example, the computer 1000 is a system on chip (SoC), a personal computer (PC), a server machine, a tablet terminal, a smartphone, or the like. The computer 1000 may be a dedicated computer designed to implement the information processing device 10, or may be a general-purpose computer. Furthermore, the information processing device 10 may be implemented by a single computer 1000, or may be implemented by a combination of multiple computers 1000.

[0101] The computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output interface 1100, and a network interface 1120. The bus 1020 is a data transmission path through which the processor 1040, the memory 1060, the storage device 1080, the input / output interface 1100, and the network interface 1120 transmit and receive data to and from each other. However, the method of interconnecting the processor 1040 and other components is not limited to bus connection. The processor 1040 may be any of various processors, such as a central processing unit (CPU), a graphics processing unit (GPU), or a field-programmable gate array (FPGA). The memory 1060 is a main storage device implemented using a random access memory (RAM) or the like. The storage device 1080 is an auxiliary storage device implemented using a hard disk, a solid state drive (SSD), a memory card, a read-only memory (ROM), or the like.

[0102] The input / output interface 1100 is an interface for connecting the computer 1000 to an input / output device. For example, an input device such as a keyboard and an output device such as a display are connected to the input / output interface 1100. The input / output interface 1100 may be connected to the input device or output device via a wireless connection or a wired connection.

[0103] The network interface 1120 is an interface for connecting the computer 1000 to a network. This communication network may be, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The network interface 1120 may be connected to the network wirelessly or by wire.

[0104] The storage device 1080 stores program modules that realize the various functional components of the information processing device 10. The processor 1040 reads these program modules into the memory 1060 and executes them to realize the functions corresponding to the respective program modules.

[0105] Furthermore, when the first image set storage unit 301 is provided inside the information processing device 10, the first image set storage unit 301 is realized, for example, using the storage device 1080. When the second image set storage unit 302 is provided inside the information processing device 10, the second image set storage unit 302 is realized, for example, using the storage device 1080. When the detector storage unit 303 is provided inside the information processing device 10, the detector storage unit 303 is realized, for example, using the storage device 1080. When the classifier storage unit 304 is provided inside the information processing device 10, the classifier storage unit 304 is realized, for example, using the storage device 1080.

[0106] As described above, according to this embodiment, the first update unit 150 generates an intermediate data set corresponding to the second image set by inputting the second image set to the intermediate data generator. The first update unit 150 then updates the intermediate data generator using the generated intermediate data set. Therefore, a detector 20 capable of detecting differences between various image sets is obtained.

[0107] (Modification 1) The information processing device 10 according to Modification 1 will be described below. Modification 1 is a modification of the first embodiment. The information processing device 10 and the information processing method according to Modification 1 are the same as the information processing device 10 and the information processing method according to the first embodiment, except for the points described below.

[0108] In the information processing device 10 according to this modification, the generation unit 130 generates a detector 20 for each combination of image types included in the image set. Furthermore, the first update unit 150 updates the intermediate data generator 210 of each detector 20. In this way, a group of detectors 20 that can handle various combinations of image types is obtained.

[0109] As described in the first embodiment, each of the two or more images included in the image set is, for example, at least one of a SAR image and an optical image.

[0110] In this modified example, for example, a detector 20 for an image set consisting of SAR images and SAR images, a detector 20 for an image set consisting of optical images and optical images, and a detector 20 for an image set consisting of SAR images and optical images are generated.

[0111] The first and second image sets used when generating and updating detector 20 for detecting differences in an image set consisting of SAR images and SAR images are both image sets consisting of SAR images and SAR images. The first and second image sets used when generating and updating detector 20 for detecting differences in an image set consisting of optical images and optical images are both image sets consisting of optical images and optical images. The first and second image sets used when generating and updating detector 20 for detecting differences in an image set consisting of SAR images and optical images are both image sets consisting of SAR images and optical images.

[0112] The detector storage unit 303 according to this modification holds a plurality of detectors 20. A user who wishes to detect a difference between two images can select and use a detector 20 that corresponds to the type of the images from the group of detectors 20.

[0113] Next, the operation and effect of this embodiment will be described. In this embodiment, the same operation and effect as in the first embodiment can be obtained. In addition, according to the information processing device 10 of this modification, the generation unit 130 generates a detector 20 for each combination of image types included in the image set. Therefore, a group of detectors 20 that can handle various combinations of image types can be obtained.

[0114] Second Embodiment Fig. 15 is a diagram for explaining the function of a discriminator 250 used in an information processing device 10 according to a second embodiment. The information processing device 10 according to this embodiment is the same as the information processing device 10 according to at least one of the first embodiment and modified example 1, except for the points described below. The information processing method according to this embodiment is the same as the information processing method according to at least one of the first embodiment and modified example 1, except for the points described below.

[0115] In the information processing device 10 according to this embodiment, the first update unit 150 inputs an intermediate data set obtained using the first image set and an intermediate data set obtained using the second image set to the discriminator 250. The discriminator 250 has a function of determining whether each of two or more intermediate data sets included in the intermediate data set is intermediate data obtained for images taken at different angles. The first update unit 150 then updates the intermediate data generator 210 so that the discrimination result by the discriminator 250 becomes a predetermined result.

[0116] In this embodiment, the input data to the discriminator 250 also includes an intermediate data set. The intermediate data set is, for example, composed of intermediate data fa and intermediate data fb. The discriminator 250 outputs a discrimination result (Vpa, Vpb) based on the input intermediate data set. The discrimination result Vpa is, for example, the likelihood that the intermediate data fa is intermediate data obtained for an image at a different angle, or the likelihood that the intermediate data fa is not intermediate data obtained for an image at a different angle. The discrimination result Vpb is, for example, the likelihood that the intermediate data fb is intermediate data obtained for an image at a different angle, or the likelihood that the intermediate data fb is not intermediate data obtained for an image at a different angle. In other words, the discriminator 250 according to this embodiment outputs a discrimination result for each intermediate data included in the intermediate data set. The output of the discriminator 250 includes discrimination results for multiple intermediate data.

[0117] 16 is a flowchart illustrating the flow of processing performed by the first update unit 150 of the information processing device 10 according to this embodiment. Fig. 16 is also a flowchart specifically illustrating the processing of step S11 in the information processing method according to this embodiment.

[0118] Steps S301 to S303 are the same as steps S201 to S203, respectively, described with reference to FIG.

[0119] In step S304, the first updating unit 150 calculates the loss Lb using the comparison result between the discrimination result for each intermediate data and the dummy target Vd. For example, the first updating unit 150 first calculates the difference between the discrimination result Vpa and the dummy target Vd as the comparison result C1. The first updating unit 150 also calculates the difference between the discrimination result Vpb and the dummy target Vd as the comparison result C2. The first updating unit 150 then calculates the loss Lb using the comparison results C1 and C2. The loss Lb can be, for example, a weighted sum of the comparison results C1 and C2. The weights for the comparison results C1 and C2 used to calculate the loss Lb are predetermined. However, any existing method can be used to calculate the loss Lb.

[0120] The first updating unit 150 then updates the intermediate data generator 210 using the calculated loss Lb (step S305). Specifically, the first updating unit 150 updates a plurality of parameters for the intermediate data generator 210 that are stored in the detector storage unit 303. As a method by which the first updating unit 150 updates the intermediate data generator 210, an existing method such as an error backpropagation algorithm can be used. In this way, the first updating unit 150 updates the intermediate data generator 210 so as to reduce the loss Lb.

[0121] Next, the operation and effect of this embodiment will be described. In this embodiment, the same operation and effect as in the first embodiment can be obtained.

[0122] (Third embodiment) Fig. 17 is a diagram illustrating an example of the functional configuration of an information processing device 10 according to a third embodiment. Fig. 18 is a flowchart illustrating an example of the processing flow of an information processing method according to this embodiment. The information processing device 10 according to this embodiment is the same as the information processing device 10 according to at least one of the first embodiment, modified example 1, and second embodiment, except for the points described below. The information processing method according to this embodiment is the same as the information processing method according to at least one of the first embodiment, modified example 1, and second embodiment, except for the points described below.

[0123] The information processing device 10 according to this embodiment further includes a second update unit 190 that uses an intermediate data set and a correct answer classification result for the intermediate data set to update the classifier 250. In this embodiment, the classifier 250 may not have undergone machine learning in advance, or may have undergone machine learning to some extent in advance.

[0124] In the example of FIG. 17, the information processing device 10 further includes a correct answer generating unit 170 that generates a correct answer determination result.

[0125] The correct answer generation unit 170 according to this embodiment generates a correct answer determination result to be used by the second update unit 190. The correct answer generation unit 170 acquires an image set. The correct answer generation unit 170 can acquire the image set in a manner similar to that used by the generation unit 130 and the first update unit 150 to acquire the image set.

[0126] Regarding the functions of the correct answer generation unit 170, an example in which the classifier 250 has the functions described in the first embodiment will be described first. Specifically, an example in which the classifier 250 outputs the likelihood that the intermediate data set is an intermediate data set obtained from the second image set as the classification result for each intermediate data set will be described. However, the functions of the classifier 250 are not limited to this example.

[0127] The correct answer generation unit 170 generates a correct answer determination result for the acquired image set. For example, when the correct answer generation unit 170 acquires an image set from the first image set storage unit 301, the correct answer generation unit 170 identifies the minimum value of the likelihood as the correct answer determination result. That is, the correct answer generation unit 170 identifies the determination result indicating that the intermediate dataset is not an intermediate dataset obtained from the second image set as the correct answer determination result. On the other hand, when the correct answer generation unit 170 acquires an image set from the second image set storage unit 302, the correct answer generation unit 170 identifies the maximum value of the likelihood as the correct answer determination result. That is, the correct answer generation unit 170 identifies the determination result indicating that the intermediate dataset is an intermediate dataset obtained from the second image set as the correct answer determination result.

[0128] The correct answer generation unit 170 can have, for example, separate input units for acquiring an image set from the first image set storage unit 301 and acquiring an image set from the second image set storage unit 302. Then, the correct answer determination result can be determined depending on which input unit the image set was acquired from.

[0129] As another example, when an imaging angle is associated with each image, the correct answer generation unit 170 may identify the correct answer determination result based on the imaging angle. As described in the first embodiment, when a learned list is generated, the correct answer generation unit 170 can identify the correct answer determination result based on the learned list and the imaging angle associated with each image.

[0130] Next, an example will be described in which the discriminator 250 has the functions described in the second embodiment. Specifically, an example will be described in which the discriminator 250 outputs the likelihood that the intermediate data is intermediate data obtained for an image at a different angle as a discrimination result for each intermediate data. However, the functions of the discriminator 250 are not limited to this example.

[0131] The correct answer generation unit 170 generates a correct answer determination result for each of the multiple images included in the acquired image set. In this example, an imaging angle is associated with each image. The correct answer generation unit 170 identifies the correct answer determination result based on the imaging angle. As described in the first embodiment, when a learned list is generated, the correct answer generation unit 170 can identify the correct answer determination result based on the learned list and the imaging angle associated with each image.

[0132] The second updating unit 190 acquires the correct answer determination result from the correct answer generation unit 170. The second updating unit 190 may also acquire the image set via the correct answer generation unit 170. That is, the second updating unit 190 may acquire, from the correct answer generation unit 170, the image set for which the correct answer generation unit 170 has generated the correct answer determination result.

[0133] However, if the correct answer determination result is prepared in advance, the information processing device 10 does not need to include the correct answer generation unit 170. For example, the correct answer determination result may be associated with each image set in advance. Alternatively, the correct answer determination result may be associated with each image in advance. In this case, the second update unit 190 can acquire the correct answer determination result associated with each image.

[0134] The process performed by the information processing device 10 according to this embodiment will be described below. In the example of Fig. 18, step S20 is the same as step S10 according to at least one of the first embodiment and the first modification.

[0135] In this embodiment, step S21 is performed after step S20. In step S21, the second update unit 190 updates the discriminator 250. Step S21 is performed after step S22. Step S22 is the same as step S11 according to at least one of the first embodiment, modified example 1, and second embodiment. While FIG. 18 shows an example in which step S22 is performed after step S21, step S21 may also be performed after step S22. That is, the order in which steps S21 and S22 are performed is not particularly limited. Steps S21 and S22 may also be performed in parallel.

[0136] 19 is a flowchart illustrating the flow of processing executed by the second updating unit 190 according to this embodiment. FIG. 19 is also a flowchart specifically illustrating an example of processing in step S21.

[0137] The second updating unit 190 acquires an image set (step S401). The second updating unit 190 can acquire the first image set from, for example, the first image set storage unit 301. The second updating unit 190 may also acquire the second image set from the second image set storage unit 302. The second updating unit 190 may also acquire an image set from another device. As described above, the second updating unit 190 may acquire the image set via the correct answer generating unit 170.

[0138] Next, the second update unit 190 generates an intermediate data set by inputting the acquired image set to the intermediate data generator 210 (step S402). The second update unit 190 can read and use a plurality of parameters for configuring the intermediate data generator 210 from the detector storage unit 303.

[0139] However, instead of performing steps S401 and S402, the second updating unit 190 may acquire the intermediate data set generated by the first updating unit 150 in step S202.

[0140] In step S403, the second updating unit 190 performs discrimination by inputting the intermediate data set into the discriminator 250. A discrimination result is obtained as an output of the discriminator 250. However, instead of performing step S403, the second updating unit 190 may obtain the discrimination result generated by the first updating unit 150 in step S203.

[0141] In step S404, the second updating unit 190 calculates the loss Lc using the discrimination result and the correct answer discrimination result. The second updating unit 190 can calculate the loss Lc by comparing the discrimination result with the correct answer discrimination result. Specifically, the loss Lc can be the difference between the discrimination result and the correct answer discrimination result. As described in the second embodiment, when a discrimination result is output for each intermediate data, the loss Lc is calculated using the comparison result (for example, the difference) between each discrimination result and the correct answer discrimination result. The loss Lc is, for example, a weighted sum of the comparison result between each discrimination result and the correct answer discrimination result. The weight for each comparison result is predetermined. However, any existing method can be used to calculate the loss Lc.

[0142] The second updating unit 190 updates the discriminator 250 using the obtained loss Lc (step S405). That is, the second updating unit 190 updates the plurality of parameters stored in the discriminator storage unit 304. As a method by which the second updating unit 190 updates the discriminator 250, an existing method such as an error backpropagation method can be used.

[0143] In this way, the second update unit 190 updates the discriminator 250 so as to reduce the loss Lc. This improves the discrimination accuracy of the discriminator 250. That is, adversarial learning can be performed between the learning for the intermediate data generator 210 and the learning for the discriminator 250. As the discrimination accuracy of the discriminator 250 improves, the learning of the intermediate data generator 210 by the first update unit 150 also progresses further. Therefore, a detector 20 that can more stably perform difference detection for images at different angles is realized.

[0144] Furthermore, the second update unit 190 may repeatedly update the classifier 250 using multiple image sets in one step S21. The update of the classifier 250 is repeated until a termination condition is met, for example. The termination condition may be at least one of the number of updates in step S21 being equal to or greater than a predetermined number and the loss Lc being equal to or less than a predetermined standard.

[0145] Note that each update of the classifier 250 may be performed by batch processing using two or more image sets. That is, a single loss Lc may be calculated using a plurality of comparison results between the classification result and the correct classification result.

[0146] However, it is preferable that the number of updates of the discriminator 250 in step S21 is the same as the number of updates of the intermediate data generator 210 in step S22. For example, it is preferable that the first update unit 150 updates the intermediate data generator 210 every time the second update unit 190 updates the discriminator 250.

[0147] 18 , after steps S21 and S22 are completed, the first update unit 150 determines whether a termination condition is satisfied (step S23). The termination condition may be, for example, that step S22 is executed a predetermined number of times or more. If the termination condition is satisfied (Yes in step S23), the processing of the information processing device 10 is terminated.

[0148] If the termination condition is not satisfied (No in step S23), the generation unit 130 updates the detector 20 (step S24). The method by which the generation unit 130 updates the detector 20 in step S24 is the same as the method by which the generation unit 130 updates the detector 20 in step S20. However, in step S24, the generation unit 130 further updates the detector 20 including the intermediate data generator 210 updated by the first update unit 150.

[0149] Following step S24, the first updating unit 150 performs step S21 again.

[0150] The hardware configuration of a computer that realizes the information processing device 10 according to this embodiment is shown in Fig. 14, for example, similarly to the information processing device 10. However, the storage device 1080 of the computer 1000 that realizes the information processing device 10 according to this embodiment further stores program modules that realize the functions of the correct answer generation unit 170 and the second update unit 190 according to this embodiment.

[0151] Next, the operation and effect of this embodiment will be described. In this embodiment, the same operation and effect as in the first embodiment can be obtained. In addition, according to this embodiment, the information processing device 10 further includes a second update unit 190 that updates the discriminator 250 using an intermediate data set and a correct answer classification result for the intermediate data set. Therefore, a detector 20 that can more stably perform difference detection for images at different angles is realized.

[0152] 20 is a diagram for explaining the functions of an intermediate data generator 210 and a difference detector 230 according to a fourth embodiment. The information processing device 10 according to this embodiment is the same as the information processing device 10 according to at least one of the first to third embodiments and Modification 1, except for the points described below. The information processing method according to this embodiment is the same as the information processing method according to at least one of the first to third embodiments and Modification 1, except for the points described below.

[0153] The intermediate data generator 210 according to this embodiment accepts input of an image set as well as information indicating the imaging conditions for each of two or more images included in the image set. The generator 130 according to this embodiment further uses information indicating the imaging conditions for each of two or more images included in a first image set to generate a detector 20. Furthermore, the first update unit 150 according to this embodiment further inputs information indicating the imaging conditions for each of two or more images included in a second image set to the intermediate data generator, thereby generating an intermediate data set corresponding to the second image set.

[0154] The imaging conditions include, for example, one or more of the imaging angle, the direction of incidence of the sun at the time of imaging, the time of imaging, the position of the satellite that captured the image, and the direction of the antenna used for imaging. Among these, the imaging conditions preferably include the imaging angle. Furthermore, if the images included in the image set are optical images, the imaging conditions for the images preferably include the direction of incidence of the sun at the time of imaging.

[0155] Similar to the intermediate data generator 210 according to the first embodiment, the intermediate data generator 210 according to this embodiment outputs an intermediate data set consisting of intermediate data fa corresponding to image Xa and intermediate data fb corresponding to image Xb. According to the intermediate data generator 210 according to this embodiment, the intermediate data set is generated based not only on the image set but also on the imaging conditions. This enables more accurate feature extraction, improving the accuracy of difference detection by the detector 20.

[0156] 20 , image Xa, image Xb, imaging angle θa of image Xa, and imaging angle θb of image Xb are input to the intermediate data generator 210. Then, the intermediate data generator 210 outputs intermediate data fa for image Xa and intermediate data fb for image Xb.

[0157] Note that information indicating the imaging conditions of all images included in the image set does not necessarily need to be input to the intermediate data generator 210. Information indicating the imaging conditions of only some of the two or more images included in the image set may also be input to the intermediate data generator 210.

[0158] In this embodiment, each functional component of the information processing device 10 can further acquire information indicating the image capture conditions for the images in a manner similar to that for acquiring the image set. In this embodiment, for example, the first image set storage unit 301 stores, in addition to the first image set, information indicating the image capture conditions for each of the two or more images included in the first image set. In this embodiment, for example, the second image set storage unit 302 stores, in addition to the second image set, information indicating the image capture conditions for each of the two or more images included in the second image set.

[0159] Next, the operation and effect of this embodiment will be described. In this embodiment, the same operation and effect as in the first embodiment can be obtained. In addition, according to this embodiment, the generation unit 130 generates the detector 20 by further using information indicating the imaging conditions of each of the two or more images included in the first image set. Therefore, the difference detection accuracy of the detector 20 is improved.

[0160] Fifth Embodiment Fig. 21 is a diagram showing an overview of a detection device 50 according to the fifth embodiment. The detection device 50 according to this embodiment includes a detection unit 510. The detection unit 510 detects differences between two or more images using an intermediate data generator 210 and a difference detector 230. The intermediate data generator 210 has been updated by a first update unit 150 included in the information processing device 10 according to at least any of the first to fourth embodiments and Modification 1.

[0161] 22 is a diagram showing an overview of the detection method according to this embodiment. The detection method according to this embodiment is executed by one or more computers. In the detection method according to this embodiment, the one or more computers detect differences between two or more images using an intermediate data generator 210 and a difference detector 230 (step S50). The intermediate data generator 210 has been updated by the information processing method according to at least one of the first to fourth embodiments and modified example 1.

[0162] The detection device 50 is a device that detects differences between images using a trained detector 20 that has been trained in the information processing device 10. The detection unit 510 acquires an image set. The image set acquired by the detection unit 510 is called a target image set. The target image set is the same as the image set described in the first embodiment. The target image set may or may not include images captured at different angles. Furthermore, the target image set may or may not include images acquired at the same imaging angle as images included in the second image set. The target image set may include images acquired at a different imaging angle from any of the images included in the first image set and any of the images included in the second image set.

[0163] The detection unit 510 inputs the target image set to the detector 20. As described in the first embodiment and the like, the detector 20 includes the intermediate data generator 210 and the difference detector 230. The detection unit 510 can read and use the intermediate data generator 210 and the difference detector 230 stored in the detector storage unit 303, for example.

[0164] As described in Modification 1, when detectors 20 are provided for different combinations of image types included in an image set, the detection unit 510 may select a detector 20 to use based on the type of image included in the target image set. For example, each image may be associated with information indicating the type of image (SAR image, optical image, etc.). Alternatively, the user may input information indicating which detector 20 should be used to the detection device 50. The detection unit 510 can select a detector 20 based on the input information.

[0165] The detection unit 510 inputs the target image set to the intermediate data generator 210 to obtain an intermediate data set corresponding to the target image set. Next, the detection unit 510 inputs the intermediate data set to the difference detector 230. Note that the output data of the intermediate data generator 210 may be automatically input to the difference detector 230. The difference detector 230 outputs estimated difference data. The estimated difference data is data indicating the difference between two or more images included in the target image set.

[0166] As described in the fourth embodiment, if the input data to the intermediate data generator 210 further includes information indicating the imaging conditions, the detection unit 510 further acquires information indicating the imaging conditions of the images included in the target image set. Then, the detection unit 510 further inputs the information indicating the imaging conditions to the intermediate data generator 210, thereby obtaining an intermediate data set.

[0167] The detector 20 used by the detection unit 510 includes the intermediate data generator 210 updated by the first update unit 150 of the information processing device 10. Therefore, even if the image set acquired by the detection unit 510 includes images taken at different angles, differences can be detected with high accuracy.

[0168] The hardware configuration of the detection device 50 will be described below. Each functional component (detection unit 510) of the detection device 50 may be realized by hardware that realizes the functional component (e.g., a hardwired electronic circuit, etc.), or may be realized by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it, etc.). Below, a case where each functional component of the detection device 50 is realized by a combination of hardware and software will be further described.

[0169] The hardware configuration of the computer that realizes the detection device 50 is shown in Fig. 14, for example, similar to that of the information processing device 10. However, the storage device 1080 of the computer 1000 that realizes the detection device 50 of this embodiment further stores a program module that realizes the functions of the detection unit 510 of this embodiment.

[0170] As described above, according to this embodiment, the detection unit 510 detects differences between two or more images using the intermediate data generator 210 updated by the first update unit 150 and the difference detector 230. Therefore, difference detection is possible for various image sets.

[0171] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various other configurations can also be adopted.

[0172] In addition, although the flowcharts used in the above description describe multiple steps (processes) in a sequential order, the order of execution of the steps performed in each embodiment and modification is not limited to the order described. In each embodiment and modification, the order of the steps shown in the drawings can be changed to the extent that the content is not affected. Furthermore, the above-described embodiments and modifications can be combined to the extent that the content is not contradictory.

[0173] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes. 1-1. An information processing device comprising: a generation means that generates, using a first image set, a detector including an intermediate data generator that accepts input of an image set including two or more images and generates an intermediate data set including two or more intermediate data, and a difference detector that detects a difference between the two or more images using the intermediate data set; and a first update means that inputs a second image set to the intermediate data generator to generate the intermediate data set corresponding to the second image set and updates the intermediate data generator using the intermediate data set, wherein the second image set includes different-angle images obtained at a different imaging angle from any of the two or more images included in the first image set. 1-2. 1-1. 1-3. The information processing device according to 1-1., wherein the first updating means inputs the intermediate data set obtained using the first image set and the intermediate data set obtained using the second image set to a classifier that determines whether or not the intermediate data set is the intermediate data set obtained from the image set including the different angle images, and updates the intermediate data generator so that the classification result by the classifier becomes a predetermined result. 1-3. The information processing device according to 1-1., wherein the first updating means inputs the intermediate data set obtained using the first image set and the intermediate data set obtained using the second image set to a classifier that determines whether or not each of the two or more intermediate data included in the intermediate data set is the intermediate data obtained for the different angle images, and updates the intermediate data generator so that the classification result by the classifier becomes a predetermined result. 1-4. The information processing device according to 1-2. or 1-3., further comprising second updating means that updates the classifier using the intermediate data set and a correct classification result for the intermediate data set. 1-5. 1-1. 1-4. The information processing device according to any one of 1 to 4, wherein each of the two or more images is a satellite image.1-6. The information processing device described in 1-5., wherein each of the two or more images is at least one of a SAR image and an optical image, and the generation means generates the detector for each combination of image types included in the image set. 1-7. The information processing device described in any one of 1-1. to 1-6., wherein the generation means generates the detector by further using information indicating the imaging conditions of each of the two or more images included in the first image set. 1-8. The information processing device described in 1-7., wherein the imaging conditions include an imaging angle. 1-9. The information processing device described in any one of 1-1. to 1-8., wherein the detector detects a difference relating to a target object between the two or more images included in the input image set. 1-10. The information processing device described in any one of 1-1. to 1-9., wherein the two or more images included in the image set all include a target region in their imaging region. 2-1. 1-1. A detection device comprising a detection means for detecting differences between two or more images using the intermediate data generator updated by the first update means provided in the information processing device described in any one of items 1 to 10. and the difference detector. 3-1. An information processing method in which one or more computers use a first set of images to generate a detector including an intermediate data generator that receives an input of an image set including two or more images and generates an intermediate data set including two or more intermediate data, and a difference detector that detects differences between the two or more images using the intermediate data set, input a second set of images to the intermediate data generator to generate the intermediate data set corresponding to the second set of images, and update the intermediate data generator using the intermediate data set, wherein the second set of images includes different-angle images obtained at a different imaging angle from any of the two or more images included in the first image set.3-2. The information processing method described in 3-1., wherein the one or more computers input the intermediate data set obtained using the first image set and the intermediate data set obtained using the second image set to a classifier that determines whether the intermediate data set is the intermediate data set obtained from the image set including the different angle images, and update the intermediate data generator so that the classification result by the classifier becomes a predetermined result. 3-3. The information processing method described in 3-1., wherein the one or more computers input the intermediate data set obtained using the first image set and the intermediate data set obtained using the second image set to a classifier that determines whether each of the two or more intermediate data included in the intermediate data set is the intermediate data obtained for the different angle images, and update the intermediate data generator so that the classification result by the classifier becomes a predetermined result. 3-4. The information processing method described in 3-2. or 3-3., wherein the one or more computers further update the classifier using the intermediate data set and a correct classification result for the intermediate data set. 3-5. The information processing method described in any one of 3-1. to 3-4., wherein each of the two or more images is a satellite image. 3-6. The information processing method described in 3-5., wherein each of the two or more images is at least one of a SAR image and an optical image, and wherein the one or more computers generate the detector for each combination of image types included in the image set. 3-7. The information processing method described in any one of 3-1. to 3-6., wherein the one or more computers generate the detector by further using information indicating the imaging conditions for each of the two or more images included in the first image set. 3-8. The information processing method described in 3-7., wherein the imaging conditions include an imaging angle.3-9. An information processing method according to any one of 3-1. to 3-8., wherein the detector detects differences relating to a target object between the two or more images included in the input image set. 3-10. An information processing method according to any one of 3-1. to 3-9., wherein the two or more images included in the image set all include a target region in their imaging region. 4-1. A detection method in which one or more computers detect differences between two or more images using the intermediate data generator updated by the information processing method according to any one of 3-1. to 3-10. and the difference detector. 5-1. 5-2. A program causing a computer to function as: a generating means that generates, using a first image set, a detector including an intermediate data generator that accepts input of an image set including two or more images and generates an intermediate data set including two or more intermediate data, and a difference detector that detects a difference between the two or more images using the intermediate data set; and a first updating means that inputs a second image set to the intermediate data generator to generate the intermediate data set corresponding to the second image set and updates the intermediate data generator using the intermediate data set, wherein the second image set includes different-angle images obtained at an imaging angle different from any of the two or more images included in the first image set. 5-2. A program in accordance with 5-1., wherein the first updating means inputs the intermediate data set obtained using the first image set and the intermediate data set obtained using the second image set to a classifier that determines whether the intermediate data set is the intermediate data set obtained from the image set including the different-angle images, and updates the intermediate data generator so that the classification result by the classifier becomes a predetermined result.5-3. A program in accordance with 5-1., wherein the first updating means inputs the intermediate data set obtained using the first image set and the intermediate data set obtained using the second image set into a classifier that determines whether each of the two or more intermediate data included in the intermediate data set is the intermediate data obtained for the different angle images, and updates the intermediate data generator so that the classification result by the classifier becomes a predetermined result. 5-4. A program in accordance with 5-2. or 5-3., wherein the computer is further caused to function as second updating means that updates the classifier using the intermediate data set and a correct classification result for the intermediate data set. 5-5. A program in accordance with any one of 5-1. to 5-4., wherein each of the two or more images is a satellite image. 5-6. A program in accordance with 5-5., wherein each of the two or more images is at least one of a SAR image and an optical image, and the generating means generates the detector for each combination of image types included in the image set. 5-7. A program in accordance with 5-1. 5-10. A program according to any one of 5-1. to 5-9., wherein the two or more images included in the image set all include a target region in their imaging region. 6-1. A program that causes a computer to function as detection means that detects differences between two or more images using the intermediate data generator updated by the information processing method according to any one of 3-1. to 3-10., and the difference detector.7-1. A computer-readable recording medium having recorded thereon the program according to any one of 5-1 to 5-10. 8-1. A computer-readable recording medium having recorded thereon the program according to 6-1.

[0174] This application claims priority based on Japanese Patent Application No. 2022-212492, filed December 28, 2022, the disclosure of which is incorporated herein by reference in its entirety.

[0175] REFERENCE SIGNS LIST 10 Information processing device 20 Detector 50 Detection device 130 Generation unit 150 First update unit 170 Correct answer generation unit 190 Second update unit 210 Intermediate data generator 230 Difference detector 250 Classifier 301 First image set storage unit 302 Second image set storage unit 303 Detector storage unit 304 Classifier storage unit 510 Detection unit 1000 Computer

Claims

1. a generating means for generating, using a first image set, a detector including: an intermediate data generator that receives an input of an image set including two or more images and generates an intermediate data set including two or more intermediate data; and a difference detector that detects a difference between the two or more images using the intermediate data set; a first updating means for generating the intermediate data set corresponding to the second image set by inputting the second image set to the intermediate data generator, and updating the intermediate data generator using the intermediate data set; The second image set includes different-angle images obtained at imaging angles different from any of the two or more images included in the first image set. Information processing device.

2. 2. The information processing device according to claim 1, The first updating means inputting the intermediate data set obtained using the first image set and the intermediate data set obtained using the second image set into a classifier that determines whether the intermediate data set is the intermediate data set obtained from the image set including the different angle image; The intermediate data generator is updated so that the discrimination result by the discriminator becomes a predetermined result. Information processing device.

3. 2. The information processing device according to claim 1, The first updating means inputting the intermediate data set obtained using the first image set and the intermediate data set obtained using the second image set into a classifier that determines whether each of the two or more intermediate data included in the intermediate data set is the intermediate data obtained for the different angle images; The intermediate data generator is updated so that the discrimination result by the discriminator becomes a predetermined result. Information processing device.

4. 4. The information processing device according to claim 2, The present invention further includes a second update means for updating the classifier using the intermediate data set and a correct classification result for the intermediate data set. Information processing device.

5. 4. The information processing device according to claim 1, each of the two or more images is at least one of a SAR image and an optical image; The generating means generates the detector for each combination of types of images included in the image set. Information processing device.

6. 4. The information processing device according to claim 1, The generating means generates the detector by further using information indicating an imaging condition of each of the two or more images included in the first image set. Information processing device.

7. 7. The information processing device according to claim 6, The imaging conditions include an imaging angle. Information processing device.

8. 4. A detection device comprising: the intermediate data generator updated by the first update means provided in the information processing device according to claim 1; and detection means for detecting a difference between two or more images using the difference detector.

9. One or more computers generating, using a first set of images, a detector including: an intermediate data generator that receives an input of an image set including two or more images and generates an intermediate data set including two or more intermediate data; and a difference detector that uses the intermediate data set to detect a difference between the two or more images; inputting a second of the image sets into the intermediate data generator to generate the intermediate data set corresponding to the second image set, and updating the intermediate data generator using the intermediate data set; The second image set includes different-angle images obtained at imaging angles different from any of the two or more images included in the first image set. Information processing methods.

10. A computer, a generating means for generating, using a first image set, a detector including: an intermediate data generator that receives an input of an image set including two or more images and generates an intermediate data set including two or more intermediate data; and a difference detector that detects a difference between the two or more images using the intermediate data set; inputting a second set of images into the intermediate data generator to generate the intermediate data set corresponding to the second set of images, and causing the intermediate data generator to function as a first update unit that updates the intermediate data generator using the intermediate data set; The second image set includes different-angle images obtained at imaging angles different from any of the two or more images included in the first image set. program.