Information processing device, detection device, information processing method, and program

Abstract

In the present invention, a generation unit generates a detector using a first image set. The detector includes an intermediate data generator and a difference detector. The intermediate data generator has a function for receiving input of an image set that includes two or more images and generating an intermediate data set that includes two or more items of intermediate data. The difference detector has a function for detecting differences between two or more images by using an intermediate data set. A first update unit, by inputting a second image set to the intermediate data generator, generates an intermediate data set corresponding to the second image set. Then, the first update unit updates the intermediate data generator by using the generated intermediate data set. The second image set includes a differently angled image that is obtained at an imaging angle that differs from that of any of the two or more images included in the first image set.

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, a detection apparatus, an information processing method, and a program.

Background Art

[0002] Detecting differences between multiple images is useful. For example, by comparing multiple images taken at the same location and detecting differences, it is possible to grasp changes in the state of that location.

[0003] Non-Patent Document 1 discloses a fully convolutional neural network for performing change detection in an image pair.

[0004] Patent Document 1 discloses an apparatus that generates a change map or the like based on an image taken at a reference time, an image taken at an arbitrary time, and metadata of those images. Further, Patent Document 1 describes that the metadata includes, for example, data indicating the position of a satellite at the time of shooting or data indicating the direction of an antenna used for shooting.

[0005] Patent Document 2 discloses an image processing apparatus that extracts an image including an object presence region where an object exists from each of the observation images, and transforms the object presence region based on the observation angle of each image and the size of the object. Further, Patent Document 2 describes that changes in the object are determined using the transformed images.

[0006] Non-Patent Document 2 describes performing adversarial learning of an encoder and a discriminator for application to an unlabeled region.

[0007] Patent Document 3 discloses a learning apparatus that executes machine learning (semi-supervised learning) using labeled data and unlabeled data. Specifically, Patent Document 3 describes that the learning apparatus executes data conversion corresponding to domain adaptation on data with domain information and data without domain information, and executes machine learning such as class discrimination.

[0008] Patent Document 4 describes a method for training a domain adversarial neural network. Specifically, Patent Document 4 discloses a feature extraction unit that extracts a first feature from labeled source data and a second feature from unlabeled target data. Patent Document 4 also discloses a label prediction unit that predicts the label of the source data based on the first feature and predicts the label of the target data based on the second feature. Furthermore, Patent Document 4 describes how to train the label prediction unit and the feature extraction unit. [Prior art documents] [Patent Documents]

[0009] [Patent Document 1] International Publication No. 2020 / 070852 [Patent Document 2] International Publication No. 2022 / 018791 [Patent Document 3] International Publication No. 2019 / 102962 [Patent Document 4] Japanese Patent Publication No. 2021-144703 [Non-patent literature]

[0010] [Non-Patent Document 1] Rodrigo Caye Daudt and 2 others, "Fully Convolutional Siamese Networks for Change Detection", online, 2018, Computing Research Repository, arXiv:1810.08462v1 [Non-Patent Document 2] Eric Tzeng and 3 others, "Adversarial Discriminative Domain Adaptation", online, 2017, Computing Research Repository, arXiv:1702.05464v1 [Overview of the Initiative] [Problems that the invention aims to solve]

[0011] In the technologies disclosed in Non-Patent Document 1 and Patent Document 1 mentioned above, the accuracy of change detection depends on the training data used for learning, and therefore, it may not be possible to detect changes well in images obtained under conditions that have not been trained. The technologies disclosed in Non-Patent Document 2, Patent Document 2, and Patent Document 4 did not enable the detection of differences in multiple images. The technology disclosed in Patent Document 3 required prior knowledge of the size of the object.

[0012] One example of the object of the present invention is to provide an information processing device, a detection device, an information processing method, and a program that enable difference detection for various image sets, in view of the above-mentioned problems. [Means for solving the problem]

[0013] According to one aspect of the present invention, A generation means for generating a detector using the first image set, which includes an intermediate data generator that accepts an image set containing two or more images as input and generates an intermediate dataset containing two or more intermediate data, and a difference detector that detects the difference between the two or more images using the intermediate dataset, The system includes a first update means that inputs the second image set to the intermediate data generator to generate the intermediate data set corresponding to the second image set, and updates the intermediate data generator using the intermediate data set, The second set of images includes images taken at different angles than any of the two or more images included in the first set of images. An information processing device is provided.

[0014] According to one aspect of the present invention, A detection device is provided that includes a detection means for detecting a difference between two or more images using the intermediate data generator updated by the first update means provided in the information processing device and the difference detector.

[0015] According to one aspect of the present invention, one or more computers generate an intermediate data generator that receives an input of an image set including two or more images and generates an intermediate data set including two or more intermediate data, and a difference detector that detects a difference between the two or more images using the intermediate data set, and generates a detector including the difference detector using the first image set. By inputting the second image set into the intermediate data generator, an intermediate data set corresponding to the second image set is generated, and the intermediate data generator is updated using the intermediate data set. The second image set includes off-angle images obtained at an imaging angle different from any of the two or more images included in the first image set. An information processing method is provided.

[0016] According to one aspect of the present invention, a computer-readable recording medium recording a program, the program causes a computer to generate a detector including an intermediate data generator that receives an input of an image set including two or more images and generates an intermediate data set including two or more intermediate data, and a difference detector that detects a difference between the two or more images using the intermediate data set, using the first image set, and function as a first update means for generating an intermediate data set corresponding to the second image set by inputting the second image set into the intermediate data generator and updating the intermediate data generator using the intermediate data set. The second image set includes off-angle images obtained at an imaging angle different from any of the two or more images included in the first image set. A recording medium is provided.

Advantages of the Invention

[0017] According to one aspect of the present invention, it is possible to provide an information processing apparatus, a detection apparatus, an information processing method, and a program that enable difference detection for various image sets.

Brief Description of the Drawings

[0018] [Figure 1] It is a diagram showing an overview of an information processing apparatus according to the first embodiment. [Figure 2] It is a diagram showing an overview of an information processing method according to the first embodiment. [Figure 3] It is a diagram for explaining the relationship between a captured image and a difference map. [Figure 4] It is a diagram for explaining the imaging of satellite images. [Figure 5] It is a diagram illustrating the functional configuration of a detector according to the first embodiment. [Figure 6] It is a diagram for explaining the functions of an intermediate data generator and a difference detector according to the first embodiment. [Figure 7] It is a diagram illustrating the functional configuration of an information processing apparatus according to the first embodiment. [Figure 8] It is a diagram illustrating the configuration of the data held in the first image set storage unit. [Figure 9] It is a diagram illustrating the configuration of the data held in the second image set storage unit. [Figure 10] It is a flowchart illustrating the flow of processing performed by a generation unit according to the first embodiment. [Figure 11] It is a flowchart illustrating the flow of processing performed by a first update unit according to the first embodiment. [Figure 12] It is a diagram for explaining the function of a discriminator used by the first update unit. [Figure 13] It is a flowchart showing a modification of the flow of an information processing method according to the first embodiment. [Figure 14] It is a diagram illustrating a computer for realizing an information processing apparatus. [Figure 15] This is a diagram illustrating the function of the discriminant used in the information processing device according to the second embodiment. [Figure 16] This is a flowchart illustrating the processing flow performed by the first update unit of the information processing device according to the second embodiment. [Figure 17] This figure illustrates the functional configuration of an information processing device according to the third embodiment. [Figure 18] This is a flowchart illustrating the processing flow of the information processing method according to the third embodiment. [Figure 19] This is a flowchart illustrating the processing flow performed by the second update unit according to the third embodiment. [Figure 20] This figure illustrates the functions of the intermediate data generator and difference detector according to the fourth embodiment. [Figure 21] This figure shows an overview of the detection device according to the fifth embodiment. [Figure 22] This figure shows an overview of the detection method according to the fifth embodiment. [Modes for carrying out the invention]

[0019] Embodiments of the present invention will be described below with reference to the drawings. In all drawings, similar components are denoted by the same reference numerals, and their descriptions are omitted as appropriate.

[0020] (First embodiment) Figure 1 is a diagram showing an overview of an information processing device 10 according to the first embodiment. The information processing device 10 comprises a generation unit 130 and a first update unit 150. The generation unit 130 generates a detector using a first image set. The detector includes an intermediate data generator and a difference detector. The intermediate data generator has the function of receiving an image set containing two or more images as input and generating an intermediate dataset containing two or more intermediate data. The difference detector has the function of detecting differences between two or more images using the intermediate dataset. The first update unit 150 generates an intermediate dataset corresponding to the second image set by inputting the second image set to the intermediate data generator. Then, the first update unit 150 updates the intermediate data generator using the generated intermediate dataset. The second image set includes different-angle images obtained at imaging angles different from any of the two or more images included in the first image set.

[0021] This information processing device 10 provides a detector capable of detecting differences across various image sets.

[0022] Figure 2 is a diagram showing an overview of the information processing method according to this embodiment. The information processing method according to this embodiment is executed by one or more computers. The information processing method according to this embodiment includes steps S10 and S11. In step S10, one or more computers generate a detector using a first image set. The detector includes an intermediate data generator and a difference detector. The intermediate data generator has the function of receiving an image set containing two or more images as input and generating an intermediate dataset containing two or more intermediate data. The difference detector has the function of detecting differences between two or more images using the intermediate dataset. In step S11, one or more computers input a second image set to the intermediate data generator to generate an intermediate dataset corresponding to the second image set and update the intermediate data generator using the generated intermediate dataset. The second image set includes different-angle images obtained at imaging angles different from any of the two or more images included in the first image set.

[0023] This information processing method yields a detector capable of detecting differences across various image sets.

[0024] This information processing method can be performed by the information processing device 10 according to this embodiment.

[0025] Figure 3 illustrates the relationship between captured images and difference maps. As an example, we will discuss the case where two images are compared and a difference map is generated. The difference map can be used, for example, to detect changes in the parking state of cars in a parking lot. The two images were obtained by capturing the same area.

[0026] Here, as shown in the upper part of Figure 3, when two images are captured at the same imaging angle, the appearance and position of the target object will be relatively consistent between the two images. Therefore, a difference map can be obtained relatively easily by taking the difference between the images.

[0027] On the other hand, as shown in the lower part of Figure 3, when two images are taken at different imaging angles, the appearance and position of the same object will differ between images. Therefore, it is difficult to obtain a difference map or to associate the object between images.

[0028] Figure 4 is a diagram illustrating satellite image acquisition. When obtaining multiple satellite images of a predetermined target area (for example, a parking lot), imaging can be performed on multiple different orbits (orbit a, orbit b, and orbit c). Furthermore, the imaging angle of the image depends on the positional relationship between the satellite and the target area at the time of imaging. Therefore, for example, if the satellite orbits at the time of imaging are different from each other, the imaging angles of the multiple images obtained (imaging angle A, imaging angle B, and imaging angle C) will be different from each other.

[0029] As mentioned above, comparing images captured at different imaging angles is difficult. However, if the multiple images used to generate the difference map are limited to those taken at the same imaging angle, the number of usable images becomes extremely small. As a result, it may not be possible to perform change detection at the required frequency. Therefore, it is important to be able to detect differences between multiple images taken at different imaging angles.

[0030] Furthermore, a pre-trained model that has undergone machine learning can be used for difference detection. For example, a model that takes a pair of images as input and outputs a change map can be considered. In the machine learning of this model, for example, a pair of images obtained at different imaging angles and their ground truth change maps can be used as training data. In this way, the imaging angles of the images used in machine learning can be treated as trained imaging angles and the model can handle them well. However, this model cannot reliably detect changes in images obtained at imaging angles other than those that have been trained, i.e., untrained imaging angles. To handle a wide range of imaging angles, it becomes necessary to prepare a very large amount of training data. In particular, preparing a very large number of ground truth change maps requires time and effort.

[0031] According to the information processing device 10 of this embodiment, a detector can be realized that can perform difference detection well regardless of whether the imaging angle of the input image is a learned imaging angle or an unlearned imaging angle.

[0032] The following describes detailed examples of the information processing device 10 and information processing method according to this embodiment.

[0033] Figure 5 is a diagram illustrating the functional configuration of the detector 20 according to this embodiment. Figure 6 is a diagram illustrating the functions of the intermediate data generator 210 and the difference detector 230 according to this embodiment. The detector 20 is composed of the intermediate data generator 210 and the difference detector 230. The intermediate data generator 210 and the difference detector 230 each include a neural network.

[0034] When an image set containing two or more images is input to the detector 20, that image set is input to the intermediate data generator 210. Although Figure 6 shows the case where the image set contains only two images, the number of images in the image set is not limited to two. The image set may contain three or more images.

[0035] The output data of the intermediate data generator 210 includes an intermediate dataset. The intermediate dataset contains intermediate data corresponding to each image in the input image set. In the example in Figure 6, the image set input to the intermediate data generator 210 consists of image Xa and image Xb. The intermediate data generator 210 then outputs an intermediate dataset consisting of intermediate data fa corresponding to image Xa and intermediate data fb corresponding to image Xb.

[0036] The input data for the difference detector 230 includes an intermediate dataset. The difference detector 230 outputs estimated difference data. The estimated difference data is data showing the difference between image Xa and image Xb.

[0037] The estimated difference data output by the difference detector 230 is the difference detection result by the detector 20. The detector 20 can, for example, detect differences in a target object between two or more images included in the input image set. Each image may contain one or more target objects. The target object is not particularly limited, but may be a moving object such as a vehicle, aircraft, ship, living organism, or article, or a non-moving object such as a building, or terrain, etc. Examples of aircraft include aircraft. Examples of ships include vessels. Examples of articles include cargo and containers.

[0038] Each of the two or more images constituting an image set is, for example, a satellite image. Furthermore, each of the two or more images is, for example, at least one of a SAR (Synthetic Aperture Radar) image and an optical image. An image set may consist only of images of the same type, or it may include images of different types. For example, in an image set consisting of image Xa and image Xb, image Xa may be a SAR image and image Xb may be an optical image. However, multiple image sets used to generate and update a single detector 20 are all the same combination of image sets. That is, a detector 20 for application to an image set consisting of a SAR image and an optical image is generated and updated using only an image set consisting of a SAR image and an optical image.

[0039] The entire region contained within an image is called the "imaging region" of that image. In other words, an image is obtained by imaging the imaging region of that image. The object to be imaged is not particularly limited, but it is preferable that two or more images included in an image set all include the target region in their imaging regions. This allows the detector 20 to detect changes in the target region over time. For example, the region included in both the imaging region of image Xa and the imaging region of image Xb can be considered the target region.

[0040] The target area is not particularly limited, but it may include, for example, a parking lot. In that case, the detector 20 can be used to detect changes in the parking status of vehicles in the parking lot. As another example, the target area may include an airport. In that case, the detector 20 can be used to detect changes in the parking status of aircraft at the airport. The target area may include, for example, a port. In that case, the detector 20 can be used to detect changes in the berthing status of ships at the port. The target area may include, for example, a container yard. In that case, the detector 20 can be used to detect changes in the arrangement of containers at the container yard.

[0041] Intermediate data, for example, provides information about the target object in an image. Specifically, intermediate data may indicate the location of the target object within the image. Intermediate data can be a vector representing features. The intermediate data generator may also be a feature extractor. Two or more images are input to the intermediate data generator 210 simultaneously as an image set. In this way, the intermediate data of one image may be influenced by other images included in the same image set. Furthermore, an intermediate dataset suitable for detecting differences between two or more images included in the image set is generated.

[0042] The estimated difference data output by the difference detector 230 may be, for example, a difference map showing the difference between two images. For example, if there is a target object at a certain position in one image, and there is no target object at that position in the other image, then it can be said that there is a difference at that position. The difference map may be a map that shows the presence or absence of a difference at each position in the target region using brightness.

[0043] If the image set contains three or more images, the difference detector 230 may output multiple estimated difference data. For example, the difference detector 230 may output estimated difference data for multiple image pairs that can be composed of three or more images included in the image set. Alternatively, one of the three or more images included in the image set may be designated as a reference image. In that case, the difference detector 230 may output estimated difference data showing the difference between the reference image and each of the other images.

[0044] Figure 7 is a diagram illustrating the functional configuration of the information processing device 10 according to this embodiment. The information processing device 10 is a device that performs machine learning on the detector 20. In the example in Figure 7, the information processing device 10 further comprises a first image set storage unit 301, a second image set storage unit 302, a detector storage unit 303, and a discriminator storage unit 304. However, one or more of the first image set storage unit 301, the second image set storage unit 302, the detector storage unit 303, and the discriminator storage unit 304 may be provided outside the information processing device 10.

[0045] The first image set storage unit 301 holds the first image set. The second image set storage unit 302 holds the second image set.

[0046] Figure 8 illustrates the structure of data stored in the first image set storage unit 301. Figure 9 illustrates the structure of data stored in the second image set storage unit 302.

[0047] Of the image sets used for generating and updating the detector 20, the image set used in the generation unit 130, that is, the image set used in step S10, is called the first image set.

[0048] The image set that includes images taken at different angles is called the second image set. Images taken at different angles are those obtained at a different imaging angle than any of the images included in the first image set. The term "image set" simply refers to both the first and second image sets.

[0049] The imaging angle indicates the orientation from which the image was captured. The imaging angle may be a combination of multiple angles. The imaging angle is the angle of the line passing through the satellite capturing the image and the center of the image area. The imaging angle may be a combination of the azimuth angle and the angle of incidence. The azimuth angle is the angle relative to the satellite's range direction, and the angle of incidence is the angle relative to the zenith direction. If the imaging angle is a combination of multiple angles, the imaging angles are considered different if at least one of the angles is different. In other words, if the imaging angles are the same, it means that all of the angles are the same.

[0050] In step S11, the first update unit 150 updates the intermediate data generator 210 using at least the second image set. The generation unit 130 does not use the second image set to update the detector 20.

[0051] As will be described in detail later, the generation unit 130 updates the intermediate data generator 210 and the difference detector 230 using the first image set and the correct difference data corresponding to the first image set. On the other hand, the correct difference data is not used when the first update unit 150, which will be described later, updates the intermediate data generator 210 using the second image set.

[0052] In the example shown in Figure 8, the first image set storage unit 301 holds multiple first image sets. Each image is associated with an imaging angle. The imaging angle may also be included in the metadata of each image.

[0053] Furthermore, each first image set is associated with ground truth difference data. This ground truth difference data is used as ground truth data in the machine learning performed in step S10. The ground truth difference data has the same structure as the estimated difference data.

[0054] In step S10, the detector 20 is trained using multiple sets of first images. Among the multiple images used for training in step S10, imaging angles that match the imaging angle of at least one of the images are called "trained imaging angles." Imaging angles other than trained imaging angles are called "untrained imaging angles."

[0055] In the example shown in Figure 9, the second image set storage unit 302 holds multiple second image sets. Each image is associated with an imaging angle. The imaging angle may also be included in the metadata of each image. Ground truth difference data does not need to be prepared for the second image sets.

[0056] For ease of understanding, the learned imaging angles shown in Figure 9 are underlined. Each second image set includes at least one image obtained at an unlearned imaging angle. That is, the second image set may be a combination of images obtained at unlearned imaging angles, or a combination of an image obtained at an unlearned imaging angle and an image obtained at a learned imaging angle. Images obtained at unlearned imaging angles are also called "different angle images."

[0057] Furthermore, as long as each second image set includes images taken at different angles, the first image set storage unit 301 and the second image set storage unit 302 do not need to contain information indicating the imaging angle of each image.

[0058] Returning to Figure 7, the processing performed by each functional component of the information processing device 10 will be explained. In step S10, the generation unit 130 generates the detector 20. Generating the detector 20 includes updating the detector 20 using machine learning.

[0059] The detector storage unit 303 holds the detector 20. Specifically, the detector storage unit 303 holds the intermediate data generator 210 and the difference detector 230. More precisely, the detector storage unit 303 holds multiple parameters for configuring the intermediate data generator 210 and multiple parameters for configuring the difference detector 230. The generation unit 130 and the first update unit 150 can each read and use multiple parameters from the detector storage unit 303.

[0060] Figure 10 is a flowchart illustrating the processing flow performed by the generation unit 130 according to this embodiment. Figure 10 is also a flowchart specifically showing an example of the processing in step S10.

[0061] In step S101, the generation unit 130 acquires a first image set. The generation unit 130 can acquire a first image set by reading it from the first image set storage unit 301. The generation unit 130 may acquire multiple first image sets at once or sequentially. The generation unit 130 may acquire a predetermined number of first image sets. Instead of reading the first image sets from the first image set storage unit 301, the generation unit 130 may acquire the first image sets from another device. The generation unit 130 also acquires the correct difference data associated with each first image set. The generation unit 130 can acquire the correct difference data in the same manner as acquiring the first image sets.

[0062] The generation unit 130 inputs the first image set to the intermediate data generator 210 of the detector 20. The intermediate data generator 210 outputs an intermediate data set corresponding to the input first image set. In this way, the generation unit 130 generates an intermediate data set (step S102).

[0063] Next, the generation unit 130 inputs the generated intermediate dataset to the difference detector 230. The difference detector 230 outputs estimated difference data corresponding to the input intermediate dataset. That is, the difference detector 230 outputs estimated difference data corresponding to the first image set input to the detector 20 (step S103). By generating estimated difference data in this way, the generation unit 130 can estimate the difference between images included in the first image set.

[0064] In step S10, the output data from the intermediate data generator 210 may be automatically input to the difference detector 230.

[0065] Next, the generation unit 130 calculates the loss La using the estimated difference data and the ground truth difference data (step S104). The loss La can be calculated using an existing method.

[0066] The generation unit 130 then updates the intermediate data generator 210 and the difference detector 230, respectively, using the calculated loss La (step S105). Specifically, the generation unit 130 updates the intermediate data generator 210 using the loss La. The generation unit 130 updates several parameters for the intermediate data generator 210 held in the detector storage unit 303. As a method for the generation unit 130 to update the intermediate data generator 210, existing methods such as backpropagation can be used. The generation unit 130 also updates the difference detector 230 using the loss La. The generation unit 130 updates several parameters for the difference detector 230 held in the detector storage unit 303. As a method for the generation unit 130 to update the difference detector 230, existing methods such as backpropagation can be used.

[0067] In this way, the generation unit 130 updates the intermediate data generator 210 and the difference detector 230, respectively, to reduce the loss La. That is, the generation unit 130 updates the detector 20 to reduce the loss La. By doing so, the generation unit 130 can update the detector 20 so that the estimated difference data output from the detector 20 approaches the ground truth difference data.

[0068] Furthermore, the generation unit 130 may repeatedly update the detector 20 using multiple first image sets in a single step S10. The updating of the detector 20 is repeated, for example, until a termination condition is met. The termination condition is, for example, at least one of the following: the number of updates in step S10 is greater than or equal to a predetermined number, and the loss La is less than or equal to a predetermined standard.

[0069] Each update of the detector 20 may be performed by batch processing using two or more first image sets. That is, a single loss La may be calculated using multiple comparison results between estimated difference data and ground truth difference data.

[0070] According to step S10, the detector 20 is trained so that it can perform difference detection effectively on the image set obtained at the learned imaging angles.

[0071] Figure 11 is a flowchart illustrating the processing flow performed by the first update unit 150 according to this embodiment. Figure 11 is also a flowchart specifically illustrating the processing of step S11. Figure 12 is a diagram for explaining the function of the discriminator 250 used by the first update unit 150.

[0072] The first update unit 150 inputs the intermediate dataset obtained using the first image set and the intermediate dataset obtained using the second image set to the discriminator 250. The discriminator 250 has the function of determining whether or not the intermediate dataset is an intermediate dataset obtained from an image set that includes images at different angles (i.e., the second image set). The first update unit 150 then updates the intermediate data generator 210 so that the discrimination result by the discriminator 250 becomes a predetermined result. The function of the discriminator 250 and the processing performed by the first update unit 150 are described in detail below.

[0073] The input data for the classifier 250 includes an intermediate dataset. The classifier 250 outputs a classification result Vp based on the input intermediate dataset. The classification result Vp is, for example, the likelihood that the intermediate dataset input to the classifier 250 is an intermediate dataset obtained from a second image set. Alternatively, the classification result Vp may be the likelihood that the intermediate dataset input to the classifier 250 is not an intermediate dataset obtained from a second image set. In this embodiment, the classifier 250 outputs one classification result Vp for each intermediate dataset.

[0074] The discriminator 250 includes a neural network. In this embodiment, the discriminator 250 includes a neural network that has been pre-machine-trained. The discriminator memory unit 304 holds multiple parameters for configuring the discriminator 250. The first update unit 150 can read and use multiple parameters from the discriminator memory unit 304.

[0075] In step S201, the first update unit 150 acquires an image set. For example, the first update unit 150 can acquire a second image set by reading it from the second image set storage unit 302. The first update unit 150 may acquire multiple second image sets at once or sequentially. The first update unit 150 may acquire a predetermined number of second image sets. Instead of reading the second image set from the second image set storage unit 302, the first update unit 150 may acquire the second image set from another device.

[0076] The first update unit 150 reads and uses multiple parameters from the detector storage unit 303 to configure the intermediate data generator 210. These multiple parameters for configuring the intermediate data generator 210 are the parameters updated by the generation unit 130 in step S10.

[0077] The first update unit 150 inputs the acquired image set to the intermediate data generator 210. The intermediate data generator 210 outputs an intermediate dataset corresponding to the input image set. In this way, the first update unit 150 generates an intermediate dataset (step S202).

[0078] Next, the first update unit 150 inputs the generated intermediate dataset to the discriminator 250 to perform discrimination (step S203). The discriminator 250 outputs a discrimination result Vp corresponding to the input intermediate dataset.

[0079] In step S11, the output data from the intermediate data generator 210 may be automatically input to the discriminator 250.

[0080] Next, the first update unit 150 calculates the loss Lb using the discrimination result Vp and the dummy target Vd (step S204).

[0081] The dummy target Vd is information that indicates a predetermined judgment result. For example, if the judgment result Vp output from the classifier 250 has a likelihood within the range of 0 to 1, the dummy target Vd is a predetermined value between 0 and 1. The dummy target Vd is not particularly limited, but for example it could be 0, 0.5, or 1.

[0082] The loss Lb is obtained, for example, by comparing the discrimination result Vp with the dummy target Vd. Specifically, the loss Lb can be the difference between the discrimination result Vp and the dummy target Vd. However, any existing method can be used to calculate the loss Lb.

[0083] The first update unit 150 then updates the intermediate data generator 210 using the calculated loss Lb (step S205). Specifically, the first update unit 150 updates several parameters for the intermediate data generator 210 held in the detector storage unit 303. The first update unit 150 can use existing methods such as backpropagation to update the intermediate data generator 210. In this way, the first update unit 150 updates the intermediate data generator 210 to reduce the loss Lb.

[0084] The above describes an example in which the first update unit 150 updates the intermediate data generator 210 using the second image set. However, the first update unit 150 may also update the intermediate data generator 210 using the first image set.

[0085] When the first update unit 150 updates the intermediate data generator 210 using the first image set, the first update unit 150 acquires the first image set in step S201 instead of acquiring the second image set. The first update unit 150 can acquire the first image set by reading it from the first image set storage unit 301. The first update unit 150 may acquire multiple first image sets at once or sequentially. The first update unit 150 may acquire a predetermined number of first image sets. The first update unit 150 may acquire the first image set from another device instead of reading it from the first image set storage unit 301.

[0086] The first image set acquired by the first update unit 150 may be the same as or different from the first image set acquired by the generation unit 130. That is, if the first update unit 150 acquires multiple first image sets, at least a portion of those first image sets may be the same as the first image set acquired by the generation unit 130. However, the first image set does not include images taken at different angles.

[0087] Even if the first update unit 150 acquires the first image set in step S201, the processing in steps S202 to S205 is the same as when the second image set is acquired. The first update unit 150 performs updates using at least the second image set. Preferably, the first update unit 150 performs updates using both the first image set and the second image set.

[0088] The dummy target Vd used is the same whether the update is performed using the first image set or the second image set. In other words, according to step S11, the intermediate data generator 210 is trained to generate an intermediate dataset that makes discrimination by the discriminator 250 more difficult. Based on the intermediate dataset obtained using the intermediate data generator 210, the difference detector 230 can accurately detect differences regardless of whether the intermediate dataset is obtained from the first image set or the second image set. In other words, a detector 20 is obtained that can accurately detect differences even for images taken at different angles.

[0089] The first update unit 150 may use multiple image sets in a single step S11 and repeatedly update the discriminator 250. The updating of the discriminator 250 is repeated, for example, until a termination condition is met. The termination condition is, for example, at least one of the following: the number of updates in step S11 is greater than or equal to a predetermined number, and the loss Lb is less than or equal to a predetermined standard.

[0090] Furthermore, each update of the discriminator 250 may be performed by batch processing using two or more image sets. That is, a single loss Lb may be calculated using multiple comparison results between the discriminant result Vp and the dummy target Vd.

[0091] Figure 13 is a flowchart showing a modified example of the information processing method according to this embodiment. In the example in Figure 2, the generation unit 130 does not update the intermediate data generator 210 and the difference detector 230 after the first update unit 150 updates the intermediate data generator 210. On the other hand, in the example in Figure 13, the generation unit 130 updates the intermediate data generator 210 again after the first update unit 150 updates the intermediate data generator 210.

[0092] In the example shown in Figure 13, the first update unit 150 performs step S11 multiple times. Each time step S11 is performed, the generation unit 130 updates the detector 20. Specifically, after the processing of step S11 is completed, the first update unit 150 determines whether or not a termination condition is met (step S12). The termination condition is, for example, that the number of times step S11 is executed is greater than or equal to a predetermined number. If the termination condition is met (Yes in step S12), the processing of the information processing device 10 is terminated.

[0093] If the termination condition is not met (No. in step S12), the generation unit 130 updates the detector 20 (step S13). The method by which the generation unit 130 updates the detector 20 in step S13 is the same as the method by which the generation unit 130 updates the detector 20 in step S10. However, in step S13, the generation unit 130 further updates the detector 20, including the intermediate data generator 210 that was updated in the first update unit 150.

[0094] Following step S13, the first update unit 150 performs step S11 again.

[0095] As mentioned above, the intermediate data generator 210 may be updated multiple times in step S11. Also, similar to step S10, the detector 20 may be updated multiple times in step S13.

[0096] Figure 7 shows an example where the first image set storage unit 301 and the second image set storage unit 302 are provided separately. However, the first image set and the second image set may be stored in a single storage unit. In that case, the first image set and the second image set are stored in the storage unit in a manner that allows them to be identified from each other. For example, classification information indicating whether each image set is the first image set or the second image set may be associated with it. The generation unit 130 can select and acquire the first image set based on the classification information.

[0097] As another example, the images included in the image set may be associated with an imaging angle. In that case, after reading the image set from the storage unit, the generation unit 130 and the first update unit 150 may determine, based on the imaging angle, whether the image set corresponds to a first image set or a second image set.

[0098] Specifically, the generation unit 130 may first select a plurality of image sets to be used in step S10 as the first image set, and identify the imaging angles of the images included in those image sets as learned imaging angles. The generation unit 130 may then register the identified learned imaging angles in a learned list for defining imaging angles. Subsequently, the generation unit 130 or the first update unit 150 can use the learned list to identify the type of image set for the image set it reads from the storage unit.

[0099] The hardware configuration of the information processing device 10 is described below. Each functional component of the information processing device 10 (generation unit 130 and first update unit 150) may be implemented by hardware that realizes each functional component (e.g., hardwired electronic circuits), or by a combination of hardware and software (e.g., a combination of electronic circuits and programs that control them). The case in which each functional component of the information processing device 10 is implemented by a combination of hardware and software will be described further below.

[0100] Figure 14 illustrates a computer 1000 for implementing the information processing device 10. Computer 1000 is any computer. For example, computer 1000 could be a System on Chip (SoC), Personal Computer (PC), server machine, tablet terminal, or smartphone. Computer 1000 may be a dedicated computer designed to implement the information processing device 10, or it may be a general-purpose computer. Furthermore, the information processing device 10 may be implemented by a single computer 1000, or by a combination of multiple computers 1000.

[0101] Computer 1000 includes a bus 1020, a processor 1040, memory 1060, a storage device 1080, an input / output interface 1100, and a network interface 1120. Bus 1020 is a data transmission path for the processor 1040, memory 1060, storage device 1080, input / output interface 1100, and network interface 1120 to send and receive data to and from each other. However, the method of connecting the processor 1040 and the other components is not limited to bus connection. The processor 1040 is a variety of processor such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), or FPGA (Field-Programmable Gate Array). Memory 1060 is a main memory device implemented using RAM (Random Access Memory), etc. Storage device 1080 is an auxiliary storage device implemented using a hard disk, SSD (Solid State Drive), memory card, or ROM (Read Only Memory), etc.

[0102] The input / output interface 1100 is an interface for connecting the computer 1000 with input / output devices. For example, input devices such as a keyboard and output devices such as a display are connected to the input / output interface 1100. The method by which the input / output interface 1100 connects to the input and output devices may be wireless or wired.

[0103] The network interface 1120 is an interface for connecting the computer 1000 to a network. This network is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The method by which the network interface 1120 connects to the network may be wireless or wired.

[0104] The storage device 1080 stores program modules that realize each functional component of the information processing device 10. The processor 1040 reads these program modules into the memory 1060 and executes them to realize the functions corresponding to each program module.

[0105] Furthermore, if the first image set storage unit 301 is provided inside the information processing device 10, for example, the first image set storage unit 301 is implemented using a storage device 1080. Also, if the second image set storage unit 302 is provided inside the information processing device 10, for example, the second image set storage unit 302 is implemented using a storage device 1080. Also, if the detector storage unit 303 is provided inside the information processing device 10, for example, the detector storage unit 303 is implemented using a storage device 1080. Also, if the discriminator storage unit 304 is provided inside the information processing device 10, for example, the discriminator storage unit 304 is implemented using a storage device 1080.

[0106] As described above, according to this embodiment, the first update unit 150 generates an intermediate data set corresponding to the second image set by inputting the second image set to the intermediate data generator. Then, the first update unit 150 updates the intermediate data generator using the generated intermediate data set. Therefore, a detector 20 capable of difference detection for various image sets is obtained.

[0107] (Variation 1) The information processing device 10 according to Modification 1 will be described below. Modification 1 is a modification of the first embodiment. The information processing device 10 and information processing method according to Modification 1 are the same as those of the information processing device 10 and information processing method according to the first embodiment, except for the points described below.

[0108] In the modified information processing device 10, the generation unit 130 generates detectors 20 according to the combination of image types included in the image set. The first update unit 150 updates the intermediate data generator 210 of each detector 20. In this way, a group of detectors 20 that can handle various combinations of image types is obtained.

[0109] As described in the first embodiment, each of the two or more images included in the image set is, for example, at least one of a SAR image and an optical image.

[0110] In this modified example, for instance, a detector 20 for an image set consisting of SAR images, a detector 20 for an image set consisting of optical images, and a detector 20 for an image set consisting of SAR images and optical images are generated, respectively.

[0111] The first and second image sets used to generate and update the detector 20 for detecting differences in an image set consisting of SAR images are both image sets consisting of SAR images. The first and second image sets used to generate and update the detector 20 for detecting differences in an image set consisting of optical images are both image sets consisting of optical images. The first and second image sets used to generate and update the detector 20 for detecting differences in an image set consisting of SAR images and optical images are both image sets consisting of SAR images and optical images.

[0112] The detector storage unit 303 in this modified example stores multiple detectors 20. A user who wants to detect the difference between two images can select and use a detector 20 corresponding to the type of image from the group of detectors 20.

[0113] Next, the operation and effects of this embodiment will be described. In this embodiment, the same operation and effects as in the first embodiment can be obtained. In addition, according to the information processing device 10 of this modified example, the generation unit 130 generates detectors 20 according to the combination of image types included in the image set. Therefore, a group of detectors 20 that can correspond to various combinations of image types can be obtained.

[0114] (Second embodiment) Figure 15 is a diagram illustrating the function of the discriminator 250 used in the information processing device 10 according to the second embodiment. The information processing device 10 according to this embodiment is the same as the information processing device 10 according to the first embodiment and at least one of the modified example 1, except for the points described below. The information processing method according to this embodiment is the same as the information processing method according to the first embodiment and at least one of the modified example 1, except for the points described below.

[0115] In the information processing device 10 according to this embodiment, the first update unit 150 inputs the intermediate data set obtained using the first image set and the intermediate data set obtained using the second image set to the discriminator 250. The discriminator 250 has the function of determining whether each of the two or more intermediate data included in the intermediate data set is intermediate data obtained for images at different angles. The first update unit 150 then updates the intermediate data generator 210 so that the determination result by the discriminator 250 becomes a predetermined result.

[0116] In this embodiment as well, the input data of the discriminator 250 includes an intermediate dataset. The intermediate dataset consists of, for example, intermediate data fa and intermediate data fb. The discriminator 250 outputs a discrimination result (Vpa, Vpb) based on the input intermediate dataset. The discrimination result Vpa is, for example, the likelihood that intermediate data fa is intermediate data obtained for images at different angles, or the likelihood that intermediate data fa is not intermediate data obtained for images at different angles. The discrimination result Vpb is, for example, the likelihood that intermediate data fb is intermediate data obtained for images at different angles, or the likelihood that intermediate data fb is not intermediate data obtained for images at different angles. In other words, the discriminator 250 according to this embodiment outputs a discrimination result for each intermediate data included in the intermediate dataset. The output of the discriminator 250 includes discrimination results for multiple intermediate data.

[0117] Figure 16 is a flowchart illustrating the processing flow performed by the first update unit 150 of the information processing device 10 according to this embodiment. Figure 16 is also a flowchart specifically illustrating the processing of step S11 in the information processing method according to this embodiment.

[0118] Steps S301 to S303 are the same as steps S201 to S203, respectively, as described with reference to Figure 11.

[0119] In step S304, the first update unit 150 calculates the loss Lb using the comparison result between the discrimination result for each intermediate data and the dummy target Vd. For example, the first update unit 150 first calculates the difference between the discrimination result Vpa and the dummy target Vd as comparison result C1. The first update unit 150 also calculates the difference between the discrimination result Vpb and the dummy target Vd as comparison result C2. Then, the first update unit 150 calculates the loss Lb using comparison result C1 and comparison result C2. The loss Lb can be, for example, a weighted sum of comparison result C1 and comparison result C2. The weights for comparison result C1 and comparison result C2, respectively, for calculating the loss Lb are predetermined. However, any existing method can be used to calculate the loss Lb.

[0120] The first update unit 150 then updates the intermediate data generator 210 using the calculated loss Lb (step S305). Specifically, the first update unit 150 updates several parameters for the intermediate data generator 210 held in the detector storage unit 303. The first update unit 150 can use existing methods such as backpropagation to update the intermediate data generator 210. In this way, the first update unit 150 updates the intermediate data generator 210 to reduce the loss Lb.

[0121] Next, the operation and effects of this embodiment will be described. In this embodiment, the same operation and effects as in the first embodiment can be obtained.

[0122] (Third embodiment) Figure 17 is a diagram illustrating the functional configuration of the information processing device 10 according to the third embodiment. Figure 18 is a flowchart illustrating the processing flow of the information processing method according to this embodiment. The information processing device 10 according to this embodiment is the same as the information processing device 10 according to at least one of the first embodiment, modified example 1, and the second embodiment, except for the points described below. The information processing method according to this embodiment is the same as the information processing method according to at least one of the first embodiment, modified example 1, and the second embodiment, except for the points described below.

[0123] The information processing device 10 according to this embodiment further includes a second update unit 190 that updates the discriminator 250 using an intermediate dataset and the correct answer discrimination result for the intermediate dataset. In this embodiment, the discriminator 250 may or may not have undergone machine learning in advance, or it may have undergone machine learning to some extent in advance.

[0124] Furthermore, in the example shown in Figure 17, the information processing device 10 further includes a correct answer generation unit 170 that generates a correct answer discrimination result.

[0125] The correct answer generation unit 170 according to this embodiment generates a correct answer determination result to be used by the second update unit 190. The correct answer generation unit 170 acquires an image set. The correct answer generation unit 170 can acquire an image set in the same manner as the generation unit 130 and the first update unit 150 acquire an image set.

[0126] First, regarding the functions of the correct answer generation unit 170, an example will be given in which the discriminator 250 has the functions described in the first embodiment. Specifically, an example will be given in which the discriminator 250 outputs the likelihood that the intermediate dataset is an intermediate dataset obtained from the second image set as the discriminator result for each intermediate dataset. However, the functions of the discriminator 250 are not limited to this example.

[0127] The correct answer generation unit 170 generates a correct answer determination result for the acquired image set. For example, if the correct answer generation unit 170 acquires an image set from the first image set storage unit 301, the correct answer generation unit 170 identifies the minimum likelihood as the correct answer determination result. That is, the correct answer generation unit 170 identifies a determination result indicating that the intermediate dataset is not an intermediate dataset obtained from the second image set as the correct answer determination result. On the other hand, if the correct answer generation unit 170 acquires an image set from the second image set storage unit 302, the correct answer generation unit 170 identifies the maximum likelihood as the correct answer determination result. That is, the correct answer generation unit 170 identifies a determination result indicating that the intermediate dataset is an intermediate dataset obtained from the second image set as the correct answer determination result.

[0128] The correct answer generation unit 170 can have, for example, separate input units for acquiring an image set from a first image set storage unit 301 and an input unit for acquiring an image set from a second image set storage unit 302. The correct answer determination result can then be determined according to which input unit the image set was acquired from.

[0129] As another example, if each image is associated with an imaging angle, the ground truth generation unit 170 may determine the ground truth result based on that imaging angle. As described in the first embodiment, if a trained list is generated, the ground truth generation unit 170 can determine the ground truth result based on the trained list and the imaging angle associated with each image.

[0130] Next, an example will be described in which the discriminator 250 has the functions described in the second embodiment. Specifically, an example will be described in which the discriminator 250 outputs the likelihood that the intermediate data is intermediate data obtained for images at different angles as the discriminator result for each intermediate data. However, the functions of the discriminator 250 are not limited to this example.

[0131] The ground truth generation unit 170 generates a ground truth determination result for each of the multiple images included in the acquired image set. In this example, each image is associated with an imaging angle. The ground truth generation unit 170 identifies the ground truth determination result based on that imaging angle. As described in the first embodiment, when a trained list is generated, the ground truth generation unit 170 can identify the ground truth determination result based on the trained list and the imaging angle associated with each image.

[0132] The second update unit 190 obtains the correct answer determination result from the correct answer generation unit 170. Alternatively, the second update unit 190 may obtain the image set via the correct answer generation unit 170. That is, the second update unit 190 may obtain the image set for which the correct answer determination result has been generated by the correct answer generation unit 170 from the correct answer generation unit 170.

[0133] However, if the correct answer determination results are prepared in advance, the information processing device 10 does not need to include the correct answer generation unit 170. For example, the correct answer determination results may be associated with each image set in advance. Alternatively, the correct answer determination results may be associated with each image in advance. In that case, the second update unit 190 can obtain the correct answer determination results associated with each image.

[0134] The processing performed by the information processing device 10 according to this embodiment will be described below. In the example in Figure 18, step S20 is the same as step S10 in at least one of the first embodiment and modified example 1.

[0135] In this embodiment, step S21 is performed after step S20. In step S21, the second update unit 190 updates the discriminator 250. Step S22 is performed after step S21. Step S22 is the same as step S11 in at least one of the first embodiment, modified example 1, and the second embodiment. Figure 18 shows an example in which step S22 is performed after step S21, but step S21 may be performed after step S22. In other words, the execution order of steps S21 and S22 is not particularly limited. Steps S21 and S22 may be performed in parallel.

[0136] Figure 19 is a flowchart illustrating the processing flow performed by the second update unit 190 according to this embodiment. Figure 19 is also a flowchart specifically showing an example of the processing in step S21.

[0137] The second update unit 190 acquires an image set (step S401). The second update unit 190 can acquire a first image set from, for example, the first image set storage unit 301. Alternatively, the second update unit 190 may acquire a second image set from the second image set storage unit 302. The second update unit 190 may also acquire an image set from another device. As described above, the second update unit 190 may acquire an image set via the correct answer generation unit 170.

[0138] Next, the second update unit 190 generates an intermediate data set by inputting the acquired image set into the intermediate data generator 210 (step S402). The second update unit 190 can read and use multiple parameters from the detector storage unit 303 to configure the intermediate data generator 210.

[0139] However, instead of performing steps S401 and S402, the second update unit 190 may obtain the intermediate dataset generated by the first update unit 150 in step S202.

[0140] In step S403, the second update unit 190 performs discrimination by inputting the intermediate dataset to the discriminator 250. The discrimination result is obtained as the output of the discriminator 250. However, instead of performing step S403, the second update unit 190 may obtain the discrimination result generated by the first update unit 150 in step S203.

[0141] In step S404, the second update unit 190 calculates the loss Lc using the discrimination result and the correct discrimination result. The second update unit 190 can calculate the loss Lc by comparing the discrimination result and the correct discrimination result. Specifically, the loss Lc may be the difference between the discrimination result and the correct discrimination result. As described in the second embodiment, when a discrimination result is output for each intermediate data, the loss Lc is calculated using the comparison result (e.g., difference) between each discrimination result and the correct discrimination result. The loss Lc is, for example, a weighted sum of the comparison results between each discrimination result and the correct discrimination result. The weights for each comparison result are predetermined. However, any existing method can be used to calculate the loss Lc.

[0142] The second update unit 190 updates the discriminator 250 using the obtained loss Lc (step S405). That is, the second update unit 190 updates the multiple parameters held in the discriminator storage unit 304. As the method for updating the discriminator 250 by the second update unit 190, existing methods such as backpropagation can be used.

[0143] In this way, the second update unit 190 updates the discriminator 250 to reduce the loss Lc. This improves the discriminant accuracy of the discriminator 250. In other words, adversarial learning can be performed between the learning of the intermediate data generator 210 and the learning of the discriminator 250. As the discriminant accuracy of the discriminator 250 improves, the learning of the intermediate data generator 210 by the first update unit 150 also progresses further. Therefore, a detector 20 that can more stably perform difference detection for images at different angles is realized.

[0144] Furthermore, the second update unit 190 may repeatedly update the discriminator 250 using multiple image sets in a single step S21. The updating of the discriminator 250 is repeated, for example, until a termination condition is met. The termination condition is, for example, at least one of the following: the number of updates in step S21 is equal to or greater than a predetermined number, and the loss Lc is less than or equal to a predetermined standard.

[0145] Furthermore, each update of the classifier 250 may be performed by batch processing using two or more image sets. That is, a single loss Lc may be calculated using multiple comparison results between the classification result and the correct classification result.

[0146] However, it is preferable that the number of updates of the discriminator 250 in step S21 and the number of updates of the intermediate data generator 210 in step S22 are the same. For example, it is preferable that the first update unit 150 updates the intermediate data generator 210 each time the second update unit 190 updates the discriminator 250.

[0147] In the example shown in Figure 18, when steps S21 and S22 are completed, the first update unit 150 determines whether or not a termination condition is met (step S23). The termination condition is, for example, that the number of times step S22 is executed is equal to or greater than a predetermined number. If the termination condition is met (Yes in step S23), the processing of the information processing device 10 is completed.

[0148] If the termination condition is not met (No. in step S23), the generation unit 130 updates the detector 20 (step S24). The method by which the generation unit 130 updates the detector 20 in step S24 is the same as the method by which the generation unit 130 updates the detector 20 in step S20. However, in step S24, the generation unit 130 further updates the detector 20, including the intermediate data generator 210 that was updated in the first update unit 150.

[0149] Following step S24, the first update unit 150 performs step S21 again.

[0150] The hardware configuration of the computer implementing the information processing device 10 according to this embodiment is shown, for example, in Figure 14, similar to the information processing device 10. However, the storage device 1080 of the computer 1000 implementing the information processing device 10 in this embodiment further stores program modules that implement the functions of the correct answer generation unit 170 and the second update unit 190 of this embodiment.

[0151] Next, the operation and effects of this embodiment will be described. In this embodiment, the same operation and effects as in the first embodiment can be obtained. In addition, according to this embodiment, the information processing device 10 further includes a second update unit 190 that updates the discriminator 250 using an intermediate dataset and the correct answer discrimination result for the intermediate dataset. Therefore, a detector 20 that can more stably perform difference detection for images at different angles is realized.

[0152] (Fourth embodiment) Figure 20 is a diagram illustrating the functions of the intermediate data generator 210 and the difference detector 230 according to the fourth embodiment. The information processing device 10 according to this embodiment is the same as the information processing device 10 according to at least one of the first to third embodiments and Modification 1, except for the points described below. The information processing method according to this embodiment is the same as the information processing method according to at least one of the first to third embodiments and Modification 1, except for the points described below.

[0153] The intermediate data generator 210 according to this embodiment accepts input of information indicating the imaging conditions for each of two or more images included in the image set, in addition to the image set. The generation unit 130 according to this embodiment further uses the information indicating the imaging conditions for each of the two or more images included in the first image set to generate the detector 20. Furthermore, the first update unit 150 according to this embodiment generates an intermediate data set corresponding to the second image set by further inputting the information indicating the imaging conditions for each of the two or more images included in the second image set to the intermediate data generator.

[0154] The imaging conditions include, for example, one or more of the following: imaging angle, direction of solar incidence at the time of imaging, time of imaging, position of the satellite that captured the image, and direction of the antenna used for imaging. Of these, it is preferable that the imaging conditions include the imaging angle. Furthermore, if the images included in the image set are optical images, it is preferable that the imaging conditions for those images include the direction of solar incidence at the time of imaging.

[0155] The intermediate data generator 210 according to this embodiment outputs an intermediate dataset consisting of intermediate data fa corresponding to image Xa and intermediate data fb corresponding to image Xb, similar to the intermediate data generator 210 according to the first embodiment. According to the intermediate data generator 210 according to this embodiment, an intermediate dataset is generated not only based on the image set but also on the imaging conditions. Therefore, more accurate feature extraction becomes possible, and the difference detection accuracy of the detector 20 is improved.

[0156] In the example shown in Figure 20, the intermediate data generator 210 receives image Xa, image Xb, the imaging angle θa of image Xa, and the imaging angle θb of image Xb as input. The intermediate data generator 210 then outputs intermediate data fa for image Xa and intermediate data fb for image Xb.

[0157] Furthermore, the intermediate data generator 210 does not necessarily need to receive information indicating the imaging conditions for all images included in the image set. It is acceptable for the intermediate data generator 210 to receive information indicating the imaging conditions for only some of the two or more images included in the image set.

[0158] In this embodiment, each functional component of the information processing device 10 can further acquire information indicating the imaging conditions of an image in the same manner as acquiring an image set. In this embodiment, for example, the first image set storage unit 301 holds information indicating the imaging conditions of two or more images included in the first image set, in addition to the first image set. In this embodiment, for example, the second image set storage unit 302 holds information indicating the imaging conditions of two or more images included in the second image set, in addition to the second image set.

[0159] Next, the operation and effects of this embodiment will be described. In this embodiment, the same operation and effects as in the first embodiment can be obtained. In addition, according to this embodiment, the generation unit 130 further uses information indicating the imaging conditions of two or more images included in the first image set to generate the detector 20. Therefore, the difference detection accuracy of the detector 20 is improved.

[0160] (Fifth embodiment) Figure 21 is a diagram showing an overview of the detection device 50 according to the fifth embodiment. The detection device 50 according to this embodiment includes a detection unit 510. The detection unit 510 detects differences between two or more images using an intermediate data generator 210 and a difference detector 230. The intermediate data generator 210 is updated by a first update unit 150 provided in an information processing device 10 according to at least one of the first to fourth embodiments and modified example 1.

[0161] Figure 22 is a diagram illustrating an overview of the detection method according to this embodiment. The detection method according to this embodiment is performed by one or more computers. In the detection method according to this embodiment, one or more computers detect the difference between two or more images using an intermediate data generator 210 and a difference detector 230 (step S50). The intermediate data generator 210 is updated by an information processing method according to at least one of the first to fourth embodiments and modified example 1.

[0162] The detection device 50 is a device that detects differences between images using a trained detector 20 that has been trained by the information processing device 10. The detection unit 510 acquires an image set. The image set acquired by the detection unit 510 is called the target image set. The target image set is the same as the image set described in the first embodiment. The target image set may or may not include images taken at different angles. The target image set may or may not include images taken at the same imaging angle as the images included in the second image set. The target image set may include images taken at different imaging angles from any of the images included in the first image set and any of the images included in the second image set.

[0163] The detection unit 510 inputs the target image set to the detector 20. As described in the first embodiment, the detector 20 includes an intermediate data generator 210 and a difference detector 230. The detection unit 510 can, for example, read and use the intermediate data generator 210 and the difference detector 230 held in the detector storage unit 303.

[0164] As explained in Modification Example 1, if detectors 20 are provided for each combination of image types included in the image set, the detection unit 510 may select the detector 20 to use based on the types of images included in the target image set. For example, each image may be associated with information indicating the type of image (SAR image, optical image, etc.). Alternatively, the user may input information to the detection device 50 indicating which detector 20 should be used. The detection unit 510 can select the detector 20 based on the input information.

[0165] The detection unit 510 inputs the target image set to the intermediate data generator 210 to obtain an intermediate data set corresponding to the target image set. Next, the detection unit 510 inputs the intermediate data set to the difference detector 230. Alternatively, the output data of the intermediate data generator 210 may be automatically input to the difference detector 230. The difference detector 230 outputs estimated difference data. The estimated difference data is data that shows the difference between two or more images included in the target image set.

[0166] As described in the fourth embodiment, if the input data to the intermediate data generator 210 further includes information indicating imaging conditions, the detection unit 510 further acquires information indicating imaging conditions for the images included in the target image set. The detection unit 510 then inputs the information indicating imaging conditions to the intermediate data generator 210 to obtain an intermediate data set.

[0167] The detector 20 used by the detection unit 510 includes an intermediate data generator 210 updated by the first update unit 150 of the information processing device 10. Therefore, even if the image set acquired by the detection unit 510 includes images at different angles, the difference can be detected with high accuracy.

[0168] The hardware configuration of the detection device 50 is described below. Each functional component of the detection device 50 (detection unit 510) may be implemented by hardware that realizes each functional component (e.g., hardwired electronic circuit), or by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it). The case in which each functional component of the detection device 50 is implemented by a combination of hardware and software will be further explained below.

[0169] The hardware configuration of the computer implementing the detection device 50 is similar to that of the information processing device 10, as shown in Figure 14, for example. However, the storage device 1080 of the computer 1000 implementing the detection device 50 in this embodiment also stores program modules that implement the functions of the detection unit 510 in this embodiment.

[0170] As described above, according to this embodiment, the detection unit 510 detects the difference between two or more images using the intermediate data generator 210 updated by the first update unit 150 and the difference detector 230. Therefore, difference detection is possible for various image sets.

[0171] The embodiments of the present invention have been described above with reference to the drawings, but these are merely examples of the present invention, and various other configurations can also be adopted.

[0172] Furthermore, while the flowcharts used in the above description show multiple steps (processes) in sequence, the execution order of the steps performed in each embodiment and modification is not limited to the order in which they are described. In each embodiment and modification, the order of the illustrated steps can be changed to the extent that it does not impede the content. Also, the above embodiments and modifications can be combined to the extent that their content is not contradictory.

[0173] Some or all of the above embodiments may also be described as follows, but are not limited to the following: 1-1. A generation means for generating a detector using the first image set, which includes an intermediate data generator that accepts an image set containing two or more images as input and generates an intermediate dataset containing two or more intermediate data, and a difference detector that detects the difference between the two or more images using the intermediate dataset, The system includes a first update means that inputs the second image set to the intermediate data generator to generate the intermediate data set corresponding to the second image set, and updates the intermediate data generator using the intermediate data set, The second set of images includes images taken at different angles than any of the two or more images included in the first set of images. Information processing device. 1-2. In the information processing device described in 1-1, The first update means is, A discriminator that determines whether the intermediate dataset is the intermediate dataset obtained from the image set including the images at different angles is given the intermediate dataset obtained using the first image set and the intermediate dataset obtained using the second image set as inputs. The intermediate data generator is updated so that the discrimination result from the discriminator becomes a predetermined result. Information processing device. 1-3. In the information processing device described in 1-1, The first update means is, A discriminator that determines whether each of the two or more intermediate data sets included in the intermediate dataset is the intermediate data obtained for the different-angle images is input to the intermediate dataset obtained using the first image set and the intermediate dataset obtained using the second image set. The intermediate data generator is updated so that the discrimination result from the discriminator becomes a predetermined result. Information processing device. 1-4. In the information processing device described in 1-2. or 1-3., The system further comprises a second update means for updating the discriminator using the intermediate dataset and the correct answer discrimination result for the intermediate dataset. Information processing device. 1-5. In the information processing device described in any one of 1-1 to 1-4, Each of the two or more images mentioned above is a satellite image. Information processing device. 1-6. In the information processing device described in 1-5, Each of the two or more images is at least one of a SAR image and an optical image. The generation means generates the detector according to the combination of image types included in the image set. Information processing device. 1-7. In the information processing device described in any one of 1-1 to 1-6, The generation means further uses information indicating the imaging conditions for each of the two or more images included in the first image set to generate the detector. Information processing device. 1-8. In the information processing device described in 1-7, The aforementioned imaging conditions include the imaging angle. Information processing device. 1-9. In the information processing device described in any one of 1-1 to 1-8, The detector detects the difference between the two or more images included in the input image set, relating to the target object. Information processing device. 1-10. In the information processing device described in any one of 1-1 to 1-9, Each of the two or more images included in the aforementioned image set includes the target region in the imaging region. Information processing device. 2-1. A detection device comprising an intermediate data generator updated by the first update means provided in any one of 1-1 to 1-10, and a difference detector, wherein a detection means is provided for detecting differences between two or more images. 3-1. One or more computers, A detector is generated using the first image set, which includes an intermediate data generator that accepts an image set containing two or more images as input and generates an intermediate dataset containing two or more intermediate data, and a difference detector that detects the difference between the two or more images using the intermediate dataset. By inputting the second image set into the intermediate data generator, an intermediate dataset corresponding to the second image set is generated, and the intermediate data generator is updated using the intermediate dataset. The second set of images includes images taken at different angles than any of the two or more images included in the first set of images. Information processing methods. 3-2. In the information processing method described in 3-1, The one or more computers mentioned above A discriminator that determines whether the intermediate dataset is the intermediate dataset obtained from the image set including the images at different angles is given the intermediate dataset obtained using the first image set and the intermediate dataset obtained using the second image set as inputs. The intermediate data generator is updated so that the discrimination result from the discriminator becomes a predetermined result. Information processing methods. 3-3. In the information processing method described in 3-1, The one or more computers mentioned above A discriminator that determines whether each of the two or more intermediate data sets included in the intermediate dataset is the intermediate data obtained for the different-angle images is input to the intermediate dataset obtained using the first image set and the intermediate dataset obtained using the second image set. The intermediate data generator is updated so that the discrimination result from the discriminator becomes a predetermined result. Information processing methods. 3-4. In the information processing method described in 3-2 or 3-3, The one or more computers further update the classifier using the intermediate dataset and the correct answer classification result for the intermediate dataset. Information processing methods. 3-5. In the information processing method described in any one of 3-1 to 3-4, Each of the two or more images mentioned above is a satellite image. Information processing methods. 3-6. In the information processing method described in 3-5, Each of the two or more images is at least one of a SAR image and an optical image. The one or more computers mentioned above generate the detectors according to the combination of image types included in the image set. Information processing methods. 3-7. In any of the information processing methods described in 3-1 to 3-6, The one or more computers further use information indicating the imaging conditions for each of the two or more images included in the first image set to generate the detector. Information processing methods. 3-8. In the information processing method described in 3-7, The aforementioned imaging conditions include the imaging angle. Information processing methods. 3-9. In the information processing method described in any one of 3-1 to 3-8, The detector detects the difference between the two or more images included in the input image set, relating to the target object. Information processing methods. 3-10. In any of the information processing methods described in 3-1 to 3-9, Each of the two or more images included in the aforementioned image set includes the target region in the imaging region. Information processing methods. 4-1. One or more computers, A detection method for detecting differences between two or more images using the intermediate data generator updated by the information processing method described in any one of 3-1 to 3-10, and the difference detector. 5-1. Computers, A generation means for generating a detector using the first image set, which includes an intermediate data generator that accepts an image set containing two or more images as input and generates an intermediate dataset containing two or more intermediate data, and a difference detector that detects the difference between the two or more images using the intermediate dataset, and By inputting the second image set into the intermediate data generator, an intermediate dataset corresponding to the second image set is generated, and the intermediate data generator is used to update the intermediate data generator as a first update means. The second set of images includes images taken at different angles than any of the two or more images included in the first set of images. program. 5-2. In the program described in 5-1, The first update means is, A discriminator that determines whether the intermediate dataset is the intermediate dataset obtained from the image set including the images at different angles is given the intermediate dataset obtained using the first image set and the intermediate dataset obtained using the second image set as inputs. The intermediate data generator is updated so that the discrimination result from the discriminator becomes a predetermined result. program. 5-3. In the program described in 5-1, The first update means is, A discriminator that determines whether each of the two or more intermediate data sets included in the intermediate dataset is the intermediate data obtained for the different-angle images is input to the intermediate dataset obtained using the first image set and the intermediate dataset obtained using the second image set. The intermediate data generator is updated so that the discrimination result from the discriminator becomes a predetermined result. program. 5-4. In the program described in 5-2. or 5-3., The computer is further configured to function as a second update means for updating the discriminator using the intermediate dataset and the correct answer discrimination result for the intermediate dataset. program. 5-5. In any of the programs described in 5-1 to 5-4, Each of the two or more images mentioned above is a satellite image. program. 5-6. In the program described in 5-5, Each of the two or more images is at least one of a SAR image and an optical image. The generation means generates the detector according to the combination of image types included in the image set. program. 5-7. In any one of the programs described in 5-1 to 5-6, The generation means further uses information indicating the imaging conditions for each of the two or more images included in the first image set to generate the detector. program. In the program described in 5-8. 5-7., The aforementioned imaging conditions include the imaging angle. program. 5-9. In any of the programs described in 5-1 through 5-8, The detector detects the difference between the two or more images included in the input image set, relating to the target object. program. 5-10. In any of the programs described in 5-1 through 5-9, Each of the two or more images included in the aforementioned image set includes the target region in the imaging region. program. 6-1. Computers, The intermediate data generator, updated by the information processing method described in any one of 3-1 to 3-10, and the difference detector are used as a detection means for detecting differences between two or more images. program. 7-1. A computer-readable recording medium containing a program described in any one of the sections 5-1 through 5-10. 8-1. A computer-readable recording medium on which the program described in 6-1. is stored.

[0174] This application claims priority based on Japanese Patent Application No. 2022-212492, filed on 28 December 2022, and incorporates all of its disclosures herein. [Explanation of Symbols]

[0175] 10 Information Processing Devices 20 detectors 50 Detection device 130 Generation part 150 1st update part 170 Correct Answer Generation Unit 190 2nd update part 210 Intermediate Data Generator 230 Difference Detectors 250 Discriminator 301 First image set storage unit 302 Second image set storage unit 303 Pixel memory unit 304 Discriminator storage unit 510 Detection unit 1000 calculator

Claims

1. A generation means for generating a detector using a first image set containing two or more images, which includes an intermediate data generator that accepts an image set containing two or more images as input and generates an intermediate dataset containing two or more intermediate data, and a difference detector that detects the difference between the two or more images using the intermediate dataset, The system includes a first update means that generates an intermediate data set corresponding to the second image set by inputting a second image set containing two or more images into the intermediate data generator, and updates the intermediate data generator using the intermediate data set, The second set of images includes images taken at different angles than any of the two or more images included in the first set of images. Information processing device.

2. In the information processing apparatus according to claim 1, The first update means is, A discriminator that determines whether the intermediate dataset is the intermediate dataset obtained from the image set including the images at different angles is given the intermediate dataset obtained using the first image set and the intermediate dataset obtained using the second image set as inputs. The intermediate data generator is updated so that the discrimination result from the discriminator becomes a predetermined result. Information processing device.

3. In the information processing apparatus according to claim 1, The first update means is, A discriminator that determines whether each of the two or more intermediate data sets included in the intermediate dataset is the intermediate data obtained for the different-angle images is input to the intermediate dataset obtained using the first image set and the intermediate dataset obtained using the second image set. The intermediate data generator is updated so that the discrimination result from the discriminator becomes a predetermined result. Information processing device.

4. In the information processing apparatus according to claim 2 or 3, The system further comprises a second update means for updating the discriminator using the intermediate dataset and the correct answer discrimination result for the intermediate dataset. Information processing device.

5. In the information processing apparatus according to any one of claims 1 to 3, Each of the two or more images is at least one of a SAR image and an optical image. The generation means generates the detector according to the combination of image types included in the image set. Information processing device.

6. In the information processing apparatus according to any one of claims 1 to 3, The generation means further uses information indicating the imaging conditions for each of the two or more images included in the first image set to generate the detector. Information processing device.

7. In the information processing apparatus described in claim 6, The aforementioned imaging conditions include the imaging angle. Information processing device.

8. A detection device comprising a detection means for detecting differences between two or more images using the intermediate data generator updated by the first update means provided in the information processing device according to any one of claims 1 to 3, and the difference detector.

9. One or more computers A detector is generated using a first image set containing two or more images, which includes an intermediate data generator that accepts an image set containing two or more images as input and generates an intermediate dataset containing two or more intermediate data, and a difference detector that detects the difference between the two or more images using the intermediate dataset. By inputting a second image set containing two or more images into the intermediate data generator, an intermediate dataset corresponding to the second image set is generated, and the intermediate data generator is updated using the intermediate dataset. The second set of images includes images taken at different angles than any of the two or more images included in the first set of images. Information processing methods.

10. Computers, A generation means for generating a detector using a first image set containing two or more images, which includes an intermediate data generator that accepts an image set containing two or more images as input and generates an intermediate dataset containing two or more intermediate data, and a difference detector that detects the difference between the two or more images using the intermediate dataset, and By inputting a second image set containing two or more images into the intermediate data generator, an intermediate dataset corresponding to the second image set is generated, and the intermediate data generator is used as a first update means to update the intermediate data generator using the intermediate dataset. The second set of images includes images taken at different angles than any of the two or more images included in the first set of images. program.

Images