Change detection system, change detection method, and change detection program

Abstract

To detect a small-to-medium-sized change using an image.SOLUTION: An input generating device 200 refers to map data to identify a target area and performs map projection on an acquired image pair to generate a map projection image pair. A learning device 300 learns learning area information indicating the target area in a learning target area, a learning image pair that is the map projection image pair in the learning target area, and learning change information indicating a notable change in the learning target area, and generates a learned model. An inference device 400 calculates the learned model using inference area information indicating the target area in an inference target area that is a notable area and an inference image pair that is the map projection image pair in the inference target area as inputs to obtain inference change information indicating the notable change in the inference target area.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] FIELD This disclosure relates to image-based change detection. [Background technology]

[0002] To maintain and manage digital map data, there is a method of extracting changes in features on the Earth's surface and reflecting the extracted changes in the digital map data.

[0003] In recent years, a method has been proposed for updating digital map data, in which the digital map data is compared with still images to detect changes in features and new features not yet shown on the map.

[0004] Patent Document 1 discloses the following technique. The method involves photographing an area using remote sensing or other methods, comparing the still images obtained from the photograph with digital map data, detecting changes in the figures and new feature information from the still images, and updating the map information. Detection is performed by comparing the features in the image with the figures on the map through matching, or by analyzing the pixel characteristic values ​​or texture of the features in the image and the figures on the map. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2005-234603 Summary of the Invention [Problem to be solved by the invention]

[0006] When the target of change detection is a large change accompanied by a structural change (for example, construction of a building), the method of Patent Document 1 is capable of detecting the change. However, in the case of a change that does not involve a structural change or is a small structural change (for example, the addition of a white line), the change cannot be detected even if the method of Patent Document 1 compares map data composed of abstracted data of structures with a still image.

[0007] The present disclosure aims to enable the detection of small to medium scale changes using images. [Means for solving the problem]

[0008] The change detection system of the present disclosure comprises: an area specifying unit that refers to map data of the area of ​​interest and specifies an area where a target feature exists as a target area; a map projection unit that performs map projection on a set of acquired images obtained by photographing the area of ​​interest at different times to generate a set of map projected images; a model generation unit that generates a trained model by performing training using, as training data, training area information indicating the target area within the training area that is the attention area, a training image set that is the map projection image set corresponding to the acquired image set of the training area, and training change information indicating notable changes in the training area; Equipped with. [Effects of the Invention]

[0009] According to the present disclosure, it is possible to detect small and medium-sized changes using images. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a configuration diagram of a change detection system 100 according to a first embodiment. [Figure 2] FIG. 1 is a configuration diagram of an input generation device 200 according to the first embodiment. [Figure 3] FIG. 1 is a block diagram of an input generation device 200 according to the first embodiment. [Figure 4] FIG. 2 is a configuration diagram of a learning device 300 according to the first embodiment. [Figure 5] FIG. 2 is a block diagram of a learning device 300 according to the first embodiment. [Figure 6] FIG. 1 is a configuration diagram of an inference device 400 according to the first embodiment. [Figure 7] FIG. 1 is a block diagram of an inference device 400 according to the first embodiment. [Figure 8] 3 is a flowchart of an input generation method according to the first embodiment. [Figure 9] FIG. 3 shows an example of target region information 119 according to the first embodiment. [Figure 10] FIG. 3 shows an example of comparison of the map projection image 131 according to the first embodiment. [Figure 11] 3 is a flowchart of a learning method according to the first embodiment. [Figure 12] 3 is a flowchart of an inference method according to the first embodiment. [Figure 13] FIG. 2 is a diagram showing an example of a neural network according to the first embodiment. [Figure 14] FIG. 10 is a configuration diagram of an input generation device 200 according to a second embodiment. [Figure 15] FIG. 10 is a block diagram of an input generation device 200 according to a second embodiment. [Figure 16] 10 is a flowchart of an input generation method according to the second embodiment. [Figure 17] FIG. 11 is a configuration diagram of an input generation device 200 according to a third embodiment. [Figure 18] FIG. 11 is a block diagram of an input generation device 200 according to a third embodiment. [Figure 19] FIG. 11 is a block diagram of a learning device 300 according to a third embodiment. [Figure 20] FIG. 11 is a block diagram of an inference device 400 according to a third embodiment. [Figure 21] 11 is a flowchart of an input generation method according to the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] In the embodiments and drawings, the same or corresponding elements are denoted by the same reference numerals. The description of elements denoted by the same reference numerals as those already described will be omitted or simplified as appropriate. Arrows in the drawings primarily indicate the flow of data or the flow of processing.

[0012] Embodiment 1 The change detection system 100 will be described with reference to FIGS.

[0013] ***Configuration Description*** The configuration of a change detection system 100 will be described with reference to FIG. The change detection system 100 includes an input generation device 200, a learning device 300, and an inference device 400. The input generation device 200 generates data to be input to the learning device 300 and the inference device 400 . The learning device 300 learns input data and generates a model for inference. The learning device 300 is also called a model generation device. The inference device 400 performs inference using input data and a model. The inference device 400 is also called an application device.

[0014] The configuration of the input generation device 200 will be described with reference to FIG. The input generation device 200 is a computer that includes hardware such as a processor 201, a memory 202, an auxiliary storage device 203, a communication device 204, and an input / output interface 205. These pieces of hardware are connected to one another via signal lines.

[0015] The processor 201 is the processor of the input generating device 200 . A processor is an integrated circuit (IC) that performs calculations and controls other hardware. For example, a processor is a CPU. IC is an abbreviation for Integrated Circuit. CPU is an abbreviation for Central Processing Unit.

[0016] The memory 202 is the memory of the input generating device 200 . Memory is a volatile or non-volatile storage device. Memory is also called primary or main memory. For example, memory is RAM. Data stored in memory is saved to secondary storage as needed. RAM is an abbreviation for Random Access Memory.

[0017] The auxiliary storage device 203 is an auxiliary storage device for the input generation device 200 . The secondary storage device is a non-volatile storage device. For example, the secondary storage device may be a ROM, a HDD, a flash memory, or a combination of these. Data stored in the secondary storage device is loaded into the memory as needed. ROM is an abbreviation for Read Only Memory. HDD is an abbreviation for Hard Disk Drive.

[0018] The communication device 204 is a communication device of the input generation device 200. The communication of the input generation device 200 is performed using the communication device 204. The communication device is a receiver and a transmitter, for example, the communication device is a communication chip or a NIC. NIC is an abbreviation for Network Interface Card.

[0019] The input / output interface 205 is an input / output interface for the input generation device 200. The input / output interface 205 is used for input and output of the input generation device 200. An input / output interface is a port to which an input device and an output device are connected. For example, an input / output interface is a USB terminal, and the input devices are a keyboard and a mouse, and the output device is a display. USB is an abbreviation for Universal Serial Bus.

[0020] The input generation device 200 includes elements such as a map acquisition unit 211, an area identification unit 212, and a map projection unit 221. These elements are realized by software.

[0021] The auxiliary storage device 203 stores an input generation program for causing the computer to function as a map acquisition unit 211, an area identification unit 212, and a map projection unit 221. The input generation program is loaded into the memory 202 and executed by the processor 201. The auxiliary storage device 203 also stores an OS. At least a part of the OS is loaded into the memory 202 and executed by the processor 201. The processor 201 executes the input generation program while running the OS. OS is an abbreviation for Operating System.

[0022] Input and output data of the input generation program are stored in the storage unit 290 . The memory 202 functions as the storage unit 290. However, a storage device such as the auxiliary storage device 203, a register in the processor 201, or a cache memory in the processor 201 may function as the storage unit 290 instead of or together with the memory 202.

[0023] The input generating device 200 may include multiple processors instead of the processor 201 .

[0024] FIG. 3 shows the functional configuration of the input generation device 200. The details of each element of the functional configuration will be described later.

[0025] The configuration of the learning device 300 will be described with reference to FIG. The learning device 300 is a computer that includes hardware such as a processor 301, a memory 302, an auxiliary storage device 303, a communication device 304, and an input / output interface 305. These pieces of hardware are connected to each other via signal lines.

[0026] The processor 301 is the processor of the learning device 300 . The memory 302 is the memory of the learning device 300 . The auxiliary storage device 303 is an auxiliary storage device for the learning device 300 . The communication device 304 is a communication device for the learning device 300. The communication device 304 is used for communication of the learning device 300. The input / output interface 305 is an input / output interface for the learning device 300. The input / output interface 305 is used for input and output of the learning device 300.

[0027] The learning device 300 includes elements such as a data receiving unit 311 and a model generating unit 312. These elements are realized by software.

[0028] The auxiliary storage device 303 stores a learning program for causing the computer to function as a data receiving unit 311 and a model generating unit 312. The learning program is loaded into the memory 302 and executed by the processor 301. The auxiliary storage device 303 also stores an OS. At least a part of the OS is loaded into the memory 302 and executed by the processor 301. The processor 301 executes the learning program while running the OS.

[0029] The input and output data of the learning program are stored in the storage unit 390. The memory 302 functions as the storage unit 390. However, a storage device such as the auxiliary storage device 303, a register in the processor 301, or a cache memory in the processor 301 may function as the storage unit 390 instead of the memory 302 or together with the memory 302.

[0030] The learning device 300 may include multiple processors replacing the processor 301.

[0031] FIG. 5 shows the functional configuration of the learning device 300. The details of each element of the functional configuration will be described later.

[0032] The configuration of the inference device 400 will be described with reference to FIG. The inference device 400 is a computer that includes hardware such as a processor 401, a memory 402, an auxiliary storage device 403, a communication device 404, and an input / output interface 405. These pieces of hardware are connected to one another via signal lines.

[0033] Processor 401 is the processor of inference device 400 . Memory 402 is the memory of reasoning device 400 . Auxiliary storage device 403 is an auxiliary storage device for inference device 400 . Communication device 404 is a communication device for inference device 400. Communication for inference device 400 is performed using communication device 404. Input / output interface 405 is an input / output interface for inference device 400. Input / output to / from inference device 400 is performed using input / output interface 405.

[0034] The inference device 400 comprises elements such as a data receiving unit 411 and an inference unit 412. These elements are realized by software.

[0035] The auxiliary storage device 403 stores an inference program for causing the computer to function as a data receiving unit 411 and an inference unit 412. The inference program is loaded into the memory 402 and executed by the processor 401. The auxiliary storage device 403 also stores an OS. At least a part of the OS is loaded into the memory 402 and executed by the processor 401. The processor 401 executes an inference program while running the OS.

[0036] Input and output data of the inference program are stored in the storage unit 490. The memory 402 functions as the storage unit 490. However, a storage unit such as the auxiliary storage unit 403, a register in the processor 401, or a cache memory in the processor 401 may function as the storage unit 490 instead of the memory 402 or together with the memory 402.

[0037] Reasoning apparatus 400 may include multiple processors replacing processor 401 .

[0038] FIG. 7 shows the functional configuration of the inference device 400. The details of each element of the functional configuration will be described later.

[0039] ***Explanation of Operation*** The procedure of operation of the change detection system 100 corresponds to a change detection method. The operation procedure of the input generation device 200 corresponds to an input generation method. The operation procedure of the learning device 300 corresponds to a learning method. The procedure of operation of inference device 400 corresponds to an inference method.

[0040] The operation procedure of the change detection system 100 corresponds to the processing procedure by the change detection program. The operation procedure of the input generation device 200 corresponds to the processing procedure according to the input generation program. The operation procedure of the learning device 300 corresponds to the processing procedure according to the learning program. The operational procedure of the inference device 400 corresponds to the processing procedure according to the inference program. The change detection program includes an input generation program, a learning program, and an inference program. The program can be recorded (stored) in a computer-readable manner on a non-volatile recording medium such as an optical disk or flash memory.

[0041] The input generation method will be described based on FIG. The area of ​​interest is photographed at multiple different times to obtain the set of acquired images 120. Specifically, the set of acquired images 120 is obtained by observing the area of ​​interest over multiple periods of time using a remote sensing technique. Acquired image set 120 consists of two or more acquired images. The acquired image is an image that shows the area of ​​interest. Information indicating various conditions during photography is called photography condition information. For example, photography condition information indicates viewpoint, LOS, field of view, magnification, position information, resolution, and pointing angle. Position information indicates the coordinate values ​​(latitude, longitude) of the points captured in the pixels at the center and four corners of the image. LOS is an abbreviation for Line of Sight.

[0042] The user specifies an area of ​​interest to the input generation device 200 . The user also inputs the acquired image set 120 and the photographing condition information to the input generation device 200 .

[0043] In step S101, the map acquisition unit 211 receives a designation of an area of ​​interest. Then, the map acquisition unit 211 acquires the map data 111 of the area of ​​interest from the map database 110.

[0044] Map data 111 for each area is registered in the map database 110. The map database 110 may be managed by the input generation device 200 or may be managed externally to the input generation device 200. The map data 111 indicates the location information of each point. The location information indicates coordinate values ​​(latitude, longitude). The map data 111 also indicates the area (element area) in which each of one or more map elements exists. The map represented by the map data 111 may be either a planar digital map or a 3D digital map. A planar digital map is composed of an (x, y) coordinate sequence. (x, y) indicates two dimensions. A 3D digital map is composed of an (x, y, z) coordinate sequence. (x, y, z) indicates three dimensions. Map elements are features that exist in an area and are registered in the map data 111. Among the map elements, a feature that is the target of change detection is called a target feature 112. The target feature 112 is determined in advance. The area in which the feature of interest 112 exists is referred to as the area of ​​interest 113 .

[0045] In step S102, the area specifying unit 212 specifies the target area 113 with reference to the acquired map data 111, and generates the target area information 119. The target area information 119 is information indicating the target area 113 .

[0046] The region specifying unit 212 outputs the target region information 119 .

[0047] FIG. 9 shows an example of the target area information 119. In the map data 111, roads are registered as target features 112. If a map is created using area polygons, the area polygons of roads are registered in the map data 111. When a map is created using linear polygons, linear polygons of the shoulder lines (edges) of roads are registered in the map data 111.

[0048] The target area information 119 indicates an area where roads exist as a target area 113 . If the map is created using area polygons, the area specifying unit 212 specifies the area polygons of the roads as the target area 113 . When the map is created using linear polygons, the area specifying unit 212 specifies, as the target area 113, an area where a road corresponding to the link (AB) from node (A) to node (B) exists. Then, the area specifying unit 212 converts the target area 113 into a binary mask. The converted map data 111 becomes the target area information 119.

[0049] Returning to FIG. 8, step S111 will be described. In step S111, the map projection unit 221 receives the acquired image set 120 and the shooting condition information. Then, the map projection unit 221 uses the photographing condition information to perform map projection on the acquired image set 120. As a result, a map-projected image set 130 is generated.

[0050] The map projection unit 221 outputs a map projection image set 130.

[0051] A map projection is performed for each acquired image in the set of acquired images 120 . Map projection is the process of projecting a captured image onto a map coordinate system, which assigns location information to each pixel of the captured image.

[0052] The map projection image set 130 consists of two or more map projection images 131 corresponding to two or more captured images. The map projection image 131 is an acquired image that has been subjected to map projection. Position information is assigned to each pixel of the map projection image 131.

[0053] FIG. 10 shows an example of a map projection image set 130. The map projection image 131A and the map projection image 131B constitute a map projection image set 130. The map projection image 131A is a map projection image 131 corresponding to an acquired image obtained by photographing at time (1). The map projection image 131B is the map projection image 131 corresponding to the acquired image obtained by photographing at time (2).

[0054] Between period (1) and period (2), changes (i) and (ii) occurred. Change (i) is an increase in the number of lanes. This change (i) is a change in the area (target area 113) of features registered in the map data 111. This change is the change to be extracted. Change (ii) is a change in a building. This change (ii) is a change in an area of ​​a feature that is not registered in the map data 111. This change is not a change that should be extracted.

[0055] In a multi-period period, in addition to the changes that should be extracted, changes that should not be extracted occur. The changes to be extracted are changes, disappearances, and new installations of features in the target area 113 . Changes that do not need to be extracted include, for example, changes due to seasonal variations, changes in solar or atmospheric conditions, changes due to dynamic subjects (clouds, shadows, automobiles, etc.), and changes in buildings (disappearance, addition, etc.).

[0056] When a change that is not desired to be extracted is considered to be a disturbance change, the target region information 119 can be used to exclude the disturbance change from the changes to be extracted.

[0057] The learning method will be explained based on FIG. A user inputs learning data 140 into learning device 300. Learning data 140 is input data for learning. The training data 140 is a set of training area information 141 , training image set 142 , and training change information 143 .

[0058] The area of ​​interest that is the subject of study is called a study area. The learning area information 141 is the target area information 119 that indicates the target area 113 within the learning target area. The training image set 142 is a map projection image set 130 corresponding to the acquired image set 120 that shows the training area. The learning change information 143 is attention change information for the learning area. The attention change information indicates an attention change in the attention area.

[0059] In step S121, the data receiving unit 311 receives the training data 140.

[0060] In step S122, the model generation unit 312 generates a trained model 149 by training the training data 140. The trained model 149 is a model that receives the target region information 119 and the acquired image set 120 as input and outputs attention change information.

[0061] The learning in step S122 will be described in detail later.

[0062] The inference method will be explained based on FIG. The trained model 149 is stored in the inference device 400. A user inputs inference data 150 into the inference device 400. The inference data 150 is input data for inference. The inference data 150 is a set of inference area information 151 and an inference image set 152 .

[0063] The area of ​​interest that is the subject of inference is called an inference subject area. The inference area information 151 is the target area information 119 that indicates the target area 113 within the inference target area. The inference image set 152 is the map projection image set 130 corresponding to the acquired image set 120 that shows the area to be inferred.

[0064] In step S131, the data receiving unit 411 receives the inference data 150.

[0065] In step S132, the inference unit 412 inputs the inference data 150 and calculates the trained model 149. This results in the inference change information 153. The inference change information 153 is attention change information for the inference target area.

[0066] The inference unit 412 outputs the inference change information 153. For example, the inference change information 153 is displayed on a display.

[0067] The learning in step S122 will be described in detail below. The model generation unit 312 can use known algorithms such as supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning.

[0068] Explain supervised learning. Supervised learning is a technique in which a set of input and result data is used as training data, features in the training data are learned, and a result is inferred from the input.

[0069] In supervised learning, the training data 140 is training data. The learning change information 143 is a correct answer to be inferred from the learning area information 141 and the learning image set 142. In FIG. 10, the learning change information 143 indicates change (i). For example, the learning change information 143 is generated by a person comparing the learning image set 142 and masking (filling in, for example) the change (i) portion. Existing map information, survey results by workers, etc. may be used to generate the learning change information 143. The changes indicated in the learned change information 143 are tagged. A tag is a label that indicates the content of the change. Examples of tags are "white line added," "white line deleted," "building added," "building deleted," and "changed."

[0070] In the case of supervised learning, the learning area information 141, the learning image set 142, and the learning change information 143 are all for the same subject. In the case of unsupervised learning, the learning area information 141, the learning image set 142, and the learning change information 143 do not need to be for the same subject.

[0071] Supervised learning, for example, uses neural networks. A neural network consists of an input layer, a hidden layer, and an output layer. Each layer consists of multiple neurons. The hidden layer can be one layer or two or more layers.

[0072] Figure 13 shows an example of a three-layer neural network. When multiple input values ​​are input to the input layer (X1 to X3), the multiple input values ​​are multiplied by first weights W1 (w11 to w16) to obtain multiple calculated values, which are then input to the hidden layer (Y1, Y2). When a plurality of calculated values ​​are input to the intermediate layers (Y1, Y2), the calculated values ​​are multiplied by the second weights W2 (w21 to w26) to obtain a plurality of output values. The plurality of output values ​​are output from the output layers (Z1 to Z3). The output values ​​vary depending on the respective values ​​of the first weight W1 and the second weight W2.

[0073] The neural network generates a trained model 149 through supervised learning according to the training data 140. The trained model 149 infers optimal attention change information corresponding to the target region information 119 and the acquired image set 120. Specifically, the neural network inputs learning domain information 141 and learning image set 142 into the input layer and adjusts the first weight W1 and the second weight W2 so that the result output from the output layer approaches learning change information 143.

[0074] The model generation unit 312 may use deep learning or machine learning as a learning algorithm. Deep learning learns to extract features themselves. Machine learning is carried out according to genetic programs, functional logic programs, or support vector machines, for example.

[0075] ***Description of Example*** The input generation device 200, the learning device 300, and the inference device 400 may be combined together. For example, the input generation device 200, the learning device 300, and the inference device 400 may be configured as a single device. The input generation device 200, the learning device 300, and the inference device 400 may be connected to each other via a network. The input generation device 200, the learning device 300, and the inference device 400 may be a cloud server.

[0076] There may be multiple change detection systems 100. Also, new change detection systems 100 may be added, or existing change detection systems 100 may be removed. The training data 140 may be generated by other change detection systems 100 . The data receiving unit 311 may collect learning data 140 from two or more change detection systems 100 used in the same area as the area in which its own change detection system 100 is used. The data receiving unit 311 may also collect learning data 140 from two or more change detection systems 100 used in areas different from the area in which its own change detection system 100 is used. The trained model 149 may be generated in another change detection system 100. The model generation unit 312 may update the trained model 149 generated in another change detection system 100 by re-training.

[0077] It is preferable that the time period of the map represented by the map data 111 and the time period of the photographing of the area of ​​interest shown in the acquired image set 120 are the same. For example, a plurality of captured images taken at different times are stored as archive data in an archive database, and a set of captured images 120 taken at the same time as the map represented by the map data 111 is retrieved from the archive database.

[0078] ***Effects of the First Embodiment*** According to the first embodiment, it becomes possible to detect small to medium-sized changes that do not involve structural changes on the earth's surface.

[0079] Embodiment 2 The following describes a mode for correcting disturbances reflected in the map projection image 131, focusing mainly on the differences from the first embodiment, with reference to FIGS. 14 to 16. FIG.

[0080] ***Configuration Description*** The configuration of the change detection system 100 is the same as that in the first embodiment. However, the input generation device 200 is partially different from the configuration in the first embodiment.

[0081] The configuration of the input generation device 200 will be described with reference to FIG. The input generating device 200 further includes a disturbance corrector 222 . The input generation program also causes the computer to function as a disturbance correction unit 222 .

[0082] FIG. 15 shows the functional configuration of the input generation device 200.

[0083] ***Explanation of Operation*** The input generation method will be described with reference to FIG. Steps S101, S102 and S111 are the same as those described in the first embodiment.

[0084] In step S112, the disturbance correction unit 222 corrects the disturbance reflected in each map projection image 131 of the map projection image set 130. In other words, the disturbance correction unit 222 corrects the portion of the map projection image 131 where the disturbance is reflected. Specifically, the disturbance correction unit 222 detects a disturbance reflected in the map projection image 131 and corrects the detected disturbance. Correction means, for example, removal of the disturbance. This generates a set of corrected images 160.

[0085] Corrected image set 160 consists of two or more corrected images corresponding to two or more acquired images. Each corrected image in the corrected image set 160 is a map-projected image that has been subjected to disturbance correction.

[0086] The acquired image is, for example, an image obtained by eight-band visible remote sensing. An example of a disturbance that may be detected is clouds and shadows. An example of correction is masking.

[0087] The learning procedure is the same as that in the first embodiment. However, the training image set 142 is a corrected image set 160 that corresponds to the acquired image set 120 that shows the training area.

[0088] The procedure of the inference method is the same as that in the first embodiment. However, the inference image set 152 is a corrected image set 160 corresponding to the acquired image set 120 that shows the inference target area.

[0089] ***Effects of the Second Embodiment*** According to the second embodiment, it is possible to correct disturbances captured in the set of acquired images 120 other than those suppressed using the target area information 119. Therefore, it is possible to suppress the influence of disturbances when performing learning, and it is possible to further improve the learning accuracy. Furthermore, by performing learning that suppresses the detection of disturbance changes, it is possible to further avoid detecting unnecessary disturbance changes.

[0090] Embodiment 3 The manner in which the target area 113 in the map projection image 131 is masked will be described below, focusing mainly on the points that differ from the first embodiment, with reference to FIGS. 17 to 21. FIG.

[0091] ***Configuration Description*** The configuration of the change detection system 100 is the same as that in the first embodiment. However, the input generation device 200 is partially different from the configuration in the first embodiment.

[0092] The configuration of the input generation device 200 will be described with reference to FIG. The input generation device 200 further includes a region masking unit 223 . The input generation program also causes the computer to function as a region masking unit 223 .

[0093] FIG. 18 shows the functional configuration of the input generation device 200. FIG. 19 shows the functional configuration of the learning device 300. FIG. 20 shows the functional configuration of the inference device 400.

[0094] ***Explanation of Operation*** The input generation method will be described with reference to FIG. Steps S101, S102 and S111 are the same as those described in the first embodiment.

[0095] In step S113, the area masking unit 223 masks, for each map projection image 131 in the map projection image set 130, the area other than the target area 113 in the map projection image 131 based on the target area information 119. In other words, the area masking unit 223 hides the area other than the target area 113 in the map projection image 131. This generates a masking image set 170.

[0096] The masking image set 170 consists of two or more masking images corresponding to two or more acquired images. Each masking image in the masking image set 170 is a map projection image that has been masked.

[0097] The learning procedure is the same as that in the first embodiment. However, the training data 140 does not include training area information 141, but is a set of a training image set 142 and training change information 143. The training image set 142 is a masking image set 170 that corresponds to the acquired image set 120 that shows the training area.

[0098] The procedure of the inference method is the same as that in the first embodiment. However, the inference data 150 does not include inference area information 151, but is an inference image set 152. The inference image set 152 is a masking image set 170 that corresponds to the acquired image set 120 that shows the area to be inferred.

[0099] ***Effects of the Third Embodiment*** According to the third embodiment, it is possible to reduce the amount of input data required for learning, thereby simplifying the learning device 300. This makes it easier to progress with learning. Also, the amount of data required for learning is reduced. As a result, high detection accuracy can be obtained with learning using a small amount of data.

[0100] ***Supplement to embodiment 3*** The third embodiment may be combined with the second embodiment. That is, the input generation device 200 may include a disturbance correction unit 222. In this case, the masking image corresponds to a map projection image that has been subjected to disturbance correction and masking.

[0101] ***Supplementary explanation of implementation form*** Each embodiment is an example of a preferred embodiment and is not intended to limit the technical scope of the present disclosure. Each embodiment may be implemented in part or in combination with other embodiments. Procedures described using flowcharts, etc. may be modified as appropriate.

[0102] Each element of the input generation device 200, the learning device 300 and the inference device 400 may be implemented in software, hardware, firmware or a combination thereof. The "unit" of each element of the input generation device 200, the learning device 300, and the inference device 400 may be read as "processing," "step," "circuit," or "circuitry." Furthermore, "information" may be read as "data."

[0103] Various aspects of the present disclosure are described below as appendices.

[0104] (Appendix 1) an area specifying unit that refers to map data of the area of ​​interest and specifies an area where a target feature exists as a target area; a map projection unit that performs map projection on a set of acquired images obtained by photographing the area of ​​interest at different times to generate a set of map projected images; a model generation unit that generates a trained model by performing training using, as training data, training area information indicating the target area within the training area that is the attention area, a training image set that is the map projection image set corresponding to the acquired image set of the training area, and training change information indicating notable changes in the training area; A change detection system comprising:

[0105] (Appendix 2) The inference unit receives as input inference area information indicating the target area within the inference target area, which is the attention area, and an inference image set, which is the map projection image set corresponding to the acquired image set of the inference target area, and calculates the trained model to obtain inference change information indicating noteworthy changes in the inference target area. 2. The change detection system of claim 1.

[0106] (Appendix 3) a map acquisition unit that acquires map data of the area of ​​interest from a map database that stores map data representing a planar digital map composed of two-dimensional coordinate sequences or a 3D digital map composed of three-dimensional coordinate sequences; 10. The change detection system of claim 1 or 2.

[0107] (Appendix 4) The acquired image set is archive data obtained by photographing at the same time as the map represented by the map data. 4. The change detection system of any one of claims 1 to 3.

[0108] (Appendix 5) Identifying an area where a target feature exists as a target area by referring to map data of the area of ​​interest; generating a set of map-projected images by performing map projection on a set of acquired images obtained by photographing the area of ​​interest at different times; A trained model is generated by performing training using training area information indicating the target area within the training area, which is the attention area, a training image set which is the map projection image set corresponding to the acquired image set of the training area, and training change information indicating notable changes in the training area as training data. Change detection methods.

[0109] (Appendix 6) an area specifying unit that refers to map data of the area of ​​interest and specifies an area where a target feature exists as a target area; a map projection unit that performs map projection on a set of acquired images obtained by photographing the area of ​​interest at different times to generate a set of map projected images; a model generation unit that generates a trained model by performing training using as training data training area information indicating the target area within the training target area that is the attention area, a training image set that is the map projection image set corresponding to the acquired image set of the training target area, and training change information that indicates notable changes in the training target area; Change detection programs to make computers function.

[0110] (Appendix 7) an area specifying unit that refers to map data of the area of ​​interest and specifies an area where a target feature exists as a target area; a map projection unit that performs map projection on a set of acquired images obtained by photographing the area of ​​interest at different times to generate a set of map projected images; a disturbance correction unit that corrects disturbances reflected in each map projection image of the map projection image set to generate a corrected image set; a model generation unit that generates a trained model by performing training using, as training data, training area information indicating the target area within the training area that is the attention area, a training image set that is the corrected image set corresponding to the acquired image set of the training area, and training change information indicating notable changes in the training area; A change detection system comprising:

[0111] (Appendix 8) The inference unit receives as input inference area information indicating the target area within the inference target area, which is the attention area, and an inference image set, which is the corrected image set corresponding to the acquired image set of the inference target area, and calculates the trained model to obtain inference change information indicating a notable change in the inference target area. 8. The change detection system of claim 7.

[0112] (Appendix 9) a map acquisition unit that acquires map data of the area of ​​interest from a map database that stores map data representing a planar digital map composed of two-dimensional coordinate sequences or a 3D digital map composed of three-dimensional coordinate sequences; 9. The change detection system of claim 7 or 8.

[0113] (Appendix 10) The acquired image set is archive data obtained by photographing at the same time as the map represented by the map data. 10. The change detection system of any one of claims 7 to 9.

[0114] (Appendix 11) Identifying an area where a target feature exists as a target area by referring to map data of the area of ​​interest; generating a set of map-projected images by performing map projection on a set of acquired images obtained by photographing the area of ​​interest at different times; correcting disturbances reflected in each map projection image of the map projection image set to generate a corrected image set; A trained model is generated by performing training using training area information indicating the target area within the training target area, which is the attention area, a training image set which is the corrected image set corresponding to the acquired image set of the training target area, and training change information indicating notable changes in the training target area as training data. Change detection methods.

[0115] (Appendix 12) an area specifying unit that refers to map data of the area of ​​interest and specifies an area where a target feature exists as a target area; a map projection unit that performs map projection on a set of acquired images obtained by photographing the area of ​​interest at different times to generate a set of map projected images; a disturbance correction unit that corrects disturbances reflected in each map projection image of the map projection image set to generate a corrected image set; a model generation unit that generates a trained model by performing training using as training data training area information indicating the target area within the training target area that is the attention area, a training image set that is the corrected image set corresponding to the acquired image set of the training target area, and training change information that indicates a notable change in the training target area; Change detection programs to make computers function.

[0116] (Appendix 13) an area specifying unit that refers to map data of the area of ​​interest and specifies an area where a target feature exists as a target area; a map projection unit that performs map projection on a set of acquired images obtained by photographing the area of ​​interest at different times to generate a set of map projected images; an area masking unit that generates a masked image set by masking an area other than the target area in each map projection image of the map projection image set; a model generation unit that generates a trained model by performing training using as training data a training image set that is the masked image set corresponding to the acquired image set of the training target area that is the attention area, and training change information that indicates notable changes in the training target area; A change detection system comprising:

[0117] (Appendix 14) an inference unit that obtains inference change information indicating a noteworthy change in the inference target area by inputting an inference image set that is the masked image set corresponding to the acquired image set of the inference target area that is the attention area and calculating the trained model; 14. The change detection system of claim 13.

[0118] (Appendix 15) a map acquisition unit that acquires map data of the area of ​​interest from a map database that stores map data representing a planar digital map composed of two-dimensional coordinate sequences or a 3D digital map composed of three-dimensional coordinate sequences; 15. The change detection system of claim 13 or 14.

[0119] (Appendix 16) The acquired image set is archive data obtained by photographing at the same time as the map represented by the map data. 16. The change detection system of any one of claims 13 to 15.

[0120] (Appendix 17) Identifying an area where a target feature exists as a target area by referring to map data of the area of ​​interest; generating a set of map-projected images by performing map projection on a set of acquired images obtained by photographing the area of ​​interest at different times; masking a region other than the target region in each map projection image of the set of map projection images to generate a set of masked images; A trained model is generated by performing training using as training data a training image set, which is the masked image set corresponding to the acquired image set of the training target area, which is the attention area, and training change information indicating notable changes in the training target area. Change detection methods.

[0121] (Appendix 18) an area specifying unit that refers to map data of the area of ​​interest and specifies an area where a target feature exists as a target area; a map projection unit that performs map projection on a set of acquired images obtained by photographing the area of ​​interest at different times to generate a set of map projected images; an area masking unit that generates a masked image set by masking an area other than the target area in each map projection image of the map projection image set; A change detection program for causing a computer to function as a model generation unit that generates a trained model by performing training using as training data a training image set, which is a masked image set corresponding to the acquired image set of the training target area, which is the attention area, and training change information that indicates noteworthy changes in the training target area. [Explanation of symbols]

[0122] 100 Change detection system, 110 Map database, 111 Map data, 112 Target feature, 113 Target area, 119 Target area information, 120 Acquired image set, 130 Map projection image set, 131 Map projection image, 140 Training data, 141 Training area information, 142 Training image set, 143 Trained change information, 149 Trained model, 150 Inference data, 151 Inference area information, 152 Inference image set, 153 Inference change information, 160 Corrected image set, 170 Masked image set, 200 Input generation device, 201 Processor, 202 Memory, 203 Auxiliary storage device, 204 Communication device, 205 Input / output interface, 211 Map acquisition unit, 212 Area identification unit, 221 Map projection unit, 222 Disturbance correction unit, 223 Area masking unit, 290 Memory unit, 300 learning device, 301 processor, 302 memory, 303 auxiliary storage device, 304 communication device, 305 input / output interface, 311 data reception unit, 312 model generation unit, 390 memory unit, 400 inference device, 401 processor, 402 memory, 403 auxiliary storage device, 404 communication device, 405 input / output interface, 411 data reception unit, 412 inference unit, 490 memory unit.

Claims

1. an area specifying unit that refers to map data of the area of ​​interest and specifies an area where a target feature exists as a target area; a map projection unit that performs map projection on a set of acquired images obtained by photographing the area of ​​interest at different times to generate a set of map projected images; a model generation unit that generates a trained model by performing training using, as training data, training area information indicating the target area within the training area that is the attention area, a training image set that is the map projection image set corresponding to the acquired image set of the training area, and training change information indicating notable changes in the training area; A change detection system comprising:

2. The inference unit receives as input inference area information indicating the target area within the inference target area, which is the attention area, and an inference image set, which is the map projection image set corresponding to the acquired image set of the inference target area, and calculates the trained model to obtain inference change information indicating noteworthy changes in the inference target area. The change detection system of claim 1 .

3. a map acquisition unit that acquires map data of the area of ​​interest from a map database that stores map data representing a planar digital map composed of two-dimensional coordinate sequences or a 3D digital map composed of three-dimensional coordinate sequences; The change detection system according to claim 1 or 2.

4. The acquired image set is archive data obtained by photographing at the same time as the map represented by the map data. The change detection system according to claim 1 or 2.

5. A change detection system comprising: Identifying an area where a target feature exists as a target area by referring to map data of the area of ​​interest; generating a set of map-projected images by performing map projection on a set of acquired images obtained by photographing the area of ​​interest at different times; A trained model is generated by performing training using training area information indicating the target area within the training area, which is the attention area, a training image set which is the map projection image set corresponding to the acquired image set of the training area, and training change information indicating notable changes in the training area as training data. Change detection methods.

6. an area specifying unit that refers to map data of the area of ​​interest and specifies an area where a target feature exists as a target area; a map projection unit that performs map projection on a set of acquired images obtained by photographing the area of ​​interest at different times to generate a set of map projected images; a model generation unit that generates a trained model by performing training using as training data training area information indicating the target area within the training target area that is the attention area, a training image set that is the map projection image set corresponding to the acquired image set of the training target area, and training change information that indicates notable changes in the training target area; Change detection programs to make computers function.

7. an area specifying unit that refers to map data of the area of ​​interest and specifies an area where a target feature exists as a target area; a map projection unit that performs map projection on a set of acquired images obtained by photographing the area of ​​interest at different times to generate a set of map projected images; a disturbance correction unit that corrects disturbances reflected in each map projection image of the map projection image set to generate a corrected image set; a model generation unit that generates a trained model by performing training using, as training data, training area information indicating the target area within the training area that is the attention area, a training image set that is the corrected image set corresponding to the acquired image set of the training area, and training change information indicating notable changes in the training area; A change detection system comprising:

8. The inference unit receives as input inference area information indicating the target area within the inference target area, which is the attention area, and an inference image set, which is the corrected image set corresponding to the acquired image set of the inference target area, and calculates the trained model to obtain inference change information indicating a notable change in the inference target area. The change detection system of claim 7 .

9. a map acquisition unit that acquires map data of the area of ​​interest from a map database that stores map data representing a planar digital map composed of two-dimensional coordinate sequences or a 3D digital map composed of three-dimensional coordinate sequences; The change detection system according to claim 7 or 8.

10. The acquired image set is archive data obtained by photographing at the same time as the map represented by the map data. The change detection system according to claim 7 or 8.

11. A change detection system comprising: Identifying an area where a target feature exists as a target area by referring to map data of the area of ​​interest; generating a set of map-projected images by performing map projection on a set of acquired images obtained by photographing the area of ​​interest at different times; correcting disturbances reflected in each map projection image of the map projection image set to generate a corrected image set; A trained model is generated by performing training using training area information indicating the target area within the training target area, which is the attention area, a training image set which is the corrected image set corresponding to the acquired image set of the training target area, and training change information indicating notable changes in the training target area as training data. Change detection methods.

12. an area specifying unit that refers to map data of the area of ​​interest and specifies an area where a target feature exists as a target area; a map projection unit that performs map projection on a set of acquired images obtained by photographing the area of ​​interest at different times to generate a set of map projected images; a disturbance correction unit that corrects disturbances reflected in each map projection image of the map projection image set to generate a corrected image set; a model generation unit that generates a trained model by performing training using as training data training area information indicating the target area within the training target area that is the attention area, a training image set that is the corrected image set corresponding to the acquired image set of the training target area, and training change information that indicates a notable change in the training target area; Change detection programs to make computers function.

13. an area specifying unit that refers to map data of the area of ​​interest and specifies an area where a target feature exists as a target area; a map projection unit that performs map projection on a set of acquired images obtained by photographing the area of ​​interest at different times to generate a set of map projected images; an area masking unit that generates a masked image set by masking an area other than the target area in each map projection image of the map projection image set; a model generation unit that generates a trained model by performing training using as training data a training image set that is the masked image set corresponding to the acquired image set of the training target area that is the attention area, and training change information that indicates notable changes in the training target area; A change detection system comprising:

14. an inference unit that obtains inference change information indicating a noteworthy change in the inference target area by inputting an inference image set that is the masked image set corresponding to the acquired image set of the inference target area that is the attention area and calculating the trained model; The change detection system of claim 13.

15. a map acquisition unit that acquires map data of the area of ​​interest from a map database that stores map data representing a planar digital map composed of two-dimensional coordinate sequences or a 3D digital map composed of three-dimensional coordinate sequences; A change detection system according to claim 13 or claim 14.

16. The acquired image set is archive data obtained by photographing at the same time as the map represented by the map data. A change detection system according to claim 13 or claim 14.

17. A change detection system comprising: Identifying an area where a target feature exists as a target area by referring to map data of the area of ​​interest; generating a set of map-projected images by performing map projection on a set of acquired images obtained by photographing the area of ​​interest at different times; masking a region other than the target region in each map projection image of the set of map projection images to generate a set of masked images; A trained model is generated by performing training using as training data a training image set, which is the masked image set corresponding to the acquired image set of the training target area, which is the attention area, and training change information indicating notable changes in the training target area. Change detection methods.

18. an area specifying unit that refers to map data of the area of ​​interest and specifies an area where a target feature exists as a target area; a map projection unit that performs map projection on a set of acquired images obtained by photographing the area of ​​interest at different times to generate a set of map projected images; an area masking unit that generates a masked image set by masking an area other than the target area in each map projection image of the map projection image set; A change detection program for causing a computer to function as a model generation unit that generates a trained model by performing training using as training data a training image set, which is a masked image set corresponding to the acquired image set of the training target area, which is the attention area, and training change information that indicates noteworthy changes in the training target area.

Images