Terrain condition change interpretation device, terrain condition change interpretation method, and computer program

The terrain state change interpretation device uses a two-stage learning model to process single-point-in-time remote sensing images, enhancing accuracy and speed in detecting terrain changes, addressing the challenges of delayed responses and inaccurate detection in existing systems.

JP7954261B1Active Publication Date: 2026-11-19PASCO CORP
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2025161828
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2025-09-29
Publication Date
2026-11-19
Estimated Expiration
2045-09-29

AI Technical Summary

Technical Problem

Existing terrain state change interpretation devices struggle to accurately and quickly interpret changes in terrain conditions, such as embankments and landslides, especially during disasters, due to the difficulty in acquiring and processing images from multiple time points, which can lead to delayed emergency responses and inaccurate detection.

Method used

A terrain state change interpretation device that utilizes a two-stage learning model to process remote sensing images captured at a single point in time, including a first learning model for improved recall and a second learning model for enhanced precision, enabling accurate detection of terrain changes.

Benefits of technology

The device can accurately interpret terrain changes with high precision and recall, reducing the time required for analysis and minimizing false detections, even when images from before the disaster are unavailable.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007954261000001_ABST
    Figure 0007954261000001_ABST
Patent Text Reader

Abstract

This invention provides a terrain state change interpretation device, a terrain state change interpretation method, and a computer program that enable more accurate interpretation of changes in terrain conditions from remote sensing images of terrain captured at a single point in time. [Solution] The terrain state change interpretation device includes an acquisition unit that acquires a remote sensing image of the terrain captured at a single point in time, and an interpretation unit that inputs an image based on the remote sensing image to a learning model that has been trained to output information about changes in the terrain state when an image is input using a training image generated by cutting out a predetermined area from an image of the terrain captured at a single point in time and upsampling it, and interprets the changes in the terrain state based on the information output from the learning model.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a terrain state change interpretation device, a terrain state change interpretation method, and a computer program.

Background Art

[0002] In recent years, disasters caused by unauthorized embankments have occurred, but many of the current monitoring activities by disaster prevention-related organizations such as local governments to detect and prevent sediment-related disasters in advance still rely on on-site patrols performed manually by local government staff and reports from residents, which has limitations in wide-area and continuous monitoring. In addition, unauthorized land modification and large-scale deforestation are factors that undermine ground stability, and there is a risk of inducing slope failures and landslides during heavy rainfall or earthquakes. For this reason, disaster prevention-related organizations such as local governments are required to establish a wide-area and continuous monitoring system that covers land modification and landslides in addition to embankment monitoring.

[0003] On the other hand, by utilizing artificial satellite remote sensing technology, it is possible to grasp the status of artificial embankments, land modifications, landslide sites at the time of disaster occurrence, and the like over a wide area on a regular basis. By making use of these technologies, it is possible to detect land modification sites at an early stage, grasp dangerous sites in advance, enable rapid response when a disaster occurs, and it is expected to reduce the labor required for daily on-site patrols by staff.

[0004] Patent Document 1 discloses an abnormality detection system in which an information processing device and an aerial vehicle can communicate with each other via communication means. This information processing device acquires an inspection difference image that is the difference between a reference orthoimage generated from a reference image obtained by photographing a target object, and a comparative orthoimage generated from a comparative image obtained by photographing the same target object at a different time point. The information processing device inputs the inspection difference image into a supervised model that has been pre-trained using pairs of a learning difference image, which is the difference between a learning reference orthoimage and a learning comparative orthoimage, and a label indicating an abnormality of the target object appearing in the learning difference image as training data, and determines whether an abnormality has occurred in the target object.

Prior Art Documents

[0005] [Patent Document 1] Japanese Patent Publication No. 2025-26181 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] In terrain state change interpretation devices that use remote sensing images to interpret changes in terrain due to embankments, landslides, land alteration, etc., there is a particular need to interpret terrain state changes more quickly and accurately, especially during disasters. However, even if one attempts to interpret terrain state changes by comparing images at two points in time, before and after a disaster, it is difficult to acquire images of all locations in advance, and there may be cases where images from before the disaster cannot be obtained when a disaster occurs. Furthermore, even if past images exist, the process of searching for and retrieving past images takes time, which may delay emergency response. On the other hand, it is theoretically possible to interpret terrain state changes using an image taken at a single point in time after a disaster, but in that case, the accuracy of interpreting terrain state changes is low, and there is a possibility of false detection or oversight. In particular, in areas where large-scale earthquakes or other disasters have occurred and landslides have occurred in many places, both "accuracy" and "speed" are required for identifying damaged areas in order to take early disaster countermeasures.

[0007] The present invention aims to provide a terrain state change interpretation device, a terrain state change interpretation method, and a computer program that can more accurately interpret changes in terrain conditions from remote sensing images in which terrain is captured at a single point in time. [Means for solving the problem]

[0008] The terrain state change interpretation device according to the present invention includes an acquisition unit that acquires a remote sensing image of terrain captured at a single point in time; an interpretation unit that inputs an image based on the remote sensing image to a learning model that has been trained to output information on terrain state changes when an image is input, using a training image generated by cutting out a predetermined area from an image of terrain captured at a single point in time and upsampling it, and interprets the terrain state change based on the information output from the learning model; and an output unit that outputs information regarding the interpretation result of the terrain state change.

[0009] Furthermore, in the terrain state change interpretation device according to the present invention, it is preferable that the interpretation unit inputs an image generated by upsampling a predetermined area from a remote sensing image as an image based on the remote sensing image into the learning model.

[0010] Furthermore, in the terrain state change interpretation device according to the present invention, it is preferable that the learning model learns images of regions extracted from a plurality of regions containing terrain state changes within an image of terrain, based on the degree to which each region contains terrain state changes, as learning images.

[0011] Furthermore, in the terrain state change interpretation device according to the present invention, it is preferable that the interpretation unit inputs an image containing candidate regions of terrain state change extracted based on information output from the learning model to a second learning model that has been trained to output the degree to which terrain state change is not included in an image when an image is input, using a second learning image that includes terrain state change, and interprets the change in terrain state based on the degree output from the second learning model.

[0012] Furthermore, in the terrain state change interpretation device according to the present invention, it is preferable that the learning model is trained using an image in which any two of the red band, green band, and blue band are swapped with each other in an image of terrain.

[0013] Furthermore, the terrain state change interpretation method according to the present invention acquires a remote sensing image of terrain captured at a single point in time, and uses a training image generated by upsampling a predetermined region from the image of terrain captured at a single point in time. The training model is trained to output information about terrain state changes when an image is input, and the image based on the remote sensing image is input to the training model. Based on the information output from the training model, the terrain state changes are interpreted, and information regarding the interpretation results of the terrain state changes is output.

[0014] Furthermore, the computer program according to the present invention acquires a remote sensing image of terrain captured at a single point in time, and uses a training image generated by cutting out a predetermined region from the image of terrain captured at a single point in time and upsampling it, to input an image based on the remote sensing image into a learning model that has been trained to output information about changes in terrain state when an image is input, and causes the computer to interpret changes in terrain state based on the information output from the learning model and output information about the result of interpreting the changes in terrain state. [Effects of the Invention]

[0015] The terrain condition change interpretation device, terrain condition change interpretation method, and computer program according to the present invention can interpret changes in terrain conditions with greater accuracy, even from remote sensing images in which terrain was captured at a single point in time. [Brief explanation of the drawing]

[0016] [Figure 1] This is a diagram illustrating an example of the configuration of image processing system 1. [Figure 2] This is a flowchart showing an example of the flow of the first learning process. [Figure 3] (A) is a schematic diagram to explain the histogram, and (B) is a schematic diagram to explain the learning domain. [Figure 4] This graph is intended to explain the technical significance. [Figure 5] This is a flowchart showing an example of the flow of the second learning process. [Figure 6] This is a flowchart showing an example of the flow of the interpretation process. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0017] Hereinafter, various embodiments of the present invention will be described with reference to the drawings. It should be noted that the technical scope of the present invention is not limited to these embodiments, and extends to the invention described in the claims and equivalents thereof.

[0018] Figure 1 is a configuration diagram showing an example of an image processing system 1 according to the present invention.

[0019] The image processing system 1 includes a terrain state change interpretation device 100 and a server device 200. The terrain state change interpretation device 100 and the server device 200 are examples of image processing devices, and are communicatively connected to each other via a network N. The network N is an intranet, the Internet, or the like.

[0020] The terrain state change interpretation device 100 is a personal computer, a laptop personal computer, a server, or the like. The terrain state change interpretation device 100 includes an operation device 101, a display device 102, a communication device 103, a storage device 110, a processing circuit 120, and the like.

[0021] The operation device 101 includes input devices such as a keyboard, a mouse, and a microphone, and an interface circuit that acquires signals from the input devices, receives operations from a user, and outputs signals corresponding to the user's inputs to the processing circuit 120.

[0022] The display device 102 is an example of an output unit. The display device 102 includes a display composed of a liquid crystal, an organic EL, or the like, and an interface circuit that outputs image data to the display, and displays the image data on the display in accordance with an instruction from the processing circuit 120.

[0023] The communication device 103 is an example of an output unit. The communication device 103 is equipped with a wired or wireless communication interface circuit and connects the terrain condition change interpretation device 100 to a communication network. The communication device 103 performs wired communication according to a communication protocol such as TCP / IP (Transmission Control Protocol / Internet Protocol). Alternatively, the communication device 103 may perform wireless communication according to the IEEE (Institute of Electrical and Electronics Engineers) 802.11 standard. The communication device 103 transmits information supplied from the processing circuit 120 to an external device. The communication device 103 also supplies information received from the external device to the processing circuit 120.

[0024] The storage device 110 includes, for example, semiconductor memory such as RAM (Random Access Memory) or ROM (Read Only Memory), a fixed disk device such as a hard disk, or a portable storage device such as an optical disc. The storage device 110 stores computer programs, data, etc., used for processing by the processing circuit 120. The computer program is installed in the storage device 110 from a server (not shown) via a communication device 103. Alternatively, the computer program may be installed in the storage device 110 from a computer-readable portable recording medium using a known setup program, etc. The portable recording medium is, for example, a CD-ROM or DVD-ROM. The computer program may also be distributed from a server or the like and installed in the storage device 110.

[0025] Furthermore, the storage device 110 stores a first learning model 111 and a second learning model 112. The first learning model 111 is an example of a learning model and outputs information regarding changes in the state of the terrain when an image of the terrain captured at a single point in time is input. Changes in the state of the terrain include embankments, landslides, land alteration, etc. Land alteration includes the construction or demolition of buildings, illegal dumping of objects, large-scale logging, conversion of agricultural land, etc.

[0026] The processing circuit 120 is, for example, a CPU (Central Processing Unit). The processing circuit 120 may also be an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), DSP (Digital Signal Processor), FPGA (Field Programmable Gate Array), etc. The processing circuit 120 is connected to the operating device 101, display device 102, communication device 103, and storage device 110, etc., and controls each of these parts. The processing circuit 120 reads the program stored in the storage device 110 and operates according to the read program, thereby functioning as a learning unit 121, acquisition unit 122, interpretation unit 123, and output control unit 124. The processing circuit 120 generates a first learning model 111 and a second learning model 112, and uses the first learning model 111 and the second learning model 112 to interpret changes in the terrain state from remote sensing images of the terrain captured at a single point in time.

[0027] Figure 2 is a flowchart showing an example of the flow of the first learning process performed by the terrain state change interpretation device 100.

[0028] The following describes an example of the operation of the first learning process of the terrain state change interpretation device 100, referring to the flowchart shown in Figure 2. The operation flow described below is mainly executed by the processing circuit 120 in cooperation with each element of the terrain state change interpretation device 100, based on a program that is pre-stored in the memory device 110.

[0029] First, the learning unit 121 acquires a base image for training (step S101). The base image for training is an aerial photograph, satellite image (or orthophoto based on an aerial photograph or satellite image) of the terrain taken from above at a single point in time. The terrain includes features such as buildings, trees, rocks, embankments, and soil. An image taken at a single point in time conceptually includes multiple images taken within a certain time interval, and specifically includes multiple images taken at different times, that is, multiple images taken discretely (discontinuously) with a predetermined time interval (frame interval) between them. Furthermore, an image taken at a single point in time may include images of different types (aerial photographs, satellite images, etc.), images taken by different imaging devices (imaging sensors), or images taken at different locations. The base image for training has a size greater than or equal to the size (patch size) of the image input to the first learning model 111, which will be described later.

[0030] The learning unit 121 acquires polygon information indicating the position of the outline (outline) of each change in the terrain to be interpreted (embankment, landslide, altered land, etc.) contained in the learning base image, along with the learning base image. The polygon information is created in advance by the operator of the terrain change interpretation device 100. The learning unit 121 acquires the learning base image and polygon information by receiving them from the server device 200 via the communication device 103. Alternatively, the learning base image and polygon information may be stored in the storage device 110 in advance, and the learning unit 121 may acquire them by reading the learning base image and change information from the storage device 110.

[0031] Next, the learning unit 121 extracts a learning region from the learning base image (step S102). Based on the polygon information, the learning unit 121 identifies a state change region for each state change included in the learning base image, by enlarging each state change or the bounding rectangle of each state change by a predetermined number of pixels. Within each identified state change region, the learning unit 121 sets multiple first sub-regions of a first predetermined size, shifting their positions by one pixel each in the mutually orthogonal X and Y directions (e.g., horizontal and vertical directions). The first predetermined size is set to be less than or equal to the size of the image (patch size) input to the first learning model 111. By training the first learning model 111 using a region smaller than the patch size input to the first learning model 111, the learning unit 121 can generate a first learning model 111 that can efficiently extract small state changes.

[0032] The learning unit 121 extracts the first subregion based on polygon information, rather than randomly extracting the first subregion from the training base image. This allows for more efficient extraction of regions containing state changes as the first subregion.

[0033] Next, the learning unit 121 calculates the degree to which each set first sub-region contains a change in the state of the terrain to be interpreted (hereinafter sometimes referred to as the degree of state change). For example, based on polygon information, the learning unit 121 calculates the degree of state change as the ratio of the number of pixels (area) that show a change in state within each first sub-region to the total number of pixels (area) of each first sub-region, i.e., the GT (Ground Truth) ratio. Note that the degree of state change for each first sub-region may be calculated by an operator or the like and set in the terrain state change interpretation device 100. Next, the learning unit 121 generates a histogram for each state change region, with the degree of state change as a class and the number of first sub-regions belonging to each class as the frequency.

[0034] Figure 3(A) is a schematic diagram illustrating histograms. The horizontal axis of Figure 3(A) represents the histogram class (degree of state change), and the vertical axis represents the histogram frequency (number of first subregions corresponding to each degree of state change). When the area of ​​state change contained in each state change region is large, many pixels within each first subregion are occupied by state changes. Therefore, as shown in Figure 3(A), the higher the degree of state change, the greater the number of first subregions corresponding to that degree of state change, and the lower the degree of state change, the fewer the number of first subregions corresponding to that degree of state change tend to be.

[0035] Next, the learning unit 121 extracts learning regions from the set first subregions for each state change region. The learning unit 121 extracts each learning region so that the degree of state change in each learning region is evenly distributed for each state change region. As mentioned above, the higher the degree of state change, the greater the number of first subregions corresponding to that degree of state change, and the lower the degree of state change, the fewer the number of first subregions corresponding to that degree of state change. Therefore, the learning unit 121 extracts learning regions so that there is no bias towards regions with a high degree of state change, and a certain number of regions with a low degree of state change are also secured. The learning unit 121 may also extract learning regions so that a relatively large number of first subregions with a low degree of state change are extracted. For example, the learning unit 121 determines the weight of each class as the reciprocal of the frequency of each class for each bin of the generated histogram, and extracts a number of first sub-regions from the first sub-regions belonging to each class in proportion to the weight of each class as the learning region.

[0036] Figure 3(B) is a schematic diagram illustrating the learning region. In the example shown in Figure 3(B), within the state change region R, which is obtained by expanding the region P1 corresponding to the state change by a predetermined number of pixels, the first sub-region A1 with a low degree of state change, the first sub-region A2 with a high degree of state change, and the first sub-region A3 with a moderate degree of state change are evenly identified as the learning region. Note that the state change region R includes not only the region P1 corresponding to the state change, but also the region P2 corresponding to the state change located in the vicinity of region P1.

[0037] In this way, the learning unit 121 extracts a learning region based on the degree of state change for each of the multiple state change regions that include changes in the state of the terrain within the image of the terrain. This learning region is used to generate a learning image for the first learning model 111 in a process described later. That is, the first learning model 111 learns the images of the regions extracted based on the degree of state change for each of the multiple state change regions that include changes in the state of the terrain within the image of the terrain, as learning images. As a result, the terrain state change interpretation device 100 can generate a learning image that evenly includes regions with a large rate of state change and regions with a small rate of state change. Therefore, the terrain state change interpretation device 100 can generate a first learning model 111 that can appropriately interpret state changes of various sizes, from small to large, and can interpret state changes well.

[0038] Next, the learning unit 121 generates a first training image based on the training region extracted from the training base image (step S103). As described above, the size of the training region is set to be less than or equal to the size of the image (patch size) input to the first training model 111. The learning unit 121 generates the first training image by cutting out the training region from the training base image and upsampling it to the patch size. The learning unit 121 upsamples the training region by increasing the resolution of the training region using known interpolation processes such as bilinear interpolation and bicubic interpolation.

[0039] This first training image is used to train the first training model 111 in the processing described later. Specifically, the first training model 111 is trained using an image generated by upsampling a predetermined region from an image of terrain as the training image.

[0040] Figure 4 is a graph illustrating the technical significance of upsampling the training region. The horizontal axis of Figure 4 shows the number of epochs in the training of the first training model 111, and the vertical axis shows the recall of the generated first training model 111. The number of epochs is the number of times the entire set of first training images is trained. Recall is the recall rate, which is the percentage of images that contain the state change of the target to be interpreted and are determined to contain that state change. Graph G1 shows the recall of a training model that was trained using images obtained by cutting out a training region of the same size as the patch size from the training base image as the first training image. Graph G2 shows the recall of a training model that was trained using images obtained by cutting out a training region smaller than the patch size in the X and Y directions from the training base image and upsampling it to the patch size as the first training image.

[0041] As shown in Figure 4, by reducing the size of the region extracted from the training base image and training using upsampled images, it is possible to further improve the recall of the trained model. In this way, the terrain state change interpretation device 100 can capture detailed information locally by reducing the size of the region extracted from the training base image. Therefore, the terrain state change interpretation device 100 can generate a first trained model 111 that can appropriately interpret small or detailed changes in terrain, and can interpret changes in terrain well. In other words, the terrain state change interpretation device 100 can reduce the omission of terrain changes by improving the recall of the first trained model 111.

[0042] Next, the learning unit 121 adds a first training image based on the generated first training image (step S104). The learning unit 121 adds a new first training image in which any two of the red, green, and blue bands in a randomly selected first training image have been swapped. That is, the learning unit 121 adds a new first training image in which any two of the R, G, and B color values ​​in the first training image have been swapped. For example, if the state change of the object to be interpreted is an embankment or a landslide, the learning unit 121 adds a new first training image in which the red band and the blue band in the first training image have been swapped. In images of terrain captured from above, the blue band tends to contain a large amount of noise and exhibit unexpected correlations. The learning unit 121 can reduce noise by swapping the red band and the blue band in the first training image.

[0043] The learning unit 121 may add an image as a new first learning image in which any two of the red, green, and blue bands of the first learning image have been swapped. The learning unit 121 may also add an image as a new first learning image in which the bands have been swapped for all first learning images. Furthermore, if the learning unit 121 adds an image as a new first learning image in which the bands have been swapped for the first learning image, it may delete the original first learning image, that is, it may replace the original first learning image with the image in which the bands have been swapped.

[0044] This first training image is used to train the first training model 111 in the processing described later. Specifically, the first training model 111 is trained using images in which any two of the red, green, and blue bands in an image of terrain are swapped with each other as training images. This allows the terrain state change interpretation device 100 to increase the number and types of training images, generate a first training model 111 that can appropriately interpret various types of state changes, and thus enable good interpretation of state changes.

[0045] Next, the learning unit 121 generates a first learning model 111 using the first training images (step S105). The first learning model 111 is a model generated using deep learning, such as a neural network. The learning unit 121 pre-trains the first learning model 111 using a first learning dataset that includes multiple combinations of each first training image and the correct information of whether or not each pixel of each first training image contains a polygon representing the change in the state of the terrain to be interpreted. The learning unit 121 pre-trains the first learning model 111 using the first learning dataset so that when an image is input, it outputs information regarding the change in the state of the terrain to be interpreted. For example, the learning unit 121 trains the first learning model 111 so that, as information regarding the change in the state of the terrain to be interpreted, the pixel value of each pixel outputs an output image that shows the degree to which the corresponding pixel of the first training image contains the change in the state of the terrain to be interpreted. For example, the degree to which the terrain being analyzed includes changes in its state is determined by the probability that such changes are included, and a higher value is set as the probability of such changes being included increases.

[0046] The learning unit 121 generates a ground truth image for each first training image, based on the polygon information acquired in step S101, indicating whether or not the pixel value of each pixel contains a change in the terrain state to be interpreted. The ground truth image may be created by an operator or the like and set in the terrain state change interpretation device 100. The learning unit 121 trains the first learning model 111 so that when each first training image is input to the first learning model 111, the output image output from the first learning model 111 approximates the ground truth image. For example, the learning unit 121 trains the first learning model 111 so that the error between the output image and the ground truth image, i.e., the loss function, is minimized. The loss function is, for example, the sum of squared errors of each pixel value of each corresponding pixel, or the sum of the absolute values ​​of the differences. Based on the error between the output image and the ground truth image, the learning unit 121 changes the values ​​of parameters such as weights in the first learning model 111 using backpropagation or the like.

[0047] The learning unit 121 may train the first learning model 111 to output an output image indicating whether or not the corresponding pixel in the first learning image contains the changes in the state of the terrain to be interpreted, as information regarding the changes in the state of the terrain to be interpreted. Alternatively, the learning unit 121 may train the first learning model 111 to output information indicating the degree to which the changes in the state of the terrain to be interpreted are included in the first learning image as a whole, or whether or not the changes in the state of the terrain to be interpreted are included in the first learning image as a whole, as information regarding the changes in the state of the terrain to be interpreted.

[0048] Next, the learning unit 121 stores the generated first learning model 111 in the storage device 110 (step S106), and terminates the first learning process.

[0049] In step S102, the learning unit 121 may randomly select learning areas. Alternatively, the learning unit 121 may select all of the set first sub-areas as learning areas.

[0050] Alternatively, in step S102, the learning unit 121 may extract a training region of the size (patch size) of the image input to the first learning model 111, and in step S103, generate a first learning image without upsampling the training region extracted from the training base image.

[0051] Furthermore, the process in step S104 may be omitted.

[0052] Figure 5 is a flowchart showing an example of the flow of the second learning process performed by the terrain state change interpretation device 100.

[0053] The following describes an example of the operation of the second learning process of the terrain state change interpretation device 100, referring to the flowchart shown in Figure 5. The operation flow described below is mainly executed by the processing circuit 120 in cooperation with each element of the terrain state change interpretation device 100, based on a program pre-stored in the memory device 110.

[0054] First, the learning unit 121 acquires the second training dataset for the second learning model 112 (step S201). The second learning model 112 is a learning model that is generated with an emphasis on improving the precision, which will be described later. The second training dataset contains multiple combinations of second training images, which include polygons (outline) of changes in the state of the terrain to be interpreted and / or polygons other than changes in the state of the terrain to be interpreted, and ground truth information indicating whether or not each second training image contains polygons of changes in the state of the terrain to be interpreted. The learning unit 121 acquires the second training dataset by receiving it from the server device 200 via the communication device 103. Alternatively, the second training dataset may be stored in the storage device 110 beforehand, and the learning unit 121 may acquire the second training dataset by reading it from the storage device 110.

[0055] Next, the learning unit 121 generates a second learning model 112 using the second learning dataset (step S202). The second learning model 112 is a model generated using deep learning, such as a neural network. The learning unit 121 pre-trains the second learning model 112 using the second learning images so that when an image is input, it outputs a first degree, which is the degree to which the image does not contain changes in the terrain state, and a second degree, which is the degree to which the image contains changes in the terrain state.

[0056] The learning unit 121 trains the second learning model 112 so that the information output from the second learning model 112 (first degree and second degree) approximates the correct information corresponding to each second learning image when each second learning image is input to the second learning model 112. For example, the learning unit 121 trains the second learning model 112 so that the error between the information output from the second learning model 112 and the correct information, i.e., the loss function, is minimized. Based on the error between the information output from the second learning model 112 and the correct information, the learning unit 121 changes the values ​​of parameters such as weights in the second learning model 112 using methods such as backpropagation.

[0057] The second learning model 112 is trained using a second learning image that includes polygons (outline) of the terrain state change to be interpreted, and a second learning image that includes polygons other than the terrain state change to be interpreted. By using the second learning model 112, the terrain state change interpretation device 100 can suppress the erroneous determination that an image containing a state change that is not the target of interpretation contains the state change to be interpreted. The learning unit 121 may also train the second learning model 112 to output only one of the first degree and the second degree, and not the other.

[0058] Next, the learning unit 121 stores the generated second learning model 112 in the storage device 110 (step S203), and terminates the second learning process.

[0059] Figure 6 is a flowchart showing an example of the interpretation process performed by the terrain condition change interpretation device 100.

[0060] The following describes an example of the operation of the terrain condition change interpretation device 100, referring to the flowchart shown in Figure 6. The operation flow described below is mainly executed by the processing circuit 120 in cooperation with each element of the terrain condition change interpretation device 100, based on a program pre-stored in the memory device 110.

[0061] First, the acquisition unit 122 acquires a remote sensing image (step S301). The remote sensing image is an aerial photograph, satellite image (or orthophoto based on the aerial photograph or satellite image) of the terrain captured from above at a single point in time. The remote sensing image is used to determine whether or not a change in the state of the object to be interpreted is included within the terrain contained in the image. The acquisition unit 122 acquires the remote sensing image by receiving it from the server device 200 via the communication device 103. Alternatively, the remote sensing image may be stored in advance in the storage device 110, and the acquisition unit 122 may acquire the remote sensing image by reading it from the storage device 110.

[0062] Next, the interpretation unit 123 generates a processed image based on the remote sensing image (step S302). The interpretation unit 123 divides the remote sensing image into second predetermined sizes and identifies all divided regions as second subregions. The second predetermined size is set to a size less than or equal to the size (patch size) of the image input to the first learning model 111. The second predetermined size is set to the same size as the first predetermined size, which is the size of the first subregion set in step S102 of the first learning process. The second predetermined size may be set to a different size from the first predetermined size. The interpretation unit 123 generates a processed image by cutting out each second subregion from the remote sensing image and upsampling it to the patch size. The learning unit 121 upsamples the second subregion by increasing the resolution of the second subregion using known interpolation processes such as bilinear interpolation and bicubic interpolation. The processed image is an example of an image generated by cutting out a predetermined region from the remote sensing image and upsampling it.

[0063] Next, the interpretation unit 123 inputs an image based on the remote sensing image to the first learning model 111 and acquires information output from the first learning model 111 (step S303). For example, the interpretation unit 123 inputs each processed image as an image based on the remote sensing image to the first learning model 111 and acquires the output image output from the first learning model 111. As described above, the first learning model 111 is pre-trained using a first learning image obtained by upsampling a first partial region within the learning base image. By inputting a processed image obtained by upsampling a second partial region within the remote sensing image to the first learning model 111, the interpretation unit 123 can match the state of the image input to the first learning model 111 to the state of the image input to the first learning model 111 during training. Therefore, the terrain state change interpretation device 100 can further improve the accuracy of interpreting terrain state changes using the first learning model 111.

[0064] Next, the interpretation unit 123 generates a candidate region image based on the information output from the first learning model 111 (step S304). The interpretation unit 123 extracts pixels in the remote sensing image whose pixel value is greater than or equal to a predetermined first judgment threshold, as candidate pixels for changes in the state of the terrain to be interpreted. The interpretation unit 123 extracts groups of mutually adjacent candidate pixels in the remote sensing image as candidate regions for changes in the state of the terrain to be interpreted. The interpretation unit 123 generates a third predetermined size image containing the extracted candidate regions as a candidate region image. The interpretation unit 123 generates the candidate region image such that pixels corresponding to the candidate regions become valid pixels and other pixels become invalid pixels. The third predetermined size is set to the same size as the image size (patch size) input to the second learning model 112. The candidate region image is an example of an image containing candidate regions for changes in the state of the terrain to be interpreted, extracted based on the information output from the first learning model 111.

[0065] Next, the interpretation unit 123 inputs the candidate region image to the second learning model 112 and obtains the information output from the second learning model 112, namely the first degree and the second degree (step S305).

[0066] Next, the interpretation unit 123 interprets the changes in terrain state based on the information output from the first learning model 111 and the information output from the second learning model 112 (step S306). If the interpretation unit 123 finds that there are no pixels with a pixel value equal to or greater than the first judgment threshold in the output image output from the first learning model 111, it determines that the area in the remote sensing image corresponding to that output image does not contain the changes in terrain state to be interpreted. On the other hand, if there are pixels in the output image with a pixel value equal to or greater than the first judgment threshold, the interpretation unit 123 determines whether the first degree output from the second learning model 112 is equal to or greater than a predetermined second judgment threshold. If the first degree is equal to or greater than the second judgment threshold, the interpretation unit 123 determines that the area in the remote sensing image corresponding to that output image does not contain the changes in terrain state to be interpreted. On the other hand, if the first degree is less than the second determination threshold, the interpretation unit 123 determines that the region in the remote sensing image corresponding to the output image contains a change in the state of the terrain to be interpreted.

[0067] Furthermore, if the first degree is less than the second judgment threshold, the interpretation unit 123 may determine whether the second degree output from the second learning model 112 is equal to or greater than a predetermined third judgment threshold. In that case, if the second degree is equal to or greater than the third judgment threshold, the interpretation unit 123 determines that the region in the remote sensing image corresponding to the output image contains a change in the state of the terrain being interpreted. On the other hand, if the second degree is less than the third judgment threshold, the interpretation unit 123 determines that the region in the remote sensing image corresponding to the output image does not contain a change in the state of the terrain being interpreted.

[0068] This section describes the case where the first and second degrees are probability values ​​greater than or equal to 0 and less than or equal to 1, the first judgment threshold is 0.6, the second judgment threshold is 0.7, and the third judgment threshold is 0.6. For example, if the maximum pixel value of the output image is 0.4, the interpretation unit 123 determines that the corresponding region does not contain a change in the state of the terrain being interpreted. On the other hand, if the maximum pixel value of the output image is 0.8, the interpretation unit 123 determines whether the first degree output from the second learning model 112 is greater than or equal to the second judgment threshold. For example, if the first degree is 0.8, the interpretation unit 123 determines that the corresponding region does not contain a change in the state of the terrain being interpreted. On the other hand, if the first degree is 0.5, the interpretation unit 123 determines that the corresponding region contains a change in the state of the terrain being interpreted. Alternatively, if the first degree is 0.5, the interpretation unit 123 determines whether the second degree is greater than or equal to the third judgment threshold. In that case, if the second degree is 0.7, the interpretation unit 123 determines that the corresponding area contains a change in the state of the terrain being interpreted. On the other hand, if the second degree is 0.4, the interpretation unit 123 determines that the corresponding area does not contain a change in the state of the terrain being interpreted.

[0069] In this way, the interpretation unit 123 interprets changes in terrain conditions based on the information output from the first learning model 111. This allows the interpretation unit 123 to determine with high accuracy whether or not changes in terrain conditions are included in the remote sensing image.

[0070] Furthermore, the interpretation unit 123 interprets changes in terrain state based on the first degree output from the second learning model 112. As described above, the first degree is the degree to which the image input to the second learning model 112 does not contain changes in the terrain state to be interpreted. By using the first degree, the interpretation unit 123 can suppress the erroneous detection of changes in the terrain state to be interpreted even when the changes in the terrain state to be interpreted are not included. The terrain state change interpretation device 100 can satisfy the required accuracy for both recall and precision by using both the first learning model 111, which is generated to prioritize improving recall, and the second learning model 112, which is generated to prioritize improving precision. Precision is the accuracy rate, which is the proportion of images that are determined to contain changes in the terrain state to be interpreted and actually contain those changes (the proportion of correct answers). Therefore, the terrain state change interpretation device 100 can interpret changes in the terrain state to be interpreted with higher accuracy.

[0071] Furthermore, the interpretation unit 123 interprets changes in terrain state based on the second degree output from the second learning model 112. As described above, the second degree is the degree to which changes in terrain state are included in the image input to the second learning model 112. The interpretation unit 123 can interpret changes in terrain state with higher accuracy by using the second learning model 112 to determine whether or not the object to be interpreted is included in the image as a whole, based on the degree to which the object to be interpreted is included calculated for each pixel in the image using the first learning model 111. In other words, the interpretation unit 123 first extracts changes in terrain state without omission using the first learning model 111 which is generated with an emphasis on improving recall, and then uses the second learning model 112 which is generated with an emphasis on improving precision to detect mis-extractions of changes in terrain state. As a result, the interpretation unit 123 can improve the detection accuracy of small-scale embankments and the like.

[0072] Next, the output control unit 124 outputs information regarding the terrain state change interpretation result by transmitting it to an external device via the communication device 103 or by displaying it on the display device 102 (step S307), and terminates the interpretation process. The output control unit 124 outputs whether or not a change in the state of the target to be interpreted was detected in the remote sensing image as information regarding the terrain state change interpretation result. The output control unit 124 may also output an image highlighting the detected change in the remote sensing image, or information indicating the location of the change in state, as information regarding the interpretation result. In addition, the output control unit 124 may output the first degree and / or second degree output from the second learning model 112 for each candidate region of the terrain state change to be interpreted as information regarding the interpretation result.

[0073] In step S302, the interpretation unit 123 may extract a second subregion of the image size (patch size) input to the first learning model 111, and use the second subregion extracted from the remote sensing image as the processing image without upsampling.

[0074] Furthermore, the processing in steps S304 to S305 may be omitted, and the interpretation unit 123 may interpret the changes in terrain state without using the second learning model 112. In that case, if the interpretation unit 123 finds that there are pixels in the output image output from the first learning model 111 whose pixel values ​​are equal to or greater than the first determination threshold, it determines that the region in the remote sensing image corresponding to that output image contains the changes in terrain state to be interpreted.

[0075] As explained above, the image processing system 1 inputs an image based on a remote sensing image of the terrain captured at a single point in time into a first learning model 111, which is trained using a training image of the terrain captured at a single point in time. Based on the information output from the first learning model 111, the system interprets changes in the terrain state. In particular, the first learning model 111 is trained using an image generated by upsampling a predetermined region extracted from an image of the terrain as the training image. This enables the image processing system 1 to interpret changes in the terrain state more accurately from a remote sensing image of the terrain captured at a single point in time.

[0076] Image processing system 1 can interpret changes in terrain conditions using only an image at a single point in time. This eliminates the need for image processing system 1 to prepare reference images for comparison, significantly expanding the range of terrain condition changes that can be interpreted. Furthermore, since image processing system 1 does not need to search for reference images for comparison, the time required for interpreting changes in terrain conditions can be reduced. In addition, image processing system 1 can interpret changes in terrain conditions such as landslides with high accuracy from only an image at a single point in time, even if no images taken in advance before the disaster occurred.

[0077] Furthermore, the image processing system 1, through a two-stage model processing including a first learning model 111 and a second learning model 112, can suppress false detections of terrain state changes while preventing oversights of terrain changes. Therefore, the image processing system 1 can improve the processing speed of interpreting terrain state changes while ensuring accuracy in interpreting terrain changes. As a result, the image processing system 1 can quickly and accurately extract terrain state changes from an image at a single point in time, contributing to a rapid response system for disaster prevention and mitigation.

[0078] While preferred embodiments have been described above, the embodiments are not limited to these. For example, instead of the terrain condition change interpretation device 100, a server device 200 may have a learning unit 121 and execute the first learning process shown in Figure 2 and / or the second learning process shown in Figure 5 to generate the first learning model 111 and / or the second learning model 112.

[0079] Alternatively, instead of the terrain state change interpretation device 100, the server device 200 may store the first learning model 111 and / or the second learning model 112. In that case, in step S303 of Figure 6, the output control unit 124 transmits the processed image to the server device 200 via the communication device 103. The server device 200 inputs the processed image received from the terrain state change interpretation device 100 into the first learning model 111 and transmits the information output from the first learning model 111 to the terrain state change interpretation device 100. The output control unit 124 acquires the information output from the first learning model 111 by receiving it from the server device 200 via the communication device 103. Similarly, in step S305 of Figure 6, the output control unit 124 transmits the region candidate image to the server device 200 via the communication device 103. The server device 200 inputs the candidate region images received from the terrain state change interpretation device 100 into the second learning model 112 and transmits the information output from the second learning model 112 to the terrain state change interpretation device 100. The output control unit 124 acquires the information output from the second learning model 112 by receiving it from the server device 200 via the communication device 103.

[0080] Furthermore, the image processing system 1 may interpret changes in the state of the terrain using images of the terrain captured at two different points in time. For example, the learning unit 121 pre-trains the first learning model 111 using a learning dataset that includes multiple combinations of a first learning image containing a specific terrain at a first point in time, a first learning image containing the same specific terrain at a second point in time after the first point in time, and correct answer information. The learning unit 121 trains the first learning model 111 so that, when an image from the first point in time and an image from the second point in time are input, the pixel value of each pixel outputs an output image that shows the degree to which the corresponding pixel in the image from the first point in time does not contain changes in the state of the terrain to be interpreted, and to which the corresponding pixel in the image from the second point in time does contain changes in the state of the terrain to be interpreted. Here, the first point in time is earlier than the second point in time, and the second point in time is later than the first point in time. Meanwhile, the interpretation unit 123 inputs an image based on a remote sensing image containing a specific terrain at a first time point and an image based on a remote sensing image containing that specific terrain at a second time point to the first learning model 111, and obtains an output image from the first learning model 111. Based on the obtained output image, the interpretation unit 123 extracts candidate areas for terrain state changes. The interpretation unit 123 may then input the extracted candidate areas for terrain state changes to the second learning model 112. The second learning model 112 outputs the degree to which the input candidate areas for terrain state changes do not contain terrain state changes (first degree) and the degree to which terrain state changes contain terrain state changes (second degree). Based on these degrees, the interpretation unit 123 can interpret terrain state changes based on the comparison of images from two time points with higher accuracy while suppressing false detections.

[0081] Those skilled in the art will understand that various changes, substitutions, and modifications can be made without departing from the spirit and scope of the present invention. For example, the embodiments and modifications described above may be combined as appropriate within the scope of the invention. [Explanation of symbols]

[0082] 1 Image processing system, 100 Terrain condition change interpretation device, 111 First learning model, 112 Second learning model, 121 Learning unit, 122 Acquisition unit, 123 Interpretation unit, 124 Output control unit

Claims

1. An acquisition unit that acquires remote sensing images of terrain captured at a single point in time, An interpretation unit interprets changes in terrain based on information output from the first learning model, which is trained to output information indicating the degree to which each pixel in an image contains changes in terrain to be interpreted when an image is input to a first learning model that is trained to output information indicating the degree to which changes in terrain to be interpreted are included in the image when an image is input to a first learning model that is trained to output information indicating the degree to which changes in terrain to be interpreted are included in the image when an image is input to a first learning model, which each pixel in the image, and interprets changes in terrain based on the information output from the first learning model, An output unit that outputs information output from a second learning model in response to inputs such as information indicating whether or not a change in the terrain state to be interpreted has been detected in the remote sensing image, an image highlighting the detected change in the remote sensing image, information indicating the location of the change in terrain, or an image containing a candidate region of the change in terrain state extracted based on the information output from the first learning model. A terrain condition change interpretation device characterized by having the following features.

2. The terrain state change interpretation device according to claim 1, wherein the interpretation unit inputs an image generated by upsampling a predetermined area from the remote sensing image as an image based on the remote sensing image to the first learning model.

3. The terrain state change interpretation device according to claim 1 or 2, wherein the first learning model learns images of regions extracted based on the ratio of the number of pixels indicating a state change within each region to the number of pixels in each region, for each of a plurality of regions containing changes in the state of the terrain in an image of the terrain, as the learning images.

4. The terrain state change interpretation device according to claim 1 or 2, wherein the interpretation unit uses a second learning image that includes changes in terrain state or a second learning image that includes shapes other than changes in terrain state, and correct information on whether or not each second learning image includes changes in terrain state, to input an image that includes candidate regions of changes in terrain state extracted based on information output from the first learning model to the second learning model, which has been trained so that the information output when a second learning image is input approximates the corresponding correct information, and interprets the changes in terrain state based on the degree output from the second learning model.

5. The terrain state change interpretation device according to claim 1 or 2, wherein the first learning model is trained to use an image obtained by swapping any two of the red band, green band, and blue band in an image of terrain as the learning image.

6. By acquiring remote sensing images in which the terrain is captured at a single point in time, Based on pre-created polygon information indicating the location of changes in terrain state contained in an image of terrain captured at a single point in time, a training image is generated by cutting out a predetermined region extracted based on the degree of change in state within an enlarged region corresponding to the change in terrain state in the image and upsampling it. This training image is used to input an image based on the remote sensing image to a first learning model that has been trained to output information indicating the degree to which each pixel in the image contains the change in terrain state to be interpreted. Based on the information output from the first learning model, the change in terrain state is interpreted. The system outputs information indicating whether or not a change in the terrain state to be interpreted has been detected in the remote sensing image, an image highlighting the detected change in the remote sensing image, information indicating the location of the change in terrain, or an image containing a candidate region of the change in terrain state extracted based on the information output from the first learning model. A method for interpreting changes in topographic conditions, characterized by the features described above.

7. By acquiring remote sensing images in which the terrain is captured at a single point in time, Based on pre-created polygon information indicating the location of changes in terrain state contained in an image of terrain captured at a single point in time, a training image is generated by cutting out a predetermined region extracted based on the degree of change in state within an enlarged region corresponding to the change in terrain state in the image and upsampling it. This training image is used to input an image based on the remote sensing image to a first learning model that has been trained to output information indicating the degree to which each pixel in the image contains the change in terrain state to be interpreted. Based on the information output from the first learning model, the change in terrain state is interpreted. The system outputs information indicating whether or not a change in the terrain state to be interpreted has been detected in the remote sensing image, an image highlighting the detected change in the remote sensing image, information indicating the location of the change in terrain, or an image containing a candidate region of the change in terrain state extracted based on the information output from the first learning model. A computer program characterized by causing a computer to perform a certain action.

Citation Information

Patent Citations

  • Transmission line geological disaster identification method, system and device and medium

    CN120747575A

  • Information processing apparatus, information processing method, and information processing program

    JP2025026181A