Targeted living system
The object extraction system uses machine learning on tagged aerial photographs to generate a universal model for extracting geographic features, reducing costs and errors by leveraging regional characteristics.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- KOKUSAI IND
- Filing Date
- 2022-10-18
- Publication Date
- 2026-06-05
AI Technical Summary
Existing methods for extracting geographic features from aerial photographs require extensive human effort and are prone to errors, and creating region-specific training data for machine learning is costly and inefficient.
An object extraction system that uses aerial photographs tagged with regional characteristic information to generate a trained model through machine learning, allowing inference across diverse regions with a small amount of training data.
Reduces the cost of creating learning data and eliminates the need for region-specific models, enabling accurate extraction of target objects like buildings across varied areas.
Smart Images
Figure 0007870705000001 
Figure 0007870705000002 
Figure 0007870705000003
Abstract
Description
Technical Field
[0005]
[0001] The invention of the present application relates to a technique for extracting a target object from an image obtained by photographing the ground surface. More specifically, it relates to a target object extraction system that can automatically extract a target object included in an image in consideration of the characteristics of the area.
Background Art
[0002] When creating a map, classifying terrain, or classifying land use, aerial photographs are used. Specifically, by interpreting aerial photographs, buildings, farmland, roads, rivers, forests, etc. are read to create a map or classify terrain and land use.
[0003] An aerial photograph, also called an air photo, is an image taken from above to faithfully reproduce the state on the ground. When acquiring an aerial photograph, it can be taken from above using an airplane or helicopter, taken from a satellite, or taken using a crane truck or balloon. In cases for map creation and the like, a vertical photograph taken vertically downward by a dedicated camera is acquired. Since land use changes year by year, aerial photographs of the same area are acquired regularly. For example, the Geospatial Information Authority of Japan takes photographs of the plains every 5 to 10 years.
[0004] As described above, various ground features such as buildings and farmland are extracted by interpreting aerial photographs. Conventionally, this photo interpretation has mainly been performed visually by an operator. However, aerial photographs cover a wide area, and visual photo interpretation is a considerably burdensome task for the operator, and it has been difficult to completely eliminate so-called human errors such as misinterpretation.
[0005] Therefore, in recent years, it has become increasingly common to use image recognition technology to sort out the types of geographic features. In other words, a computer performs software-related processing to automatically extract the types of geographic features from aerial photographs. Sometimes the automatically extracted geographic feature types are treated as the final decision, and sometimes the results are used as a preliminary judgment (i.e., as a screening process) before the operator makes a final visual judgment.
[0006] Furthermore, when automatically extracting the types of features from aerial photographs, machine learning (ML) techniques are sometimes used. This machine learning is one of the technologies that underpin artificial intelligence (AI), and it generates a model that can output the desired result by applying machine learning (for example, deep learning) to training data. For example, Patent Document 1 proposes a technique for creating window images by cropping aerial photographs to a predetermined size, classifying these window images, and then storing them as training data. [Prior art documents] [Patent Documents]
[0007] [Patent Document 1] Japanese Patent Publication No. 2019-204258 [Overview of the Initiative] [Problems that the invention aims to solve]
[0008] Naturally, aerial photographs have different characteristics depending on the area in which they are taken. For example, aerial photographs taken in urban areas show a high concentration of buildings that make up commercial areas and office districts, while aerial photographs taken in rural areas often include farmland and forests. Therefore, when using aerial photographs (or data based on aerial photographs) as training data for machine learning, urban training data was used when inference was performed in urban areas, and rural training data was used when inference was performed in rural areas. The technology disclosed in Patent Document 1 is also a technology that creates training data according to the building situation, that is, it divides one aerial photograph into multiple window images, classifies each window image (for example, window images with a high percentage of S-sized buildings), and then stores a training dataset for each class. Then, in areas with a high percentage of S-sized buildings, the system learns window images of the class with a high percentage of S-sized buildings, and in areas with a high percentage of L-sized buildings, it learns window images of the class with a high percentage of L-sized buildings.
[0009] Thus, creating training data for each region and then applying machine learning to that data would result in exorbitant costs. Therefore, there has been a strong demand for a technology that can generate models applicable to regions with diverse characteristics by applying machine learning to a small number of types of training data. For example, a technology that can generate models that can be used for inference in various regions by applying machine learning to aerial photographs of the Tokyo metropolitan area.
[0010] The object to be addressed by the present invention is to solve the problems of the conventional methods, namely, to provide an object extraction system that can extract a target object from an image and can be commonly applied to regions with different characteristics. [Means for solving the problem]
[0011] The present invention focuses on the fact that it uses aerial photographs tagged with characteristic information about a region as training data, and generates a trained model by machine learning on this training data. This invention is based on a completely new idea.
[0012] The object extraction system of the present invention is a system for extracting target objects from aerial photographs, and comprises a model generation means and an object discrimination means. The model generation means is a means for generating a trained model by machine learning on training data. The object discrimination means is a means for discriminating against objects contained in an aerial photograph by inputting the aerial photograph into the trained model. The training data consists of an "aerial photograph" and "tag information" attached to the aerial photograph, and this tag information includes information on the presence or absence of the target object and regional characteristic information (information on the region related to the aerial photograph). Of the tag information, the information on the presence or absence of the target object is attached to each pixel that makes up the aerial photograph, while the regional characteristic information is attached to each mesh (an area composed of multiple pixels). The object discrimination means then discriminates against the presence or absence of the target object for each pixel.
[0013] The object extraction system of the present invention can also be configured so that the object is a building. In this case, the tag information includes information on the presence or absence of a building and regional characteristic information. Furthermore, in this case, the object discrimination means determines the presence or absence of a building for each pixel.
[0014] The object extraction system of the present invention can also use the green cover level (an index representing the range of green cover ratio) as the regional characteristic information. In this case, the tag information includes information on the presence or absence of buildings and the green cover level.
[0015] The object extraction system of the present invention can also be configured such that regional characteristic information is the building area level (an index representing the range of average building areas). In this case, the tag information includes information on the presence or absence of buildings and the building area level. [Effects of the Invention]
[0016] The object extraction system of the present invention has the following effects: (1) By simply preparing a small number of types of learning data, it is possible to extract the target object from aerial photos related to regions with different characteristics. As a result, the cost of creating learning data can be reduced. (2) In order to generate a pre-trained model that can be commonly applied to regions with different characteristics, it is not necessary to create a model for each task, and thus the cost of generating the pre-trained model can be reduced.
Brief Description of Drawings
[0017] [Figure 1] A block diagram showing the main configuration of the target object extraction system of the present invention. [Figure 2] A model diagram schematically showing a plurality of meshes set in an aerial photo. [Figure 3] A model diagram explaining tag information consisting of "presence or absence of target object" and "regional characteristic information". [Figure 4] An image plan view showing an aerial photo in which a five-level green coverage level is assigned to each mesh. [Figure 5] An image plan view showing an aerial photo in which a five-level building area level is assigned to each mesh. [Figure 6] A flowchart showing an example of the main processing flow of the target object extraction system of the present invention.
Embodiments for Carrying Out the Invention
[0018] An example of an embodiment of the target object extraction system of the present invention will be described based on the drawings.
[0019] FIG. 1 is a block diagram showing the main configuration of the target object extraction system 100 of the present invention. As shown in this figure, the target object extraction system 100 includes a model generation means 101 and a target object discrimination means 102, and may further include a teacher data generation means 103, an image data storage means 104, a tag information storage means 105, a teacher data storage means 106, a model storage means 107, and the like.
[0020] The model generation means 101, the target object discrimination means 102, and the teacher data generation means 103 that constitute the target object extraction system 100 can be manufactured as dedicated ones, or general-purpose computer devices can also be used. That is, by causing the computer device to execute arithmetic processing by a predetermined program, the processing peculiar to each means is performed. This computer device includes a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), and a memory such as a ROM or a RAM, and some also include input means such as a mouse and a keyboard and a display, and can be configured by, for example, a personal computer (PC) or a server.
[0021] Also, the image data storage means 104, the tag information storage means 105, the teacher data storage means 106, and the model storage means 107 can use the storage device of a general-purpose computer (for example, a personal computer), or can be constructed in a database server. When constructing in a database server, it can be placed on a local network (LAN: Local Area Network), or can be a cloud server stored via the Internet.
[0022] Hereinafter, each main element constituting the target object extraction system 100 of the present invention will be described in detail.
[0023] (Teacher data generation means) One of the features of the present invention is to generate a model that can make inferences without depending on the characteristics of a region (hereinafter simply referred to as "regional characteristics"), that is, common to various regional characteristics. Therefore, the teacher data for machine learning reflects the regional characteristics. And that teacher data can be generated by the teacher data generation means 103. Hereinafter, the procedure for generating the teacher data applied to the present invention will be described.
[0024] First, divided regions (hereinafter referred to as "mesh MS") as shown in Figure 2 are set for the image (for example, an aerial photograph) stored in the image data storage means 104 (Figure 1). However, training data is not generated for each mesh MS, but rather based on one aerial photograph as the unit. That is, in the example in Figure 2, one training data is generated based on an aerial photograph composed of a mesh MS of 12 horizontally and 15 vertically. This mesh MS is set by the training data generation means 103, and specifically, it is set at predetermined intervals based on the coordinates of the aerial photograph. A mesh MS can be a collection of pixels that make up the aerial photograph; for example, 500 x 500 pixels to 1,000 x 1,000 pixels can be one mesh MS.
[0025] When a mesh MS is set for an aerial photograph, tag information is assigned to that aerial photograph. As shown in Figure 3, this tag information consists of "presence or absence of target object" and "regional characteristic information." "Presence or absence of target object" is assigned to each pixel that makes up the aerial photograph, while "regional characteristic information" is assigned to each mesh MS. In other words, the training data of the present invention consists of an aerial photograph, "presence or absence of target object" assigned to each pixel that makes up the aerial photograph, and "regional characteristic information" assigned to each mesh MS. Of the tag information, "presence or absence of target object" literally indicates whether or not a target object "is" present at that pixel, and "target object" is the feature that is to be output by inference. Various features can be selected as the target object, such as farmland (rice paddies and fields, etc.), green spaces such as parks, roads, railways, and rivers, but for convenience, the example of a "building" as the target object will be explained here.
[0026] Among the tag information, "regional characteristic information" is information that indicates regional characteristics, and can be, for example, "green cover level" or "building area level." Here, green cover level is the level obtained when the proportion of the green area (e.g., area ratio) in a mesh MS is calculated as "green cover ratio" and the green cover ratio is pre-classified into several stages (ranges). For example, in Figure 4, the green cover ratio is pre-classified into five ranges, meaning that each mesh MS is assigned one of the five green cover levels.
[0027] Furthermore, the building area level refers to the level at which the area of the building portion within the mesh MS is calculated as "building area" and then pre-classified into several stages (ranges). For example, in Figure 5, the building area is pre-classified into five ranges, meaning that each mesh MS is assigned one of the five building area levels.
[0028] Regional characteristic information is not limited to green cover level or building area level; various types of information that indicate regional characteristics can be adopted. For example, Figure 3 shows not only green cover level and building area level, but also building density level, building height level, tree species, tree distribution level, geological classification, and topographic classification. In any case, each pixel is assigned "presence or absence of target object," and each mesh MS is assigned "regional characteristic information." When assigning tag information to each pixel or mesh MS, the operator can do so using the training data generation means 103. In this case, it is preferable that the operator assigns the desired tag information by operating the training data generation means 103, which consists of a pointing device (mouse, touch panel, pen tablet, touchpad, trackpad, trackball, etc.) or keyboard, while visually viewing an aerial photograph displayed on a display means such as a display. Alternatively, if the tag information storage means 105 (Figure 1) stores tag information ("presence or absence of target object" for each pixel and "regional characteristic information" for each mesh MS), the training data generation means 103 can be configured to automatically assign tag information to the corresponding pixels or mesh MS. The training data generated here is stored in the training data storage means 106 (Figure 1).
[0029] (Model generation means) The model generation means 101 is a means for generating a "trained model" by performing machine learning using training data. For machine learning to generate the trained model, various conventional machine learning techniques can be employed, including deep learning such as CNN (Convolutional Neural Network). However, the trained model is generated in such a way that it outputs (infers) the presence or absence of a building (target object) for each pixel constituting the aerial photograph. The trained model generated by the model generation means 101 is stored in the model storage means 107 (Figure 1).
[0030] (object discrimination means) The object discrimination means 102 is configured to include a trained model generated by the model generation means 101, and is a means for outputting (inferring) the presence or absence of buildings (objects) in an input aerial photograph. However, the object discrimination means 102 discriminates and outputs the presence or absence of buildings for each pixel that makes up the aerial photograph. Of course, it is also possible to configure the system to first discriminate the presence or absence of buildings for each pixel, and then make a final decision on whether or not a building is present based on the discrimination results. For example, when pixels that are determined to have buildings are clustered together in a certain number (above a pre-set threshold), it can be determined that a building exists in that cluster. Furthermore, the object discrimination means 102 can also be configured to output (infer) regional characteristic information in addition to the presence or absence of buildings.
[0031] (Process flow) The main processes of the object extraction system 100 of the present invention will be described in detail below with reference to Figure 6. Figure 6 is a flowchart showing an example of the main processing flow of the object extraction system 100 of the present invention from determining the presence or absence of a building for each pixel to outputting the result. The central column shows the processes to be executed, the left column shows what is necessary for those processes, and the right column shows what results from those processes.
[0032] To determine the presence or absence of a building for each pixel, training data is first generated as shown in Figure 6 (Step 201 in Figure 6). The training data is generated by the training data generation means 103, which specifically assigns "presence or absence of a target object" to each pixel constituting the aerial photograph, and also assigns "regional characteristic information" to each mesh MS set for the aerial photograph.
[0033] Once training data is generated, the model generation means 101 generates a trained model by performing machine learning using the training data (Step 202 in Figure 6). Then, when the aerial photograph to be inferred is input to the object discrimination means 102, the object discrimination means 102 discriminates and outputs whether or not there is a building for each pixel (Step 203 in Figure 6). Alternatively, as described above, it is also possible to output regional characteristic information in addition to the presence or absence of buildings. The trained model is generated as a result of learning training data that reflects regional characteristics, and since the object discrimination means 102 is equipped with this trained model, it can infer the presence or absence of buildings without depending on regional characteristics. For example, even if a trained model is generated by performing machine learning on training data based on aerial photographs of the Tokyo metropolitan area, it can appropriately infer the presence or absence of buildings for aerial photographs taken in other urban areas (e.g., the Kansai area) or rural areas. [Industrial applicability]
[0034] The object extraction system of the present invention can be used to extract various geographical features, including buildings, roads and railway areas, farmland by type, and trees by type. Because the present invention allows for the general identification of geographical features across various regions, it can be effectively utilized in social infrastructure planning and disaster prevention planning. Therefore, it is an invention that can not only be used industrially but also is expected to make a significant social contribution. [Explanation of Symbols]
[0035] 100 Target material extraction system of the present invention 101 Model generation means (of the target extraction system) 102 Target object discrimination means (of the target object extraction system) 103 (Training data generation means for the target object extraction system) 104 Image data storage means (of the target object extraction system) 105 Tag information storage means (of the target object extraction system) 106 (Training data storage means for the target object extraction system) 107 Model storage means (of the target object extraction system) MS Mesh
Claims
1. A system for extracting target objects from aerial photographs, A model generation means that generates a trained model by applying machine learning to training data, The system includes object discrimination means for identifying the object contained in the aerial photograph by inputting the aerial photograph into the trained model, The aforementioned training data consists of the aerial photograph and tag information assigned to the aerial photograph. The tag information includes information on the presence or absence of the object and regional characteristic information relating to the area covered by the aerial photograph. Of the tag information, the information regarding the presence or absence of the target object is assigned to each pixel constituting the aerial photograph, and of the tag information, the regional characteristic information is assigned to each mesh composed of multiple pixels. The object determination means determines the presence or absence of the object for each pixel. A target extraction system characterized by the following features.
2. The aforementioned object is a building, The aforementioned tag information includes information on the presence or absence of the building and the aforementioned regional characteristics information. The object determination means determines whether the building is present or absent for each pixel. The target extraction system according to claim 1, characterized in that it is a target substance extraction system.
3. The aforementioned regional characteristic information is a green cover level that represents the range of green coverage ratios. The tag information includes information on the presence or absence of the building and the green cover level. The target substance extraction system according to claim 2, characterized in that it is a feature of the present invention.
4. The aforementioned regional characteristic information is a building area level that represents the range of average building areas in the mesh, The aforementioned tag information includes information on the presence or absence of the building and the building area level. The target substance extraction system according to claim 2, characterized in that it is a feature of the present invention.