A method and device for identifying building use

Abstract

The present application discloses a method and device for identifying building use, which includes obtaining a remote sensing image of an identified area and a social attribute image of the identified area; splicing the remote sensing image and the social attribute image in the channel dimension to obtain heterogeneous data, wherein the remote sensing image includes m channels, the social attribute image includes n channels, and the heterogeneous data includes m channels of the remote sensing image and n channels of the social attribute image; inputting the heterogeneous data into a building use identification model, wherein the building use identification model includes an encoder and a decoder; encoding the heterogeneous data by the encoder to obtain fusion features of the remote sensing image and the social attribute image, wherein the fusion features include visual features of the building in the remote sensing image and social features of the building in the social attribute image; decoding the fusion features by the decoder to obtain the building use identification result of the identified area. The present application is conducive to improving the accuracy of building use identification.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for identifying building usage. Background Art

[0002] Building use identification has strong social and economic value. In the insurance industry, different uses of buildings often reflect different vulnerabilities of buildings. Accurately knowing the building use of the insured building can help better assess the potential loss risk of the insured building, obtain more reasonable insurance pricing, and better share risks. In the field of urban planning, accurate knowledge of building use information can enable better urban analysis, simulation, and policy formulation.

[0003] At present, some studies have trained artificial intelligence models based on remote sensing images. The input of the model includes remote sensing images, and the model identifies the use of buildings based on the visual features of the buildings contained in the remote sensing images. However, the use of buildings has social attributes, and it is difficult to accurately judge the use of buildings based only on the visual features of buildings. Summary of the invention

[0004] The embodiments of the present application provide a method and device for identifying building use, which improve the accuracy of building use identification.

[0005] In a first aspect, a method for identifying building use is provided, the method comprising:

[0006] Acquire a remote sensing image of the identified area and a social attribute image of the identified area;

[0007] splicing the remote sensing image and the social attribute image in a channel dimension to obtain heterogeneous data, wherein the remote sensing image includes m channels, the social attribute image includes n channels, and the heterogeneous data includes the m channels of the remote sensing image and the n channels of the social attribute image;

[0008] Inputting the heterogeneous data into a building use identification model, wherein the building use identification model includes an encoder and a decoder;

[0009] The heterogeneous data is encoded by the encoder to obtain fusion features of the remote sensing image and the social attribute image, wherein the fusion features include visual features of the building in the remote sensing image and social features of the building in the social attribute image;

[0010] The fusion feature is decoded by the decoder to obtain a building use recognition result of the identified area.

[0011] In some embodiments, the social attribute image includes point of interest (POI) data, and the POI data includes the location of the POI in the identified area and the category of the POI;

[0012] The step of acquiring the remote sensing image of the identified area and the social attribute image of the identified area includes:

[0013] Creating a plurality of POI point maps based on the POI data, wherein the size of each of the POI point maps is the same as the size of the remote sensing image, each of the POI point maps corresponds to a category of POI, the pixel values ​​of the pixel points of the POI of the corresponding category in each of the POI point maps being a first pixel value, and the pixel values ​​of the pixel points of the POI of the corresponding category not being in each of the POI point maps being a second pixel value;

[0014] Creating a plurality of POI color temperature maps based on the distance between each pixel point in each of the POI point maps and the nearest adjacent POI pixel point;

[0015] Accordingly, the step of splicing the remote sensing image and the social attribute image in the channel dimension includes:

[0016] The remote sensing image and the plurality of POI color temperature maps are spliced ​​in a channel dimension.

[0017] In some implementations, creating multiple POI color temperature maps based on the distance between each pixel in each POI point map and the nearest adjacent POI pixel includes:

[0018] For each pixel in each of the POI point maps, determine the distance between the pixel and each POI pixel belonging to the maximum range of influence in the POI point map to which the pixel belongs, wherein in each of the POI point maps, the maximum range of influence of the POI point map of the industrial category is greater than the maximum range of influence of the POI point map of the residential category, and greater than the maximum range of influence of the POI point map of the commercial category;

[0019] The closest distance value between the pixel point and all POI pixels is selected as the pixel value of the pixel point in the POI color temperature map.

[0020] In some implementations, the step of splicing the remote sensing image and the social attribute image in the channel dimension includes:

[0021] Dividing the remote sensing image into blocks according to a preset image size to obtain a plurality of remote sensing image blocks;

[0022] Dividing the social attribute image into blocks according to the preset image size to obtain a plurality of social attribute image blocks;

[0023] Splicing the remote sensing image blocks and the social attribute image blocks with the same geographical location coordinates in the channel dimension;

[0024] Accordingly, the heterogeneous data includes a plurality of heterogeneous data blocks, each of which includes m channels of a remote sensing image block and n channels of a social attribute image block with the same geographical location coordinates;

[0025] Accordingly, encoding the heterogeneous data by the encoder to obtain the fusion features of the remote sensing image and the social attribute image includes:

[0026] The encoder encodes the plurality of heterogeneous data blocks in parallel to obtain a plurality of fused features, each of the fused features comprising a feature of the remote sensing image block and a feature of the social attribute image block;

[0027] The decoding of the fusion feature by the decoder to obtain the building use recognition result of the identified area includes:

[0028] Decoding the multiple fusion features in parallel by the decoder to obtain multiple recognition result data blocks;

[0029] The multiple recognition result data blocks are spliced ​​according to the geographical location coordinates to obtain the building use recognition result of the recognized area.

[0030] In some embodiments, the method further comprises:

[0031] Acquire a sample remote sensing image of a sample area and a sample social attribute image of the sample area;

[0032] The sample remote sensing image and the sample social attribute image are spliced ​​in the channel dimension to obtain sample heterogeneous data, wherein the sample remote sensing image includes m channels, the sample social attribute image includes n channels, and the sample heterogeneous data includes the m channels of the sample remote sensing image and the n channels of the sample social attribute image;

[0033] Inputting the sample heterogeneous data into the building use identification model;

[0034] Encoding the sample heterogeneous data by the encoder to obtain fusion features of the sample remote sensing image and the sample social attribute image;

[0035] Decoding the fusion features of the sample remote sensing image and the sample social attribute image through the decoder to obtain a building use recognition result of the sample remote sensing image;

[0036] Determine a building use loss based on a building use identification result of the sample area and a building use label of the sample area, wherein the building use label is used to indicate the building use of the sample area, and the building use loss is used to characterize a deviation between the building use identification result and the building use label;

[0037] The parameters of the building use identification model are adjusted based on the building use loss so that the deviation of the building use identification result output by the building use identification model meets the conditions.

[0038] In some implementations, decoding the fusion features of the sample remote sensing image and the sample social attribute image by the decoder to obtain a building use recognition result of the sample remote sensing image includes:

[0039] Decoding the fused features by the decoder to obtain a building use recognition result for each pixel in the sample remote sensing image;

[0040] Accordingly, the pixel value of each pixel point in the building use label is a first pixel value, a second pixel value, a third pixel value or a fourth pixel value, the first pixel value represents the background, the second pixel value represents that the building use is residential, the third pixel value represents that the building use is commercial, and the fourth pixel value represents that the building use is industrial. The determination of the building use loss based on the building use recognition result of the sample area and the building use label of the sample area includes:

[0041] The building use loss is determined based on the probability that the building use of each pixel point in the building use identification result is identified as residential, the probability that the building use of each pixel point is identified as commercial, the probability that the building use of each pixel point is identified as industrial, and the pixel value of each pixel point in the building use label.

[0042] In some embodiments, the sample remote sensing images of the sample area include remote sensing images of provincial administrative regions across the country, remote sensing images with building distribution density higher than a density threshold, remote sensing images with building distribution density lower than a density threshold, and remote sensing images with building use identification result deviations higher than a deviation threshold output by the building use identification model in the test set.

[0043] In some embodiments, the social attribute image includes at least one of point of interest (POI) data, population distribution data of the identified area, age distribution data of the identified area, GDP data of the identified area, or county annals of the identified area.

[0044] In a second aspect, a building use identification device is provided, the device comprising:

[0045] An acquisition unit, used to acquire a remote sensing image of the identified area and a social attribute image of the identified area;

[0046] A stitching unit, used for stitching the remote sensing image and the social attribute image in a channel dimension to obtain heterogeneous data, wherein the remote sensing image includes m channels, the social attribute image includes n channels, and the heterogeneous data includes the m channels of the remote sensing image and the n channels of the social attribute image;

[0047] A model processing unit is used to input the heterogeneous data into a building use recognition model, the building use recognition model includes an encoder and a decoder; the heterogeneous data is encoded by the encoder to obtain a fusion feature of the remote sensing image and the social attribute image, the fusion feature includes the visual features of the building in the remote sensing image and the social features of the building in the social attribute image; the fusion feature is decoded by the decoder to obtain a building use recognition result of the identified area.

[0048] In some embodiments, the social attribute image includes point of interest (POI) data, and the POI data includes the location of the POI in the identified area and the category of the POI;

[0049] The acquisition unit is used to create a plurality of POI point maps based on the POI data, wherein the size of each of the POI point maps is the same as the size of the remote sensing image, each of the POI point maps corresponds to a category of POI, the pixel values ​​of the pixel points of the POI of the corresponding category in each of the POI point maps are first pixel values, and the pixel values ​​of the pixel points of the POI of the corresponding category that do not exist in each of the POI point maps are second pixel values; and create a plurality of POI color temperature maps based on the distance between each pixel point in each of the POI point maps and the nearest adjacent POI pixel point;

[0050] Correspondingly, the stitching unit is used to stitch the remote sensing image and the plurality of POI color temperature maps in a channel dimension.

[0051] In some embodiments, the acquisition unit is used to determine, for each pixel point in each of the POI point maps, the distance between the pixel point and each POI pixel point within the maximum range of influence in the POI point map to which the pixel point belongs, wherein in each of the POI point maps, the maximum range of influence of the POI point map of the industrial category is greater than the maximum range of influence of the POI point map of the residential category, and greater than the maximum range of influence of the POI point map of the commercial category; and select the closest distance value between the pixel point and all POI pixel points as the pixel value of the pixel point in the POI color temperature map.

[0052] In some embodiments, the stitching unit is used to divide the remote sensing image into blocks according to a preset image size to obtain a plurality of remote sensing image blocks; divide the social attribute image into blocks according to the preset image size to obtain a plurality of social attribute image blocks; and stitch the remote sensing image blocks and the social attribute image blocks having the same geographical location coordinates in the channel dimension;

[0053] Accordingly, the heterogeneous data includes a plurality of heterogeneous data blocks, each of which includes m channels of a remote sensing image block and n channels of a social attribute image block with the same geographical location coordinates;

[0054] Correspondingly, the model processing unit is used to encode the multiple heterogeneous data blocks in parallel through the encoder to obtain multiple fusion features, each of which includes a feature of the remote sensing image block and a feature of the social attribute image block; decode the multiple fusion features in parallel through the decoder to obtain multiple recognition result data blocks; and splice the multiple recognition result data blocks according to the geographic location coordinates to obtain the building use recognition result of the identified area.

[0055] In some embodiments, the acquisition unit is further used to acquire a sample remote sensing image of the sample area and a sample social attribute image of the sample area;

[0056] The stitching unit is further used to stitch the sample remote sensing image and the sample social attribute image in the channel dimension to obtain sample heterogeneous data, the sample remote sensing image includes m channels, the sample social attribute image includes n channels, and the sample heterogeneous data includes the m channels of the sample remote sensing image and the n channels of the sample social attribute image;

[0057] The model processing unit is also used to input the sample heterogeneous data into the building use identification model; encode the sample heterogeneous data through the encoder to obtain the fusion features of the sample remote sensing image and the sample social attribute image; decode the fusion features of the sample remote sensing image and the sample social attribute image through the decoder to obtain the building use identification result of the sample remote sensing image; determine the building use loss based on the building use identification result of the sample area and the building use label of the sample area, the building use label is used to indicate the building use of the sample area, and the building use loss is used to characterize the deviation between the building use identification result and the building use label; adjust the parameters of the building use identification model based on the building use loss so that the deviation of the building use identification result output by the building use identification model meets the conditions.

[0058] In some embodiments, the model processing unit is used to decode the fusion feature through the decoder to obtain a building use identification result for each pixel in the sample remote sensing image; accordingly, the pixel value of each pixel in the building use label is a first pixel value, a second pixel value, a third pixel value or a fourth pixel value, the first pixel value represents the background, the second pixel value represents that the building use is residential, the third pixel value represents that the building use is commercial, and the fourth pixel value represents that the building use is industrial; based on the probability of the building use of each pixel in the building use identification result being identified as residential, the probability of the building use of each pixel being identified as commercial, the probability of the building use of each pixel being identified as industrial, and the pixel value of each pixel in the building use label, the building use loss is determined.

[0059] In some embodiments, the sample remote sensing images of the sample area include remote sensing images of provincial administrative regions across the country, remote sensing images with building distribution density higher than a density threshold, remote sensing images with building distribution density lower than a density threshold, and remote sensing images with building use identification result deviations higher than a deviation threshold output by the building use identification model in the test set.

[0060] In some embodiments, the social attribute image includes at least one of point of interest (POI) data, population distribution data of the identified area, age distribution data of the identified area, GDP data of the identified area, or county annals of the identified area.

[0061] According to a third aspect, a computer device is provided, comprising: a processor, the processor being coupled to a memory, the memory storing at least one computer program instruction, the at least one computer program instruction being loaded and executed by the processor, so that the computer device implements the method described in the first aspect or any optional method of the first aspect.

[0062] In a fourth aspect, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, and when the instruction is executed on a computer, the computer executes the method described in the first aspect or any optional method of the first aspect.

[0063] In a fifth aspect, a computer program product is provided, which includes one or more computer program instructions. When the computer program instructions are loaded and executed by a computer, the computer executes the method described in the first aspect or any optional method of the first aspect.

[0064] In a sixth aspect, a chip is provided, which includes a programmable logic circuit and / or program instructions, and when the chip is running, it is used to implement the method described in the first aspect or any optional method of the first aspect.

[0065] It can be seen that the embodiments of the present application have the following beneficial effects:

[0066] The method provided in the embodiment of the present application obtains a remote sensing image of the identified area and a social attribute image of the identified area, and splices the remote sensing image and the social attribute image in the channel dimension to obtain heterogeneous data. Due to the splicing of the two types of data, the fusion of the remote sensing image and the social attribute image is achieved, and the obtained heterogeneous data can contain both the visual features of the building in the remote sensing image and the social features of the building in the social attribute image. The heterogeneous data is input into the building use recognition model, and the heterogeneous data is encoded by the encoder in the model, so that the visual features of the building in the heterogeneous data and the social features of the building are further fused in the shallow layer of the model (such as the convolutional layer in the encoder) along with the encoding process. The fusion is achieved by data splicing before inputting the model, and the method of further feature fusion through the encoding process can be regarded as information fusion based on pixel points. This fusion method is more conducive to the decoder's recognition of the use of independent buildings and improves the accuracy of building use recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 It is a schematic diagram of a building use identification architecture provided by an embodiment of the present application;

[0068] Figure 2 It is a schematic diagram of a U-Net structure for implementing a building use recognition model provided in an embodiment of the present application;

[0069] Figure 3 It is a structural diagram of a ResNet residual block provided in an embodiment of the present application;

[0070] Figure 4 This is a comparison diagram of heterogeneous data before and after use provided by an embodiment of the present application;

[0071] Figure 5 is a flow chart of a method for identifying building use provided in an embodiment of the present application;

[0072] Figure 6 This is a comparison chart of 10-meter and 1-meter resolution remote sensing images provided in an embodiment of the present application;

[0073] Figure 7 This is a POI point map conversion effect diagram provided by an embodiment of the present application;

[0074] Figure 8This is a diagram showing the effect of implementing a 1-meter resolution POI distance color temperature map provided in an embodiment of the present application;

[0075] Fig. 9 It is a structural schematic diagram of a building use identification device provided in an embodiment of the present application;

[0076] Fig.10 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0077] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0078] The following is an explanation of some terminology concepts involved in the embodiments of the present application.

[0079] A mega-disaster is a disaster event with extreme intensity, wide impact and far-reaching follow-up effects. These events exceed the normal response capacity of the affected communities or countries, and have a major impact on human society, economic systems, natural environment and even global stability. The United Nations International Strategy for Disaster Reduction (UNISDR) calls it a disaster that has a serious impact on society and requires national or international assistance. The Organization for Economic Cooperation and Development (OECD) defines it as a disaster that causes a large number of casualties, property losses and extensive damage to infrastructure, leaving the governments of the affected areas and neighboring areas helpless.

[0080] Catastrophes have a huge impact on life and property, social structure, economic development potential, ecological and environmental balance, etc., which makes it impossible for government emergency response and various industries to ignore the risks brought by catastrophes. Among all industries, the financial insurance industry has the most direct perception of catastrophe risks. Catastrophes directly affect the financial stability and profitability of financial insurance companies, and even affect their normal operations. Therefore, the financial insurance industry is also the most eager to manage catastrophe risks. However, compared with traditional risks, the extraordinary risks brought by catastrophes often present the characteristics of low frequency and high loss, wide and long-term impact, high complexity and uncertainty. Therefore, financial insurance catastrophe risk management often requires a comprehensive system framework, covering multiple key links such as risk identification, risk quantification, risk control and loss compensation. For the financial insurance industry, the core of the comprehensive system framework of catastrophe risk management is the catastrophe model, which helps insurance companies quantify potential losses by simulating the occurrence and impact of catastrophe events, thereby conducting risk management and pricing. Catastrophe models usually include: (1) Disaster module: using historical disaster data combined with physical mechanisms to simulate random event sets. (2) Risk exposure module: Starting from the perspective of spatial distribution, it collects detailed spatial information and attribute information of the disaster-prone objects. (3) Engineering module: This module focuses on vulnerability, and the vulnerability model considers the sensitivity of different disaster-prone objects to disasters. (4) Financial module: It is used to assess the financial impact of disasters on insurance companies and other financial institutions.

[0081] Risk exposure is the data basis for converting hazard calculations and vulnerability calculations into economic value loss calculations. It is one of the most basic data requirements for catastrophe models. It accurately reflects the economic value and geographic spatial distribution of the main disaster-bearing bodies, which is crucial to the scientificity and accuracy of model simulation results. Risk exposure data is an important input requirement for the insurance industry to further calculate catastrophe economic losses, and is also the data basis for reinsurance companies to reasonably split and calculate unified policies based on economic value distribution.

[0082] Building use refers to the intended function and purpose of a building or group of buildings. Different building uses will have different design requirements, structural characteristics and usage specifications. In the insurance industry, the different uses of buildings often reflect the different vulnerabilities of buildings, and are often divided into three categories: residential, commercial and industrial. Building use is a very important attribute, but there is currently no nationwide work on building use identification.

[0083] The national scope refers to the land area across China, covering an area of ​​approximately 9.6 million square kilometers, including urban and rural areas.

[0084] Building use identification refers to identifying the functional use of a single building. In this document, it specifically refers to identifying three types of building uses: residential, commercial, and industrial. These building uses meet the insurance use requirements. The identified building uses can be linked to specific building roofs through geographic location.

[0085] POI data refers to a point of interest (Place Of Interest). In a geographic information system, a POI can be a house, a shop, a mailbox, a bus stop, etc. Generally, POI data includes latitude and longitude information, name information, address information, purpose information, etc.

[0086] Remote sensing interpretation refers to the process of identifying and extracting information about land objects by analyzing and processing remote sensing images. It includes methods such as visual interpretation and computer digital image processing, using image features such as shape, size, color, texture, combined with professional knowledge and experience to determine the attributes and status of land objects. Remote sensing interpretation is widely used in geological surveys, environmental monitoring, resource management and other fields.

[0087] A neural network is a computational model that mimics the way neurons in the human brain connect and process information. It consists of a large number of nodes (called neurons), which are usually arranged in multiple layers. Each neuron receives an input signal, applies a nonlinear activation function after weighted summation, and generates an output signal to the next layer of neurons. Neural networks learn complex relationships and patterns between input data by adjusting the weights of connections between neurons, and are widely used in image recognition, speech processing, natural language processing and other fields. With the development of deep learning, the layers of neural networks have become more complex and in-depth, enabling the capture of more abstract data features.

[0088] Multi-task learning (MTL) is a machine learning method that improves the performance of each task by learning multiple related tasks simultaneously. The core idea of ​​multi-task learning is to use the shared information between tasks to improve the generalization ability of the model. Specifically, MTL can be achieved by sharing model parameters or feature representations, so that different tasks can learn from each other and pass useful information to improve the recognition accuracy of learning tasks.

[0089] Hidden layers are one or more layers between the input layer and the output layer of a neural network. Their purpose is to capture and learn nonlinear features and complex patterns in the input data. In these layers, each neuron receives weighted input from the previous layer, processes it through an activation function, and passes the result to the next layer. Since this information processing process of the hidden layer is invisible to the outside world, they are called "hidden" layers. The number of hidden layers and the number of neurons in each layer are important hyperparameters that need to be determined when building a neural network. They directly affect the learning ability and complexity of the model.

[0090] Forward propagation is the process of passing input data layer by layer and calculating output results in a neural network. Specifically, the input data is processed by weighted summation and activation function of each layer, and then gradually passed to the next layer until the output layer generates the final recognition result. Forward propagation is the basic step for neural network recognition and classification, ensuring that the input data can be calculated through each layer of the network to obtain the output result.

[0091] Model inference refers to the process of using a trained model to identify or classify new input data in machine learning or deep learning. Simply put, it is to apply the trained model to actual data to generate output results. For example, in image recognition, model inference is to input an unseen picture into the model and obtain the recognition of the category to which the picture belongs.

[0092] High-definition remote sensing data refers to the image data of the earth's surface obtained through high-resolution remote sensing technology, which has extremely high spatial resolution (usually 1 meter or less) and can clearly capture the details of the ground objects. These data are widely used in land use, urban planning, environmental monitoring and other fields, providing rich ground object texture information and multi-spectral band data, supporting fine ground object classification and identification.

[0093] Building vector data refers to the geometric information and attribute data of buildings expressed in vector form, which usually includes the location, shape, size and other related attribute information of the building, such as height, material, purpose, etc. These data can be edited and analyzed by geographic information system (GIS) software, and are often used in urban planning, architectural design, asset management and geospatial analysis. Building vector data can be obtained by professional surveying and mapping personnel through field measurement, or it can be automatically or semi-automatically extracted through technical means such as remote sensing images and aerial photography.

[0094] Ground truth data refers to accurate and error-free data sets used to train and validate models in the fields of machine learning and pattern recognition. These data contain input features and their corresponding correct output labels, which are the benchmarks for building and evaluating model performance. Ground truth data are usually obtained through manual annotation, historical records, or verified measurements, and are critical for supervised learning algorithms because they ensure that the model can learn the correct mapping relationship during training and accurately measure the model's recognition ability in subsequent evaluations.

[0095] Image Channel refers to an independent information source that represents color information in a digital image. In a color image, there are usually three color channels: red (R), green (G), and blue (B). Each channel contains the intensity information of the color in the image. In a grayscale image, there is only one channel, which represents brightness information. With the diversification of applications, the channels of an image can be very diverse. For example, in the field of remote sensing, other optical bands besides visible light can also be put into remote sensing images, so the file format of some remote sensing images can support ten or even dozens of channels. Image channel is a basic concept in image processing and analysis, and is often used in operations such as image enhancement, filter effects, and image segmentation.

[0096] CNN, or Convolutional Neural Network, is a deep learning model that excels in image and video recognition, natural language processing, and other fields. CNN simulates the perception mechanism of biological visual cortex on images and uses convolutional layers to automatically and layer by layer extract features of input data, which gradually evolve from simple edges and textures to complex patterns. The key features of CNN include local receptive field, weight sharing, and pooling operations, which make it very effective in processing data with grid structures, and can reduce the number of model parameters, improve training efficiency, and improve generalization capabilities.

[0097] Urban Functional Zone refers to a specific area divided according to the different functions and uses of various activities within the city. They are the basic units of urban structure and layout. These areas usually include residential areas, commercial areas, industrial areas, office areas, educational areas, entertainment areas, etc. Each functional area carries specific social and economic activities and has corresponding architectural characteristics, land use patterns and population distribution. The division of urban functional zones is of great significance for urban planning, traffic management, environmental monitoring, public service facilities configuration and urban sustainable development. Urban functional zones can be identified and divided through remote sensing images, geographic information systems (GIS) and social perception data.

[0098] Semantic Segmentation is a technique in computer vision that aims to assign each pixel in an image to a specific category label. In this way, each pixel in the image is accurately classified into a semantic category, such as road, building, pedestrian, etc. Semantic segmentation usually uses deep learning models, especially convolutional neural networks (CNN), to achieve this goal. It has a wide range of applications in autonomous driving, medical image analysis, robotic vision and other fields, helping machines better understand and parse visual scenes by providing pixel-level classification information.

[0099] The technical problem to be solved by some embodiments of the present application is to design a modeling solution for identifying the use of buildings throughout China. To solve this major technical problem, there are two core sub-technical problems that need to be solved:

[0100] (1) Design an artificial intelligence model framework suitable for nationwide modeling of building uses. After researching a large number of literature, it is found that the largest application area of ​​the current building use identification solution is one or more cities, and the modeling framework used is often difficult to promote nationwide. The reasons for the difficulty in promoting include difficulty in data acquisition and high computational complexity. Therefore, it is necessary to design a model framework that has the possibility of nationwide building use identification, the model input data used can cover the whole of China, the framework structure is simple and easy to use, the computational workload is moderate, and it can be calculated nationwide.

[0101] (2) Design an effective multi-source heterogeneous data fusion method. The use of buildings has social attributes, and it is difficult to accurately judge the use of buildings based solely on their visual appearance. Therefore, it is necessary to fuse multiple data so that the fused data includes both visual features and social features. The fused data is used as model input to enable the model to better identify the use of buildings. Existing heterogeneous data fusion methods often require a lot of data preprocessing and rule-based geographic association, which leads to a lot of information loss in the processing of heterogeneous data features. The subsequent fusion substructure can only fuse limited information, which limits the effect of the model.

[0102] The following is an example of an application scenario of the embodiment of the present application.

[0103] The embodiments of the present application are applied to the scenario of identifying the use of a building. At present, the identification of building use is relatively active in the field of academic research, but its application and promotion in social institutions and industries is relatively lacking. The test area for building use identification in the field of academic research generally covers an area of ​​several square kilometers to tens of square kilometers. A variety of finely processed data can be used in a relatively small range, without worrying about data acquisition bottlenecks and computing power bottlenecks. From the existing literature on building use identification, the integration and use of multiple heterogeneous data can effectively improve the accuracy of building use identification. At the same time, in order to realize industrial applications, it is necessary to upgrade and expand the existing building use identification algorithm so that it can be applied to a larger area and solve practical social problems. At the same time, there is work on building interpretation on a national scale in the field of remote sensing interpretation, which can be used as a reference for upgrading the building use algorithm to a larger area.

[0104] To identify building attributes nationwide, we need feature data that covers the entire country and an application framework that is suitable for model reasoning across the country. The aerospace information industry has developed rapidly in recent years. There are thousands of remote sensing satellites in orbit around the world, and a single satellite can transmit up to 20TB of data per day. Therefore, remote sensing data can cover the entire China. At the same time, as the resolution increases, remote sensing data (meter-level resolution) can provide rich ground object information to assist in interpreting building attributes.

[0105] After the model training is completed and the evaluation is available, the model can be inferred in the national area. The basic principle of the nationwide model inference is, for example, to divide the large remote sensing image into small remote sensing images that can be processed by the artificial intelligence model. After the artificial intelligence model completes the recognition, the recognition results of the small blocks are pieced back into the recognition results of the complete large geographical area. Among them, the remote sensing image itself carries accurate longitude and latitude information, and the recognition results also carry the corresponding longitude and latitude information, which can be associated with the longitude and latitude of the actual administrative area.

[0106] The shortcomings of current building attribute reasoning frameworks include:

[0107] (1) The existing building use identification framework is a small-scale regional framework and does not consider the scalability of large-scale reasoning applications.

[0108] Specifically, existing building use identification frameworks usually rely on street view image information, building vector information and building roof data as model inputs, and before model processing, perform data preprocessing steps to find the building's street view data and building roof based on the building's location, and associate the building roof with other attributes such as the building's street view. However, street view image information and building roof data are not necessarily available nationwide, and the data preprocessing steps are more dependent on manpower, resulting in technical difficulties when the solution is applied to nationwide building use prediction.

[0109] In some embodiments of the present application, heterogeneous data obtained by splicing remote sensing images and social attribute images are used as input to the model. Remote sensing images and social attribute images are data that are easy to obtain and process automatically nationwide, and the model does not rely on building roof data as input to the model, and there is no need to perform preprocessing steps to associate the building roof with other attributes of the building. This expands the scope of application of the solution and improves the scalability of the solution.

[0110] Furthermore, some embodiments of the present application perform training and identification of building uses at a pixel point granularity, and the building use corresponding to each pixel point in the sample remote sensing image is marked in the label used in training, so that the model can identify the building use corresponding to each pixel point in the remote sensing image during the inference stage. Compared with the method of training and identifying building uses at a functional area granularity, the training and identification method is more refined, reducing the impact of the variable number of buildings in a single image block in the input remote sensing image on the recognition results.

[0111] (2) The existing national building attribute recognition framework does not take into account heterogeneous data input and is difficult to use for accurate building use identification.

[0112] Some embodiments of the present application use remote sensing data of some regions and social attribute images of some regions to train models based on heterogeneous data input, and use national remote sensing data and national social attribute images to identify national building uses based on heterogeneous data input.

[0113] The shortcomings of the heterogeneous data fusion model structure of building use in related technologies are as follows: the model extracts the visual information and social information from the remote sensing data and POI data in the same functional area separately, and then fuses the two types of information. This method can be regarded as information fusion based on functional areas, which may be conducive to the division of functional areas, but is not conducive to the identification of the uses of independent buildings.

[0114] In some embodiments of the present application, since the remote sensing images and the social attribute images are spliced ​​and the heterogeneous data obtained by the splicing is used as the input of the model, when the shallow layer of the model (such as the convolution layer close to the input layer in the encoder) encodes the heterogeneous data (feature extraction), the visual features of the buildings in the remote sensing images and the social features of the buildings in the social attribute images can be fully integrated in the encoding process, and the amount of information of the pixels in the remote sensing images and the social attribute images in the shallow layer of the model is better retained, which reduces the risk of losing texture information compared to feature fusion in the deep layer of the model. This fusion method can be regarded as information fusion based on pixels, and the fusion features obtained by the fusion can fully express the visual features and social features of the buildings. After the decoder decodes the fusion features, the obtained building use recognition results are more accurate and more conducive to the recognition of the use of independent buildings.

[0115] The following is an example of the design of the building use identification framework in the embodiment of the present application.

[0116] Attached Figure 1 This is a schematic diagram of a building use identification architecture provided by an embodiment of the present application, as shown in the attached Figure 1As shown in the figure, in terms of data usage, remote sensing images are mainly used, and social attribute images are added as auxiliary. Remote sensing images and social attribute images can achieve nationwide coverage. Since remote sensing images contain longitude and latitude information, the building use identified based on remote sensing images has its own longitude and latitude information, which can be easily associated with the target building in subsequent use. For example, when the user inputs the longitude and latitude of the target building, the input longitude and latitude of the target building and the longitude and latitude information of the identified building use are matched to determine the building use of the target building corresponding to the input longitude and latitude.

[0117] In addition, a heterogeneous input fusion module is introduced for joint optimization. For example, the feature extraction of remote sensing images and the feature extraction of social attribute images can be performed simultaneously to achieve efficient feature utilization. For example, since heterogeneous data are encoded at the shallow level of the model, the visual features of buildings in remote sensing images and the social features of buildings in social attribute images can be fused at the shallow level of the model, reducing the risk of losing texture information due to feature fusion in the deep level of the model and improving the efficiency of feature utilization. This feature is not available in the heterogeneous national remote sensing interpretation framework.

[0118] In addition, batch recognition and end-to-end recognition are used in the recognition method. For example, batch recognition is to divide and splice the remote sensing images and social attribute images into blocks, input multiple heterogeneous data blocks into the model in parallel, and the encoder encodes multiple heterogeneous data blocks in parallel. The decoder decodes the multiple fusion features obtained by the encoder in parallel, and then splices the multiple recognition result data blocks obtained by parallel decoding to obtain the overall building use recognition result of the identified area. No assumptions are made about where the buildings are, and the architecture is simple and scalable. This method is not available in the existing heterogeneous data fusion building function recognition framework.

[0119] The following is an example of the model structure design of the embodiment of the present application.

[0120] In real life, buildings are large and small, so there are small building roofs in remote sensing images taken from satellites. Since what needs to be identified is the purpose of each building, the model needs to be able to have good recognition effect in small areas of the image. U-Net also has good effect on small object semantic tasks, so this model structure is used as the basic model.

[0121] U-Net was proposed by Olaf Ronneberger, Phillip Fischer, and Thomas Brox in 2015. The structure of U-Net is U-shaped. It consists of an encoder (Encoder, downsampling path) and a decoder (Decoder, upsampling path). The feature map in the encoder is combined with the feature map in the decoder through skip connections to retain more spatial information and improve positioning accuracy. U-Net combines the global and local features of the image through skip connections, which makes it perform well in detail recovery. At the same time, it also retains multi-level feature maps, which is very friendly to the semantic segmentation task of remote sensing data with a lot of image features. For example, please refer to the attached Figure 2 , attached Figure 2 is a schematic diagram of a U-Net structure for implementing a building use recognition model provided in an embodiment of the present application, such as Figure 2 As shown, the building use recognition model includes an encoder and a decoder. The encoder receives input heterogeneous data. The output layer of the encoder is connected to the input layer of the decoder, and there is a jump connection between the encoder and the decoder. The decoder performs classification based on the features of the heterogeneous data output by the encoder output layer and the shallow features of the heterogeneous data transmitted from the jump connection to output the building use recognition result.

[0122] The encoder part of the building use recognition model uses ResNet (residual network). Specifically, considering that the embodiment of the present application needs to realize use mining in addition to building location information mining, a deeper network structure is required to extract image features of multiple channels, and the features need to cover the bottom layer edge information; the middle layer texture information and the high layer use information.

[0123] Please refer to the attached Figure 3 , attached Figure 3It is a structural diagram of a ResNet residual block provided in an embodiment of the present application. ResNet is a relatively good choice as an encoder. It was proposed by Kaiming He et al. in 2015 and achieved excellent results in the ImageNet Large Scale Visual Recognition Challenge (ILSVRC). ResNet solves the gradient vanishing / exploding problem and degradation problem in deep networks by introducing "residual blocks", making it possible to train very deep networks. ResNet34 is a deep residual network containing 34 layers. Its basic building block is a residual block, each of which usually contains two 3x3 convolutional layers, each of which is followed by a batch normalization layer and a ReLU activation function. The key to the residual block is the introduction of a jump connection (or shortcut connection), which bypasses one or more layers and directly adds the input to the output after several layers of transformation. This structure allows information and gradients to propagate forward and backward more easily. ResNet34 uses a unique residual block design to solve the network degradation problem, gradient vanishing and explosion problem, and parameter efficiency problem caused by the deepening of the network depth.

[0124] In the embodiment of the present application, the model identifies the building purpose of each pixel, so the fusion between the visual features of the building in the remote sensing image and the social features of the building in the social attribute data (such as POI image) can be started in the shallow network to realize the identification of the building purpose of each pixel.

[0125] Considering that the processed POI image and remote sensing image have been aligned in terms of geographical location, this embodiment directly splices the channels of the POI image and the channels of the remote sensing image together and inputs them into the building use recognition model (such as the UNet network), so that the building use recognition model can fully extract and fuse the features of the POI image and the remote sensing image at every extraction depth from shallow to deep layers.

[0126] Please refer to the attached Figure 4 , attached Figure 4 is a comparison chart of heterogeneous data before and after use provided by the embodiment of the present application, Figure 4 (a) is a schematic diagram of single-source input of remote sensing images. Figure 4 (b) is a schematic diagram of the heterogeneous data fusion structure. Figure 4 Here, encoder represents the encoder in the model, decoder represents the decoder in the model, and remote sensing images are images with a length and width of 224 pixels and three channels (red channel, green channel, and blue channel). POI data are images with a length and width of 224 pixels and three layers (residential layer, commercial layer, and industrial layer).

[0127] As attached Figure 4 As shown in (a), before using heterogeneous data, the remote sensing image is input to the encoder, and the encoder encodes the remote sensing image to obtain the visual features of the building in the remote sensing image. The visual features of the building obtained by the encoder are output to the decoder, and the decoder decodes the visual features of the building to obtain the building use recognition result. Figure 4 As shown in (b), after using heterogeneous data, the remote sensing image and POI data are spliced ​​in the channel dimension to obtain heterogeneous fusion data, which contains 6 channels, of which 3 channels are the red channel, green channel and blue channel of the remote sensing image, and the other 3 channels are the residential layer, commercial layer and industrial layer of the POI data. The heterogeneous data is input to the encoder, and the encoder encodes the heterogeneous data to obtain the visual features and social features of the building in the heterogeneous data. The visual features and social features of the building obtained by the encoder are output to the decoder, and the decoder decodes the visual features and social features of the building to obtain the building use recognition result.

[0128] Attached Figure 5 : is a flow chart of a method for identifying building use provided in an embodiment of the present application, the method is applied to a computer device, and the method includes:

[0129] Step S510, obtaining a remote sensing image of the identified area and a social attribute image of the identified area.

[0130] The remote sensing image contains visual features of buildings in the identified area. In some embodiments, remote sensing images provided by remote sensing data sources that meet the conditions are screened by randomly sampling remote sensing data samples. For example, remote sensing images with a resolution of the target resolution, no obstructions (such as clouds), and light illumination brightness that meets the requirements are screened to make the remote sensing images clear and usable. The target resolution can be determined based on engineering performance and image clarity. For example, the target resolution is 1 meter. The 1-meter resolution remote sensing image source generally provides processed RGB images, which are not suitable for data clarity processing. Please refer to the attached Figure 6 , attached Figure 6 This is a comparison diagram of 10-meter and 1-meter resolution remote sensing images provided in the embodiment of the present application. Figure 6 It can be seen that in the same area, Figure 6 The 1-meter resolution remote sensing image shown in (b) is compared Figure 6 The 10-meter resolution remote sensing image shown in (a) contains more information from the surrounding Figure 6 In (b), each building and its surroundings can be seen more clearly.

[0131] The social attribute image contains the social features of the buildings in the identified area. The size of the social attribute image is the same as the size of the remote sensing image. For example, the number of pixels in the social attribute image is the same as the number of pixels in the remote sensing image. The geographic coordinates of the social attribute image correspond to the geographic coordinates of the remote sensing image. For example, the social attribute image and the remote sensing image are images of the same area.

[0132] There are many types of social attribute images. In some embodiments, the social attribute image includes point of interest POI data, population distribution data of the identified area, age distribution data of the identified area, GDP data of the identified area, or at least one item in the county annals of the identified area. These types of social attribute images are all related to the use of buildings. Taking population distribution data as an example, for example, at night, the probability that the use of buildings in densely populated areas is residential is relatively high, and during the day, the probability that the use of buildings in densely populated areas is commercial or industrial is relatively high. Taking GDP data as an example, for example, in areas with larger GDPs, most buildings are commercial in use. The following article focuses on using social attribute images as examples of POI data.

[0133] POI (Point Of Interest) data, usually translated into Chinese as "point of interest" or "information point", can be provided by map-related cooperative suppliers. POI data includes the location of POI in the identified area (such as latitude and longitude information), POI name information, POI address information, POI category and POI use information. From the category point of view, POI categories include "residential", "commercial" and "industrial" which are of concern to the insurance industry. In addition, where POIs of a certain category are concentrated, the nearby buildings are often of the same category.

[0134] In addition to remote sensing images from the sky, street view image information, point of interest (POI) data, and building vector information are all helpful for identifying building uses. However, street view image information and building vector information are difficult to obtain in full across the country, so the solution of fusing remote sensing images with POI data is considered. POI data is introduced in the process of model learning to assist the model in determining the use of buildings. The original POI data storage format is a table. Considering that the final modeling result is output in the form of an image, the POI data is converted into an image format, and the POI image and remote sensing image are fused with the machine vision model structure, which is conducive to the automatic identification of large-area building uses. In the form of unified input data and output data, and retaining the spatial correlation between POI and buildings. POI data in image form is, for example, a POI point map, and another example is a POI color temperature map.

[0135] For the generation process of the POI point map, for example, based on the position of the POI in the identified area in the POI data and the category of the POI, multiple POI point maps are created, the size of each POI point map is the same as the size of the remote sensing image, and each POI point map corresponds to a category of POI. Each POI point map is, for example, a binary image. The pixel value of the pixel point of the POI of the corresponding category in each POI point map is a first pixel value, and the pixel value of the pixel point of the POI of the corresponding category that does not exist in each POI point map is a second pixel value. The first pixel value and the second pixel value are different. For example, the first pixel value is 1, and the second pixel value is 0. After obtaining multiple POI point maps, the remote sensing image and the multiple POI point maps can be spliced ​​in the channel dimension to obtain heterogeneous data, and the heterogeneous data includes m channels of the remote sensing image and each channel of each POI point map in the multiple POI point maps.

[0136] As a specific example, the multiple POI point maps created include a residential POI point map, a commercial POI point map, and an industrial POI point map. The residential POI point map corresponds to POIs of the residential category, and the pixel values ​​of the pixels of the POIs of the residential category that exist in the residential POI point map are 1, and the pixel values ​​of the pixels of the POIs of the residential category that do not exist in the residential POI point map are 0. The commercial POI point map corresponds to POIs of the commercial category, and the pixel values ​​of the pixels of the POIs of the commercial category that exist in the commercial POI point map are 1, and the pixel values ​​of the pixels of the POIs of the commercial category that do not exist in the commercial POI point map are 0. The industrial POI point map corresponds to POIs of the industrial category, and the pixel values ​​of the pixels of the POIs of the industrial category that exist in the industrial POI point map are 1, and the pixel values ​​of the pixels of the POIs of the industrial category that do not exist in the industrial POI point map are 0. The residential POI point map, the commercial POI point map, and the industrial POI point map are three layers with the same size as the remote sensing image.

[0137] The POI point map conversion effect is as follows Figure 7 As shown, attached Figure 7 (a) shows the remote sensing image, Figure 7 (b) and Figure 7 (c) and Figure 7 (d) are all POI point maps with a resolution of 10 meters in the corresponding areas of remote sensing images. Figure 7 (b) shows a residential POI point map, and it can be seen that residential POI points are basically distributed in residential areas. Figure 7 (c) is a commercial POI point map. It can be seen that commercial POI points are more densely distributed along the streets and in densely populated areas. Figure 7 (d) is the industrial POI point map. It can be seen that there are relatively few industrial POIs, which are concentrated in the areas corresponding to the red or blue remote sensing roofs.

[0138] In some embodiments, the POI point map is a direct reflection of the POI location, but the distribution of values ​​in the POI point map is not continuous. When POI points are sparse, most of the values ​​in the entire POI point map are 0, which increases the difficulty of information extraction for the AI ​​model: it is difficult for the model to infer the distance from the rest of the image to the neighboring POI points based on the sparse POI points, and it is difficult to use POI data for building type recognition. In view of this, the distance color temperature map method is used to convert the POI point map into a POI color temperature map, thereby effectively describing the feature that "the use classification of adjacent POIs and buildings is correlated".

[0139] The POI color temperature map is also called the POI distance heat map. In the process of converting the POI point map into the POI color temperature map, specifically, based on the distance between each pixel point in each POI point map and the nearest adjacent POI pixel point, multiple POI color temperature maps are created, and each created POI color temperature map corresponds to a category of POI; each POI color temperature map includes the POI distance information in the corresponding POI point map, that is, the distance between each pixel point representing the POI in the POI point map and the nearest adjacent pixel point representing the POI (such as Euclidean distance).

[0140] For example, based on the distance between each pixel in the residential POI point map and the nearest adjacent POI pixel, a residential POI color temperature map is created; the residential POI color temperature map includes the distance between each pixel in the residential POI point map and the nearest adjacent residential POI pixel. Based on the distance between each pixel in the commercial POI point map and the nearest adjacent POI pixel, a commercial POI color temperature map is created; the commercial POI color temperature map includes the distance between each pixel in the commercial POI point map and the nearest adjacent commercial POI pixel. Based on the distance between each pixel in the industrial POI point map and the nearest adjacent POI pixel, an industrial POI color temperature map is created; the industrial POI color temperature map includes the distance between each pixel in the industrial POI point map and the nearest adjacent industrial POI pixel.

[0141] For example, the specific calculation formula for the pixel value of each pixel in the POI color temperature map is as follows:

[0142]

[0143] DH stands for distance heat map, which indicates the coordinates of a pixel representing a POI within a specified range. The Euclidean distance formula is used to calculate the distance from the pixel to the POI. After the distances from the pixel to all surrounding POIs are calculated, the closest distance from the pixel to all POIs is selected.

[0144] In some embodiments, different maximum ranges of influence are configured for different categories of POI, and a POI color temperature map is generated based on the distance between POI pixels in the POI point map and the maximum range of influence. For example, considering that there are more data in the unit range of the accommodation area and the commercial area, it can be converted into a relatively limited influence of a certain accommodation area or commercial area on the surrounding area, and the industrial area has less data and is located in a more remote location, which can be converted into a greater influence of the industrial area on the surrounding area. Based on this, the maximum range of influence of the POI point map of the industrial category is configured to be greater than the maximum range of influence of the POI point map of the residential category, and greater than the maximum range of influence of the POI point map of the commercial category. In the process of generating the POI color temperature map, for each pixel point in each POI point map, the distance between the pixel point and each POI pixel point belonging to the maximum range of influence in the POI point map to which the pixel point belongs is determined; the closest distance value between the pixel point and all POI pixels is selected as the pixel value of the pixel point in the POI color temperature map. Exemplarily, the maximum range of influence of the accommodation area or commercial area is set to 500 meters, and the maximum range of influence of the industrial layer is set to 1000 meters.

[0145] For example, please refer to the attached Figure 8 , attached Figure 8 This is a diagram showing the effect of implementing a 1-meter resolution POI distance color temperature map provided in an embodiment of the present application, Figure 8 (a) shows the remote sensing image, Figure 8 (b) shows the residential POI distance color temperature map of the corresponding area of ​​the remote sensing image. It can be seen that the residential POI points are basically distributed in the residential area; Figure 8 (c) shows the commercial POI distance color temperature map. It can be seen that the commercial POI points are more densely distributed on the street side and in densely populated areas. Figure 8 (d) shows the industrial POI distance color temperature map. It can be seen that there are relatively few industrial POIs, and most of them appear in areas where blue greenhouses are concentrated.

[0146] In actual modeling, using distance color temperature maps has two advantages. First, if the POI point information is not accurate, the surrounding buildings can still be affected by the POI point according to the closest distance to the POI, thereby increasing the accuracy of building use classification; second, it also solves the problem of buildings with similar uses at adjacent POI points, and the distance can be used to determine whether the nearest building is of the same type. At the same time, even after cropping, blocks with POI points can still affect surrounding blocks without POI point data, making the most of POI data for use classification.

[0147] Step S520, splicing the remote sensing image and the social attribute image in the channel dimension to obtain heterogeneous data.

[0148] Remote sensing images include m channels, social attribute images include n channels, and heterogeneous data include m channels of remote sensing images and n channels of social attribute images. For example, heterogeneous data contains a total of (m+n) channels, of which the first m channels are channels of remote sensing images, and the last n channels are channels of social attribute images. Among them, m is, for example, the number of color channels (or visible light channels) in the remote sensing image, and n is the number of social attribute categories in the social attribute image. For example, m is 3, and the m channels are the red channel, green channel, and blue channel of the remote sensing image. n is 3, the social attribute image is a POI image, and the n channels are residential POI images, commercial POI images, and industrial POI images.

[0149] Step S530, inputting the heterogeneous data into a building use identification model, the building use identification model including an encoder and a decoder;

[0150] The structure of the building use recognition model is described above. In some embodiments, the building use recognition model has a U-Net network structure, and the building use recognition model includes an encoder and a decoder. Optionally, the encoder is a residual network. Alternatively, the encoder is a transformer.

[0151] Step S540, encoding the heterogeneous data through an encoder to obtain fusion features of the remote sensing image and the social attribute image.

[0152] The encoder includes one or more convolutional layers, and the encoder convolves (forward propagation) the heterogeneous data through the convolutional layers. During the convolution process, the visual features of the buildings in the remote sensing images and the social features of the buildings in the social attribute images are further integrated. The final encoded fusion features include the visual features of the buildings in the remote sensing images and the social features of the buildings in the social attribute images.

[0153] Step S550: decode the fused features through a decoder to obtain a building use recognition result of the identified area.

[0154] In some embodiments, pixel-level recognition is adopted, and the fusion features are decoded by a decoder to obtain a building use recognition result for each pixel in the sample remote sensing image. For example, the building use recognition result is in the form of a mask image, and the pixel value of each pixel in the building use recognition result is used to characterize the building use of the corresponding pixel. For example, if the pixel value of the pixel with coordinates (i, j) in the building use recognition result is 1, it means that the building use of the pixel with coordinates (i, j) in the remote sensing image is residential, if the pixel value of the pixel with coordinates (i, j) in the building use recognition result is 2, it means that the building use of the pixel with coordinates (i, j) in the remote sensing image is commercial, if the pixel value of the pixel with coordinates (i, j) in the building use recognition result is 3, it means that the building use of the pixel with coordinates (i, j) in the remote sensing image is industrial.

[0155] The method provided in the embodiment of the present application obtains a remote sensing image of the identified area and a social attribute image of the identified area, and splices the remote sensing image and the social attribute image in the channel dimension to obtain heterogeneous data. Due to the splicing of the two types of data, the fusion of the visual features in the remote sensing image and the social features in the social attribute image is achieved, and the obtained heterogeneous data contains both the visual features in the remote sensing image and the social features in the social attribute image. The heterogeneous data is input into the building use recognition model, so that the input obtained by the model contains both visual features and social features. The heterogeneous data is encoded by the encoder in the model, so that the visual features in the remote sensing image and the social features in the social attribute image can be further fused in the shallow layer of the model (such as the convolution layer in the encoder), which can be regarded as information fusion based on pixel points, which is more conducive to the use recognition of independent buildings, thereby improving the accuracy of building use recognition.

[0156] In some embodiments, the input image is cropped into small blocks, and then the output results of the model are spliced ​​back into a large block, so as to reduce the risk of the input image exceeding the range that the model can handle and causing the solution to be unavailable, and to achieve batch recognition, thereby improving the efficiency of recognition of large-sized remote sensing images. For example, after executing step S510, the remote sensing image is divided into blocks according to the preset image size to obtain multiple remote sensing image blocks; the social attribute image is divided into blocks according to the preset image size to obtain multiple social attribute image blocks; in the process of executing step S520, the remote sensing image blocks and the social attribute image blocks with the same geographical location coordinates are spliced ​​in the channel dimension to obtain multiple heterogeneous data blocks, each heterogeneous data block includes m channels of a remote sensing image block with the same geographical location coordinates and n channels of a social attribute image block. In the process of executing step S530, multiple heterogeneous data blocks are input into the building use recognition model in parallel, and the multiple heterogeneous data blocks are equivalent to a batch. During the execution of step S540, multiple heterogeneous data blocks are encoded in parallel by an encoder to obtain multiple fused features, each fused feature includes a feature of a remote sensing image block and a feature of a social attribute image block; during the execution of step S550, multiple fused features are decoded in parallel by a decoder to obtain multiple recognition result data blocks; multiple recognition result data blocks are spliced ​​according to geographic location coordinates to obtain a building use recognition result of the identified area.

[0157] The following is an example of the training method of the building use recognition model.

[0158] Exemplarily, the training method of the building use recognition model includes the following steps.

[0159] Step 1: Obtain a sample remote sensing image of the sample area and a sample social attribute image of the sample area.

[0160] Regarding the selection of sample remote sensing images, in some embodiments, the sample remote sensing images of the sample area include remote sensing images of provincial-level administrative regions across the country, remote sensing images with building distribution density higher than a density threshold, remote sensing images with building distribution density lower than a density threshold, and remote sensing images with building use identification result deviations higher than a deviation threshold output by a building use identification model in a test set.

[0161] For example, when selecting sample remote sensing images, considering that the buildings in regions with different terrains and different cultural preferences in China are very different, in order to make the sample remote sensing images cover the buildings in various terrains and various cultural preferences, remote sensing images of various provincial administrative regions across the country (data for Taiwan, China is temporarily unavailable) are sampled as sample remote sensing images. In a possible implementation, the remote sensing images of each provincial administrative region are randomly cut into blocks, for example, into 2 km*2 km remote sensing image blocks, and 3 blocks are randomly selected from the remote sensing image blocks of each provincial administrative region as sample remote sensing images.

[0162] For example, in order to improve the recognition ability of the model, difficult samples are added to the sample remote sensing images. Remote sensing images with too sparse building distribution and remote sensing images with too dense building distribution are both difficult samples for the model, so remote sensing images with building distribution density higher than the density threshold and remote sensing images with building distribution density lower than the density threshold are selected as sample remote sensing images.

[0163] For example, during the i-th round of training of the model, the performance of the model on the test set can be tested. If the model performs poorly on some sample remote sensing images in the test set (such as the deviation of the building use recognition result is higher than the deviation threshold), these sample remote sensing images can be used as difficult samples. During the (i+1)-th round of training of the model, these sample remote sensing images can be used to train the model.

[0164] Step 2: Splice the sample remote sensing image and the sample social attribute image in the channel dimension to obtain sample heterogeneous data.

[0165] The sample remote sensing image includes m channels, the sample social attribute image includes n channels, and the sample heterogeneous data includes m channels of the sample remote sensing image and n channels of the sample social attribute image.

[0166] Step 3, input the sample heterogeneous data into the building use recognition model; encode the sample heterogeneous data through the encoder to obtain the fusion features of the sample remote sensing image and the sample social attribute image; decode the fusion features of the sample remote sensing image and the sample social attribute image through the decoder to obtain the building use recognition result of the sample remote sensing image.

[0167] In some embodiments, during the model training phase, the model is used to perform pixel-by-pixel recognition, for example, by decoding the fused features through a decoder to obtain a building use recognition result for each pixel in the sample remote sensing image.

[0168] In some embodiments, during the model training phase, the input image is cropped into small blocks, and then the output results of the model are spliced ​​back into large blocks. For example, the sample remote sensing image is divided into blocks according to a preset image size to obtain multiple sample remote sensing image blocks; the sample social attribute image is divided into blocks according to a preset image size to obtain multiple sample social attribute image blocks; the sample remote sensing image blocks and the sample social attribute image blocks with the same geographical location coordinates are spliced ​​in the channel dimension to obtain multiple heterogeneous data blocks, each heterogeneous data block includes m channels of a sample remote sensing image block with the same geographical location coordinates and n channels of a sample social attribute image block. Multiple heterogeneous data blocks are input into the building use recognition model in parallel, and the multiple heterogeneous data blocks are equivalent to a batch. The encoder encodes multiple heterogeneous data blocks in parallel to obtain multiple fused features, each of which includes the features of a sample remote sensing image block and the features of a sample social attribute image block; the decoder decodes the multiple fused features in parallel to obtain multiple building use data blocks, each of which includes a building use recognition result in a remote sensing image of a preset image size; the multiple building use data blocks are spliced ​​according to the geographic location coordinates to obtain the building use recognition result of the sample area.

[0169] Step 4: Determine the building use loss based on the building use identification result of the sample area and the building use label of the sample area.

[0170] The building use label is used to indicate the building use of the sample area. In some embodiments, pixel-level annotation is used, that is, the building use corresponding to each pixel is annotated in the building use label. For example, the pixel value of each pixel in the building use label is a first pixel value, a second pixel value, a third pixel value, or a fourth pixel value. Among them, the first pixel value represents the background, the second pixel value represents that the building use is residential, the third pixel value represents that the building use is commercial, and the fourth pixel value represents that the building use is industrial. For example, the pixel value of the pixel in the building use label is 0, 1, 2, or 3, 0 represents the background, 1 represents residential, 2 represents commercial, and 3 represents industry.

[0171] Building label data can be provided by a map-related cooperative supplier. The data format of building label data is a shp surface feature file, covering the core areas of 68 core cities in China, including information such as the building location, roof shape, building purpose, and building height. Among them, the building purpose is the content that the model needs to learn and output, so this building data is used as the label for model learning. At present, the model outputs a raster image format that is easy to calculate, so it is necessary to convert the building shp surface file into a raster image format, and align the converted raster position and raster size to the remote sensing raster file of the corresponding area. The label layers to be converted include the building purpose layer. In the building purpose layer, use a mask to set the file to single-band data, and use a grayscale image to express the labeled data.

[0172] The building use loss is used to characterize the deviation between the building use recognition result and the building use label. In some embodiments, the building use loss is determined based on the probability that the building use of each pixel in the building use recognition result is recognized as residential, the probability that the building use of each pixel is recognized as commercial, the probability that the building use of each pixel is recognized as industrial, and the pixel value of each pixel in the building use label.

[0173] Step 5: Adjust the parameters of the building use identification model based on the building use loss so that the deviation of the building use identification result output by the building use identification model meets the conditions.

[0174] For example, based on the building use loss, the back propagation method is used to adjust the parameters of the building use recognition model, so that the deviation of the building use recognition result output by the building use recognition model continues to decrease until the deviation of the building use recognition result output by the building use recognition model is less than the deviation threshold, and the training is terminated.

[0175] After the building use recognition model is trained, the building use recognition model can be used to perform building use recognition tasks for remote sensing images of unknown use, that is, to perform the attached Figure 5 The method shown.

[0176] In order to test the help of heterogeneous data fusion in building use identification, an ablation experiment was conducted based on the remote sensing image of the identified area of ​​500m*500m. The specific experimental results are shown in the following table. In Experiment 1, the input data of the model is the remote sensing image, and the output of the model is the building use identification result; on the basis of Experiment 1, Experiment 2 fuses the POI image and the remote sensing image into heterogeneous data as the input of the model, and the output of the model is the building use identification result. Among them, the F1 index is a binary classification evaluation index that can measure both accuracy and recall.

[0177] Building use identification ablation experiment table

[0178]

[0179] By comparing Experiment 1 and Experiment 2, it can be found that the classification effects of various building uses are improved after the introduction of POI data, which shows that POI data is beneficial to the task of building use classification and also illustrates the effectiveness of the currently designed heterogeneous data fusion structure.

[0180] The national building use identification model framework provided in this embodiment utilizes nationwide remote sensing data and nationwide social attribute data, adopts a partial area training and nationwide reasoning approach, and takes into account both performance and accuracy.

[0181] In the visual feature and social feature fusion model provided in this embodiment, by directly performing channel stitching on the input remote sensing image and POI data, the visual features in the remote sensing image and the social features in the POI data can be effectively fused, thereby improving the accuracy of pixel-level granularity building use recognition.

[0182] This embodiment uses social attribute data in the process of executing the national building use identification task, thereby improving the accuracy of building use identification while ensuring that the identification scope remains unchanged.

[0183] Fig. 9 9 is a schematic diagram of a structure of a building use identification device provided in an embodiment of the present application, wherein the device 900 comprises:

[0184] An acquisition unit 910 is used to acquire a remote sensing image of the identified area and a social attribute image of the identified area;

[0185] A stitching unit 920 is used to stitch the remote sensing image and the social attribute image in a channel dimension to obtain heterogeneous data, wherein the remote sensing image includes m channels, the social attribute image includes n channels, and the heterogeneous data includes the m channels of the remote sensing image and the n channels of the social attribute image;

[0186] The model processing unit 930 is used to input the heterogeneous data into a building use recognition model, and the building use recognition model includes an encoder and a decoder; the heterogeneous data is encoded by the encoder to obtain a fusion feature of the remote sensing image and the social attribute image, and the fusion feature includes the visual features of the building in the remote sensing image and the social features of the building in the social attribute image; the fusion feature is decoded by the decoder to obtain a building use recognition result of the identified area.

[0187] In some embodiments, the social attribute image includes point of interest (POI) data, and the POI data includes the location of the POI in the identified area and the category of the POI;

[0188] The acquisition unit 910 is used to create a plurality of POI point maps based on the POI data, wherein the size of each of the POI point maps is the same as the size of the remote sensing image, each of the POI point maps corresponds to a category of POI, the pixel values ​​of the pixel points of the POI of the corresponding category in each of the POI point maps are first pixel values, and the pixel values ​​of the pixel points of the POI of the corresponding category that do not exist in each of the POI point maps are second pixel values; and create a plurality of POI color temperature maps based on the distance between each pixel point in each of the POI point maps and the nearest adjacent POI pixel point;

[0189] Correspondingly, the stitching unit 920 is used to stitch the remote sensing image and the plurality of POI color temperature maps in the channel dimension.

[0190] In some embodiments, the acquisition unit 910 is used to determine, for each pixel point in each of the POI point maps, the distance between the pixel point and each POI pixel point within the maximum range of influence in the POI point map to which the pixel point belongs, wherein in each of the POI point maps, the maximum range of influence of the POI point map of the industrial category is greater than the maximum range of influence of the POI point map of the residential category, and greater than the maximum range of influence of the POI point map of the commercial category; and select the closest distance value between the pixel point and all POI pixel points as the pixel value of the pixel point in the POI color temperature map.

[0191] In some embodiments, the stitching unit 920 is used to divide the remote sensing image into blocks according to a preset image size to obtain a plurality of remote sensing image blocks; divide the social attribute image into blocks according to the preset image size to obtain a plurality of social attribute image blocks; and stitch the remote sensing image blocks and the social attribute image blocks having the same geographical location coordinates in the channel dimension;

[0192] Accordingly, the heterogeneous data includes a plurality of heterogeneous data blocks, each of which includes m channels of a remote sensing image block and n channels of a social attribute image block with the same geographical location coordinates;

[0193] Correspondingly, the model processing unit 930 is used to encode the multiple heterogeneous data blocks in parallel through the encoder to obtain multiple fusion features, each of which includes a feature of the remote sensing image block and a feature of the social attribute image block; decode the multiple fusion features in parallel through the decoder to obtain multiple recognition result data blocks; and splice the multiple recognition result data blocks according to the geographic location coordinates to obtain the building use recognition result of the identified area.

[0194] In some implementations, the acquisition unit 910 is further used to acquire a sample remote sensing image of the sample area and a sample social attribute image of the sample area;

[0195] The stitching unit 920 is further used to stitch the sample remote sensing image and the sample social attribute image in the channel dimension to obtain sample heterogeneous data, the sample remote sensing image includes m channels, the sample social attribute image includes n channels, and the sample heterogeneous data includes the m channels of the sample remote sensing image and the n channels of the sample social attribute image;

[0196] The model processing unit 930 is also used to input the sample heterogeneous data into the building use recognition model; encode the sample heterogeneous data through the encoder to obtain the fusion features of the sample remote sensing image and the sample social attribute image; decode the fusion features of the sample remote sensing image and the sample social attribute image through the decoder to obtain the building use recognition result of the sample remote sensing image; determine the building use loss based on the building use recognition result of the sample area and the building use label of the sample area, the building use label is used to indicate the building use of the sample area, and the building use loss is used to characterize the deviation between the building use recognition result and the building use label; adjust the parameters of the building use recognition model based on the building use loss so that the deviation of the building use recognition result output by the building use recognition model meets the conditions.

[0197] In some embodiments, the model processing unit 930 is used to decode the fusion feature through the decoder to obtain a building use identification result for each pixel in the sample remote sensing image; accordingly, the pixel value of each pixel in the building use label is a first pixel value, a second pixel value, a third pixel value or a fourth pixel value, the first pixel value represents the background, the second pixel value represents that the building use is residential, the third pixel value represents that the building use is commercial, and the fourth pixel value represents that the building use is industrial; based on the probability of the building use of each pixel in the building use identification result being identified as residential, the probability of the building use of each pixel being identified as commercial, the probability of the building use of each pixel being identified as industrial, and the pixel value of each pixel in the building use label, the building use loss is determined.

[0198] In some embodiments, the sample remote sensing images of the sample area include remote sensing images of provincial administrative regions across the country, remote sensing images with building distribution density higher than a density threshold, remote sensing images with building distribution density lower than a density threshold, and remote sensing images with building use identification result deviations higher than a deviation threshold output by the building use identification model in the test set.

[0199] In some embodiments, the social attribute image includes at least one of point of interest (POI) data, population distribution data of the identified area, age distribution data of the identified area, GDP data of the identified area, or county annals of the identified area.

[0200] Fig. 9 600 is a schematic diagram of a computer device provided in an embodiment of the present application. The computer device 600 includes: a processor 601, the processor 601 is coupled to a memory 602, the memory 602 stores at least one computer program instruction, and the at least one computer program instruction is loaded and executed by the processor 601, so that the computer device 600 implements Figure 5 The method provided by the embodiment.

[0201] In some embodiments, a computer-readable storage medium is further provided, wherein the storage medium stores at least one instruction, and when the instruction is executed on a computer, the computer executes the above-mentioned Figure 5 The method provided by the embodiment.

[0202] In some embodiments, a computer program product is also provided. The computer program product includes one or more computer program instructions. When the computer program instructions are loaded and executed by a computer, the computer executes the above Figure 5 The method provided by the embodiment.

[0203] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0204] A refers to B, which means that A is the same as B or A is a simple variant of B.

[0205] The terms "first" and "second" in the description and claims of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order of objects, and cannot be understood as indicating or implying relative importance. For example, a first pixel value and a second pixel value are used to distinguish different pixel values, rather than to describe a specific order of pixel values, and cannot be understood as meaning that the first pixel value is more important than the second pixel value.

[0206] In the embodiments of the present application, unless otherwise specified, "at least one" means one or more, and "a plurality" means two or more. For example, a plurality of pixel values ​​means two or more pixel values.

[0207] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in accordance with the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk SolidState Disk (SSD)), etc.

[0208] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for identifying building use, characterized in that: The method comprises: Acquire a remote sensing image of the identified area and a social attribute image of the identified area; splicing the remote sensing image and the social attribute image in a channel dimension to obtain heterogeneous data, wherein the remote sensing image includes m channels, the social attribute image includes n channels, and the heterogeneous data includes the m channels of the remote sensing image and the n channels of the social attribute image; Inputting the heterogeneous data into a building use identification model, wherein the building use identification model includes an encoder and a decoder; The heterogeneous data is encoded by the encoder to obtain fusion features of the remote sensing image and the social attribute image, wherein the fusion features include visual features of the building in the remote sensing image and social features of the building in the social attribute image; The fusion feature is decoded by the decoder to obtain a building use recognition result of the identified area.

2. The method according to claim 1, characterized in that The social attribute image includes point of interest (POI) data, and the POI data includes the location of the POI in the identified area and the category of the POI; The step of acquiring the remote sensing image of the identified area and the social attribute image of the identified area includes: Creating a plurality of POI point maps based on the POI data, wherein the size of each of the POI point maps is the same as the size of the remote sensing image, each of the POI point maps corresponds to a category of POI, the pixel values ​​of the pixel points of the POI of the corresponding category in each of the POI point maps being a first pixel value, and the pixel values ​​of the pixel points of the POI of the corresponding category not being in each of the POI point maps being a second pixel value; Creating a plurality of POI color temperature maps based on the distance between each pixel in each of the POI point maps and the nearest adjacent POI pixel; Accordingly, the step of splicing the remote sensing image and the social attribute image in the channel dimension includes: The remote sensing image and the plurality of POI color temperature maps are spliced ​​in a channel dimension.

3. The method according to claim 2, characterized in that The step of creating a plurality of POI color temperature maps based on the distance between each pixel point in each POI point map and the nearest adjacent POI pixel point comprises: For each pixel in each of the POI point maps, determine the distance between the pixel and each POI pixel belonging to the maximum range of influence in the POI point map to which the pixel belongs, wherein in each of the POI point maps, the maximum range of influence of the POI point map of the industrial category is greater than the maximum range of influence of the POI point map of the residential category, and greater than the maximum range of influence of the POI point map of the commercial category; The closest distance value between the pixel point and all POI pixels is selected as the pixel value of the pixel point in the POI color temperature map.

4. The method according to claim 1, characterized in that: The step of splicing the remote sensing image and the social attribute image in the channel dimension includes: Dividing the remote sensing image into blocks according to a preset image size to obtain a plurality of remote sensing image blocks; Dividing the social attribute image into blocks according to the preset image size to obtain a plurality of social attribute image blocks; Splicing the remote sensing image blocks and the social attribute image blocks with the same geographical location coordinates in the channel dimension; Accordingly, the heterogeneous data includes a plurality of heterogeneous data blocks, each of which includes m channels of a remote sensing image block and n channels of a social attribute image block with the same geographical location coordinates; Accordingly, encoding the heterogeneous data by the encoder to obtain the fusion features of the remote sensing image and the social attribute image includes: The encoder encodes the plurality of heterogeneous data blocks in parallel to obtain a plurality of fused features, each of the fused features comprising a feature of the remote sensing image block and a feature of the social attribute image block; The decoding of the fusion feature by the decoder to obtain the building use recognition result of the identified area includes: Decoding the multiple fusion features in parallel by the decoder to obtain multiple recognition result data blocks; The multiple recognition result data blocks are spliced ​​according to the geographical location coordinates to obtain the building use recognition result of the recognized area.

5. The method according to claim 1, characterized in that The method further comprises: Acquire a sample remote sensing image of a sample area and a sample social attribute image of the sample area; The sample remote sensing image and the sample social attribute image are spliced ​​in the channel dimension to obtain sample heterogeneous data, wherein the sample remote sensing image includes m channels, the sample social attribute image includes n channels, and the sample heterogeneous data includes the m channels of the sample remote sensing image and the n channels of the sample social attribute image; Inputting the sample heterogeneous data into the building use identification model; Encoding the sample heterogeneous data by the encoder to obtain fusion features of the sample remote sensing image and the sample social attribute image; Decoding the fusion features of the sample remote sensing image and the sample social attribute image through the decoder to obtain a building use recognition result of the sample remote sensing image; Determine a building use loss based on a building use identification result of the sample area and a building use label of the sample area, wherein the building use label is used to indicate the building use of the sample area, and the building use loss is used to characterize a deviation between the building use identification result and the building use label; The parameters of the building use identification model are adjusted based on the building use loss so that the deviation of the building use identification result output by the building use identification model meets the conditions.

6. The method according to claim 5, characterized in that The decoding of the fusion features of the sample remote sensing image and the sample social attribute image by the decoder to obtain the building use recognition result of the sample remote sensing image includes: Decoding the fused features by the decoder to obtain a building use recognition result for each pixel in the sample remote sensing image; Accordingly, the pixel value of each pixel point in the building use label is a first pixel value, a second pixel value, a third pixel value or a fourth pixel value, the first pixel value represents the background, the second pixel value represents that the building use is residential, the third pixel value represents that the building use is commercial, and the fourth pixel value represents that the building use is industrial. The determination of the building use loss based on the building use recognition result of the sample area and the building use label of the sample area includes: The building use loss is determined based on the probability that the building use of each pixel point in the building use identification result is identified as residential, the probability that the building use of each pixel point is identified as commercial, the probability that the building use of each pixel point is identified as industrial, and the pixel value of each pixel point in the building use label.

7. The method according to claim 5, characterized in that The sample remote sensing images of the sample area include remote sensing images of provincial administrative regions across the country, remote sensing images with building distribution density higher than a density threshold, remote sensing images with building distribution density lower than a density threshold, and remote sensing images with building use recognition result deviations higher than a deviation threshold output by the building use recognition model in the test set.

8. A building use identification device, characterized in that: The device comprises: An acquisition unit, used to acquire a remote sensing image of the identified area and a social attribute image of the identified area; A stitching unit, used for stitching the remote sensing image and the social attribute image in a channel dimension to obtain heterogeneous data, wherein the remote sensing image includes m channels, the social attribute image includes n channels, and the heterogeneous data includes the m channels of the remote sensing image and the n channels of the social attribute image; A model processing unit is used to input the heterogeneous data into a building use recognition model, the building use recognition model includes an encoder and a decoder; the heterogeneous data is encoded by the encoder to obtain a fusion feature of the remote sensing image and the social attribute image, the fusion feature includes the visual features of the building in the remote sensing image and the social features of the building in the social attribute image; the fusion feature is decoded by the decoder to obtain a building use recognition result of the identified area.

9. A computer device, characterized in that: The computer device comprises: a processor, the processor is coupled to a memory, the memory stores at least one computer program instruction, and the at least one computer program instruction is loaded and executed by the processor, so that the computer device implements the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one instruction, and when the instruction is executed on a computer, the computer executes the method according to any one of claims 1 to 7.

Images