Urban land use function recognition method and system based on visual and semantic information

Abstract

The present invention discloses a method for identifying urban land use functions based on visual and semantic information. The method comprises acquiring remote sensing image data and building-related data of a specified area to obtain a training dataset; constructing a regional visual representation network; constructing a regional semantic representation network; constructing a classification layer and combining the regional visual representation network and the regional semantic representation network to construct an original model for identifying urban land use functions; training the original model for identifying urban land use functions using the training dataset to obtain an urban land use function identification model; and using the urban land use function identification model to identify urban land use functions within a target area. The present invention also discloses a system for implementing the method for identifying urban land use functions based on visual and semantic information. The present invention not only identifies urban land use functions but also does so with greater reliability and accuracy.

Description

Technical Field

[0001] The present invention belongs to the field of geographic information technology, and in particular relates to a method and system for identifying urban land use functions based on visual and semantic information. Background Art

[0002] Urban land use patterns are the functional attributes of specific areas within urban space. The various land use patterns interact with each other to form the functional structure of urban space. These functional attributes reflect the state of the city and provide crucial foundational data for urban planning and management, infrastructure development, and travel planning. Therefore, accurate and reliable identification of urban land use functions is crucial.

[0003] Early identification of urban land use functions mostly relied on field surveys: statistically analyzing and compiling survey data to generate a series of indicators to characterize a city's functional structure. This approach was not only time-consuming and labor-intensive, but the data obtained was also highly subjective and lacked timeliness. With the advancement of remote sensing technology, researchers have proposed methods based on remote sensing interpretation. These methods map superficial visual features of land cover (such as spectral and texture characteristics) obtained from high-resolution remote sensing imagery to urban land use function types, thereby enabling identification of urban land use functions. Deep learning provides technical support for extracting deeper visual information from high-resolution remote sensing imagery. A range of deep learning-based methods for identifying urban land use functions from remote sensing imagery have emerged. However, due to the complexity of urban land use function structures, these methods rely solely on visual information to distinguish between different land use functions, resulting in poor reliability and accuracy. Summary of the Invention

[0004] One of the objectives of the present invention is to provide a method for identifying urban land use functions based on visual and semantic information with high reliability and good accuracy.

[0005] A second object of the present invention is to provide a system for realizing the urban land use function identification method based on visual and semantic information.

[0006] The method for identifying urban land use functions based on visual and semantic information provided by the present invention comprises the following steps:

[0007] S1. Acquire remote sensing image data and building data of a set area and produce a training data set; the remote sensing image data is used to provide visual information, and the building data is used to provide semantic information;

[0008] S2. Based on the Inception V3 convolutional neural network, prune and supplement the Inception V3 convolutional neural network to build a regional visual representation network for extracting visual features from remote sensing image data;

[0009] S3. Based on the language representation model BERT, a regional semantic representation network is constructed to extract semantic features from building data.

[0010] S4. Constructing a classification layer based on the fully connected layer and the softmax function, and combining the regional visual representation network constructed in step S2 and the regional semantic representation network constructed in step S3 to construct an original model for urban land use function recognition;

[0011] S5. Using the training data set obtained in step S1, the original model for urban land use function identification constructed in step S4 is trained to obtain an urban land use function identification model;

[0012] S6. Use the urban land use function identification model obtained in step S5 to complete the identification of urban land use functions in the target area.

[0013] The step S1 specifically includes the following steps:

[0014] Obtain remote sensing image data and building-related data of the set area;

[0015] Slice the remote sensing image data and building data to obtain input image data for land use function identification and input building data for land use function identification respectively;

[0016] Marking several land use function recognition input image data to obtain visual training samples;

[0017] According to the spatial range of the visual training samples, the corresponding building data is extracted to construct semantic training samples.

[0018] The step S1 specifically includes the following steps:

[0019] Acquire remote sensing image data, 3D building outline data, POI (Point of Interest) data, and AOI (Area of ​​Interest) data within the vector boundary of the set area;

[0020] Slice and pre-process the remote sensing image data to generate the corresponding input image data for land use function identification;

[0021] According to the categories of industrial land, commercial land, residential land, green land, urban village land, public management service land and unused land, a number of land use function identification input image data are selected for annotation to obtain visual training samples;

[0022] Using the overlay analysis method, the functional attributes of buildings are assigned values ​​according to the POI and AOI attributes;

[0023] Slice the building data and pre-process it to generate input semantic data for land use function recognition corresponding to the input image data;

[0024] According to the spatial range of each remote sensing image in the visual training sample, the corresponding building data is extracted to obtain semantic training samples that correspond one to one with the visual training samples;

[0025] The visual training samples and the corresponding semantic training samples together constitute a training data set.

[0026] The step S2 specifically includes the following steps:

[0027] Get the network data of the Inception V3 architecture convolutional neural network;

[0028] The three Inception A modules, one Inception B module, four Inception C modules, one Inception D module, and two Inception E modules in the Inception V3 convolutional neural network were retained. The GlobalAveragePooling2D layer and the fully connected layer at the top of the Inception V3 convolutional neural network were then removed. Finally, the Dropout layer, the Flatten layer, and the Dense layer were added. Finally, a regional visual representation network was constructed.

[0029] In the constructed regional visual representation network: for the input image data, the regional visual representation network first passes through the convolution layer and pooling layer to gradually reduce the size of the feature map, helping to extract low-level image features such as edges and textures; further, multiple Inception modules are used to extract features of different scales. Each Inception module consists of multiple parallel convolution layers and pooling layers to capture feature information at different scales. After the last Inception module, the network passes through the Dropout layer to prevent overfitting, and the Flatten layer expands it into a one-dimensional vector. Finally, the Dense layer undergoes nonlinear transformation and maps it to the output space.

[0030] The visual training samples are input into the regional visual representation network, and the regional visual representation network outputs the corresponding visual representation vector.

[0031] The step S3 specifically includes the following steps:

[0032] Calculate the floor area of ​​each building;

[0033] Based on the obtained building area of ​​each building, the building attributes are converted into semantic text and used as the input of the regional semantic representation prediction network;

[0034] Based on the language representation model BERT, a regional semantic representation network is constructed;

[0035] The input semantic data is input into the regional semantic representation network, and the regional semantic representation network outputs the corresponding semantic representation vector.

[0036] The step S3 specifically includes the following steps:

[0037] The building area of ​​each building is calculated using the following formula:

[0038]

[0039] In the formula is the building area of ​​the jth building; L bf is the number of floors of the j-th building; LS i is the area of ​​the i-th floor of the j-th building;

[0040] Given a building dataset {bf1, bf2, ..., bf n}, where bf n For the nth building in the set area, the semantic text S_T in the set area is calculated using the following formula:

[0041]

[0042] In the formula is the building area of ​​the nth building; is the functional attribute of the nth building; × represents the attribute Recurring Second-rate; Concatenate symbols for text;

[0043] Based on the BERT language representation model, a regional semantic representation prediction network is constructed. The regional semantic representation prediction network includes 12 encoding layers consisting of self-attention heads and feedforward neural networks, as well as a dense layer. The encoding layer uses a self-attention mechanism to capture the dependencies between different positions in the text sequence. The feedforward neural network performs nonlinear transformations and integration on the features at each position to extract rich semantic features. The dense layer uses nonlinear transformations to map the extracted semantic features to the output space.

[0044] The semantic text is input into the regional semantic representation network, and the regional semantic representation network outputs the corresponding semantic representation vector.

[0045] The step S4 specifically includes the following steps:

[0046] Integrate the visual representation vector and the semantic representation vector to obtain the land use function representation vector;

[0047] The classification layer is constructed based on the fully connected layer and the softmax function to build the original model for urban land use function recognition;

[0048] The land use function representation vector is input into the classification layer, and the output of the classification layer is the type probability of the urban land use function;

[0049] The following formula is used to express the processing process of the original model for urban land use function identification:

[0050]

[0051] Where n k is the probability of the type of use function of the kth piece of urban land; g k is the visual representation vector corresponding to the kth piece of urban land; h k is the semantic representation vector corresponding to the k-th piece of urban land; W is a text concatenation operation; (f) is the weight matrix corresponding to the fully connected layer; soft max() is the softmax function.

[0052] The training described in step S5 specifically includes the following steps:

[0053] The following formula is used as the loss function L in the training process:

[0054]

[0055] Where N is the number of samples; M is the number of categories; y p，c is a symbolic function. If the true category of the pth sample is c, then y p，c =1, otherwise y p，c =0;p p,c is the predicted probability that the pth sample belongs to category c; log() is the logarithmic operation.

[0056] The present invention also provides a system for realizing the urban land use function identification method based on visual and semantic information, comprising a data acquisition module, a visual network construction module, a semantic network construction module, a prediction model construction module, a training module and an identification module; the output end of the data acquisition module is simultaneously connected to the input end of the visual network construction module and the input end of the semantic network construction module; the output end of the visual network construction module and the output end of the semantic network construction module are simultaneously connected to the input end of the prediction model construction module; the output end of the prediction model construction module is sequentially connected to the training module and the identification module; the data acquisition module is used to acquire remote sensing image data and building data of a set area, and to prepare a training data set, and upload it to the visual network construction module and the semantic network construction module; the visual network construction module is used to perform convolutional neural network training on the Inception V3 architecture. The V3 architecture convolutional neural network is pruned and supplemented to construct a regional visual representation network, and the data is uploaded to the prediction model construction module; the semantic network construction module is used to extract semantic features from building data based on the language representation model BERT, construct a regional semantic representation network, and upload the data to the prediction model construction module; the prediction model construction module is used to construct a classification layer based on the fully connected layer and the softmax function, and combine the received regional visual representation network and regional semantic representation network to construct the original model for urban land use function recognition, and upload the data to the training module; the training module is used to use the obtained training data set to train the constructed original model for urban land use function recognition, obtain the urban land use function recognition model, and upload the data to the recognition module; the recognition module is used to use the obtained urban land use function recognition model to complete the recognition of urban land use functions in the target area.

[0057] The urban land use function identification method and system based on visual and semantic information provided by the present invention characterizes the functional attributes of urban land by integrating visual and semantic features, and at the same time identifies the complex functional structure of land use types through deep learning methods. Therefore, the present invention can not only identify urban land use functions, but also has higher reliability and better accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 Schematic diagram of the process of the present invention.

[0059] Figure 2 Schematic diagram of an example of image slicing of the present invention.

[0060] Figure 3 This is a schematic diagram of an example of building slicing according to the present invention.

[0061] Figure 4 This is a schematic diagram of the visual network module of the present invention.

[0062] Figure 5 Schematic diagram of the semantic network module of the present invention.

[0063] Figure 6 Schematic diagram of the functional modules of the system of the present invention. DETAILED DESCRIPTION

[0064] like Figure 1 The figure shows a flow chart of the method of the present invention: The method for identifying urban land use functions based on visual and semantic information disclosed in the present invention comprises the following steps:

[0065] S1. Acquire remote sensing image data and building data of a set area and create a training data set; the remote sensing image data is used to provide visual information, and the building data is used to provide semantic information; specifically, the steps include:

[0066] Obtain remote sensing image data and building-related data of the set area;

[0067] Slice the remote sensing image data and building data to obtain input image data for land use function identification and input building data for land use function identification respectively;

[0068] Marking several land use function recognition input image data to obtain visual training samples;

[0069] According to the spatial range of the visual training samples, the corresponding building data is extracted to construct semantic training samples;

[0070] The specific implementation includes the following:

[0071] Acquire remote sensing image data, 3D building outline data, POI (Point of Interest) data, and AOI (Area of ​​Interest) data within the vector boundary of the set area;

[0072] The remote sensing image data is sliced ​​and preprocessed to generate the corresponding land use function identification input image data, as shown in the following example: Figure 2 As shown;

[0073] According to the categories of industrial land, commercial land, residential land, green land, urban village land, public management service land and unused land, a number of land use function identification input image data are selected for annotation to obtain visual training samples;

[0074] Using the overlay analysis method, the functional attributes of buildings are assigned values ​​according to the POI and AOI attributes;

[0075] The building data is subjected to the same slice preprocessing as the remote sensing image data (such as Figure 3 As shown), generating land use function recognition input semantic data corresponding to the input image data;

[0076] According to the spatial range of each remote sensing image in the visual training sample, the corresponding building data is extracted to obtain semantic training samples that correspond one to one with the visual training samples;

[0077] The visual training samples and the corresponding semantic training samples together constitute a training data set;

[0078] S2. Based on the Inception V3 architecture convolutional neural network, the Inception V3 architecture convolutional neural network is pruned and supplemented to build a regional visual representation network for extracting visual features from remote sensing image data; specifically, the following steps are included (such as Figure 4 shown):

[0079] Get the network data of the Inception V3 architecture convolutional neural network;

[0080] The three Inception A modules, one Inception B module, four Inception C modules, one Inception D module, and two Inception E modules in the Inception V3 convolutional neural network were retained. The GlobalAveragePooling2D layer and the fully connected layer at the top of the Inception V3 convolutional neural network were then removed. Finally, the Dropout layer, the Flatten layer, and the Dense layer were added. Finally, a regional visual representation network was constructed.

[0081] In the constructed regional visual representation network, the input image data first passes through convolutional and pooling layers to gradually reduce the size of the feature map, helping to extract low-level image features such as edges and textures. It then uses multiple Inception modules to extract features at different scales. Each Inception module consists of multiple parallel convolutional and pooling layers to capture feature information at different scales. After the last Inception module, the network passes through a Dropout layer to prevent overfitting. The Flatten layer expands the data into a one-dimensional vector, and finally, a Dense layer undergoes nonlinear transformations before mapping it to the output space.

[0082] Inputting the input image data into the regional visual representation network, the regional visual representation network will output the corresponding visual representation vector;

[0083] S3. Based on the language representation model BERT, a regional semantic representation network is constructed to extract semantic features from building data; specifically, the following steps are included (e.g. Figure 5 shown):

[0084] Calculate the floor area of ​​each building;

[0085] According to the obtained building area of ​​each building, the building attributes are converted into semantic text and used as the input of the regional semantic representation network;

[0086] Based on the language representation model BERT, a regional semantic representation prediction network is constructed;

[0087] Inputting the input semantic data into the regional semantic representation prediction network, the regional semantic representation network will output the corresponding semantic representation vector;

[0088] The specific implementation includes the following:

[0089] The building area of ​​each building is calculated using the following formula:

[0090]

[0091] In the formula is the building area of ​​the jth building; L bf is the number of floors of the j-th building; LS i is the area of ​​the i-th floor of the j-th building;

[0092] Given a building dataset {bf1, bf2, ..., bf n}, where bf n For the nth building in the set area, the semantic text S_T in the set area is calculated using the following formula:

[0093]

[0094] In the formula is the building area of ​​the nth building; is the functional attribute of the nth building; × represents the attribute Recurring Second-rate; Concatenate symbols for text;

[0095] Based on the BERT language representation model, a regional semantic representation prediction network is constructed. The regional semantic representation prediction network consists of 12 encoding layers with self-attention heads and a dense layer. The encoding layer uses a self-attention mechanism to capture the dependencies between different positions in the text sequence and uses a feedforward neural network to perform nonlinear transformations and integration on the features at each position to extract rich semantic features. The dense layer is used to map the extracted semantic features to the output space through nonlinear transformations.

[0096] Inputting the semantic text into the regional semantic representation network, the regional semantic representation network will output the corresponding semantic representation vector;

[0097] S4. Construct a classification layer based on the fully connected layer and the softmax function, and combine the regional visual representation network constructed in step S2 and the regional semantic representation network constructed in step S3 to construct an original model for urban land use function recognition. Specifically, the steps include:

[0098] Integrate the visual representation vector and the semantic representation vector to obtain the land use function representation vector;

[0099] The classification layer is constructed based on the fully connected layer and the softmax function to build the original model for urban land use function recognition;

[0100] The land use function representation vector is input into the classification layer, and the output of the classification layer is the type probability of the urban land use function;

[0101] The following formula is used to express the processing process of the original model for urban land use function identification:

[0102]

[0103] Where n k is the probability of the type of use function of the kth piece of urban land; g k is the visual representation vector corresponding to the kth piece of urban land; h k is the semantic representation vector corresponding to the k-th piece of urban land; W is a text concatenation operation; (f) is the weight matrix corresponding to the fully connected layer; soft max() is the softmax function;

[0104] S5. Using the training data set obtained in step S1, the original model for urban land use function identification constructed in step S4 is trained to obtain an urban land use function identification model;

[0105] During training, the following formula is used as the loss function L of the training process:

[0106]

[0107] Where N is the number of samples; M is the number of categories; y p,c is a symbolic function. If the true category of the pth sample is c, then y p,c =1, otherwise y p,c =0;p p,c is the predicted probability that the pth sample belongs to category c; log() is the logarithm operation;

[0108] S6. Use the urban land use function identification model obtained in step S5 to complete the identification of urban land use functions in the target area.

[0109] The method of the present invention is further described below with reference to an embodiment:

[0110] After obtaining remote sensing imagery (from Google Maps) and building distribution, point of interest (POI), and area of ​​interest (AOI) data for the study area (from Baidu Maps), we further divided the remote sensing imagery into three scales: 96×96, 128×128, and 164×164 pixel slices. Building vector data within each remote sensing image slice was extracted, and building attributes were converted into input semantic text based on the area of ​​each building within the spatial range. 200 representative remote sensing image slices and their corresponding semantic text were selected as training samples for each of the seven functional types: industrial land, commercial land, residential land, green space, urban villages, public management services, and unused land, for a total of 1,400 samples.

[0111] The processed input image data and the corresponding input semantic text are fed into the visual representation network and the semantic representation network, respectively, to generate a 128-dimensional visual feature vector and a 128-dimensional semantic feature vector. The visual feature vector and the semantic feature vector are further concatenated to obtain a 256-dimensional fused feature vector, which is then fed into a classification network consisting of a fully connected layer and a softmax layer for land use function classification.

[0112] The model was trained using samples of the seven functional types mentioned above. 120 samples were selected for each land use type as training data, and the remaining samples were used as test data. The cross entropy loss was used as the loss function, the learning rate was set to 0.001, the maximum number of iterations was 100, and the batch size was set to 3. The results of the proposed method were compared with those of other models. The comparative data are shown in Table 1:

[0113] Table 1 Effect comparison data diagram

[0114]

[0115] In Table 1, the indicator "OA" represents overall accuracy, which is used to quantify the ratio of correct matches of samples in the entire test dataset; the indicator "Kappa" represents the Karpaty coefficient, which is used to quantify the degree to which the model recognition performance exceeds random assignment. The Karpaty coefficient is a commonly used indicator for consistency testing; the Inception V3+LSTM method is a commonly used semantic feature extraction technology LSTM (proposed by Sepp Hochreiter and Jurgen Schmidhuber in the article "Long Short-Term Memory"). This technology is compared with the prevention of this application to reflect the performance advantages and disadvantages of the method of the present invention in semantic feature extraction. The ResNet18+BERT method and the VGG16+BERT method are compared using the commonly used visual feature extraction technologies ResNet18 (proposed by Kaiming He, Xiangyu Zhang et al. in the article "Deep Residual Learning for Image Recognition") and VGG16 (proposed by Karen Simonyan and Andrew Zisserman in the article "Very deep convolutional networks for large-scale image recognition") to reflect the performance advantages and disadvantages of the method of the present invention in visual feature extraction.

[0116] Table 1 shows that the proposed method achieves the best performance product for samples of varying scales. Based on the comparison results of the trained models, the model trained with the best 128×128 sample size was used to classify land use types for the entire study area. The trained model, the Urban Land Use Function Recognition Model, was used to identify land use functions within the study area based on preprocessed data.

[0117] like Figure 6The figure shows a schematic diagram of the functional modules of the system of the present invention: the system disclosed by the present invention for realizing the urban land use function identification method based on visual and semantic information includes a data acquisition module, a visual network construction module, a semantic network construction module, a prediction model construction module, a training module and an identification module; the output end of the data acquisition module is connected to the input end of the visual network construction module and the input end of the semantic network construction module at the same time; the output end of the visual network construction module and the output end of the semantic network construction module are connected to the input end of the prediction model construction module at the same time; the output end of the prediction model construction module is connected in series with the training module and the identification module in sequence; the data acquisition module is used to acquire remote sensing image data and building data of a set area, and to prepare a training data set, and upload it to the visual network construction module and the semantic network construction module; the visual network construction module is used to perform convolutional neural network training on the Inception V3 architecture. The V3 architecture convolutional neural network is pruned and supplemented to construct a regional visual representation network, and the data is uploaded to the prediction model construction module; the semantic network construction module is used to extract semantic features from building data based on the language representation model BERT, construct a regional semantic representation network, and upload the data to the prediction model construction module; the prediction model construction module is used to construct a classification layer based on the fully connected layer and the softmax function, and combine the received regional visual representation network and regional semantic representation network to construct the original model for urban land use function recognition, and upload the data to the training module; the training module is used to use the obtained training data set to train the constructed original model for urban land use function recognition, obtain the urban land use function recognition model, and upload the data to the recognition module; the recognition module is used to use the obtained urban land use function recognition model to complete the recognition of urban land use functions in the target area.

Claims

1. A method for identifying urban land use functions based on visual and semantic information, characterized by The steps include: S1. Acquire remote sensing image data and building data of a set area and produce a training data set; the remote sensing image data is used to provide visual information, and the building data is used to provide semantic information; S2. Based on the Inception V3 convolutional neural network, prune and supplement the Inception V3 convolutional neural network to build a regional visual representation network for extracting visual features from remote sensing image data; S3. Based on the language representation model BERT, a regional semantic representation network is constructed to extract semantic features from building data. The specific steps include: Get the network data of the Inception V3 architecture convolutional neural network; Keep the three Inception A modules, one Inception B module, four Inception C modules, one Inception D module, and two Inception E modules in the Inception V3 convolutional neural network. Then, remove the GlobalAveragePooling2D layer and the fully connected layer at the top of the Inception V3 convolutional neural network. Finally, add the Dropout layer, the Flatten layer, and the Dense layer. Finally, a regional visual representation network is constructed; In the constructed regional visual representation network, the input image data first passes through convolutional and pooling layers to reduce the size of the feature map, helping to extract low-level features of edge and texture images. It then uses several Inception modules to extract features at different scales. Each Inception module consists of several parallel convolutional and pooling layers to capture feature information at different scales. After the last Inception module, the network passes through a Dropout layer to prevent overfitting, then expands it into a one-dimensional vector through a Flatten layer. Finally, it undergoes nonlinear transformations in a Dense layer and maps it to the output space. Inputting the visual training sample into the regional visual representation network, the regional visual representation network will output the corresponding visual representation vector; S4. Constructing a classification layer based on the fully connected layer and the softmax function, and combining the regional visual representation network constructed in step S2 and the regional semantic representation network constructed in step S3 to construct an original model for urban land use function recognition; S5. Using the training data set obtained in step S1, the original model for urban land use function identification constructed in step S4 is trained to obtain an urban land use function identification model; S6. Use the urban land use function identification model obtained in step S5 to complete the identification of urban land use functions in the target area.

2. The urban land use function identification method based on visual and semantic information according to claim 1 is characterized in that The step S1 specifically includes the following steps: Obtain remote sensing image data and building-related data of the set area; Slice the remote sensing image data and building data to obtain input image data for land use function identification and input building data for land use function identification respectively; Marking several land use function recognition input image data to obtain visual training samples; According to the spatial range of the visual training samples, the corresponding building data is extracted to construct semantic training samples.

3. The urban land use function identification method based on visual and semantic information according to claim 2 is characterized in that The step S1 specifically includes the following steps: Acquire remote sensing image data, 3D building outline data, POI data, and AOI data within the vector boundary of the set area; Slice and pre-process the remote sensing image data to generate the corresponding input image data for land use function identification; According to the categories of industrial land, commercial land, residential land, green land, urban village land, public management service land and unused land, a number of land use function identification input image data are selected for annotation to obtain visual training samples; Using the overlay analysis method, the functional attributes of buildings are assigned values ​​according to the POI and AOI attributes; Slice the building data and pre-process it to generate input semantic data for land use function recognition corresponding to the input image data; According to the spatial range of each remote sensing image in the visual training sample, the corresponding building data is extracted to obtain semantic training samples that correspond one to one with the visual training samples; The visual training samples and the corresponding semantic training samples together constitute a training data set.

4. The urban land use function identification method based on visual and semantic information according to claim 1 is characterized in that The step S3 specifically includes the following steps: Calculate the floor area of ​​each building; Based on the obtained building area of ​​each building, the building attributes are converted into semantic text and used as the input of the regional semantic representation prediction network; Based on the language representation model BERT, a regional semantic representation network is constructed; The input semantic data is input into the regional semantic representation network, and the regional semantic representation network outputs the corresponding semantic representation vector.

5. The urban land use function identification method based on visual and semantic information according to claim 4 is characterized in that The step S3 specifically includes the following steps: The building area of ​​each building is calculated using the following formula: In the formula is the building area of ​​the jth building; L bf is the number of floors of the j-th building; LS i is the area of ​​the i-th floor of the j-th building; Given a building dataset {bf1, bf2, ..., bf n }where bf n For the nth building in the set area, the semantic text S_T in the set area is calculated using the following formula: In the formula is the building area of ​​the nth building; is the functional attribute of the nth building; × represents the attribute Recurring Second-rate; Concatenate symbols for text; Based on the BERT language representation model, a regional semantic representation prediction network is constructed. The regional semantic representation prediction network consists of 12 encoding layers consisting of self-attention heads and feedforward neural networks, as well as a dense layer. The encoding layer uses a self-attention mechanism to capture the dependencies between different positions in the text sequence. The feedforward neural network performs nonlinear transformations and integration on the features at each position to extract semantic features. The dense layer uses nonlinear transformations to map the extracted semantic features to the output space. The semantic text is input into the regional semantic representation network, and the regional semantic representation network outputs the corresponding semantic representation vector.

6. The urban land use function identification method based on visual and semantic information according to claim 5 is characterized in that The step S4 specifically includes the following steps: Integrate the visual representation vector and the semantic representation vector to obtain the land use function representation vector; The classification layer is constructed based on the fully connected layer and the softmax function to build the original model for urban land use function recognition; The land use function representation vector is input into the classification layer, and the output of the classification layer is the type probability of the urban land use function; The following formula is used to express the processing process of the original model for urban land use function identification: Where n k is the probability of the type of use function of the kth piece of urban land; g k is the visual representation vector corresponding to the kth piece of urban land; h k is the semantic representation vector corresponding to the k-th piece of urban land; W is a text concatenation operation; (f) is the weight matrix corresponding to the fully connected layer; soft max() is the softmax function.

7. The urban land use function identification method based on visual and semantic information according to claim 6 is characterized in that The training described in step S5 specifically includes the following steps: The following formula is used as the loss function L in the training process: Where N is the number of samples; M is the number of categories; y p，c is a symbolic function. If the true category of the pth sample is c, then y p，c =1, otherwise y p，c =0p p,c is the predicted probability that the pth sample belongs to category c; log() is the logarithmic operation.

8. A system for implementing the urban land use function identification method based on visual and semantic information according to any one of claims 1 to 7, characterized in that It includes a data acquisition module, a visual network construction module, a semantic network construction module, a prediction model construction module, a training module and a recognition module; the output end of the data acquisition module is connected to the input end of the visual network construction module and the input end of the semantic network construction module at the same time; the output end of the visual network construction module and the output end of the semantic network construction module are connected to the input end of the prediction model construction module at the same time; the output end of the prediction model construction module is connected in series with the training module and the recognition module in sequence; the data acquisition module is used to obtain remote sensing image data and building data of a set area, and to prepare a training data set, and upload it to the visual network construction module and the semantic network construction module; the visual network construction module is used to prune and supplement the Inception V3 architecture convolutional neural network based on the Inception V3 architecture convolutional neural network, build a regional visual representation network, and upload the data to the prediction model construction module; The semantic network construction module is used to extract semantic features from building data based on the language representation model BERT, construct a regional semantic representation network, and upload the data to the prediction model construction module. The prediction model construction module is used to build a classification layer based on the fully connected layer and the softmax function, and combine the received regional visual representation network and regional semantic representation network to build the original model for urban land use function recognition, and upload the data to the training module. The training module is used to use the obtained training data set to train the constructed urban land use function identification original model, obtain the urban land use function identification model, and upload the data to the identification module; The identification module is used to use the obtained urban land use function identification model to complete the identification of urban land use functions in the target area.

Images