Method and device for automatically extracting urban functional areas based on multi-source data

Abstract

The invention discloses an urban functional area automatic extraction method and device based on multi-source data, a storage medium and electronic equipment. The method comprises the following steps: acquiring a high-resolution remote sensing image, a hyperspectral image and positioning data of a corresponding urban functional area; inputting the high-resolution remote sensing image into the trained deep learning model for instance segmentation to obtain an image segmentation result; wherein the image segmentation result corresponds to an urban functional area instance segmentation result; performing multi-source remote sensing feature extraction on the hyperspectral image to obtain a plurality of remote sensing features; performing space-time behavior extraction on the positioning data to obtain track features; and performing multi-source feature fusion on the multiple remote sensing features and the track features, and inputting the fused features and an image segmentation result into a machine learning classification model to obtain a classification result of the urban functional area. According to the invention, the classification accuracy of the urban functional areas can be improved.

Description

Technical Field

[0001] This application relates to the field of urban planning and remote sensing applications, and particularly to a method, device, storage medium, and electronic device for automatically extracting urban functional areas based on multi-source data. Background Art

[0002] Accurately identifying and dividing urban functional areas is of great significance for urban planning, land use management, and urban development decision-making. The division of urban functional areas can reflect the urban spatial structure and land use characteristics, providing important decision-making basis for urban managers.

[0003] Traditional methods for dividing urban functional areas mainly rely on manual field surveys and empirical judgments, suffering from problems such as low efficiency, strong subjectivity, and long update cycles. With the development of remote sensing technology and the accumulation of urban big data, it has become possible to automatically identify and extract urban functional areas using multi-source data.

[0004] Currently, functional area identification generally relies on a single data source (such as remote sensing images or POI data). This identification method lacks accurate extraction of urban functional area spatial entities, often using regular grids or administrative units as basic analysis units, affecting the identification accuracy, and not fully considering the time dimension characteristics in the classification process of urban functional areas, making it difficult to accurately distinguish regions with similar physical characteristics but different functions. Summary of the Invention

[0005] Embodiments of this application provide a method, device, storage medium, and electronic device for automatically extracting urban functional areas based on multi-source data, which can improve the classification accuracy of urban functional areas.

[0006] Embodiments of this application provide a method for automatically extracting urban functional areas based on multi-source data, including: Obtaining high-resolution remote sensing images, hyperspectral images, and positioning data corresponding to urban functional areas; Inputting the high-resolution remote sensing images into a trained deep learning model for instance segmentation to obtain an image segmentation result; wherein, the image segmentation result corresponds to the instance segmentation result of urban functional areas; Performing multi-source remote sensing feature extraction on the hyperspectral images to obtain various remote sensing features; Performing spatio-temporal behavior extraction on the positioning data to obtain trajectory features; Performing multi-source feature fusion on the various remote sensing features and the trajectory features, and inputting the fused features and the image segmentation result into a machine learning classification model to obtain the classification result of urban functional areas.

[0007] Furthermore, in the above method for automatically extracting urban functional areas based on multi-source data, the deep learning model includes a backbone network, a region proposal network, a Fast R-CNN detector, and a mask prediction branch; Inputting the high-resolution remote sensing image into the trained deep learning model for instance segmentation to obtain an image segmentation result, which includes: Inputting the high-resolution remote sensing image into the backbone network for multi-scale feature extraction to obtain a first feature map; Inputting the first feature map into the region proposal network to generate candidate regions; Inputting the candidate regions into the Fast R-CNN detector for bounding box prediction and classification to obtain prediction and classification results; Inputting the prediction and classification results into the mask prediction branch for mask prediction to obtain an image segmentation result.

[0008] Furthermore, in the above method for automatically extracting urban functional areas based on multi-source data, the multi-source remote sensing feature extraction of the hyperspectral image to obtain various remote sensing features includes: Performing spectral feature extraction on the hyperspectral image; Calculating the normalized difference vegetation index information and the normalized difference water index information of the hyperspectral image; Performing night light feature extraction on the hyperspectral image.

[0009] Furthermore, in the above method for automatically extracting urban functional areas based on multi-source data, the spectral feature extraction of the hyperspectral image includes: Extracting the mean and standard deviation of the blue, green, red, and near-infrared bands through the following formula:

[0010]

[0011]

[0012]

[0013]

[0014]

[0015]

[0016]

[0017] where N represents the number of pixels within each urban functional area instance, 、 , , respectively refer to the blue band, green band, red band and near-infrared band values of the i-th pixel, , , , respectively represent the means of the blue, green, red and near-infrared bands of each urban functional area instance, , , , respectively represent the standard deviations of the blue, green, red and near-infrared bands of each urban functional area instance.

[0018] Furthermore, for the above-mentioned automatic extraction method of urban functional areas based on multi-source data, wherein, calculating the normalized difference vegetation index information and normalized difference water index information of the hyperspectral image includes: Calculating the mean and standard deviation of the normalized difference vegetation index through the following formula:

[0019]

[0020]

[0021] wherein, N represents the number of pixels within each urban functional area instance, , , represent the near-infrared band, red band value and normalized difference vegetation index value of the i-th pixel within each urban functional area instance, , respectively represent the mean and standard deviation of the normalized difference vegetation index of each urban functional area instance; Calculating the mean and standard deviation of the normalized difference water index (NDWI) through the following formula:

[0022]

[0023]

[0024] wherein, N represents the number of pixels within each urban functional area instance, , , represent the green band, near-infrared band value and normalized difference water index value of the i-th pixel within each urban functional area instance, , respectively represent the mean and standard deviation of the normalized difference water index of each urban functional area instance.

[0025] Furthermore, for the above-mentioned method for automatically extracting urban functional areas based on multi-source data, the extraction of the night brightness feature from the hyperspectral image includes: Calculating the average night brightness through the following formula:

[0026] where N represents the number of pixels within each functional area instance, represents the night brightness value of the i-th pixel within each functional area instance, respectively represent the average night brightness of each functional area instance.

[0027] Furthermore, for the above-mentioned method for automatically extracting urban functional areas based on multi-source data, the trajectory features include temporal activity features and facility distribution features; The extraction of spatio-temporal behaviors from the positioning data to obtain trajectory features includes: Based on the positioning data, calculating the population activity levels on weekdays and weekends respectively, and taking the population activity levels as the temporal activity features; Based on the positioning data, conducting an analysis of the total number of points of interest and an analysis of the proportion of points of interest, and taking the analysis results of the total number of points of interest and the analysis results of the proportion of points of interest as the facility distribution features.

[0028] The embodiment of the present application also provides an apparatus for automatically extracting urban functional areas based on multi-source data, including: An acquisition module, configured to acquire high-resolution remote sensing images, hyperspectral images, and positioning data of corresponding urban functional areas; An image segmentation module, configured to input the high-resolution remote sensing image into a trained deep learning model for instance segmentation to obtain an image segmentation result; wherein, the image segmentation result corresponds to the instance segmentation result of the urban functional area; A multi-source remote sensing feature extraction module, configured to extract multi-source remote sensing features from the hyperspectral image to obtain various remote sensing features; A spatio-temporal behavior extraction module, configured to extract spatio-temporal behaviors from the positioning data to obtain trajectory features; A classification module, configured to perform multi-source feature fusion on the various remote sensing features and the trajectory features, and input the fused features and the image segmentation result into a machine learning classification model to obtain a classification result of the urban functional area.

[0029] The embodiment of the present application also provides a computer-readable storage medium, in which multiple instructions are stored, and the instructions are suitable for being loaded by a processor to execute any one of the above-mentioned methods for automatically extracting urban functional areas based on multi-source data.

[0030] An embodiment of the present application also provides an electronic device, including a processor and a memory. The processor is electrically connected to the memory. The memory is used to store instructions and data, and the processor is used for the steps in the method for automatically extracting urban functional areas based on multi-source data described in any one of the above items.

[0031] The method, device, storage medium and electronic device for automatically extracting urban functional areas based on multi-source data provided by the present application. The present application extracts multi-source remote sensing features from high-resolution images and extracts spatio-temporal behaviors from positioning data, and classifies the image segmentation results of urban functional areas through the fused multi-source data. The present application has the following beneficial effects: 1) Through the instance segmentation technology, the precise extraction of spatial entities in urban functional areas is realized, avoiding the limitations of traditional grid division methods; 2) Fusing multi-source data such as remote sensing images, population activities, night lights and POIs comprehensively depicts the physical, social and economic characteristics of functional areas; 3) Introducing temporal activity features improves the discrimination ability of areas with similar functions; 4) The method has a high degree of automation, can be updated quickly, and is applicable to dynamic monitoring of large-scale urban functional areas. Description of the Drawings

[0032] The following will combine the drawings and describe the specific embodiments of the present application in detail, making the technical solutions and other beneficial effects of the present application obvious.

[0033] Figure 1 It is a flowchart of the method for automatically extracting urban functional areas based on multi-source data provided by an embodiment of the present application.

[0034] Figure 2 It is another flowchart of the method for automatically extracting urban functional areas based on multi-source data provided by an embodiment of the present application.

[0035] Figure 3 It is the image segmentation result of the case area provided by an embodiment of the present application.

[0036] Figure 4 It is the classification result of urban functional areas provided by an embodiment of the present application.

[0037] Figure 5 It is a schematic structural diagram of the device for automatically extracting urban functional areas based on multi-source data provided by an embodiment of the present application.

[0038] Figure 6 It is a schematic structural diagram of the electronic device provided by an embodiment of the present application. Detailed Embodiments

[0039] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present application.

[0040] The embodiments of the present application provide a method, device, storage medium, and electronic device for automatically extracting urban functional areas based on multi-source data. A device for automatically extracting urban functional areas based on multi-source data provided by the embodiments of the present application can be integrated in an electronic device, and the electronic device can be a device such as a terminal or a server. Among them, the terminal can include a tablet computer, a notebook computer, a personal computer (PC), a microprocessing box, or other devices, etc.

[0041] Please refer to Figure 1 With Figure 2 , Figure 1 is a flowchart of the method for automatically extracting urban functional areas based on multi-source data provided by the embodiments of the present application. Figure 2 is another flowchart of the method for automatically extracting urban functional areas based on multi-source data provided by the embodiments of the present application, which is applied to an electronic device. The method for automatically extracting urban functional areas based on multi-source data includes the following steps: S1, obtain high-resolution remote sensing images, hyperspectral images, and positioning data of corresponding urban functional areas.

[0042] Specifically, obtain remote sensing images with a resolution of 3 meters or higher, and perform preprocessing operations such as image enhancement, noise elimination, and spatial registration on the high-resolution remote sensing images to provide high-quality input data for the subsequent deep learning model.

[0043] S2, input the high-resolution remote sensing image into the trained deep learning model for instance segmentation to obtain an image segmentation result; among them, the image segmentation result corresponds to the instance segmentation result of the urban functional area.

[0044] First, before step S2, construct a deep learning model framework based on Mask R-CNN, design a network structure suitable for extracting urban functional areas, and use the labeled sample data for model training and optimization.

[0045] Among them, transfer learning is carried out based on the pre-trained weights of MaskRCNN_ResNet50_FPN_V2_Weights provided by the official PyTorch to improve the feature extraction ability and segmentation accuracy. The SGD optimizer is used to optimize the parameters of the deep learning model. The learning rate is set to 0.005, the momentum coefficient is set to 0.9, and the weight decay coefficient is 0.0005 to prevent overfitting. During the training process, only the parameters that require gradients are updated to improve the training efficiency. The model prediction output includes: target bounding box coordinates, class prediction probabilities, instance segmentation masks, and prediction confidence scores.

[0046] Then, the trained Mask R-CNN model is used to perform instance segmentation on the preprocessed remote sensing images, and independent surface feature objects with clear boundaries are output as the spatial entities of the functional areas.

[0047] In one embodiment, the deep learning model includes a backbone network, a region proposal network, a Fast R-CNN detector, and a mask prediction branch. The backbone network is a Mask R-CNN model structure based on ResNet50-FPN, which includes a Feature Pyramid Network (FPN) for multi-scale feature extraction. Region Proposal Network (RPN): used to generate candidate regions. Fast R-CNN detector: performs object detection and classification. Mask prediction branch: generates instance segmentation masks.

[0048] Step S2 includes the following steps: S21, input the high-resolution remote sensing image into the backbone network for multi-scale feature extraction to obtain the first feature map.

[0049] S22, input the first feature map into the region proposal network to generate candidate regions.

[0050] S23, input the candidate regions into the Fast R-CNN detector for bounding box prediction and classification to obtain the prediction and classification results.

[0051] Specifically, the candidate regions output by the RPN network are sent to the ROI Pooling layer for feature alignment. Then, the Fast R-CNN detector (Fast RCNN Predictor) is used for bounding box prediction and classification, and its input feature dimension matches the backbone network feature dimension.

[0052] S24, input the prediction and classification results into the mask prediction branch for mask prediction to obtain the image segmentation result.

[0053] Specifically, a mask prediction is performed using a Mask RCNN Predictor, where the number of input channels of the convolutional layer matches the feature dimension of the backbone network.

[0054] Finally, the detection threshold is adjustable and is used to control the confidence requirement for prediction.

[0055] S3. Extract multi-source remote sensing features from the hyperspectral image to obtain various remote sensing features.

[0056] In one embodiment, step S3 includes the following steps: S31. Extract spectral features from the hyperspectral image.

[0057] Specifically, the mean and standard deviation of the blue, green, red, and near-infrared bands are extracted through the following formulas:

[0058]

[0059]

[0060]

[0061]

[0062]

[0063]

[0064]

[0065] where N represents the number of pixels within each urban functional area instance, 、 、 、 respectively refer to the blue band, green band, red band, and near-infrared band values of the i-th pixel, 、 、 、 respectively represent the means of the blue, green, red, and near-infrared bands of each urban functional area instance, 、 、 、 respectively represent the standard deviations of the blue, green, red, and near-infrared bands of each urban functional area instance.

[0066] S32. Calculate the normalized difference vegetation index information and the normalized difference water index information of the hyperspectral image.

[0067] Specifically, the mean and standard deviation of the normalized difference vegetation index (NDVI) are calculated by the following formula:

[0068]

[0069]

[0070] where N represents the number of pixels within each urban functional area instance, 、 、 represent the near-infrared band value, red band value, and normalized difference vegetation index value of the i-th pixel within each urban functional area instance, 、 represent the mean and standard deviation of the normalized difference vegetation index of each urban functional area instance, respectively; The mean and standard deviation of the normalized difference water index (NDWI) are calculated by the following formula:

[0071]

[0072]

[0073] where N represents the number of pixels within each urban functional area instance, 、 、 represent the green band value, near-infrared band value, and normalized difference water index value of the i-th pixel within each urban functional area instance, 、 represent the mean and standard deviation of the normalized difference water index of each urban functional area instance, respectively.

[0074] S33. Extract the night-time brightness features from the hyperspectral image.

[0075] The night-time brightness mean is calculated by the following formula:

[0076] where N represents the number of pixels within each functional area instance, represents the night-time brightness value of the i-th pixel within each functional area instance, represents the mean of the night-time brightness of each functional area instance, respectively.

[0077] S4. Extract the spatio-temporal behavior from the positioning data to obtain trajectory features.

[0078] In one embodiment, step S4 includes the following steps: S41. Based on the positioning data, calculate the population activity levels on weekdays and weekends respectively, and use the population activity levels as the time-series activity features.

[0079] Specifically, step S41 includes: (1) The average population activity trajectory during 8 hours on weekdays.

[0080] Calculation method: Count the number of independent users in the functional area instance per hour from 8:00 to 16:00 on weekdays, and finally take the average number of people in the eight hours as the statistical value to reflect the population activity level in this functional area during working hours.

[0081] (2) The average population activity trajectory during 8 hours on weekends.

[0082] Calculation method: Count the number of independent users in the functional area instance per hour from 8:00 to 16:00 on weekends, and finally take the average number of people in the eight hours as the statistical value to reflect the population activity level in this functional area on rest days.

[0083] S42. Based on the positioning data, conduct the total number analysis and proportion analysis of points of interest (POIs), and use the total number analysis result and proportion analysis result of POIs as the facility distribution features.

[0084] (1) Total number statistics of points of interest (POIs) Calculation method: Count the total number of all POI points falling within the scope of the functional area instance, which reflects the facility density of this functional area.

[0085] (2) Analysis of the number and proportion of various types of points of interest (POIs) Calculation method: First, classify and count the number according to the POI categories, and then calculate the percentage of each category in the total number of POIs, which reflects the functional composition characteristics of this functional area.

[0086] Furthermore, in the spectral feature extraction and night brightness feature extraction processes in steps S3 and S4, spatial statistical methods are used to summarize and calculate the data within each urban functional area instance to ensure the representativeness and comparability of the features.

[0087] S5. Perform multi-source feature fusion on multiple remote sensing features and trajectory features, and input the fused features and the image segmentation results into the machine learning classification model to obtain the classification results of urban functional areas.

[0088] Specifically, standardize and fuse the spectral features, index information (normalized difference vegetation index information and normalized difference water index information), night brightness features, time-series activity features, and facility distribution features to construct a comprehensive feature vector as the input of the machine learning classification model.

[0089] Based on the fused multi-source features, a machine learning method is used to classify urban functional area instances (i.e., the image segmentation results), divide them into five categories: residential area, entertainment area, transportation area, industrial area, and office area, and conduct classification accuracy evaluation and result optimization.

[0090] In this application, multi-source remote sensing features are extracted from high-altitude images, and spatio-temporal behaviors are extracted from positioning data. The classification of urban functional areas is carried out on the image segmentation results through the fused multi-source data. This application has the following beneficial effects: 1) Through the instance segmentation technology, the precise extraction of spatial entities in urban functional areas is realized, avoiding the limitations of traditional grid division methods; 2) Fusing multi-source data such as remote sensing images, population activities, night lights, and POIs comprehensively depicts the physical, social, and economic characteristics of functional areas; 3) Introducing temporal activity features improves the discrimination ability of areas with similar functions; 4) The method has a high degree of automation, can be updated quickly, and is applicable to dynamic monitoring of large-scale urban functional areas.

[0091] The following is a specific embodiment: [1] Step 1: Based on Google image data with a resolution of 3 meters, preprocessing operations such as image enhancement, noise elimination, and spatial registration are carried out.

[0092] [2] Step 2: Construct a deep learning model framework based on Mask R-CNN, use ResNet50-FPN as the backbone network, and conduct transfer learning based on the pre-trained weights of MaskRCNN_ResNet50_FPN_V2_Weights provided by the official PyTorch.

[0093] [3] Step 3: Use the trained Mask R-CNN model to perform instance segmentation on the preprocessed Google images, and extract spatial entities of functional areas with clear boundaries. Figure 3 The image segmentation results of the case area provided by the embodiment of this application are as Figure 3 shown.

[0094] [4] Step 4: Extract spectral features based on the 10-meter resolution multi-spectral images of Sentinel-2A / B satellites, including: ① The mean and standard deviation of the blue, green, red, and near-infrared bands ② Normalized Difference Vegetation Index (NDVI) ③ Normalized Difference Water Index (NDWI) [5] Step 5: At the same time, extract the night brightness features of the 130-meter resolution night light remote sensing image of Luojia-1 satellite to obtain the mean value of the digital number (DN).

[0095] [6] Step Six: Extract temporal activity features based on Tencent mobile device location data: ① Average population activity trajectory during 8 hours on weekdays ② Average population activity trajectory during 8 hours on weekends And extract facility distribution features by combining with Amap POI data: ② Total number of POIs ② Quantities and proportions of various types of POIs Rearrange the data classification of Amap POIs into the following categories: food and beverage services, road affiliated facilities, scenic spots, public facilities, companies and enterprises, shopping services, transportation facility services, financial and insurance services, science, education and culture services, vehicle repair and services, biological residences, life services, sports and leisure services, medical and health services, government agencies and social organizations, accommodation services, others.

[0096] [7] Step Seven: Standardize and fuse the above four categories of features to construct a comprehensive feature vector.

[0097] [8] Step Eight: Based on the random forest model, classify the functional areas into five categories: residential areas, entertainment areas, transportation areas, industrial areas, and office areas. Figure 4 This is the classification result of urban functional areas provided by the embodiments of this application, as Figure 4 shown.

[0098] According to the method described in the above embodiments, this embodiment will further describe from the perspective of an automatic extraction device for urban functional areas based on multi-source data. The automatic extraction device for urban functional areas based on multi-source data can be specifically implemented as an independent entity, or integrated in an electronic device. The electronic device can be a terminal, a server, etc. devices. Among them, the terminal can include a tablet computer, a laptop computer, a personal computer (PC), a microprocessing box, or other devices, etc.

[0099] Please refer to Figure 5 , Figure 5 which specifically describes the automatic extraction device for urban functional areas based on multi-source data provided by the embodiments of this application, applied in an electronic device. The automatic extraction device for urban functional areas based on multi-source data can include: An acquisition module, configured to acquire high-resolution remote sensing images, hyperspectral images, and location data of corresponding urban functional areas; An image segmentation module, configured to input the high-resolution remote sensing image into a trained deep learning model for instance segmentation to obtain an image segmentation result; wherein, the image segmentation result corresponds to the instance segmentation result of the urban functional area; A multi-source remote sensing feature extraction module, which is used to perform multi-source remote sensing feature extraction on the hyperspectral image to obtain various remote sensing features; A spatio-temporal behavior extraction module, which is used to perform spatio-temporal behavior extraction on the positioning data to obtain trajectory features; A classification module, which is used to perform multi-source feature fusion on the various remote sensing features and the trajectory features, and input the fused features and the image segmentation result into a machine learning classification model to obtain the classification result of the urban functional area.

[0100] In specific implementation, each of the above modules and / or units can be implemented as an independent entity, or can be combined arbitrarily to be implemented as the same or several entities. For the specific implementation of each of the above modules and / or units, reference can be made to the foregoing method embodiments. For the specific beneficial effects that can be achieved, reference can also be made to the beneficial effects in the foregoing method embodiments, which will not be elaborated herein.

[0101] In addition, an embodiment of the present application further provides an electronic device, which can be a device such as a computer or a tablet computer. The electronic device can implement the steps in any of the embodiments of the method for automatically extracting urban functional areas based on multi-source data provided by the embodiments of the present application. Therefore, the beneficial effects that can be achieved by any of the methods for automatically extracting urban functional areas based on multi-source data provided by the embodiments of the present invention can be achieved. For details, refer to the foregoing embodiments, which will not be elaborated herein.

[0102] Figure 6 The specific structural block diagram of the electronic device provided by the embodiment of the present invention is shown. The electronic device can be used to implement the method for automatically extracting urban functional areas based on multi-source data provided in the foregoing embodiments. The electronic device 500 can be a device such as a terminal or a server. Among them, the terminal can include a tablet computer, a notebook computer, a personal computer (PC), a micro-processing box, or other devices, etc.

[0103] The RF circuit 510 is used to receive and transmit electromagnetic waves, realizing the mutual conversion between electromagnetic waves and electrical signals, so as to communicate with a communication network or other devices. The RF circuit 510 may include various existing circuit elements for performing these functions. For example, antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, subscriber identity module (SIM) cards, memories, and so on. The RF circuit 510 can communicate with various networks such as the Internet, enterprise intranets, wireless networks or communicate with other devices through wireless networks. The above-mentioned wireless networks may include cellular phone networks, wireless local area networks or metropolitan area networks. The above-mentioned wireless networks can use various communication standards, protocols and technologies, including but not limited to Global System for Mobile Communication (GSM), Enhanced Data GSM Environment (EDGE), Wideband Code Division Multiple Access (WCDMA), Code Division Access (CDMA), Time Division Multiple Access (TDMA), Wireless Fidelity (Wi-Fi) (such as Institute of Electrical and Electronics Engineers standards IEEE 802.11a, IEEE 802.11b, IEEE 802.11g and / or IEEE 802.11n), Voice over Internet Protocol (VoIP), Worldwide Interoperability for Microwave Access (Wi-Max), other protocols for email, instant messaging and short messages, and any other suitable communication protocols, and may even include those protocols that have not yet been developed currently.

[0104] The memory 520 can be used to store software programs and modules, such as the corresponding program instructions / modules in the above embodiments. The processor 580 executes various functional applications and data processing by running the software programs and modules stored in the memory 520, that is, to implement functions such as taking pictures with the front camera, processing the captured images, and switching the display colors of the display content on the display screen. The memory 520 may include a high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 520 may further include a memory remotely disposed relative to the processor 580, and these remote memories can be connected to the electronic device 500 through a network. Examples of the above network include but are not limited to the Internet, enterprise intranet, local area network, mobile communication network, and combinations thereof.

[0105] The input unit 530 can be used to receive input digital or character information, and generate a keyboard and a mouse related to user settings and function controls. The display unit 540 can be used to display information input by the user or information provided to the user, as well as various graphical user interfaces, and these graphical user interfaces can be composed of graphics, text, icons, videos, and any combination thereof. The display unit 540 may include a display panel 541. Optionally, the display panel 541 can be configured in the form of an LCD (Liquid Crystal Display) or an OLED (Organic Light-Emitting Diode).

[0106] The audio circuit 560, the speaker 561, and the microphone 562 can provide an audio interface between the user and the electronic device 500. The audio circuit 560 can transmit the electrical signal converted from the received audio data to the speaker 561, and the speaker 561 converts it into a sound signal for output; on the other hand, the microphone 562 converts the collected sound signal into an electrical signal, which is received by the audio circuit 560 and then converted into audio data. After the audio data is output to the processor 580 for processing, it is sent to another terminal, for example, through the RF circuit 510, or the audio data is output to the memory 520 for further processing. The audio circuit 560 may further include an earphone jack to provide communication between the external peripheral earphone and the electronic device 500.

[0107] The electronic device 500 can help the user receive requests, send information, etc. through the transmission module 570 (such as a Wi-Fi module), and it provides the user with wireless broadband Internet access. Although the transmission module 570 is shown in the figure, it can be understood that it does not belong to the essential components of the electronic device 500 and can be omitted completely as needed without changing the essence of the invention.

[0108] The processor 580 is the control center of the electronic device 500, connecting various parts of the entire mobile phone through various interfaces and circuits. By running or executing software programs and / or modules stored in the memory 520, and by calling the data stored in the memory 520, it executes various functions of the electronic device 500 and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 580 may include one or more processing cores; in some embodiments, the processor 580 may integrate an application processor and a modem processor. Among them, the application processor mainly processes the operating system, user interface, application programs, etc., and the modem processor mainly processes wireless communication. It can be understood that the above-mentioned modem processor may not be integrated into the processor 580 either.

[0109] The electronic device 500 also includes a power source 590 (such as a battery) that supplies power to each component. In some embodiments, the power source can be logically connected to the processor 580 through a power management system, so as to realize functions such as management of charging, discharging, and power consumption management through the power management system. The power source 590 may also include any components such as one or more DC or AC power sources, a recharge system, a power failure detection circuit, a power converter or inverter, and a power status indicator.

[0110] Although not shown, the electronic device 500 also includes a camera (such as a front camera, a rear camera), a Bluetooth module, etc., which will not be elaborated here. Specifically, in this embodiment, the display unit of the electronic device is a touch screen display, and the mobile terminal also includes a memory, and one or more programs, where one or more programs are stored in the memory and are configured to be executed by one or more processors. The one or more programs include instructions for performing the following operations: Obtain high-resolution remote sensing images, hyperspectral images, and the positioning data of the corresponding urban functional areas; Input the high-resolution remote sensing image into the trained deep learning model for instance segmentation to obtain an image segmentation result; wherein, the image segmentation result corresponds to the instance segmentation result of the urban functional area; Extract multi-source remote sensing features from the hyperspectral image to obtain various remote sensing features; Extract spatio-temporal behaviors from the positioning data to obtain trajectory features; Perform multi-source feature fusion on the various remote sensing features and the trajectory features, and input the fused features and the image segmentation result into a machine learning classification model to obtain the classification result of the urban functional area.

[0111] In specific implementation, each of the above modules can be implemented as an independent entity, or can be arbitrarily combined and implemented as the same or several entities. For the specific implementation of each of the above modules, reference can be made to the foregoing method embodiments, which will not be elaborated herein.

[0112] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructions, or by controlling relevant hardware through instructions. The instructions can be stored in a computer-readable storage medium and loaded and executed by a processor. To this end, an embodiment of the present invention provides a storage medium in which multiple instructions are stored, and the instructions can be loaded by a processor to execute the steps of any one of the embodiments of the method for automatically extracting urban functional areas based on multi-source data provided by the embodiments of the present invention.

[0113] Among them, the computer-readable storage medium may include: read-only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.

[0114] Since the instructions stored in the storage medium can execute the steps in any one of the embodiments of the method for automatically extracting urban functional areas based on multi-source data provided by the embodiments of the present invention, the beneficial effects that can be achieved by any of the methods for automatically extracting urban functional areas based on multi-source data provided by the embodiments of the present invention can be realized. For details, see the foregoing embodiments, which will not be elaborated herein.

[0115] The above has introduced in detail a method, device, storage medium, and electronic device for automatically extracting urban functional areas based on multi-source data provided by the embodiments of the present application. Specific examples are used in this article to elaborate on the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those skilled in the art, according to the idea of the present application, there will be changes in the specific implementation manner and application scope. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A method for automatically extracting urban functional areas based on multi-source data, characterized in that: The method comprises: Obtain high-resolution remote sensing images, hyperspectral images and positioning data of corresponding urban functional areas; Inputting the high-resolution remote sensing image into a trained deep learning model for instance segmentation to obtain an image segmentation result; wherein the image segmentation result corresponds to an instance segmentation result of an urban functional area; Performing multi-source remote sensing feature extraction on the hyperspectral image to obtain a variety of remote sensing features; Extracting spatiotemporal behaviors from the positioning data to obtain trajectory features; The multiple remote sensing features and the trajectory features are subjected to multi-source feature fusion, and the fused features and the image segmentation results are input into a machine learning classification model to obtain classification results of urban functional areas.

2. The method for automatically extracting urban functional areas based on multi-source data according to claim 1 is characterized in that: The deep learning model includes a backbone network, a region proposal network, a fast R-CNN detector, and a mask prediction branch; The high-resolution remote sensing image is input into the trained deep learning model for instance segmentation to obtain image segmentation results, including: Inputting the high-resolution remote sensing image into the backbone network to extract multi-scale features to obtain a first feature map; Inputting the first feature map into the region proposal network to generate a candidate region; Inputting the candidate region into the fast R-CNN detector for bounding box prediction and classification to obtain prediction and classification results; The prediction and classification results are input into the mask prediction branch to perform mask prediction to obtain an image segmentation result.

3. The method for automatically extracting urban functional areas based on multi-source data according to claim 1 is characterized in that: The multi-source remote sensing feature extraction is performed on the hyperspectral image to obtain a variety of remote sensing features, including: Extracting spectral features from the hyperspectral image; Calculating normalized vegetation index information and normalized water index information of the hyperspectral image; Nighttime brightness features are extracted from the hyperspectral image.

4. The method for automatically extracting urban functional areas based on multi-source data according to claim 3 is characterized in that: The extracting spectral features of the hyperspectral image comprises: The mean and standard deviation of the blue, green, red, and near-infrared bands are extracted using the following formulas: Where N represents the number of pixels in each urban functional area instance, , , , They refer to the blue band, green band, red band and near-infrared band values ​​of the i-th pixel respectively. , , , Represents the mean values ​​of the blue, green, red and near-infrared bands of each urban functional area instance, , , , Represents the standard deviation of the blue, green, red and near-infrared bands for each urban functional area instance.

5. The method for automatically extracting urban functional areas based on multi-source data according to claim 3 is characterized in that: Calculating the normalized vegetation index information and the normalized water index information of the hyperspectral image, including: The mean and standard deviation of the normalized difference vegetation index were calculated using the following formula: Where N represents the number of pixels in each urban functional area instance, , , Represents the near infrared band, red band value and normalized vegetation index value of the i-th pixel in each urban functional area instance. , Respectively represent the mean and standard deviation of the normalized difference vegetation index for each urban functional area instance; The mean and standard deviation of the normalized water index (NDWI) were calculated using the following formula: Where N represents the number of pixels in each urban functional area instance, , , Represents the green band, near-infrared band value and normalized water index value of the i-th pixel in each urban functional area instance, , Represent the mean and standard deviation of the normalized water index of each urban functional area instance.

6. The method for automatically extracting urban functional areas based on multi-source data according to claim 3 is characterized in that: The extracting of nighttime brightness features from the hyperspectral image comprises: The mean nighttime brightness is calculated using the following formula: Where N represents the number of pixels in each functional area instance, Represents the night brightness value of the i-th pixel in each functional area instance, Represents the mean night brightness of each functional area instance.

7. The method for automatically extracting urban functional areas based on multi-source data according to claim 1 is characterized in that: The trajectory characteristics include temporal activity characteristics and facility distribution characteristics; The extracting the spatiotemporal behavior of the positioning data to obtain trajectory features includes: Based on the positioning data, respectively calculate the population activity levels on weekdays and weekends, and use the population activity levels as the time series activity features; Based on the positioning data, a total number analysis of points of interest and a proportion analysis of points of interest are performed, and the total number analysis results of points of interest and the proportion analysis results of points of interest are used as the facility distribution characteristics.

8. An automatic extraction device for urban functional areas based on multi-source data, characterized in that: include: Acquisition module, used to obtain high-resolution remote sensing images, hyperspectral images and positioning data of corresponding urban functional areas; An image segmentation module is used to input the high-resolution remote sensing image into a trained deep learning model to perform instance segmentation to obtain an image segmentation result; wherein the image segmentation result corresponds to an instance segmentation result of an urban functional area; A multi-source remote sensing feature extraction module is used to extract multi-source remote sensing features from the hyperspectral image to obtain a variety of remote sensing features; A spatiotemporal behavior extraction module, used to extract spatiotemporal behavior from the positioning data to obtain trajectory features; The classification module is used to perform multi-source feature fusion on the multiple remote sensing features and the trajectory features, and input the fused features and the image segmentation results into a machine learning classification model to obtain classification results of urban functional areas.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the method for automatically extracting urban functional areas based on multi-source data as described in any one of claims 1 to 7.

10. An electronic device, characterized in that: It includes a processor and a memory, the processor is electrically connected to the memory, the memory is used to store instructions and data, and the processor is used to execute the steps in the method for automatic extraction of urban functional areas based on multi-source data as described in any one of claims 1 to 7.

Images