A system for dynamically monitoring changes in natural resources

Abstract

The application discloses a system for dynamically monitoring natural resource changes, which is realized based on WebGIS and comprises a server end and a local browser end; the server end is provided with a map service subsystem, a natural resource monitoring subsystem and a management subsystem; the natural resource monitoring subsystem comprises a natural resource change monitoring module; the natural resource change monitoring module is configured to perform the following steps: acquiring aerial remote sensing images collected by a UAV; using PHOTOMOD software to make single image of the aerial remote sensing images, correcting image distortion through aerial triangulation encryption processing, and obtaining a plurality of single image orthographic maps; splicing the plurality of single image orthographic maps to obtain an orthographic image map of a target monitoring area; acquiring a historical orthographic image map of the target monitoring area; segmenting the orthographic image map based on a preset segmentation algorithm to obtain an image segmentation result; and obtaining a change detection result by using a preset change analysis algorithm according to the historical orthographic image map and the image segmentation result.

Description

Technical Field

[0001] This invention belongs to the field of natural resource monitoring technology, specifically relating to a system for dynamically monitoring changes in natural resources. Background Technology

[0002] Natural resources play a vital role in human survival and production. However, with rapid economic development, the exploitation and utilization of natural resources have increased, leading to increasingly prominent problems of resource degradation and depletion. Therefore, the scientific and rational monitoring and management of natural resources is crucial for sustainable human development.

[0003] However, the applicant's research found that current monitoring of natural resource changes faces many challenges and shortcomings, mainly including:

[0004] (1) Limited monitoring methods: Traditional manual monitoring is time-consuming, labor-intensive, and inefficient, and cannot meet the monitoring needs of large-scale changes in natural resources. The coverage of manual monitoring is limited, the monitoring frequency is low, and it cannot provide timely and comprehensive monitoring information.

[0005] (2) Limitations of remote sensing data: Satellite remote sensing data is affected by many factors, such as cloud cover, revisit period, and climate, leading to instability and uncertainty in data acquisition. The resolution and frequency of remote sensing data are relatively low, making it difficult to capture subtle and rapid changes in resource conditions.

[0006] (3) Data processing and analysis challenges: Massive amounts of remote sensing data require efficient processing and analysis to extract useful information and discover patterns and trends in resource changes. However, current data processing technologies and analysis methods are not yet mature enough, and there are problems such as slow data processing speed and low analysis accuracy.

[0007] In summary, current monitoring of natural resource changes faces numerous shortcomings and challenges, necessitating improvements in the efficiency and accuracy of such monitoring to better support the management and control of natural resources. Summary of the Invention

[0008] The purpose of this invention is to solve the above-mentioned technical problems and provide a system for dynamically monitoring changes in natural resources.

[0009] To solve the above problems, the present invention is implemented according to the following technical solution:

[0010] This invention provides a system for dynamically monitoring changes in natural resources. The monitoring system is based on WebGIS and includes a server and a local browser. The server deploys a map service subsystem, a natural resource monitoring subsystem, and a management subsystem.

[0011] The natural resource monitoring subsystem includes a natural resource change monitoring module, which is configured to perform the following steps:

[0012] Acquire aerial remote sensing images collected by drones;

[0013] The aerial remote sensing images were processed using PHOTOMOD software to create multiple orthophotos of the single images.

[0014] Multiple single-image orthophotos are stitched together to obtain an orthophoto map of the target monitoring area;

[0015] Obtain historical orthophotos of the target monitoring area;

[0016] The orthophoto image is segmented based on a preset segmentation algorithm to obtain the image segmentation result;

[0017] Based on historical orthophotos and image segmentation results, change detection results are obtained using a preset change analysis algorithm.

[0018] Preferably, the preset segmentation algorithm includes the UPerNet network model, which adopts an FPN network with Swing Transformer as the backbone network in the feature extraction stage; Stage4 in the Swing Transformer backbone network is connected to the PPM module of the UPerNet network model.

[0019] Preferably, the preset segmentation algorithm obtains the image segmentation result through the following segmentation steps:

[0020] The backbone network of the Swin Transformer extracts and constructs multi-scale feature maps from the input image to be processed.

[0021] The PPM module is used to perform pooling and convolution operations on the last layer feature map output by the backbone network of the Swin Transformer, and then connect and fuse it with the feature maps of the same scale in the bottom-up branches to output feature maps of the corresponding scale, thereby forming a top-down multi-scale feature pyramid.

[0022] Each feature map is adjusted to 1 / 4 resolution using bilinear interpolation, and each corresponding layer is connected. Then, convolutional layers are used to fuse these feature maps from different layers.

[0023] The result of feature map fusion is input into the detection head module. After passing through a separate 3×3 convolutional layer and a classifier, the detection result is output, thus completing semantic segmentation.

[0024] Preferably, the preset change analysis algorithm includes the following steps:

[0025] Obtain historical image maps of the target monitoring area, wherein the historical image maps correspond to the orthophoto map;

[0026] The historical image and the orthophoto image are input into the CNN backbone network to extract semantic features and generate a historical image feature map and an orthophoto image feature map of the same size as the original image.

[0027] The historical image feature map and orthophoto image feature map are respectively converted into a set of semantic label information, and global information modeling is performed by an encoder. After the global modeling information is processed by a decoder, it is mapped to their respective historical image feature map and orthophoto image feature map to enhance the semantic feature representation.

[0028] Preliminary change prediction maps are derived from the enhanced historical image feature maps and orthophoto image feature maps using the difference calculation method.

[0029] The image segmentation results are fused with the preliminary change prediction map to obtain the final single-element change detection results.

[0030] Preferably, the natural resource change monitoring module is configured to perform the following surface water monitoring steps:

[0031] Acquire optical satellite remote sensing images of the area to be monitored, including 2m panchromatic images and 8m multispectral images from the same period;

[0032] The Pansharpening algorithm was used to fuse 2m panchromatic image and 8m multispectral image to obtain the current fused image with 2m spatial resolution.

[0033] Acquire historical fused images of the area to be monitored at a spatial resolution of 2m.

[0034] The historical fused image and the current fused image of the area to be monitored are divided into 16 frames at a scale of 1:50000, resulting in a total of 16 historical frame images and 16 current frame images.

[0035] The trained DeepLabv3+ network was used to automatically extract data from 16 frames of images to obtain preliminary historical water features and preliminary current water features.

[0036] By comparing preliminary historical water body patches with preliminary current water body patches, the results of surface water body changes can be obtained.

[0037] Preferably, the natural resource change monitoring module is configured to perform the following DeepLabv3+ network update steps:

[0038] Acquire optical satellite remote sensing images of the area to be monitored, including 2m panchromatic images and 8m multispectral images from the same period;

[0039] The Pansharpening algorithm was used to fuse 2m panchromatic image and 8m multispectral image to obtain the current fused image with 2m spatial resolution.

[0040] Acquire historical fused images of the area to be monitored at a spatial resolution of 2m.

[0041] The historical fused image and the current fused image of the area to be monitored are divided into 16 frames at a scale of 1:50000, resulting in a total of 16 historical frame images and 16 current frame images.

[0042] Four historical sectional images and the current sectional image were randomly selected, along with the preliminary historical water body patches and the preliminary current water body patches corresponding to the four historical sectional images and the current sectional image.

[0043] Two historical frame images and the current frame image are randomly selected and output for manual edge trimming; the two historical frame images and the current frame image used for manual edge trimming are different from the historical frame images and the current frame images of the above four scenes.

[0044] Acquire the historical and current frame images of two scenes after manual trimming, and obtain the true historical and current water body patches of the two scenes' historical and current frame images;

[0045] A training set is constructed by combining the historical and current imagery of the four scenes, as well as the corresponding preliminary historical and current water feature patches.

[0046] The historical and current imagery of the two scenes, along with the corresponding real historical and current water features, are used as the validation set.

[0047] Using the above training and validation sets, the DeepLabv3+ network was trained again.

[0048] Compared with the prior art, the beneficial effects of the present invention are:

[0049] This invention provides a system for dynamically monitoring changes in natural resources. The monitoring system is based on WebGIS and includes a server and a local browser. The server deploys a map service subsystem, a natural resource monitoring subsystem, and a management subsystem.

[0050] In this invention, the natural resource monitoring subsystem includes a natural resource change monitoring module, which is configured to perform the following steps: acquiring aerial remote sensing images collected by a UAV; using PHOTOMOD software to create single-image orthophotos from the aerial remote sensing images to obtain multiple single-image orthophotos; stitching together the multiple single-image orthophotos to obtain an orthophoto map of the target monitoring area; acquiring historical orthophoto maps of the target monitoring area; segmenting the orthophoto map based on a preset segmentation algorithm to obtain image segmentation results; and obtaining change detection results based on the historical orthophoto map and the image segmentation results using a preset change analysis algorithm.

[0051] This invention utilizes a combination of advanced technologies such as UAV aerial remote sensing imagery, GIS technology, and computer image interpretation technology to dynamically monitor natural resources through aerial photography, thereby enabling efficient and accurate effective supervision. It can promptly detect and track changes in natural resources, providing crucial data support for resource management and protection, and offering better technical support for the management and monitoring of natural resources. Attached Figure Description

[0052] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings, wherein:

[0053] Figure 1 This is a technical roadmap of the monitoring system of the present invention;

[0054] Figure 2 This is a platform architecture diagram of the monitoring system of the present invention;

[0055] Figure 3 This is a schematic diagram of the monitoring steps of the natural resource change monitoring module of the present invention:

[0056] Figure 4 This is a model architecture diagram of the UPerNet semantic segmentation network of the present invention;

[0057] Figure 5 This is a model structure diagram of the Swing Transformer network of the present invention;

[0058] Figure 6 This is a schematic diagram of the feature extraction module structure of the present invention;

[0059] Figure 7 This is a schematic diagram of channel attention generation according to the present invention;

[0060] Figure 8 This is a schematic diagram of spatial attention generation according to the present invention;

[0061] Figure 9 This is a schematic diagram of the Semantic Tokens generation process of the present invention;

[0062] Figure 10 This is a schematic diagram of the input, encoding, and output structure of the encoder of the present invention. Detailed Implementation

[0063] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0064] Currently, monitoring of changes in natural resources faces many challenges and shortcomings, mainly including:

[0065] (1) Limited monitoring methods: Traditional manual monitoring is time-consuming, labor-intensive, and inefficient, and cannot meet the monitoring needs of large-scale changes in natural resources. The coverage of manual monitoring is limited, the monitoring frequency is low, and it cannot provide timely and comprehensive monitoring information.

[0066] (2) Limitations of remote sensing data: Satellite remote sensing data is affected by many factors, such as cloud cover, revisit period, and climate, leading to instability and uncertainty in data acquisition. The resolution and frequency of remote sensing data are relatively low, making it difficult to capture subtle and rapid changes in resource conditions.

[0067] (3) Data processing and analysis challenges: Massive amounts of remote sensing data require efficient processing and analysis to extract useful information and discover patterns and trends in resource changes. However, current data processing technologies and analysis methods are not yet mature enough, and there are problems such as slow data processing speed and low analysis accuracy.

[0068] In summary, current monitoring of natural resource changes faces numerous shortcomings and challenges, necessitating improvements in the efficiency and accuracy of such monitoring to better support the management and control of natural resources.

[0069] Therefore, this invention utilizes a combination of advanced technologies such as UAV aerial remote sensing imagery, GIS technology, and computer image interpretation technology to dynamically monitor natural resources through aerial photography, thereby achieving efficient and accurate effective supervision. It can promptly detect and track changes in natural resources, providing crucial data support for resource management and protection, and offering better technical support for the management and monitoring of natural resources.

[0070] In this invention, natural resources refer to the general term for various environmental factors and conditions that, under certain historical conditions, can be developed and utilized by humankind to improve their welfare or survival capabilities, possess a certain degree of scarcity, and are subject to social constraints. Based on spatial distribution, they can be divided into surface resources, underground resources, and marine resources; based on their renewability, they can be classified into renewable resources and non-renewable resources.

[0071] The monitoring system of this invention primarily targets natural resources including land resources, forest and grassland resources, and surface water resources. It can also monitor illegal mining activities such as mining beyond permitted boundaries.

[0072] Example 1

[0073] like Figures 1-2 As shown, the system for dynamically monitoring changes in natural resources according to the present invention is implemented based on WebGIS, and the monitoring system includes a server and a local browser.

[0074] In this invention, WebGIS (Web Geographic Information System) is a geographic information system that operates using a B / S architecture. Compared to traditional desktop geographic information systems, it is based on the Internet, uses the HTTP protocol, and the program runs on the server side and the local browser side respectively, thus overcoming platform limitations and improving the accessibility of geographic information services.

[0075] In this specific technical process, the browser and server interact with each other via HTTP requests. The browser sends a service request to the GIS server, which receives and parses the request, calls relevant middleware to perform calculations to generate the response data, and then returns it to the browser over the network. The browser then renders the returned data and presents it to the user. By adopting WebGIS design, users can access geospatial data application services published by the server anytime, anywhere through a browser.

[0076] In this invention, the server-side is equipped with a map service subsystem, a natural resource monitoring subsystem, and a management subsystem.

[0077] In one implementation, mature WebGIS solutions can be used, such as the SuperMapIS.net platform provided by Beijing SuperMap, as well as open-source solutions such as GeoServer.

[0078] In a specific implementation, such as Figure 1 The technical approach of this monitoring system is shown, and as follows: Figure 2 The platform architecture of this system is shown below.

[0079] In this invention, the operational support layer includes the hardware and software infrastructure and network environment for platform operation. The data layer includes remote sensing image tile data, data from the Third National Land Survey, geographic national conditions data, business data, 3D real-scene data, and user data. The service support layer is the supporting environment for platform data storage, tiling, publishing, and program execution, including GeoServer, PostgreSQL, MySQL, Tomcat, Nginx, and ArcGIS Server. The service layer includes data resource services and platform general interface services. Data resource services, with compatibility with open spatial data protocols as their core, provide data management services, data processing services, element query services, tile services, and permission services with interoperable interfaces. The platform general interface is used to uniformly encapsulate common basic functions, including query and analysis interfaces developed for front-end business functions and resource service interfaces customized for data access. In the business layer, the platform, based on business divisions, conducts thematic business construction for specific functions, namely a 3D natural resource map, a remote sensing image data center, natural resource change monitoring, and dynamic natural resource supervision. In the presentation layer, users access the 3D dynamic natural resource supervision platform through a browser.

[0080] Example 2

[0081] This second embodiment provides a system for dynamically monitoring changes in natural resources. The system architecture and composition of the monitoring system are exactly the same as those in the first embodiment.

[0082] In this embodiment, as Figure 3 The flowchart shown illustrates that the natural resource monitoring subsystem of this invention includes a natural resource change monitoring module, which is configured to perform the following steps:

[0083] S100: Acquire aerial remote sensing images collected by drones.

[0084] S200: Use PHOTOMOD software to create single-image images from the aerial remote sensing images to obtain multiple single-image orthophotos.

[0085] S300: Stitch together multiple single-image orthophotos to obtain an orthophoto map of the target monitoring area.

[0086] S400: Acquire historical orthophotos of the target monitoring area.

[0087] S500: The orthophoto image is segmented based on a preset segmentation algorithm to obtain the image segmentation result.

[0088] S600: Based on historical orthophoto maps and image segmentation results, change detection results are obtained using a preset change analysis algorithm.

[0089] Specifically, the present invention provides a detailed description of each step as follows:

[0090] S100: Acquire aerial remote sensing images collected by drones.

[0091] In this invention, aerial remote sensing images are obtained by real-time acquisition of the target monitoring area by drones, and then transmitted by the drones or drone hangars to a cloud server for storage and access by the server.

[0092] This design allows the drone to be highly maneuverable, less affected by air traffic control and weather conditions, and capable of acquiring photographic data directly in harsh environments. Even if the equipment malfunctions, there will be no personnel casualties, ensuring high safety. The drone can fly at extremely low altitudes below clouds, compensating for the limitations of satellite remote sensing due to cloud cover, and can acquire higher resolution images than satellite remote sensing. Furthermore, low-altitude multi-angle photography can obtain high-resolution texture images of multiple surfaces of buildings.

[0093] On the other hand, drones fly at low altitudes, between 50 and 1000 meters, which is close-range aerial photogrammetry. The photogrammetry accuracy reaches the sub-meter level, with an accuracy range of 0.1 to 0.5 meters, which greatly improves the efficiency and accuracy of monitoring changes in natural resources.

[0094] In a specific implementation, before using drones for aerial photography, it is necessary to base the work on the specific task requirements and be familiar with the natural geographical features, topography, and other conditions of the aerial photography area. A clear aerial photography plan should be formulated based on the specific conditions of the area. Before the aerial photography work begins, the following should be done: 1) First, apply for airspace in the aerial photography area; 2) Second, conduct on-site reconnaissance to conduct safety risk assessment, flight route design, and control point deployment, write a flight plan, and formulate an emergency plan; 3) Finally, try to choose clear weather with sufficient sunlight and atmospheric transparency to improve the quality of the flight and photography, and ensure the work cycle.

[0095] In a preferred embodiment, considering the potential deviation caused by the misalignment between the UAV remote sensing equipment's lens and the surveying center, a method for calculating lens error is used to compensate for the surveying data and improve its accuracy. The calculation formula is as follows:

[0096] V x =(x-x0)(k1r2+k2r4+k3r6+L);

[0097] V y =(y-y0)(k1r2+k2r4+k3r6+L);

[0098] V z = (x-x0)2(y-y0)2;

[0099] In the formula: Vx, Vy, and Vz represent the deviations in the three directions of horizontal, vertical, and elevation generated by UAV remote sensing mapping; x and y represent the horizontal and vertical coordinates of the mapping point; x0 and y0 represent the correction point deviation with the origin as the reference point; k1, k2, and k3 represent the radial distortion coefficients, and in the calculation process, only k1 is usually considered; r represents the image point radial direction. Based on the above calculation results, preliminary compensation is performed on the mapping data.

[0100] Furthermore, based on the above, a univariate linear regression equation is established, and this equation is used to perform overall processing of remote sensing images. The equation expression is as follows: Y i =α+bX i +ε i In the formula: Y i This represents the equation for a univariate linear regression; α represents the estimated value, which is a constant in the calculation; b represents the empirical regression coefficient; X i ε represents the linear prediction slope of the i-th feedback data; i Let represent the random error of the i-th feedback data.

[0101] In one preferred embodiment, the control points are obtained through the previous imaging period and then connected for measurement. Based on the connected control points, the aerial images taken by the UAV are processed and subjected to aerial triangulation.

[0102] In one specific implementation, three types of drones—the P316 VTOL fixed-wing UAV equipped with remote sensing sensors, the X-Chimera compound-wing UAV, and the DJIPHANTOM 4RTK multi-rotor UAV—were used for aerial data collection.

[0103] It should be noted that the deployment of ground control points is not mandatory. The images acquired by UAV aerial photography are characterized by small image size and a large number of photos. They are then quickly stitched together using fast stitching software, and control points that meet the requirements are selected using the simple mosaicked image map of the photographed area.

[0104] S200: Use PHOTOMOD software to create single-image images from the aerial remote sensing images to obtain multiple single-image orthophotos.

[0105] In one specific implementation, after aerial imagery data is acquired, image distortion is first corrected through aerial triangulation. Next, single-image images are created using PHOTOMOD software, and these images are then corrected and homogenized to form orthophoto maps. These processed orthophoto maps within the monitoring area are then stitched together to form an orthophoto map covering the entire monitoring area. Next, image correction and geometric correction are performed on the orthophoto map within the monitoring area to reduce positional errors caused by terrain undulations. Finally, image enhancement and mosaicking are performed on the orthophoto map within the monitoring area to average positional errors and eliminate significant color differences between different areas of the image, facilitating subsequent image application and management.

[0106] In another preferred embodiment, given the distortions such as compression and warping present in the original image, preprocessing is essential. The preprocessing platform is ENVI software. First, orthorectification is performed on both the panchromatic and multispectral images to correct topographic distortions. Second, geometric correction is performed on the multispectral image using the panchromatic image as a reference to eliminate geometric errors between corresponding points in the panchromatic and multispectral images. Then, the GS fusion method is used to fuse the panchromatic and multispectral images, improving image interpretation accuracy and image quality. Finally, the study area is cropped. Specifically:

[0107] (1) Orthorectification: Orthorectification of images requires the use of a topographic elevation model (DEM) to correct topographic distortions in the image, ensuring the image meets the requirements of orthophoto projection. Orthorectification of aerial remote sensing images can correct the impact of geometric distortions caused by topography on the aerial remote sensing images. Based on RPC information, orthorectification was performed on a one-meter panchromatic image and a four-meter multispectral image. The operating platform for this experiment was ENVI, combined with a cubic convolution resampling method, utilizing...

[0108] The RPCOrthorectificationWorkflow tool performs orthorectification on multispectral and panchromatic data.

[0109] (2) Geometric Correction: Geometric correction addresses geometric distortions such as compression, deformation, and twisting between pixels in an image and their corresponding ground features. Geometric correction can eliminate geometric errors between pixels with the same name in panchromatic and multispectral images. Using the panchromatic image from aerial remote sensing as a reference, the multispectral image is corrected. This is achieved using the ImageRegistrationWorkflow tool in ENVI. The key point of this operation is that the multispectral and panchromatic images are fully registered. First, approximately 30 control points are manually or automatically selected on both the multispectral and panchromatic images. Then, control points with significant errors are checked and deleted. Finally, the corrected image is previewed, and the correction operation is performed.

[0110] (3) Data fusion and trimming

[0111] Remote sensing image fusion improves the temporal and spatial resolution of original images; it also facilitates ground feature identification and information extraction; and it enhances image interpretation accuracy and image quality. The purpose of remote sensing image fusion is to integrate remote sensing information from different types of images to create a new image with richer spatial features. For high-resolution images, Gram-Schmidt fusion offers high image fidelity and is an efficient image fusion method.

[0112] S300: Stitch together multiple single-image orthophotos to obtain an orthophoto map of the target monitoring area.

[0113] In one specific implementation, orthophotos can be generated using ContextCapture, Pix4D, or PhotoScan. Then, the generated orthophotos can be stitched together in ArcGIS or Global Mapper to obtain an orthophoto map of the target monitoring area.

[0114] S400: Acquire historical orthophotos of the target monitoring area.

[0115] In this invention, historical orthophoto maps and orthophoto maps are essentially remote sensing data from different periods, with historical orthophoto maps preceding orthophoto maps. By comparing the two, the monitoring of changes in natural resources can be achieved.

[0116] In practice, the historical orthophoto map of the target monitoring area is data historically used for natural resource monitoring and is stored in a server database. In another implementation, the historical orthophoto map can be historical spaceborne remote sensing data or historical airborne remote sensing data.

[0117] S500: The orthophoto image is segmented based on a preset segmentation algorithm to obtain the image segmentation result.

[0118] In this invention, the preset segmentation algorithm includes the UPerNet network model, which adopts an FPN network with Swing Transformer as the backbone network in the feature extraction stage; Stage4 in the Swing Transformer backbone network is connected to the PPM module of the UPerNet network model.

[0119] Specifically, this invention provides a detailed description of the UpperNet network model as follows:

[0120] like Figure 4As shown, this is the basic architecture of the UperNet network model. Its processing flow is as follows: (1) First, the backbone network in FPN extracts the input image and constructs a multi-scale feature pyramid from bottom to top. Then, the PPM module is added to the last layer of the backbone network. (2) Then, the PPM module is used to perform pooling and convolution operations on the last layer feature map output by the backbone network, and they are connected and fused with the feature maps of the same scale in the bottom-up branch. The corresponding scale feature map is then output, thus forming a multi-scale feature pyramid from top to bottom. (3) Next, in order to fuse each layer feature map with different scales in the next step, UperNet first adjusts each layer feature map to 1 / 4 resolution by using bilinear interpolation and connects each corresponding layer. Then, the convolutional layer is used to fuse these feature maps from different layers, thereby reducing the number of channels. (4) Finally, the result after feature map fusion is input into the detection head module. After passing through a separate 3×3 convolutional layer and classifier, the detection result is output, completing the semantic segmentation task.

[0121] In the natural resource segmentation and extraction task studied in this invention, the types of surface features within the study area are complex and diverse, typically including roads, land, rivers, vegetation, etc. Therefore, the UPerNet semantic segmentation network is suitable for such complex scenarios with diverse land feature types and rich feature information. However, the applicant's research found that:

[0122] While the UperNet network model can achieve good segmentation results for multi-layered visual concepts in complex scenes, it still faces the following challenges when applied to the natural resource segmentation and extraction task of this invention:

[0123] Compared to other images, UAV remote sensing imagery has a high spatial resolution, resulting in richer details and more complex and diverse features. However, the backbone network of the UPerNet network model lacks effective capture of target features when extracting complex and abundant feature information from the imagery, exhibiting weak feature representation capabilities and low computational efficiency.

[0124] Therefore, to address the above issues, the UpperNet network model in this patent is an optimization of the basic UpperNet network model. The core innovation lies in the use of an FPN network with a Swing Transformer as the backbone network in the feature extraction stage. Figure 4 The bottom-up network structure on the left side of the middle section was replaced with... Figure 5The improved model first uses the Swing Transformer to extract and construct multi-scale feature maps to represent the corresponding feature pyramid levels. Then, Stage 4 of the Swing Transformer backbone is connected to the PPM module. Finally, after processing by the PPM module, the data is fed into the top-down branch of the FPN, using the same downsampling rate as the Swing Transformer backbone. Subsequent network processing flows are consistent with the basic UPerNet network model. The feature pyramid structure of the FPN in the UPerNet network model is highly compatible with the multi-level feature representation structure obtained by the Swing Transformer backbone, achieving effective fusion of high, medium, and low-level semantic information. Simultaneously, the multi-head self-attention mechanism module in the Swing Transformer network enhances the model's global modeling and feature representation capabilities while maintaining low computational complexity, emphasizing the focus on target pixels such as ground features and water bodies, effectively eliminating the impact of non-target noise on the network model, thus enabling the improved network to achieve better semantic segmentation results. It also has the following advantages:

[0125] (1) By adopting a hierarchical construction method similar to that in CNN, multi-scale and hierarchical feature representations can be constructed. Its receptive field also gradually expands from shallow to deep layers, thus enabling the model to be applicable to pixel-level dense prediction visual tasks such as semantic segmentation.

[0126] (2) Self-attention is calculated locally in the non-overlapping window of the image to be segmented, and cross-window connection is introduced, which greatly reduces the computational complexity and improves the computational efficiency.

[0127] (3) A self-attention mechanism module based on moving windows is adopted, which realizes the transmission and interaction of information between different windows while maintaining low computational complexity, greatly enhancing the model's global modeling ability and feature expression ability.

[0128] Compared to backbone networks such as ResNet and AlexNet, the Swing Transformer, as a backbone network, can provide more compact feature representations and richer semantic information, thereby helping subsequent networks to better locate building targets and enhancing the model's modeling capabilities while reducing computational complexity.

[0129] like Figure 5As shown, firstly, the image input block segmentation layer PatchPartition, with a size of [H, W, 3], is divided into blocks of adjacent 4×4 pixels, and then flattened in the channel direction, so the image size becomes [H / 4, W / 4, 48]. Next, the network constructs feature maps of different scales through four consecutive stages: In Stage 1, the output of the previous module is input into the linear embedding layer LinearEmbedding, which performs a linear transformation on the channel data of each pixel, so the image size becomes [H / 4, W / 4, C], and then the image is input into the SwinTransformer module; the remaining three stages differ from Stage 1 in that they no longer need to go through LinearEmbedding, but instead use a block merging layer PatchMerging to downsample the input feature map.

[0130] In another implementation, Stage 3 has 6 SwingTransformer modules instead of 2, meaning there are 3 pairs in Stage 3. Figure 5 The structure shown in (b) is as follows. In the subsequent three stages, the image size will change to [H / 8, W / 8, 2C], [H / 16, W / 16, 4C] and [H / 32, W / 32, 8C] respectively, thus constructing a multi-scale feature map.

[0131] Regarding the loss function of the UpperNet network model of this invention, the loss function refers to the difference between the predicted value and the true value of the training sample in the deep learning model. The smaller the difference, the smaller the loss, that is, the closer the predicted value is to the true value, and the better the model's performance.

[0132] In this invention, a combined loss function, integrating the binary cross-entropy loss (BCE loss) and the Dice loss, is employed to improve the stability of model training. The combined loss function is: L = aL BCE +(1-a)L Dice Where a is the weight coefficient of BCE loss, and its value range is [0,1], which is determined to be 0.8.

[0133] (1) BCE loss: Cross-entropy is mainly used to measure the degree of difference between two probability distributions. The smaller the cross-entropy value, the better the model's performance. The definition of BCE loss is as follows: Where a is the model prediction value, y is the corresponding true value of the sample label, n is the total number of samples, and x is the sample value.

[0134] (2) Dice loss: The Dice coefficient is a widely used metric in the field of computer vision. It is mainly used to measure the similarity between two images. Its calculation formula is as follows: Here, A and B represent the datasets of the model's predicted results and the sample's true values, respectively. The Dice coefficient is the similarity between the two datasets, ranging from [0,1]. A larger Dice value indicates a higher degree of overlap and similarity between the two datasets, meaning a better model performance. The Dice loss is defined as:

[0135] In one specific implementation, the preset segmentation algorithm obtains the image segmentation result through the following segmentation steps:

[0136] S510: The backbone network of the Swin Transformer extracts and constructs multi-scale feature maps from the input image to be processed.

[0137] S520: The PPM module is used to perform pooling and convolution operations on the last layer feature map output by the backbone network of the Swin Transformer, and then connect and fuse it with the feature maps of the same scale in the bottom-up branches to output feature maps of the corresponding scale, thereby forming a top-down multi-scale feature pyramid.

[0138] S530: Use bilinear interpolation to adjust the feature map of each layer to 1 / 4 resolution, connect each corresponding layer, and then use convolutional layers to fuse these feature maps from different layers.

[0139] S540: The result of feature map fusion is input into the detection head module. After passing through a separate 3×3 convolutional layer and classifier, the detection result is output, completing semantic segmentation.

[0140] S600: Based on historical orthophoto maps and image segmentation results, change detection results are obtained using a preset change analysis algorithm.

[0141] In one specific implementation, the preset change analysis algorithm includes the following steps:

[0142] S610: Obtain historical image maps of the target monitoring area, wherein the historical image maps correspond to the orthophoto maps.

[0143] In specific implementation, the corresponding data refers to the historical image map and the orthophoto image map as remote sensing monitoring data of the target monitoring area at different times.

[0144] S620: Input the historical image and the orthophoto image into the CNN backbone network, extract semantic features, and generate a historical image feature map and an orthophoto image feature map of the same size as the original image.

[0145] In practical implementation, the network structure for feature extraction is as follows: Figure 6 As shown, the feature extraction module uses the encoder of the proposed FRUNetbase as the backbone network, followed by a 2x upsampling and 1×1 convolutional structure, and finally outputs a feature map F containing deeper information. This approach avoids the problem of losing some feature information due to the feature map being too small, and at the same time expands the receptive field to a certain extent.

[0146] To further represent the feature map and make it more prominent in the region of interest, this invention incorporates a combined module of channel attention and spatial attention.

[0147] (1) The input to the channel attention generation module is the feature map F, which is the final output of the feature extraction network in the previous step. C×1×1 vectors are obtained through max pooling and average pooling respectively. Then, two C×1×1 vectors are generated using the same shallow MLP module with two convolutions. The two vectors are added together and used as input to the sigmoid activation function, setting its mean to 0 and variance to 1. Finally, the channel attention is obtained. The calculation for adding the generated channel attention to the feature map is as follows: Features are the extracted feature maps, W c (Features) are the channel attention weight parameters used in network training. That is, all pixels in the feature map of each channel are multiplied by the corresponding weight, and Features′ is the feature map with channel attention added.

[0148] (2) Spatial Attention: After convolving the original C×1×1 feature map with 1×1 layers to obtain an M×H×W feature map (M H×W spatial attention weights), each channel generates a spatial attention matrix using the sigmoid activation function. The calculation method for adding the generated spatial attention to the feature map is as follows: W s (Features′) represents the spatial attention weights during network training. That is, all pixels in the feature maps that have received channel attention are multiplied by the corresponding weight. Features′ is the map with channel attention and spatial attention added.

[0149] S630: The historical image feature map and orthophoto image feature map are converted into a set of semantic label information, and global information modeling is performed by the encoder. After the global modeling information is processed by the decoder, it is mapped to the corresponding historical image feature map and orthophoto image feature map to enhance the semantic feature representation.

[0150] like Figure 8 As shown, the specific process of adding channel attention and spatial attention mechanisms and converting them into tokens is as follows: First, channel attention is added to the original feature map to obtain Features. CA Then, the feature map of spatial attention is transformed into M vectors of length HW after dimensionality reduction, and then compared with the Feature Map. CA Multiplying these together yields K semantic tokens, which are used as input to the encoder, similar to the feature vectors representing each word in the target sentence in a text translation task.

[0151] tokens represent the feature vectors of the image, and their calculation is as follows:

[0152] MDR represents the feature map F that receives channel attention. CA Expanded into matrix Me CA , MDR' represents the spatial attention weight matrix W. SA Expanded into matrix Me SA , Multiplying the two matrices yields the semantic information label (a set of M sub-label vectors of length C).

[0153] Then, the semantically rich tokens obtained in the above steps are output to the encoder, which uses its self-attention mechanism to perform correlation analysis on the features (the features represented by different geographic locations differ). The encoder's input consists of the original feature maps before and after transformation, and the encoder's output. It performs a text-translation-like operation on the information at each location in the original feature map, making the vector representation of the differences more significant. Finally, it outputs two transformed feature maps. The encoder structure is as follows... Figure 10 As shown.

[0154] The decoder's task is similar to translation. The input is the serialized result of the original feature map, which is transformed into a Q-tensor through a linear transformation function. The encoder's output tokens are then transformed into K and V tensors, respectively. These are then processed by a multi-head cross-attention module and a feedforward module. Finally, after linear transformation and activation functions, the output is an enhanced feature map with the same size as the original feature map. The calculation is as follows:

[0155] MCA(F iTokens new )=Concat(head1, head2,..., head N W A ;

[0156]

[0157] Where N is the number of attention heads, This is the linear mapping parameter matrix.

[0158] In this invention, the key aspect of the encoder lies in the multi-head self-attention mechanism. Its main idea is as follows: First, each linear input is mapped to three tensors—Q (Query), K (Key), and V (Value)—using a linear function, calculated as follows:

[0159] Q = Tokens i W q

[0160] K = Tokens i W k

[0161] V = Tokens i W v ;

[0162] Among them, W q W k W v There are three trainable parameter weight matrices. Dimensions of Q, K, and V and Tokens i Consistent. For any one of the input tokens, calculate the attention for all input tokens Q and K and the current V, as shown in the following formula: The similarity between them is represented by the dot product of Q and K. The purpose is to transform the attention matrix into a standard state distribution, which is normalized using softmax so that the sum of the attention weights of each input and the other inputs is 1.

[0163] S640: Based on the enhanced historical image feature map and orthophoto image feature map, a preliminary change prediction map is obtained through the difference calculation method;

[0164] In practical implementation, aerial remote sensing images from different phases belonging to the same target monitoring area, after passing through both the feature extraction and encoding / decoding modules, output two enhanced feature map sets. Based on this, the difference between each feature map in these two feature sets is calculated separately, as shown in the following formula: Let i, j, and r represent the pixel values ​​at coordinates (i, j) in channel c of feature maps a, b, and r, respectively. The difference result is obtained by subtracting the pixel values ​​at the corresponding channel and position in a and b and taking the absolute value. After the difference calculation, the feature maps are passed through a shallow convolutional structure and a softmax function to generate a preliminary single-channel difference result map.

[0165] S650: The image segmentation results are fused with the preliminary change prediction map to obtain the final single-element change detection results.

[0166] In practice, the output only shows the differences between the two remote sensing images. In order to achieve single-element change detection in the remote sensing images, the preliminary single-channel difference result map and the image segmentation result map are jointly judged. That is, only the regions that meet both the conditions of having a change and belonging to the image segmentation result are retained, and finally the single-element change detection result map is obtained.

[0167] To further improve change detection efficiency, a preliminary change detection result is first judged between the image segmentation result images, that is, its average pixel value is calculated. When the average pixel value is 0, it means that there are no changed patches. At this time, there is no need to perform subsequent image segmentation and single-feature change detection. The preliminary change detection result is directly used as the final result.

[0168] Example 3

[0169] This embodiment three provides a system for dynamically monitoring changes in natural resources. The system architecture and composition of the monitoring system are exactly the same as those in embodiments one and two.

[0170] In this embodiment, the natural resource change monitoring module is configured to perform the following surface water monitoring steps:

[0171] S1000: Acquire optical satellite remote sensing images of the area to be monitored, including 2m panchromatic images and 8m multispectral images from the same period.

[0172] In this invention, optical satellite remote sensing imagery requires data preprocessing, including orthorectification, registration and fusion, regional adjustment, mosaicking and color balancing, and downsizing and cropping. Specifically, 2m panchromatic imagery and 8m multispectral imagery are fused using the Pansharpening algorithm to obtain a fused image with 2m spatial resolution. Data orthorectification, registration and fusion, regional adjustment, and downsizing are performed using GXL software, ultimately yielding basic imagery data with identical geographic coordinates, consistent geometric positions, and unified color tones.

[0173] S2000: The Pansharpening algorithm is used to fuse 2m panchromatic image and 8m multispectral image to obtain the current fused image with 2m spatial resolution;

[0174] S3000: Acquire historical fused images of the area to be monitored with a spatial resolution of 2m;

[0175] S4000: Divide the historical fused image and the current fused image of the area to be monitored into 16 frames at a scale of 1:50000, resulting in a total of 16 historical frame images and 16 current frame images;

[0176] S5000: The trained DeepLabv3+ network is used to automatically extract 16 frames of images to obtain preliminary historical water features and preliminary current water features.

[0177] In this invention, remote sensing images from the domestically produced Gaofen-1 (GF-1) series optical satellites are used as the basic data source.

[0178] In this invention, DeepLabv3+, widely used in deep learning semantic segmentation, is adopted as the basic network architecture. DeepLabv3+ is an encoder-decoder structure. The encoder is a deep convolutional network with atrous spatial pyramid pooling (ASPP) modules, which enables DeepLabv3+ to effectively identify objects at different spatial scales in images. Considering the multi-scale characteristics of lakes in remote sensing images, this paper uses DeepLabv3+ as a framework to extract water targets from GF1 series satellite images.

[0179] The DeepLabv3+ model architecture comprises an encoder and a decoder. Assume the network input X is an N×N remote sensing image (3 bands, N pixels in width and height). For ease of description, this paper denotes the scale ratio of the input image to the feature map in the network as δ. For the encoding module, the input image is downsampled to extract features. Specifically, the input image X first passes through a backbone network. The backbone network can be any mainstream network, such as ResNet or Xception; this paper uses ResNet18 as the backbone network for the encoder, resulting in a 512-channel N16×N16 feature map. This feature map then passes through the ASPP module to introduce multi-scale information. The ASPP module mainly consists of a 1×1 convolutional layer, three 3×3 dilated convolutions, and a global average pooling layer. It merges features of different scales along the feature dimension and then feeds them into a 1×1 convolution for feature fusion, resulting in a new 256-dimensional N16×N16 feature map.

[0180] For the decoding module, the feature map is upsampled to reconstruct a segmentation map of the same size as the input image. Specifically, the feature map is first upsampled bilinearly with a scaling factor of 4 to obtain a 256-dimensional 4×N4 output feature (δ=4); then, the low-level features with the same spatial resolution as those in the Backbone encoder (such as...) are... Figure 3 The Low-Level Features shown are merged to obtain a 256-dimensional N4×N4 output feature set (δ=4). Then, a 3×3 convolution and bilinear upsampling with a scaling factor of 4 are used to obtain the final segmentation result. The network's output image is a single-band binary classification result image with the same size as the input image. Pixels with a value of 1 are water target pixels identified by the network, and pixels with a value of 0 are background pixels identified by the network.

[0181] S6000: Compare preliminary historical water body patches with preliminary current water body patches to obtain the results of surface water changes.

[0182] In one specific implementation, the changes in surface water are obtained by calculating the area of ​​preliminary historical water body patches and preliminary current water body patches, and comparing the numerical differences between the two. Alternatively, the preliminary historical water body patches and preliminary current water body patches can be overlaid to obtain non-overlapping areas, i.e., areas of surface water change.

[0183] In a preferred embodiment, the natural resource change monitoring module is configured to perform the following DeepLabv3+ network update steps:

[0184] S01: Acquire optical satellite remote sensing images of the area to be monitored, including 2m panchromatic images and 8m multispectral images from the same period;

[0185] S02: Use the Pansharpening algorithm to fuse the 2m panchromatic image and the 8m multispectral image to obtain the current fused image with a spatial resolution of 2m;

[0186] S03: Acquire historical fused images of the area to be monitored with a spatial resolution of 2m.

[0187] S04: Divide the historical fused image and the current fused image of the area to be monitored into 16 frames according to a scale of 1:50000, resulting in a total of 16 historical frame images and 16 current frame images;

[0188] S05: Randomly select 4 historical sectional images and the current sectional image, as well as the preliminary historical water body patches and preliminary current water body patches corresponding to the 4 historical sectional images and the current sectional image;

[0189] S06: Randomly select two historical frame images and the current frame image, and output them for manual edge trimming; wherein, the two historical frame images and the current frame image used for manual edge trimming are different from the historical frame images and the current frame images of the above four scenes.

[0190] S07: Obtain the historical and current frame images of the two scenes after manual trimming, and obtain the true historical water body patches and the true current water body patches of the two scenes' historical and current frame images;

[0191] S08: Construct a training set by combining the historical sectional images and current sectional images of the above four scenes, as well as the corresponding preliminary historical water body patches and preliminary current water body patches of the historical sectional images and current sectional images of the four scenes;

[0192] S09: The historical and current segmented images of the above two scenes, as well as the corresponding real historical and current water body patches of the historical and current segmented images of the two scenes, are used as the verification set;

[0193] S10: Using the above training and validation sets, train the DeepLabv3+ network again.

[0194] Other structures of the system for dynamically monitoring changes in natural resources described in this embodiment are available in the prior art.

[0195] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Therefore, any modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A system for dynamically monitoring changes in natural resources, characterized in that, The monitoring system is based on WebGIS and includes a server and a local browser. The server deploys a map service subsystem, a natural resource monitoring subsystem, and a management subsystem. The natural resource monitoring subsystem includes a natural resource change monitoring module, which is configured to perform the following steps: S100: Acquire aerial remote sensing images collected by drones; S200: Use PHOTOMOD software to create single-image plots from the aerial remote sensing images to obtain multiple single-image orthophotos. S300: Stitch together multiple single-image orthophotos to obtain an orthophoto map of the target monitoring area; S400: Acquire historical orthophotos of the target monitoring area; S500: The orthophoto image is segmented based on a preset segmentation algorithm to obtain the image segmentation result; S600: Based on historical orthophoto maps and image segmentation results, a pre-defined change analysis algorithm is used to obtain change detection results; Step S200 further includes: S210 uses aerial triangulation encryption processing to correct image distortion. The S220 uses PHOTOMOD software to create single-image production, and then corrects and homogenizes the single-image results to form an orthophoto of the single image. S230 stitches together the processed single-image orthophotos within the monitoring area to form an orthophoto map within the monitoring area; then it performs image correction and geometric correction on the orthophoto map within the monitoring area to reduce the positional error caused by terrain undulations; The S240 performs image enhancement and mosaicking on orthophotos within the monitoring area, averaging the positional error of the images and eliminating significant color differences between different areas of the images, thus facilitating the application and management of the images in the later stages. The preset segmentation algorithm in the S500 step includes the UPerNet network model, which uses an FPN network with a Swing Transformer as the backbone network in the feature extraction stage; Stage 4 of the Swing Transformer backbone network is connected to the PPM module of the UPerNet network model; The UPerNet network model includes: The S510 uses the backbone network in the FPN to extract input images and construct a bottom-up multi-scale feature pyramid, and then adds the PPM module to the last layer of the backbone network; The S520 uses the PPM module to perform pooling and convolution operations on the last layer feature map output by the backbone network, and then connects and fuses them with feature maps of the same scale from the bottom-up branches, thereby outputting feature maps of the corresponding scale, thus forming a top-down multi-scale feature pyramid; To enable the fusion of feature maps from different layers with varying scales in the next step, UPerNet first adjusts each layer's feature map to 1 / 4 resolution using bilinear interpolation, then connects the corresponding layers, and finally uses convolutional layers to fuse these feature maps from different layers, thereby reducing the number of channels. The S540 inputs the result of feature map fusion into the detection head module. After passing through a separate 3×3 convolutional layer and a classifier, it outputs the detection result, thus completing the semantic segmentation task.

2. The system for dynamically monitoring changes in natural resources according to claim 1, characterized in that: The distortion correction involves preprocessing the original image using ENVI software. Orthorectification was performed on panchromatic and multispectral images to correct topographic distortion in the images. Geometric correction is performed on the multispectral image based on the panchromatic image to eliminate geometric errors between corresponding points in the panchromatic and multispectral images. The panchromatic and multispectral images were fused using the GS fusion method to improve image interpretation accuracy and image quality; finally, the study area was cropped out.

3. A system for dynamically monitoring changes in natural resources according to claim 2, characterized in that, The orthorectification requires the use of a terrain elevation model to correct terrain distortion in the image, so that the image meets the requirements of orthophoto projection. Orthorectification of aerial remote sensing images can correct the influence of geometric distortion caused by terrain on aerial remote sensing images. Orthorectification was performed on one-meter panchromatic images and four-meter multispectral images based on RPC information. The experimental platform was ENVI. The RPCorthorectificationWorkflow tool was used to perform orthorectification on multispectral and panchromatic data, combined with the cubic convolution resampling method.

4. A system for dynamically monitoring changes in natural resources according to claim 2, characterized in that, The geometric correction addresses geometric distortions such as compression, deformation, and twisting between pixels in the image and their corresponding ground features. Geometric correction can eliminate geometric errors between pixels with the same name in panchromatic and multispectral images. Using the panchromatic image of aerial remote sensing as a reference, the multispectral image is corrected using the ImageRegistrationWorkflow tool in ENVI. The basic principle of the operation is that the multispectral and panchromatic images are fully registered. First, about 30 control points are manually or automatically selected on the multispectral and panchromatic images. Then, control points with large errors are checked and deleted. Finally, the corrected image is previewed and the correction operation is performed.

5. A system for dynamically monitoring changes in natural resources according to claim 1, characterized in that, The feature extraction is achieved through a feature extraction module, which uses the FRUNetbase encoder as the backbone network and adds a 2x upsampling and 1×1 convolutional structure after it, and finally outputs a feature map containing deeper information.

6. A system for dynamically monitoring changes in natural resources according to claim 5, characterized in that, Highlighting the regions of interest in the feature map, including a channel attention generation module and a spatial attention generation module; The input to the channel attention generation module is the feature map F, which is the final output of the feature extraction network in the previous step. The C×1×1 vectors are obtained by max pooling and average pooling respectively. Then, two C×1×1 vectors are obtained by using the same shallow MLP module with two convolutions. The two vectors are added together and used as the input of the sigmoid activation function, so that its mean is 0 and its variance is 1. Finally, the channel attention is obtained.

7. A system for dynamically monitoring changes in natural resources according to claim 6, characterized in that, The spatial attention generation module, after obtaining an M×H×W feature map by 1×1 convolution on the original C×1×1 feature map, generates a spatial attention matrix for each channel using the sigmoid activation function. The calculation method for adding the generated spatial attention to the feature map is as follows: ; The spatial attention weights are used for network training. That is, all pixels in the feature maps that receive channel attention are multiplied by their corresponding weights. The graph for incorporating channel attention and spatial attention.

Images