Land resource monitoring method and device based on remote sensing image and electronic equipment

Abstract

The application belongs to the technical field of remote sensing image processing, and specifically provides a land resource monitoring method and device based on remote sensing images and electronic equipment. The method comprises the following steps: obtaining a target remote sensing image of a land resource to be monitored, the target remote sensing image comprising at least one mask area, the mask area being an area historically determined as a same land type; dividing the target remote sensing image into multiple sub-image blocks, and segmenting at least one map patch in the mask area of each sub-image block; determining, for each map patch, a plurality of target frame maps of different scales, wherein each target frame map of each scale comprises at least the each map patch; and identifying a target land type of a land resource in each target frame map based on the plurality of target frame maps of different scales through a pre-trained land type recognition model. The technical scheme provided by the application can improve the accuracy of land resource monitoring.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of remote sensing image processing, and particularly relates to a land resource monitoring method and device based on remote sensing images and electronic equipment. BACKGROUND

[0002] With the continuous development of remote sensing technology, change detection of multi-temporal remote sensing images plays an increasingly key role in the field of land resource monitoring and has become an indispensable important tool. However, in the prior art, the land resource monitoring method based on remote sensing images is mainly divided into an image-image method and a vector-image method. The image-image method has certain defects. Pseudo-changes caused by seasonal changes, changes in light conditions, and sensor differences will cause the detection model to incorrectly determine unchanged areas as changed areas, thereby resulting in low accuracy of land resource monitoring. In addition, the training of the detection model of the image-image method requires a large amount of manually labeled training data, which consumes a lot of time and effort. The vector-image method has not yet formed a mature technical framework, and it is difficult to achieve high-precision detection of land resources in relatively complex scenes. In addition, with the improvement of the resolution of remote sensing images to sub-meter level, the target of land resource monitoring is shifted from traditional large-scale objects to multi-scale objects, and the accuracy requirement of land resource monitoring is increasingly improved. Therefore, how to improve the accuracy of land resource monitoring has become a technical problem to be solved. SUMMARY

[0003] Embodiments of the present application provide a land resource monitoring method and device based on remote sensing images and electronic equipment, thereby improving the accuracy of land resource monitoring.

[0004] Other characteristics and advantages of the present application will become apparent from the following detailed description, or will be learned by practice of the present application.

[0005] According to a first aspect of the embodiments of the present application, a land resource monitoring method based on remote sensing images is provided, characterized in that the method comprises: acquiring a target remote sensing image of a land resource to be monitored, the target remote sensing image comprising at least one mask area, the mask area being an area historically determined as the same land type; dividing the target remote sensing image into a plurality of sub-image blocks, and segmenting at least one graph patch in the mask area of each sub-image block; for each graph patch, determining a plurality of target frame graphs of different scales, wherein each scale of the target frame graph contains at least the each graph patch; and identifying a target land type of the land resource in each target frame graph based on the plurality of target frame graphs of different scales through a pre-trained land type recognition model.

[0006] In some embodiments of the present application, based on the foregoing scheme, the obtaining of the target remote sensing image of the land resource to be monitored comprises: obtaining an original remote sensing image of the land resource to be monitored; and dividing, in the original remote sensing image, areas in the land resource to be monitored that are historically determined to be of the same land type into mask areas to obtain the target remote sensing image.

[0007] In some embodiments of the present application, based on the foregoing scheme, the dividing of the target remote sensing image into a plurality of divided image blocks comprises: performing division processing on the target remote sensing image according to a preset specification of a division grid to obtain a plurality of divided image blocks; and if an edge area of the target remote sensing image cannot completely cover the preset specification of the division grid, making the edge area completely cover the preset specification of the division grid by using a sliding window method and a boundary filling method.

[0008] In some embodiments of the present application, based on the foregoing scheme, the dividing of at least one graph region in the mask area of each divided image block comprises: determining at least one seed point in the mask area of each divided image block; and dividing, based on a superpixel segmentation method, a graph region covering each seed point in the mask area with each seed point as the center.

[0009] In some embodiments of the present application, based on the foregoing scheme, the identifying of a target land type of a land resource in each target frame by using a pre-trained land type identification model based on the plurality of target frames of different scales comprises: inputting the plurality of target frames of different scales into the land type identification model; obtaining a confidence degree of each target frame in which a land resource is of each land type output by the land type identification model; and taking a land type with the highest confidence degree as a target land type of a land resource in each target frame.

[0010] In some embodiments of the present application, based on the foregoing scheme, the method further comprises: obtaining a confidence degree of a target land type of a land resource in each target frame; saving, as a sample frame, a target frame with a confidence degree greater than or equal to a first preset confidence threshold to a sample library, wherein the sample frames in the sample library are used for iteratively training the land type identification model; saving, as a pending frame, a target frame with a confidence degree less than the first preset confidence threshold and greater than or equal to a second preset confidence threshold to the sample library after checking the pending frame; and removing, as a noise frame, a target frame with a confidence degree less than the second preset confidence threshold.

[0011] In some embodiments of the present application, based on the foregoing scheme, the method further comprises: obtaining a historical land type of the land resource in each mask region in the target remote sensing image, and a target land type of the land resource in each target frame; if the target land type of the land resource in any one target frame is different from the historical land type of the land resource in the mask region where the any one target frame is located, marking the any one target frame as an abnormal frame.

[0012] In some embodiments of the present application, based on the foregoing scheme, the method further comprises: obtaining a confidence of the target land type of the land resource in each abnormal frame; dividing the abnormal frames with the same target land type into the same abnormal frame set, and determining an abnormal frame with the highest confidence of the target land type in the abnormal frame set as a reference frame; calculating an intersection over union of each candidate frame in the abnormal frame set with the reference frame, and removing the candidate frame with the intersection over union greater than a preset intersection over union threshold, wherein the candidate frame is an abnormal frame in the abnormal frame set except the reference frame; and performing global spatial data integration on the reference frame and the remaining candidate frames in the abnormal frame set, to obtain a global land type change detection vector result of the land resource to be monitored.

[0013] According to a second aspect of the embodiments of the present application, a land resource monitoring device based on remote sensing images is provided, and the device comprises: an acquisition unit configured to acquire a target remote sensing image of a land resource to be monitored, wherein the target remote sensing image comprises at least one mask region, and the mask region is a region historically determined to be of the same land type; a segmentation unit configured to segment the target remote sensing image into a plurality of segmented image blocks, and segment at least one plot in the mask region of each segmented image block; a determination unit configured to determine, for each plot, a plurality of target frames of different scales, wherein each target frame of each scale comprises at least the each plot; and an identification unit configured to identify a target land type of the land resource in each target frame based on the plurality of target frames of different scales by using a pre-trained land type identification model.

[0014] According to a third aspect of the embodiments of the present application, a computer program product is provided, which comprises computer instructions stored in a computer readable storage medium and adapted to be read and executed by a processor, so as to enable a computer device with the processor to perform the operations performed by the method according to any one of the above-mentioned first aspect embodiments.

[0015] According to a fourth aspect of the embodiments of the present application, a computer readable storage medium is provided, and the computer readable storage medium stores at least one computer program instruction. The at least one computer program instruction is loaded and executed by a processor to implement the operations performed by the method according to any one of the first aspect of the embodiments.

[0016] According to a fifth aspect of the embodiments of the present application, an electronic device is provided, and the electronic device includes one or more processors and one or more memories. The one or more memories store at least one computer program instruction. The at least one computer program instruction is loaded and executed by the one or more processors to implement the operations performed by the method according to any one of the first aspect of the embodiments.

[0017] Based on the technical solutions proposed in the present application, by accurately dividing the mask area, refining the patch segmentation, introducing the multi-scale target frame diagram analysis, and using the pre-trained land type recognition model, not only the accuracy and pertinence of land resource monitoring can be effectively improved, and different scale land type changes can be accurately captured, but also the adaptability of the model to complex scenes can be enhanced, and misjudgment and omission can be reduced, thereby improving the accuracy of land resource monitoring, and providing comprehensive and accurate land change information for the land resource management department to make reasonable land resource management strategies.

[0018] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. BRIEF DESCRIPTION OF DRAWINGS

[0019] The drawings incorporated into the specification and forming a part thereof illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the application. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor. In the drawings:

[0020] Figure 1 A flowchart of a land resource monitoring method based on remote sensing images in an embodiment of the present application is shown;

[0021] Figure 2 A schematic diagram of image framing in an embodiment of the present application is shown;

[0022] Figure 3 A schematic diagram of superpixel segmentation in an embodiment of the present application is shown;

[0023] Figure 4 A schematic diagram of target frame diagrams of different scales in an embodiment of the present application is shown;

[0024] Figure 5 FIG. 1 shows a schematic diagram of a land type identification model according to an embodiment of the present application;

[0025] Figure 6 FIG. 2 shows a schematic diagram of integration of an abnormality block diagram according to an embodiment of the present application;

[0026] Figure 7 FIG. 3 shows a block diagram of a land resource monitoring device based on remote sensing images according to an embodiment of the present application;

[0027] Figure 8 FIG. 4 shows a structural schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0028] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0029] In addition, the described features, structures or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, many specific details are provided to give a full understanding of the embodiments of the present application. However, those skilled in the art will realize that the technical solutions of the present application can be practiced without one or more of the specific details, or other methods, components, devices, steps, etc. can be used. In other cases, well-known methods, devices, implementations or operations are not shown or described in detail to avoid obscuring the aspects of the present application.

[0030] The block diagrams shown in the drawings are only functional entities, which do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0031] The flowcharts shown in the drawings are only exemplary illustrations, which do not necessarily include all the contents and operations / steps, and do not necessarily be executed in the described order. For example, some operations / steps can be further decomposed, and some operations / steps can be combined or partially combined, so that the actual execution order can be changed according to the actual situation.

[0032] It should also be noted that the terms "first", "second", etc. in the specification and claims of the present application and in the above drawings are used to distinguish similar objects, and do not necessarily have to be used to describe a specific order or sequence. It should be understood that the objects thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described.

[0033] With the continuous development of remote sensing technology, change detection of multi-temporal remote sensing images plays an increasingly key role in land resource monitoring field and has become an indispensable important tool. However, in the prior art, the land resource monitoring method based on remote sensing images is mainly divided into image-image method and vector-image method. The image-image method has certain defects. The pseudo change caused by factors such as seasonal change, illumination condition change and sensor difference will cause the detection model to incorrectly determine the unchanged area as the changed area, thereby resulting in low accuracy of land resource monitoring. In addition, the training of the detection model of the image-image method requires a large amount of manually labeled training data, which consumes a lot of time and effort. The vector-image method has not formed a mature technical framework, and it is difficult to realize high-precision detection of land resources in a relatively complex scene. In addition, with the improvement of the resolution of remote sensing images to sub-meter level, the target of land resource monitoring is shifted from traditional large-scale objects to multi-scale objects, and the accuracy requirement of land resource monitoring is also increasing. Therefore, the present application proposes a land resource monitoring method based on remote sensing images to improve the accuracy of land resource monitoring.

[0034] Next, the land resource monitoring method based on remote sensing images proposed by the present application will be described in detail. Figure 1 The land resource monitoring method based on remote sensing images proposed by the present application will be described in detail.

[0035] Referring to Figure 1 , a flowchart of the land resource monitoring method based on remote sensing images in an embodiment of the present application is shown, specifically, the method can be executed by a device with computing processing function. As Figure 1 shown, the land resource monitoring method based on remote sensing images can at least include steps 110 to 140, which are described in detail as follows:

[0036] In step 110, a target remote sensing image of a land resource to be monitored is acquired, the target remote sensing image includes at least one mask area, and the mask area is an area historically determined as the same land type.

[0037] In the present application, the land resource to be detected refers to a set of land surface space units dynamically observed and identified by remote sensing technology, for example, it can be a farmland area, it can also be a forest coverage area, and it can also be a city expansion change area, and the present application does not make specific limitation on this.

[0038] In the present application, the target remote sensing image refers to a remote sensing image to be analyzed after mask processing, and the mask region is a closed region marked by historical land resource monitoring data in the target remote sensing image, and the region is historically confirmed to be of the same land type (for example, all are farmland).

[0039] Further, in step 110, the target remote sensing image of the land resource to be monitored is obtained, which can be specifically performed according to steps 111 to 112 as follows:

[0040] Step 111, obtaining the original remote sensing image of the land resource to be monitored.

[0041] Step 112, in the original remote sensing image, the regions of the land resource to be monitored which are historically determined to be of the same land type are divided into mask regions to obtain the target remote sensing image.

[0042] In the present application, the original remote sensing image is an initial remote sensing image without processing, for example, a multispectral / hyperspectral image collected by a satellite, a drone or an aerial platform.

[0043] In the present application, the same land type block can be extracted as the mask region by calling a land survey database, historical remote sensing interpretation map and other data sources.

[0044] In the present application, by pre-dividing the mask region, specific regions can be monitored, avoiding comprehensive analysis of the entire remote sensing image, reducing the amount of data to be processed, and improving the data processing efficiency.

[0045] In the present application, the mask region is divided based on historical remote sensing image data, so that the monitoring result of the land type of the current land resource can be compared with the historical remote sensing image data, which can intuitively show whether the land resource has changed, and provide strong data support for land resource monitoring.

[0046] Continuing to refer to Figure 1 In step 120, the target remote sensing image is divided into a plurality of sub-image blocks, and at least one graph patch is segmented in the mask region of each sub-image block.

[0047] In the present application, by dividing the target remote sensing image into a plurality of sub-image blocks, each sub-image block can be independently input into a subsequent land type recognition model, so that the land type recognition model can process multiple sub-image blocks in parallel, thereby speeding up the processing process.

[0048] In the present application, by dividing at least one map spot in the mask area of each sub-image block, the distribution and change of land resources can be described more finely, so as to reduce the misjudgment caused by false changes due to seasonal changes, changes in light conditions, and sensor differences, thereby improving the accuracy of land resource monitoring. At the same time, by identifying the land type of each map spot, the land type of the land resources in each map spot can be more accurately judged, thereby improving the accuracy of land type identification of the entire land resource to be monitored.

[0049] Further, in the above step 120, the target remote sensing image is divided into a plurality of sub-image blocks, which can be performed according to the following steps 121 to 122:

[0050] Step 121, the target remote sensing image is divided according to the preset specification of the framing grid, and a plurality of sub-image blocks are obtained.

[0051] Step 122, if the edge region of the target remote sensing image cannot completely cover the preset specification of the framing grid, the edge region is completely covered by the preset specification of the framing grid by using the sliding window method and the boundary filling method.

[0052] In one specific embodiment of the present application, please refer to Figure 2 , which shows a schematic diagram of image framing in an embodiment of the present application, as shown in subgraph (a) in Figure 2 , the grid composed of lines 201 is the framing grid of the preset specification (1024x1024 pixels), as shown in subgraph (b) in Figure 2 , a rectangular region divided by lines 201 is a sub-image block, in addition, in subgraph (a) and subgraph (b) in Figure 2 , the area outlined by line 202 is the mask area, and the land type of the land resource in the mask area is cultivated land.

[0053] In the present application, a preset specification of framing grid can be set according to actual needs and processing capacity. The size of this grid can be determined according to the resolution of remote sensing image, processing efficiency and subsequent analysis needs, please refer to Figure 2 , which can be a 1024x1024 pixel window framing grid.

[0054] In the present application, by sliding window method, the preset specification window is slid on the image, gradually covering the entire remote sensing image, to ensure that the edge area can also be effectively processed and covered, thereby avoiding the irregularity of the edge area, in addition, by the boundary filling method, additional pixels or data are added to the edge area to completely fill the preset specification framing grid, which can be simply copying the edge pixels, or performing interpolation or predicting the edge area. In this way, it can ensure that the specification of each framing image block obtained by framing processing is uniform, and it can also effectively avoid the monitoring blind area caused by incomplete or missing edge area processing, thereby improving the accuracy of land resource monitoring.

[0055] Further, in the above step 120, at least one graph is segmented in the mask area of each framing image block, which can be performed according to the following steps 123 to 124:

[0056] Step 123, at least one seed point is determined in the mask area of each framing image block.

[0057] Step 124, based on the superpixel segmentation method, a graph covering each seed point is segmented in the mask area with each seed point as the center.

[0058] In the present application, please refer to Figure 3 , which shows a schematic diagram of superpixel segmentation in an embodiment of the present application; first, in Figure 3 , a plurality of seed points are generated in the mask area outlined by the line 202 in the subgraph (a), and then a plurality of graphs are generated with each seed point as the center, the graph is the polygon 203 shown in the subgraph (b) in Figure 3

[0059] In the present application, the remote sensing image in the mask area is segmented into a plurality of connected regions with similar color, texture and other characteristics by the superpixel segmentation method. During the segmentation process, the superpixel segmentation algorithm will expand around the seed point to its surroundings, gather the surrounding pixels similar to the seed point region together, form a graph covering the seed point, so that the graph can represent a relatively uniform area of land type in the mask area, which is helpful for subsequent land type identification and change detection.

[0060] In the present application, at least one seed point needs to be determined in the mask area of each framing image block, which can be determined based on the features (such as color, texture, shape, etc.) in the mask area, or can be set according to prior knowledge (such as known land type distribution), or can be specified according to user demand The number of seed points in the mask area, for which the present application does not make specific limitation.

[0061] ​In the present application, by determining seed points in the mask area of each sub-image block, and using superpixel segmentation method to segment multiple patches with the seed points as the center, not only can the false change interference caused by natural factors such as seasonal replacement and light condition fluctuation be avoided, but also the accuracy of land type identification can be optimized through patch division, so that the change detection of land resources is more reliable. In addition, by dividing large-scale remote sensing images into smaller and more easily processed patches, the complexity of calculation can be significantly reduced, thereby improving the processing speed.

[0062] With reference to the foregoing Figure 1 In step 130, for each patch, a plurality of target frame maps of different scales are determined, wherein each scale of the target frame maps at least contains the each patch.

[0063] In the present application, the plurality of target frame maps of different scales may, for example, be four different scales, which can be a 64x64 frame map, a 128x128 frame map, a 256x256 frame map, and a 512x512 frame map, respectively. For details, please refer to Figure 4 , which shows a schematic diagram of target frame maps of different scales in an embodiment of the present application, as shown in subgraph (b) of Figure 4 , four target frame maps of different scales are determined, wherein the first target frame map 204 is a 64x64 frame map, the second target frame map 205 is a 128x128 frame map, the third target frame map 206 is a 256x256 frame map, and the fourth target frame map 207 is a 512x512 frame map.

[0064] In the present application, by determining target frame maps of different scales and ensuring that each target frame map at least contains one patch, the land type identification model can effectively capture the characteristics and change trends of land resources at different spatial scales, thereby improving the accuracy and comprehensiveness of the land type identification model, and also enhancing the adaptability and reliability of land resource monitoring in complex scenarios, providing multi-dimensional analysis data for decision-makers to develop land resource management strategies.

[0065] With reference to the foregoing Figure 1 In step 140, based on the plurality of target frame maps of different scales, the target land type of land resources in each target frame map is identified by a pre-trained land type identification model.

[0066] Further, in the above step 140, the target land type of land resources in each target frame map is identified by a pre-trained land type identification model based on the plurality of target frame maps of different scales, which can further include steps 141 to 143 as follows:

[0067] Step 141, input the plurality of target frame maps of different scales into the land type identification model.

[0068] Step 142, obtain the confidence of the land resource in each target frame map output by the land type identification model as various land types.

[0069] Step 143, take the land type with the highest confidence as the target land type of the land resource in each target frame map.

[0070] In the present application, the land type identification model can be a feature extraction network of a CNN-Transformer hybrid architecture, can also be a machine learning model, and can also be a random forest model, and the present application does not make specific limitations.

[0071] In the present application, please refer to Figure 5 , which shows a schematic diagram of the land type identification model in an embodiment of the present application, as shown in Figure 5 After inputting the target frame map into the land type identification model, first, the feature extraction module is used to process the target frame map to obtain the overall features and the detailed features of the target frame map.

[0072] In a specific embodiment of the present application, please continue to refer to Figure 5After the 512x512 target frame diagram is input into the feature extraction module, it first enters the first convolutional layer, and after being processed by the first convolutional layer CNN module C and the Transformer module T, the 512x512 target frame diagram is converted into multiple 256x256 frame data, which is input into the second convolutional layer, and after being processed by the second convolutional layer CNN module C and the Transformer module T, each 256x256 target frame diagram is converted into multiple 128x128 frame data, which is input into the third convolutional layer, and after being processed by the third convolutional layer CNN module C and the Transformer module T, each 128x128 target frame diagram is converted into multiple 64x64 frame data, which is input into the fourth convolutional layer, and after being processed by the third convolutional layer CNN module C and the Transformer module T, each 64x64 target frame diagram is converted into multiple 32x32 frame data, which is input into the feature recognition module. The feature recognition module can output the confidence of the land resources in the target frame diagram as various land types based on the multiple 32x32 frame data. Finally, the land type with the highest confidence is taken as the target land type of the land resources in the target frame diagram. For example, if the confidence of the land resources in the target frame diagram as various land types is 0.3 for roads, 0.4 for buildings, 0.2 for vegetation, and 0.1 for cars, the land type with the highest confidence, i.e., buildings with a confidence of 0.4, is selected as the land type of the land resources in the target frame diagram.

[0073] In some other specific embodiments of the present application, the target frame diagram can also be a 256x256 target frame diagram, which is input into the feature extraction module, first enters the first convolutional layer, and after being processed by the first convolutional layer CNN module C and the Transformer module T, the 256x256 target frame diagram is converted into a plurality of 128x128 frame diagram data, which is input into the second convolutional layer, and after being processed by the second convolutional layer CNN module C and the Transformer module T, each 128x128 target frame diagram is converted into a plurality of 64x64 frame diagram data, which is input into the third convolutional layer, and after being processed by the third convolutional layer CNN module C and the Transformer module T, each 64x64 target frame diagram is converted into a plurality of 32x32 frame diagram data, which is input into the fourth convolutional layer, and after being processed by the third convolutional layer CNN module C and the Transformer module T, each 32x32 target frame diagram is converted into a plurality of 16x16 frame diagram data, and the plurality of 16x16 frame diagram data is input into the feature recognition module, the confidence of the land resources in the 256x256 target frame diagram for various land types is output by the feature recognition module, and the land type with the highest confidence is taken as the target land type of the land resources in the 256x256 target frame diagram.

[0074] In the present application, reference is continued to be made to Figure 5 The CNN module C of each convolutional layer can be used to analyze the detailed features of the target frame diagram, and the Transformer module T can be used to process the overall features in the target frame diagram. Through the fusion of overall features and detailed features, the recognition accuracy of the land type recognition model can be effectively improved.

[0075] In the present application, by inputting a plurality of target frame diagrams, the model can understand the distribution and changes of land resources from different perspectives, improve the comprehensiveness and accuracy of the recognition ability of the land type recognition model, so that the land type recognition model can comprehensively consider the features of land resources at different scales, and further more accurately recognize the land type. In this way, the misjudgment of land type caused by monitoring land resources at a single scale can be reduced, and the accuracy of the land type recognition model can be improved.

[0076] In the present application, by accurately dividing the mask area, refining the spot segmentation, introducing multi-scale target frame analysis, and using a pre-trained land type recognition model, the accuracy and pertinence of land resource monitoring can be effectively improved, different scale land type changes can be accurately captured, the adaptability of the model to complex scenes can be enhanced, misjudgment and omission can be reduced, and the accuracy of land resource monitoring can be improved, thereby providing comprehensive and accurate land change information for land resource management departments to develop reasonable land resource management strategies.

[0077] Based on the land resource monitoring method proposed in the present application, the method can further perform steps 210 to 240 as follows:

[0078] In step 210, the confidence of the target land type of the land resource in each target frame is obtained.

[0079] In step 220, the target frame with a confidence greater than or equal to a first preset confidence threshold is saved as a sample frame to a sample library, and the sample frame in the sample library is used for iterative training of the land type recognition model.

[0080] In step 230, the target frame with a confidence less than the first preset confidence threshold and greater than or equal to a second preset confidence threshold is saved to the sample library after checking the target frame.

[0081] In step 240, the target frame with a confidence less than the second preset confidence threshold is removed as a noise frame.

[0082] In the present application, the first preset confidence threshold can be 0.8, 0.85, or 0.75, and the second preset confidence threshold can be 0.3, 0.35, or 0.25, and the present application does not make specific limitations.

[0083] In the present application, if the confidence is greater than or equal to the first preset confidence threshold, it means that the target frame recognition accuracy is high, and it is a reliable recognition result, which can be used for subsequent model iterative training to improve the accuracy and robustness of the model.

[0084] In the present application, the target bounding box with a confidence less than the first preset confidence threshold and greater than or equal to the second preset confidence threshold is regarded as a pending bounding box, which means that the pending bounding box has certain uncertainty and needs to be checked. The checking can be performed by screening to determine the bounding box with accurate recognition, or by modifying to change the bounding box with incorrect recognition, for example, by optimizing the semantic boundary of the bounding box, adding a rare change category of the bounding box, etc. In this way, the accuracy of the pending bounding box recognition result can be determined, and the pending bounding box recognition result can be saved in the sample library to further enrich and optimize the training data.

[0085] In the present application, the target bounding box with a confidence less than the second preset confidence threshold is regarded as a noise bounding box and is removed. These noise samples may have problems such as being unclear and difficult to identify features, and cannot be used as training data to optimize the land type recognition model. Therefore, the noise samples can be removed.

[0086] In the present application, by retaining high-confidence sample bounding boxes for direct model optimization and iterative training, and storing medium-confidence pending bounding boxes in the sample library after secondary checking, and removing low-confidence noise bounding boxes, the quality of the training data can be ensured, thereby improving the accuracy of the land type recognition model in identifying land resources, and further improving the accuracy of land resource monitoring.

[0087] In the present application, it is also necessary to mention that when constructing the initial sample library of the land type recognition model, a small amount of training data can be used for construction, and the number of samples of various land types in the small amount of training data is basically kept consistent to ensure that the land type recognition model can recognize different land types.

[0088] Further, based on the land resource monitoring method proposed in the present application, the method can further perform steps 250 to 260 as follows:

[0089] Step 250: obtaining the historical land type of the land resource in each mask region in the target remote sensing image, and the target land type of the land resource in each target bounding box.

[0090] Step 260: if the target land type of the land resource in any one target bounding box is different from the historical land type of the land resource in the mask region where the any one target bounding box is located, marking the any one target bounding box as an abnormal bounding box.

[0091] In the present application, by comparing the land resource in the historical land type with the current land type in each target graph, it can be judged whether the land type of the land resource in the target graph changes, and the target graph with changed land type is marked as an abnormal graph, which can improve the efficiency and accuracy of land resource monitoring, and provide reliable monitoring data for land resource management department, and provide data basis for formulating reasonable land resource management policy.

[0092] In the present application, by identifying the land type of the land resource in the target graph through the land type identification model, and determining whether the land type of the land resource in the target graph changes, the limitations of manual inspection or simple image comparison method in traditional method can be reduced, so that the efficiency and accuracy of land resource monitoring can be effectively improved.

[0093] Further, based on the above scheme, the land resource monitoring method can further include steps 261 to 264 as follows:

[0094] Step 261, obtaining the confidence of the target land type of the land resource in each abnormal graph.

[0095] Step 262, dividing the abnormal graphs with the same target land type into the same abnormal graph set, and determining the abnormal graph with the highest confidence of the target land type in the abnormal graph set as the reference graph.

[0096] Step 263, calculating the intersection over union of each candidate graph in the abnormal graph set and the reference graph respectively, and removing the candidate graph with intersection over union greater than a preset intersection over union threshold, wherein the candidate graph is the abnormal graph in the abnormal graph set except the reference graph.

[0097] Step 264, performing global spatial data integration on the reference graph and the remaining candidate graph in the abnormal graph set to obtain the global land type change detection vector result of the land resource to be monitored.

[0098] In the present application, please refer to Figure 6 , which shows the integration schematic diagram of the abnormal graph in an embodiment of the present application. As Figure 6As shown in the subgraph (a) of FIG. 2, the first target frame 204, the second target frame 205, the third target frame 206 and the fourth target frame 207 are all determined as abnormal frames, and the land types of the land resources in the above frames are the same land type, so the first target frame 204, the second target frame 205, the third target frame 206 and the fourth target frame 207 can be regarded as an abnormal frame set, and the third target frame 206 with the highest confidence of the target land type is determined as the reference frame 208, and the first target frame 204, the second target frame 205 and the fourth target frame 207 are regarded as candidate frames, and the intersection-over-union ratios between the reference frame and the first target frame 204, the second target frame 205 and the fourth target frame 207 are calculated respectively, and the candidate frames with the intersection-over-union ratios greater than a preset intersection-over-union threshold (i.e., the high overlap degree with the reference frame) are removed, and the result is shown in the subgraph (b) of FIG. 2. Figure 6 As shown in the subgraph (b) of FIG. 2, the intersection-over-union ratios between the reference frame and the first target frame 204, the second target frame 205 and the fourth target frame 207 are all greater than the preset intersection-over-union threshold, so only the reference frame 208 is reserved. Finally, as shown in the subgraph (c) of FIG. 2, the reference frame 208 is subjected to global spatial data integration to obtain the global land type change detection vector result of the land resources to be monitored. Figure 6

[0099] In the present application, by processing abnormal frame data of the same land type and removing redundant abnormal frames (i.e., candidate frames with high overlap degree with the reference frame), the intuitiveness of the land resource monitoring data can be improved, so that the land resource monitoring efficiency and accuracy can be greatly improved, and comprehensive and detailed monitoring information can be provided for the land resource management department to make reasonable land resource management strategies.

[0100] Based on the technical solutions provided in the present application, by accurately dividing the mask area, refining the plot segmentation, introducing multi-scale target frame analysis and using the pre-trained land type recognition model, not only the accuracy and pertinence of land resource monitoring can be effectively improved, and different scale land type changes can be accurately captured, but also the adaptability of the model to complex scenes can be enhanced, and misjudgment and omission can be reduced, so that the accuracy of land resource monitoring can be improved, and comprehensive and accurate land change information can be provided for the land resource management department to make reasonable land resource management strategies.

[0101] The device embodiments of the present application are described below, which can be used to execute the land resource monitoring method based on remote sensing images in the above embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the above embodiments of the land resource monitoring method based on remote sensing images.

[0102] Figure 7 ​A block diagram of a land resource monitoring device based on remote sensing images in an embodiment of the present application is shown.

[0103] With reference to Figure 7 According to the land resource monitoring device based on remote sensing images 700 in an embodiment of the present application, it comprises an acquisition unit 701, a segmentation unit 702, a determination unit 703, and an identification unit 704.

[0104] The acquisition unit 701 is configured to acquire a target remote sensing image of a land resource to be monitored, wherein the target remote sensing image comprises at least one mask area, and the mask area is an area historically determined to be of the same land type.

[0105] In some embodiments of the present application, based on the foregoing scheme, the acquisition unit 701 is configured to acquire an original remote sensing image of the land resource to be monitored; and in the original remote sensing image, the area historically determined to be of the same land type in the land resource to be monitored is divided into a mask area to obtain a target remote sensing image.

[0106] In some embodiments of the present application, based on the foregoing scheme, the segmentation unit 702 is configured to perform framing processing on the target remote sensing image according to a preset specification of a framing grid to obtain a plurality of framed image blocks; and if an edge area of the target remote sensing image cannot completely cover the preset specification of the framing grid, the edge area is made to completely cover the preset specification of the framing grid by using a sliding window method and a boundary filling method.

[0107] In some embodiments of the present application, based on the foregoing scheme, the segmentation unit 702 is further configured to determine at least one seed point in the mask area of each framed image block; and based on a superpixel segmentation method, a graphon covering each seed point is segmented in the mask area with each seed point as the center.

[0108] In some embodiments of the present application, based on the foregoing scheme, the identification unit 704 is configured to: input the plurality of target frame maps of different scales into the land type identification model; obtain the confidence of the land resource in each target frame map being of various land types output by the land type identification model; and take the land type with the highest confidence as the target land type of the land resource in each target frame map.

[0109] In some embodiments of the present application, based on the foregoing scheme, the device further comprises a saving unit configured to: obtain the confidence of the target land type of the land resource in each target frame map; save the target frame map with a confidence greater than or equal to a first preset confidence threshold to a sample library as a sample frame map, wherein the sample frame map in the sample library is used for iterative training of the land type identification model; save the target frame map with a confidence less than the first preset confidence threshold and greater than or equal to a second preset confidence threshold to the sample library after checking the target frame map; and remove the target frame map with a confidence less than the second preset confidence threshold as a noise frame map.

[0110] In some embodiments of the present application, based on the foregoing scheme, the device further comprises a marking unit configured to: obtain the historical land type of the land resource in each mask region in the target remote sensing image and the target land type of the land resource in each target frame map; and mark any one target frame map as an abnormal frame map if the target land type of the land resource in the any one target frame map is different from the historical land type of the land resource in the mask region where the any one target frame map is located.

[0111] In some embodiments of the present application, based on the foregoing scheme, the device further comprises an integration unit configured to: obtain the confidence of the target land type of the land resource in each abnormal frame map; divide abnormal frame maps with the same target land type into the same abnormal frame map set, and determine the abnormal frame map with the highest confidence of the target land type in the abnormal frame map set as a reference frame map; calculate the intersection over union of each candidate frame map in the abnormal frame map set with the reference frame map, and remove the candidate frame map with an intersection over union greater than a preset intersection over union threshold, wherein the candidate frame map is an abnormal frame map in the abnormal frame map set except the reference frame map; and perform global spatial data integration on the reference frame map and the remaining candidate frame maps in the abnormal frame map set to obtain a global land type change detection vector result of the land resource to be monitored.

[0112] As another embodiment of the present application, a computer program product or computer program is also provided, which includes computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method described in the above embodiments.

[0113] As another embodiment of the present application, a computer readable storage medium is also provided, which can be included in the electronic device described in the above embodiments, or can exist separately and not be assembled into the electronic device. The computer readable storage medium carries one or more programs, which, when executed by the electronic device, enable the electronic device to implement the method described in the above embodiments.

[0114] Based on the same inventive concept, the embodiments of the present application also provide an electronic device. Referring to Figure 8 , a structural schematic diagram of an electronic device in an embodiment of the present application is shown. The electronic device includes one or more memories 804, one or more processors 802, and at least one computer program (program code) stored in the memory 804 and executable on the processor 802, and the processor 802 implements the method as described above when executing the computer program.

[0115] In the above embodiments, the electronic device includes one or more processors 802, one or more memories 804, and at least one computer program (program code) stored in the memory 804 and executable on the processor 802, and the processor 802 implements the method as described above when executing the computer program. Figure 8 In the above embodiments, the electronic device includes one or more processors 802, one or more memories 804, and at least one computer program (program code) stored in the memory 804 and executable on the processor 802, and the processor 802 implements the method as described above when executing the computer program.

[0116] The functions described herein can be implemented in hardware, software executed by a processor, firmware, or any combination thereof. If implemented in software executed by a processor, the functions can be stored on or transferred over as one or more instructions or code on a computer-readable medium. Other examples and implementations are within the scope and spirit of the disclosure and appended claims. For example, due to the nature of software, functions described above can be implemented using software executed by a processor, hardware, firmware, hardwiring, or combinations of any of these. Features implementing functions can also be physically located at various positions, including being distributed such that portions of functions are implemented at different physical locations. Also, as used herein, including in the claims, "or" as used in a list of items prefaced by "comprising" to indicate a disjunctive list means each single item in the list has been recited before "or" one or more additional disjunctive items also have been recited. However, "or" in such a phrase does not mean that the list is inclusive of at least one of the items. Further, as used herein, "comprising" is to be interpreted as including the more restrictive terms "consisting of" and "consisting essentially of."

[0117] In several embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. Among them, the above-mentioned device embodiments are only schematic, for example, the division of the units can be a logical function division, and other division manners can be used in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, units or modules, which can be electrical or other forms.

[0118] The units described as separate components can or can not be physically separated, and the components of the control device can or can not be physical units, i.e. can be located in one place or can be distributed on a plurality of units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0119] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part of the prior art that makes a contribution or the whole or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium, including instructions for making a computer device (which can be a personal computer, a server or a network device, etc.) execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic disk or an optical disk, etc. Various media that can store program codes.

[0120] The above merely provides an example of the present application, and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall fall into the scope of claims of the present application.

Claims

1. A land resource monitoring method based on remote sensing images, characterized in that, The method comprises: acquiring a target remote sensing image of a land resource to be monitored, the target remote sensing image comprising at least one mask region, the mask region being a region historically determined as a same land type; dividing the target remote sensing image into a plurality of divided image blocks, and segmenting at least one graph patch in the mask region of each divided image block; for each graph patch, determining a plurality of target frame graphs of different scales, wherein each scale of the target frame graph comprises at least the each graph patch; based on the plurality of target frame graphs of different scales, identifying a target land type of the land resource in each target frame graph by using a pre-trained land type recognition model.

2. The method of claim 1, wherein, The method further comprises: acquiring a target remote sensing image of a land resource to be monitored, the target remote sensing image comprising at least one mask region, the mask region being a region historically determined as a same land type; dividing the target remote sensing image into a plurality of divided image blocks, and segmenting at least one graph patch in the mask region of each divided image block; 3. The method of claim 1, wherein, for each graph patch, determining a plurality of target frame graphs of different scales, wherein each scale of the target frame graph comprises at least the each graph patch; based on the plurality of target frame graphs of different scales, identifying a target land type of the land resource in each target frame graph by using a pre-trained land type recognition model. The method further comprises:

4. The method of claim 1, wherein, acquiring a target remote sensing image of a land resource to be monitored, the target remote sensing image comprising at least one mask region, the mask region being a region historically determined as a same land type; dividing the target remote sensing image into a plurality of divided image blocks, and segmenting at least one graph patch in the mask region of each divided image block; for each graph patch, determining a plurality of target frame graphs of different scales, wherein each scale of the target frame graph comprises at least the each graph patch; 5. The method of claim 1, wherein, based on the plurality of target frame graphs of different scales, identifying a target land type of the land resource in each target frame graph by using a pre-trained land type recognition model. The method further comprises: acquiring a target remote sensing image of a land resource to be monitored, the target remote sensing image comprising at least one mask region, the mask region being a region historically determined as a same land type; dividing the target remote sensing image into a plurality of divided image blocks, and segmenting at least one graph patch in the mask region of each divided image block; 6. The method of claim 5, wherein, for each graph patch, determining a plurality of target frame graphs of different scales, wherein each scale of the target frame graph comprises at least the each graph patch; based on the plurality of target frame graphs of different scales, identifying a target land type of the land resource in each target frame graph by using a pre-trained land type recognition model. The method further comprises: acquiring a target remote sensing image of a land resource to be monitored, the target remote sensing image comprising at least one mask region, the mask region being a region historically determined as a same land type; dividing the target remote sensing image into a plurality of divided image blocks, and segmenting at least one graph patch in the mask region of each divided image block; 7. The method according to any one of claims 1 to 5, characterized in that, for each graph patch, determining a plurality of target frame graphs of different scales, wherein each scale of the target frame graph comprises at least the each graph patch; based on the plurality of target frame graphs of different scales, identifying a target land type of the land resource in each target frame graph by using a pre-trained land type recognition model. The method further comprises: acquiring a target remote sensing image of a land resource to be monitored, the target remote sensing image comprising at least one mask region, the mask region being a region historically determined as a same land type; dividing the target remote sensing image into a plurality of divided image blocks, and segmenting at least one graph patch in the mask region of each divided image block; for each graph patch, determining a plurality of target frame graphs of different scales, wherein each scale of the target frame graph comprises at least the each graph patch; based on the plurality of target frame graphs of different scales, identifying a target land type of the land resource in each target frame graph by using a pre-trained land type recognition model. If a target land type of a land resource in any one target block diagram is different from a historical land type of the land resource in a mask region where the any one target block diagram is located, the any one target block diagram is marked as an abnormal block diagram.

8. The method of claim 7, wherein, The method further includes: obtaining a confidence degree of the target land type of the land resource in each abnormal block diagram; dividing the abnormal block diagrams with the same target land type into a same abnormal block diagram set, and determining an abnormal block diagram with a highest confidence degree of the target land type in the abnormal block diagram set as a reference block diagram; calculating an intersection-over-union ratio of each candidate block diagram in the abnormal block diagram set and the reference block diagram, and removing a candidate block diagram with an intersection-over-union ratio greater than a preset intersection-over-union ratio threshold, wherein the candidate block diagram is an abnormal block diagram in the abnormal block diagram set except the reference block diagram; performing global spatial data integration on the reference block diagram and the remaining candidate block diagrams in the abnormal block diagram set to obtain a global land type change detection vector result of the land resource to be monitored.

9. A land resource monitoring device based on remote sensing images, characterized in that, The device includes: an acquisition unit configured to acquire a target remote sensing image of a land resource to be monitored, the target remote sensing image including at least one mask region, and the mask region being a region historically determined as a same land type; a segmentation unit configured to segment the target remote sensing image into a plurality of segmented image blocks, and segment at least one map patch in the mask region of each segmented image block; a determination unit configured to determine, for each map patch, a plurality of target block diagrams of different scales, wherein each target block diagram of a scale includes at least the each map patch; an identification unit configured to identify, based on the plurality of target block diagrams of different scales, a target land type of a land resource in each target block diagram by using a pre-trained land type identification model.

10. An electronic device, comprising: The electronic device includes one or more processors and one or more memories, the one or more memories storing at least one program code, the at least one program code being loaded and executed by the one or more processors to implement the method according to any one of claims 1 to 8.

Images