Non-agriculturalization Monitoring Device, Method, Equipment and Storage Medium

Abstract

The present application discloses a non-agriculturalization monitoring device, method, equipment and storage medium. The method includes: when a non-agriculturalization monitoring instruction for a non-agriculturalization area is detected, determining a remote sensing image set of the non-agriculturalization area and determining a reference image from the remote sensing image set; extracting patch change information of each change patch between each other image and the reference image, and determining a target comparison image of the change patch based on the patch change information, where the other image is an image in the remote sensing image set except the reference image; displaying the target comparison image of the change patch on a preset monitoring interface for an operator to perform non-agriculturalization monitoring on the change patch based on the target comparison image. The present application improves the monitoring accuracy of non-agriculturalization monitoring.

Description

Technical Field

[0001] This application relates to the technical field of remote sensing applications, and particularly to a non-agriculturalization monitoring device, method, equipment and storage medium. Background Art

[0002] Cultivated land is an important natural resource, and the reasonable utilization and protection of cultivated land are important guarantees for social and economic development. And the monitoring of the non-agriculturalization of cultivated land (it is strictly prohibited to illegally occupy cultivated land for afforestation, lake excavation, non-agricultural construction, etc.) is an important link in the reasonable utilization and protection of cultivated land.

[0003] Currently, remote sensing technology is generally used to monitor the non-agriculturalization of cultivated land. Remote sensing technology is a non-contact long-distance detection technology. Generally, by using modern transportation tools and sensors, the electromagnetic wave characteristics of the target area or target object obtained from a long distance are presented in remote sensing images or remote sensing photos.

[0004] In the prior art, when using remote sensing images for cultivated land change monitoring, adjacent remote sensing images obtained by sensors are analyzed based on algorithms to extract suspected change information for operators to review. However, the types of illegal occupation on each cultivated land change area, that is, each patch, may be different, and the timeliness of illegal occupation of the corresponding cultivated land is different. Each patch is extracted based on adjacent remote sensing images, and only suspected change information is extracted, which may lead to inaccurate monitoring of non-agriculturalization. Summary of the Invention

[0005] In view of this, embodiments of this application provide a non-agriculturalization monitoring device, method, equipment and storage medium, aiming to solve the technical problem of inaccurate non-agriculturalization monitoring.

[0006] Embodiments of this application provide a non-agriculturalization monitoring method, and the method includes:

[0007] When a non-agriculturalization monitoring instruction for a non-agriculturalization area is detected, determine a set of remote sensing images of the non-agriculturalization area, and determine a reference image from the set of remote sensing images;

[0008] Extract the patch change information of each change patch between each other image and the reference image, and determine the target comparison image of the change patch based on the patch change information, where the other image is an image in the set of remote sensing images except the reference image;

[0009] Display the target comparison image of the change patch on a preset monitoring interface for the operator to perform non-agriculturalization monitoring on the change patch based on the target comparison image.

[0010] In a possible implementation manner of the present application, the step of determining the remote sensing image set of the non-agriculturalization area and determining the reference image from the remote sensing image set when detecting the non-agriculturalization monitoring instruction for the non-agriculturalization area includes:

[0011] When detecting the non-agriculturalization monitoring instruction for the non-agriculturalization area, determine the multi-temporal remote sensing image set of the non-agriculturalization area, and determine the reference image from the remote sensing image set.

[0012] In a possible implementation manner of the present application, the step of extracting the patch change information of each change patch between the other image and the reference image and determining the target comparison image of the change patch based on the patch change information includes:

[0013] Determine the patch area, change confidence, and classification confidence of each change patch between the other image and the reference image;

[0014] Based on the area proportion of the patch area in all change patches and its first influence weight, the change confidence and its second influence weight, and the classification confidence and its third influence weight, determine the change degree score of the change patch;

[0015] Based on the change degree score, determine the target comparison image of the change patch.

[0016] In a possible implementation manner of the present application, the step of determining the patch area of each change patch, the change confidence of each change patch, and the classification confidence of each change patch between the other image and the reference image includes:

[0017] Input the other image and the reference image into a preset change detection model, and perform comparison processing on the other image and the reference image based on the preset change detection model to obtain the patch area and change confidence of each change patch;

[0018] Input each change patch into a preset classification model, and perform classification processing on the change patch based on the preset classification model to obtain the classification confidence.

[0019] In a possible implementation manner of the present application, the step of displaying the target comparison image of the change patch on a preset monitoring interface includes:

[0020] Determine the patch type of the change patch;

[0021] For change patches of the same type, display the change patch and the target comparison image of the change patch based on the high and low change degree scores of the change patch.

[0022] In a possible implementation manner of the present application, after the step of displaying the target comparison image of the changed patch on the preset monitoring interface, the method includes:

[0023] If it is determined that the change degree score of the changed patch is less than the preset score value, prompt the operator to compare the comparison granularity of the changed patch, so that the operator can perform non-agriculturalization monitoring on the changed patch based on the target comparison image and the granularity.

[0024] In a possible implementation manner of the present application, the step of extracting the patch change information of each changed patch between each other image and the reference image, and determining the target comparison image of the changed patch based on the patch change information includes:

[0025] Based on the extraction models with different model structures, extract the patch change information of each changed patch between each other image and the reference image to determine different change scores of the changed patch;

[0026] Determine the average value of the different change scores as the target score of the changed patch;

[0027] Based on the target score, determine the target comparison image of the changed patch.

[0028] The present application also provides a non-agriculturalization monitoring device, and the device includes:

[0029] The first determination module is used to determine the remote sensing image set of the non-agriculturalized area and determine the reference image from the remote sensing image set when detecting the non-agriculturalization monitoring instruction of the non-agriculturalized area;

[0030] The second determination module is used to extract the patch change information of each changed patch between each other image and the reference image, and determine the target comparison image of the changed patch based on the patch change information, where the other image is the image in the remote sensing image set except the reference image;

[0031] The display module is used to display the target comparison image of the changed patch on the preset monitoring interface for the operator to perform non-agriculturalization monitoring on the changed patch based on the target comparison image.

[0032] In a possible implementation manner of the present application, the non-agriculturalization monitoring device is further used to: when detecting the non-agriculturalization monitoring instruction of the non-agriculturalized area, determine the multi-temporal remote sensing image set of the non-agriculturalized area and determine the reference image from the remote sensing image set.

[0033] In a possible implementation manner of the present application, the non-agriculturalization monitoring device is further configured to: determine the patch area, change confidence, and classification confidence of each changed patch between the other image and the reference image; determine the change degree score of the changed patch based on the area ratio of the patch area in all changed patches and its first influence weight, the change confidence and its second influence weight, and the classification confidence and its third influence weight; and determine the target comparison image of the changed patch based on the change degree score.

[0034] In a possible implementation manner of the present application, the non-agriculturalization monitoring device is further configured to: input the other image and the reference image into a preset change detection model, perform comparison processing on the other image and the reference image based on the preset change detection model to obtain the patch area and change confidence of each changed patch; and input each changed patch into a preset classification model, and perform classification processing on the changed patch based on the preset classification model to obtain the classification confidence.

[0035] In a possible implementation manner of the present application, the non-agriculturalization monitoring device is further configured to: determine the patch type of the changed patch; for the changed patches of the same type, display the changed patches and the target comparison images of the changed patches based on the high and low change degree scores of the changed patches.

[0036] In a possible implementation manner of the present application, the non-agriculturalization monitoring device is further configured to: if it is determined that the change degree score of the changed patch is less than a preset score value, prompt the operator to compare the comparison granularity of the changed patch, so that the operator can perform non-agriculturalization monitoring on the changed patch based on the target comparison image and the granularity.

[0037] In a possible implementation manner of the present application, the non-agriculturalization monitoring device is further configured to: extract the patch change information of each changed patch between each other image and the reference image based on an extraction model with different model structures to determine different change scores of the changed patch; determine the average value of the different change scores as the target score of the changed patch; and determine the target comparison image of the changed patch based on the target score.

[0038] The present application further provides a non-agriculturalization monitoring device. The non-agriculturalization monitoring device is an entity node device, and the non-agriculturalization monitoring device includes: a memory, a processor, and a program of the non-agriculturalization monitoring method stored on the memory and executable on the processor. When the program of the non-agriculturalization monitoring method is executed by the processor, the steps of the non-agriculturalization monitoring method as described above can be implemented.

[0039] To achieve the above object, a storage medium is further provided. A non-agriculturalization monitoring program is stored on the storage medium. When the non-agriculturalization monitoring program is executed by a processor, the steps of any one of the above non-agriculturalization monitoring methods are implemented.

[0040] The present application provides a non-agriculturalization monitoring device, method, equipment and storage medium. Compared with the prior art in which the suspected change information extracted from adjacent remote sensing images is provided to the staff for review, resulting in inaccurate non-agriculturalization monitoring, in the present application, when a non-agriculturalization monitoring instruction for a non-agriculturalized area is detected, a remote sensing image set of the non-agriculturalized area is determined, and a reference image is determined from the remote sensing image set; the patch change information of each change patch between each other image and the reference image is extracted, and a target comparison image of the change patch is determined based on the patch change information, wherein the other image is an image in the remote sensing image set other than the reference image; the target comparison image of the change patch is displayed on a preset monitoring interface for the operator to perform non-agriculturalization monitoring on the change patch based on the target comparison image. It can be understood that when the non-agriculturalization monitoring instruction is detected in the present application, for each change patch (suspected) between the other image and the reference image in the remote sensing image set, the corresponding patch change information is extracted, rather than only being extracted from adjacent remote sensing images. Therefore, the inaccuracy caused by the different timeliness of illegal occupation of corresponding cultivated land is avoided. In addition, in the present application, based on the corresponding patch change information, the best target comparison image is determined and displayed to the operator for review. Since not only the patch change information is displayed to the operator, that is, in the present application, the best target comparison image of each change patch is conveniently displayed to the operator, thus enabling the operator to perform targeted review (avoiding inaccurate review caused by the operator randomly comparing images), and improving the accuracy of the operator's review and monitoring. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a schematic diagram of the scenario of the non-agriculturalization monitoring system of the present application;

[0042] Figure 2 It is a schematic flowchart of the first embodiment of the non-agriculturalization monitoring method of the present application;

[0043] Figure 3 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment solution of the present application;

[0044] Figure 4 It is a schematic diagram of the first scenario involved in the non-agriculturalization monitoring method of the present application;

[0045] Figure 5 It is a schematic diagram of the second scenario involved in the non-agriculturalization monitoring method of the present application;

[0046] Figure 6 This is a schematic diagram of the third scenario involved in the non - agricultural conversion monitoring method of this application. Detailed implementation manners

[0047] It should be understood that the specific embodiments described herein are only used to explain this application and are not used to limit this application.

[0048] The embodiments of this application provide a non - agricultural conversion monitoring method. In the first embodiment of the non - agricultural conversion monitoring method of this application, with reference to Figure 1 , the non - agricultural conversion monitoring method includes:

[0049] Step S10, when a non - agricultural conversion monitoring instruction for a non - agricultural conversion area is detected, determine the remote sensing image set of the non - agricultural conversion area, and determine a reference image from the remote sensing image set;

[0050] Step S20, extract the patch change information of each change patch between each other image and the reference image, and determine the target comparison image of the change patch based on the patch change information, where the other image is an image in the remote sensing image set except the reference image;

[0051] Step S30, display the target comparison image of the change patch on a preset monitoring interface for an operator to perform non - agricultural conversion monitoring on the change patch based on the target comparison image.

[0052] In this embodiment, it should be noted that:

[0053] Remote sensing technology is a non - contact long - distance detection technology. Remote sensing technology uses modern transportation tools and sensors to obtain the electromagnetic wave characteristics of the target area or target object from a long distance.

[0054] In this embodiment, remote sensing images or remote sensing pictures generally refer to films or photos that can record the electromagnetic wave magnitudes of various target areas or target objects. Remote sensing images are mainly divided into aerial photos and satellite photos.

[0055] As an example, remote sensing images can be divided into satellite remote sensing images, aerial remote sensing images, unmanned aerial vehicle (UAV) remote sensing images, aircraft remote sensing images, and hot - air balloon remote sensing images according to the type of sensor.

[0056] As an example, remote sensing images can be divided into optical images (visible light, multispectral, hyperspectral) and SAR (Synthetic Aperture Radar) images according to different detection ranges.

[0057] As an example, remote sensing images generally have three types of resolutions: temporal resolution, spectral resolution, and spatial resolution.

[0058] Among them, the time resolution is: the minimum time interval between two adjacent remote sensing observations carried out in the same area;

[0059] The spectral resolution is: the minimum wavelength interval that the sensor can resolve when receiving the spectrum of the target radiation;

[0060] The spatial resolution is: the minimum size of the target ground object that the sensor can distinguish, usually represented by the actual ground size corresponding to a pixel in the image.

[0061] It should be noted that the applications of remote sensing technology mainly include the following categories:

[0062] First, land survey: including but not limited to mineral resource exploration, land resource survey and evaluation;

[0063] Second, agricultural applications: extraction of crop planting area, growth monitoring, protection of arable land from non-agricultural use;

[0064] Third, ecological environment monitoring: monitoring of forest and grassland coverage;

[0065] Fourth, water conservancy applications: monitoring of four types of chaos in rivers and lakes (illegal construction and sand mining), changes in the water area shoreline, etc.;

[0066] Fifth, monitoring and early warning of geological disasters, etc.

[0067] In this embodiment, after obtaining the remote sensing image, the algorithms for analyzing the remote sensing image mainly include:

[0068] 1. Change detection model: The input of this model is multi-temporal remote sensing images, and the output is the changed area or changed light spot. The change detection model is usually used for monitoring illegal construction and ecological protection;

[0069] 2. Target extraction model: The input of this model is remote sensing images, and the output is the target area or target object, such as greenhouses, buildings, water areas, etc.;

[0070] 3. Land cover classification model: The input of this model is remote sensing images, and the output is various geographical elements, including forests, grasslands, roads, crops, etc.

[0071] Arable land is an important natural resource. Rational utilization and protection of arable land are important guarantees for social and economic development.

[0072] Currently, the country has put forward the "non-agriculturalization" policy for arable land. The "non-agriculturalization" policy clearly states that it is strictly prohibited to illegally occupy arable land for afforestation, digging lakes for scenery, illegal construction, and engaging in non-agricultural construction, etc.

[0073] Such as Figure 4 is an example of illegal construction on arable land, and the arrow indicates the corresponding illegal change in the corresponding arable land.

[0074] More strictly, the state has put forward the "non - grain - oriented" policy for cultivated land. The "non - grain - oriented" policy clearly states that it is strictly prohibited to illegally occupy cultivated land to plant other plants except rice, corn, and wheat, etc.

[0075] To avoid situations such as illegally occupying cultivated land for afforestation, digging lakes for landscaping, illegally building houses, and engaging in non - agricultural construction, it is necessary to monitor cultivated land. The monitoring can generally be divided into monthly, quarterly, and annual monitoring. In this embodiment, taking the implementation of non - agriculturalization monitoring on a quarterly basis as an example for specific illustration.

[0076] If non - agriculturalization monitoring is carried out quarterly, then 4 - period images are captured annually through remote sensing technology. The specific goal is to monitor the changes in cultivated land, output the changed areas and change types, etc., for the operators to review.

[0077] Specifically, multiple scenes of images are captured annually through remote sensing technology, such as spring remote - sensing images, summer remote - sensing images, autumn remote - sensing images, and winter remote - sensing images. The information of the changed patches (including the changed areas, types, and scores, etc.) of the summer, autumn, and winter remote - sensing images relative to the spring remote - sensing image are extracted respectively.

[0078] In this embodiment, the research and development background is as follows:

[0079] In the prior art, when using remote - sensing images for cultivated - land change monitoring, adjacent remote - sensing images obtained by sensors are analyzed based on algorithms to extract suspected change information for the operators to review. However, the types of possible illegal occupation on each cultivated - land change area, that is, each patch, may be different, and the timeliness of illegal occupation of the corresponding cultivated land is different. Each patch is extracted based on adjacent remote - sensing images, and only extracting suspected change information may lead to inaccurate non - agriculturalization monitoring.

[0080] Specifically, for example, when illegal occupation of cultivated land starts in summer with a small area, and the area of illegal occupation becomes larger in autumn. Based on adjacent remote - sensing images, such as comparing the spring remote - sensing image with the summer remote - sensing image, the extracted suspected change information is obviously different from that obtained by comparing the spring remote - sensing image with the autumn remote - sensing image. The difference between the two may lead to inaccurate non - agriculturalization monitoring.

[0081] Specifically, for another example, only extracting suspected change information will also lead to inaccurate non - agriculturalization monitoring. This is because when the operator reviews, if only the suspected change information is displayed without showing the corresponding remote - sensing images, it will cause the operator to select the wrong remote - sensing images for review and confirmation, resulting in inaccurate monitoring.

[0082] The purpose of this application is to avoid inaccurate monitoring caused by the different timeliness of illegal occupation of the corresponding cultivated land.

[0083] Overall, in this embodiment, as Figure 6 shown, receive the input multi-scene images (pictures) or directly input multi-scene images (pictures), select the reference image (picture), and then sequentially perform change monitoring and scoring on other images (pictures) with the reference image (picture), and output the best image pair for auditing the changed patches. For example, take the spring image as the reference image, and compare the images taken in summer, autumn, and winter with the spring image in sequence, extract the changed patch information (including the changed area, type, score), and select the best image pair for the operator to audit by comparing the scores of the changed patches.

[0084] Specifically, in this embodiment, Figure 5 As shown on the left is a schematic diagram of the non-agriculturalization monitoring device. After the non-agriculturalization monitoring device detects the input data, it is processed by the controller, stored in the memory, and displayed on the interface (audit interface). Figure 5 As shown on the right is the audit interface. The image (picture) currently being audited, i.e., Image (Picture) 1, is displayed on this interface, and the best comparison image (picture) is also displayed on this interface, which is convenient for the staff to audit. In this embodiment, the audit interface also displays the image list and patch information. In addition, in this embodiment, an artificial audit button and an automatic interpretation button are also displayed on the audit interface, so that the staff can click the artificial audit button to trigger the start of the audit, and click the automatic interpretation button to trigger the automatic interpretation. After the automatic interpretation is triggered, the interface will display the automatic interpretation results such as the location of the patch, the area of the patch, etc.

[0085] In addition, in this application, based on the corresponding patch change information, the best target comparison image is determined specifically and displayed to the operator for auditing. Since not only the patch change information is displayed to the operator, that is, in this application, the best target comparison image of each changed patch is also conveniently displayed to the operator, thus enabling the operator to conduct targeted audits (avoiding inaccurate audits caused by the operator randomly comparing images), and improving the accuracy of the operator's audit and monitoring.

[0086] The specific steps are as follows:

[0087] Step S10, when detecting the non-agriculturalization monitoring instruction for the non-agriculturalization area, determine the remote sensing image set of the non-agriculturalization area, and determine the reference image from the remote sensing image set;

[0088] As an example, the non-agriculturalization monitoring method can be applied to the non-agriculturalization monitoring device, and the non-agriculturalization monitoring device belongs to the non-agriculturalization monitoring system.

[0089] As an example, the non-agriculturalization area includes:

[0090] The first type: It is strictly prohibited to illegally occupy areas used for afforestation, lake excavation for landscaping, illegal construction of houses, and non-agricultural construction, etc.

[0091] The second type: It is strictly prohibited to illegally occupy areas used for growing plants other than rice, corn, and wheat, etc.

[0092] The triggering conditions for the non-agriculturalization monitoring instruction when detecting a non-agriculturalized area can be:

[0093] First, trigger based on pre-set triggering conditions, such as triggering the non-agriculturalization monitoring instruction at a determined time point;

[0094] Second, trigger manually on the monitoring interface.

[0095] When detecting the non-agriculturalization monitoring instruction for a non-agriculturalized area, determine the remote sensing image set of the non-agriculturalized area. In this embodiment, the remote sensing image set of the non-agriculturalized area includes the spring remote sensing image, summer remote sensing image, autumn remote sensing image, and winter remote sensing image of the non-agriculturalized area.

[0096] Among them, the spring remote sensing image, summer remote sensing image, autumn remote sensing image, and winter remote sensing image can each be multiple images.

[0097] After determining the remote sensing image set, manually determine a reference image from the remote sensing image set, or randomly determine a reference image from the remote sensing image set. This reference image can be a spring remote sensing image. Of course, it can also be other remote sensing images.

[0098] The steps of determining the remote sensing image set of the non-agriculturalized area and determining a reference image from the remote sensing image set when detecting the non-agriculturalization monitoring instruction for the non-agriculturalized area include:

[0099] Step S11, when detecting the non-agriculturalization monitoring instruction for a non-agriculturalized area, determine the multi-temporal remote sensing image set of the non-agriculturalized area, and determine a reference image from the remote sensing image set.

[0100] In this embodiment, the remote sensing image set is a multi-temporal remote sensing image set. Multi-temporal usually refers to the characteristics exhibited by a set of remote sensing images in a time series.

[0101] In this embodiment, by combining multi-temporal image cross-verification, the probability of false detection can be reduced, and the review workload can be reduced.

[0102] Specifically, for example, if there is illegal construction on summer cultivated land, but the illegal construction is demolished by itself in autumn. Since the different temporal phases of the images are considered, in this embodiment, such situations can be filtered to avoid false detection (if the temporal phase is not considered, based on the illegal construction on summer cultivated land, information of suspected illegal construction will still be output, and then, it is necessary for the operators to verify, which increases the workload of the operators and may cause incorrect monitoring).

[0103] Step S20: Extract the patch change information of each change patch between each other image and the reference image, and determine the target comparison image of the change patch based on the patch change information, where the other images are the images in the remote sensing image set except the reference image.

[0104] In this embodiment, extract the patch change information of each change patch between each other image and the reference image. For example, extract the patch change information of each change patch between image 1 (reference image) and image 3 (other image), extract the patch change information of each change patch between image 1 (reference image) and image 4 (other image), and extract the patch change information of each change patch between image 1 (reference image) and image 5 (other image).

[0105] In this embodiment, there may be multiple change patches between image 1 (reference image) and image 3 (other image), such as change patch A and change patch B.

[0106] In this embodiment, the patch change information includes the changed area, change type, change score, etc.

[0107] In the ordinary change patch review method, after determining change patch A, the monitoring interface displays image 2 adjacent to image 1 for the operator to compare change patch A in image 1 with adjacent image 2. Correspondingly, after determining change patch B, the monitoring interface displays image 2 adjacent to image 1 for the operator to compare change patch B in image 1 with adjacent image 2.

[0108] The change patch review method in this embodiment comprehensively considers multiple-phase images, adaptively selects the best comparison image, that is, the target comparison image, for display for the operator to compare. In this embodiment, the best comparison image, that is, the target comparison image, is adaptively selected for display, improving the comparison efficiency and comparison accuracy.

[0109] Specifically, for example, for change patch A in image 1, the best comparison image is image 3, and image 1 and image 3 will be displayed on the interface. However, for change patch B in image 1, the best comparison image is image 4, and image 1 and image 4 will be displayed on the interface.

[0110] Among them, the step of extracting the patch change information of each change patch between each other image and the reference image and determining the target comparison image of the change patch based on the patch change information includes:

[0111] Step S21: Determine the patch area, change confidence level, and classification confidence level of each change patch between the other image and the reference image.

[0112] Step S22: Determine the change degree score of the changed patch based on the area ratio of the patch area in all changed patches, its first influence weight, the change confidence and its second influence weight, and the classification confidence and its third influence weight.

[0113] Step S23: Determine the target comparison image of the changed patch based on the change degree score.

[0114] In this embodiment, the patch change information specifically includes the patch area, change confidence, and classification confidence of each changed patch between the other image and the reference image.

[0115] Specifically, the first influence weight of the patch area is W1, the second influence weight of the change confidence is W2, and the classification confidence and its third influence weight are W3.

[0116] In this embodiment, based on the area ratio of the patch area in all changed patches, its first influence weight, the change confidence and its second influence weight, and the classification confidence and its third influence weight, determine the change degree score of the changed patch.

[0117] For example, the change degree score score = W1 * ratio_areai + W2 * cd_thi + W3 * cl_thi);

[0118] Where ratio_areai is the area ratio of the changed patch (comparing the areas of all detected changed patches, value range 0 - 1), cd_th is the change confidence (value range 0 - 1), cl_th is the classification confidence (value range 0 - 1), and w1, w2, w3 are the corresponding weight coefficients or influence weights (value range 0 - 1).

[0119] The steps of determining the patch area, change confidence of each changed patch, and classification confidence of each changed patch between the other image and the reference image include:

[0120] Step D1: Input the other image and the reference image into a preset change detection model, and perform comparison processing on the other image and the reference image based on the preset change detection model to obtain the patch area and change confidence of each changed patch.

[0121] Step D2: Input each changed patch into a preset classification model, and perform classification processing on the changed patch based on the preset classification model to obtain the classification confidence.

[0122] In this embodiment, the other image and the reference image are input into a preset change detection model. Based on the preset change detection model, the other image and the reference image are compared to obtain the patch area and change confidence of each changed patch. After determining the patch area ratio and change confidence, each changed patch is input into a preset classification model. Based on the preset classification model, the changed patch is classified to obtain the classification confidence. The preset change detection model and the preset classification model are pre-trained models.

[0123] The preset change detection model and the preset classification model are processed in the same way as general deep learning algorithms, both divided into two steps: training and prediction. The training process mainly involves collecting data, performing annotation, then selecting the corresponding network for learning, and finally obtaining the weights of the network to generate the output of the network model, that is, the model can be used for prediction.

[0124] Step S30: Display the target comparison image of the changed patch on a preset monitoring interface for the operator to perform non-agriculturalization monitoring based on the target comparison image.

[0125] In this embodiment, when displaying the target comparison image of the changed patch on the preset monitoring interface, specifically, the target comparison image and the reference image can be displayed simultaneously for the operator to perform non-agriculturalization monitoring based on the target comparison image.

[0126] The present application provides a non-agriculturalization monitoring device, method, equipment, and storage medium. Compared with the prior art where only the suspected change information extracted from adjacent remote sensing images is provided to the staff for review, resulting in inaccurate non-agriculturalization monitoring, in the present application, when a non-agriculturalization monitoring instruction for a non-agriculturalized area is detected, a set of remote sensing images of the non-agriculturalized area is determined, and a reference image is determined from the set of remote sensing images; the patch change information of each change patch between each other image and the reference image is extracted, and a target comparison image of the change patch is determined based on the patch change information, where the other images are the images in the set of remote sensing images except the reference image; the target comparison image of the change patch is displayed on a preset monitoring interface for the operator to perform non-agriculturalization monitoring on the change patch based on the target comparison image. It can be understood that in the present application, when a non-agriculturalization monitoring instruction is detected, for each change patch (suspected) between the other images and the reference image in the set of remote sensing images, the corresponding patch change information is extracted, rather than only extracting from adjacent remote sensing images. Therefore, it avoids the inaccuracy of monitoring caused by the different timeliness of illegal occupation of corresponding cultivated land. In addition, in the present application, based on the corresponding patch change information, the best target comparison image is specifically determined and displayed to the operator for review. Since not only the patch change information is displayed to the operator, that is, in the present application, the best target comparison image of each change patch is conveniently displayed to the operator, thus enabling the operator to conduct targeted review (avoiding inaccurate review caused by the operator randomly comparing images), and improving the accuracy of the operator's review and monitoring.

[0127] Further, based on the first embodiment of the present application, another embodiment of the present application is provided. In this embodiment, the step of extracting the patch change information of each change patch between each other image and the reference image and determining the target comparison image of the change patch based on the patch change information includes:

[0128] Step S24, based on extraction models with different model structures, extract the patch change information of each change patch between each other image and the reference image to determine different change scores of the change patch;

[0129] Step S25, determine the average value of the different change scores as the target score of the change patch;

[0130] Step S26, determine the target comparison image of the change patch based on the target score.

[0131] Specifically, to avoid errors caused by accidental factors, in this embodiment, based on extraction models with different model structures, extract the patch change information of each change patch between each other image and the reference image to determine different change scores of the change patch.

[0132] Specifically, the other image and the reference image are respectively input into a plurality of preset change detection models, and the patch area of each change patch and the change confidence of each change patch are determined based on each preset change detection model; wherein, the model structures of each preset change detection model are different (the model structures of siamese networks and semantic segmentation); each change patch is respectively input into a plurality of preset classification models, and the classification confidence of the change patch is determined based on each preset classification model; wherein, the model structures of each preset classification model are different (the basic structures of models such as Resnet and VGG).

[0133] The step of determining the score of the change patch based on the area ratio of the patch area in all change patches and its first influence weight, the change confidence and its second influence weight, and the classification confidence and its third influence weight includes:

[0134] Based on the area ratio of the patch area in all change patches and its first influence weight, the change confidence and its second influence weight, and the classification confidence and its third influence weight corresponding to different models, different change scores of the change patch are determined; the mean value of the different change scores is used as the target score of the change patch; based on the target score, the target comparison image of the change patch is determined.

[0135] Specifically, in this embodiment, the target score is:

[0136]

[0137] Wherein, N is the number of monitoring times, ratio_areai is the area ratio of the patch area of the change patch (comparing the areas of all detected change patches, value range 0-1), cd_th is the change confidence (value range 0-1), cl_th is the classification confidence (value range 0-1), and w1, w2, and w3 are the corresponding weight coefficients or influence weights (value range 0-1).

[0138] In this embodiment, based on the extraction models with different model structures, the patch change information of each change patch between each other image and the reference image is extracted to determine different change scores of the change patch; the mean value of the different change scores is determined as the target score of the change patch; based on the target score, the target comparison image of the change patch is determined. In this embodiment, the error caused by accidental factors is avoided, and the monitoring accuracy is improved.

[0139] Further, based on the first embodiment and the second embodiment of the present application, another embodiment of the present application is provided. In this embodiment, the step of displaying the target comparison image of the change patch on the preset monitoring interface includes:

[0140] Step E1, determine the patch type of the changed patch;

[0141] Step E2, for the changed patches of the same type, based on the change degree score of the changed patches from high to low, display the shown changed patches and the target comparison images of the changed patches.

[0142] In this embodiment, first determine the patch type of the changed patch. Specifically, one example can be:

[0143] 1. When Image 1 is the reference image, record the changed patch information A, B, and C of other images (Image 2, Image 3, Image 4) respectively with the reference image 1.

[0144] 2. Compare the intersection over union (IOU) of the changed patch information A, B, and C. If the IOU is greater than a certain threshold, such as 0.5, then classify these three patches into the same category.

[0145] For the changed patches of the same type, based on the change degree score of the changed patches from high to low, display the shown changed patches and the target comparison images of the changed patches, that is, screen according to the patch score and display the shown changed patches and the target comparison images of the changed patches.

[0146] After the step of displaying the target comparison image of the changed patch on the preset monitoring interface, the method includes:

[0147] Step M1, if it is determined that the change degree score of the changed patch is less than the preset score value, prompt the operator to compare the comparison granularity of the changed patch for the operator to perform non-agricultural monitoring on the changed patch based on the target comparison image and the granularity.

[0148] In this embodiment, if it is determined that the change degree score of the changed patch is less than the preset score value, prompt the operator to compare the comparison granularity of the changed patch for the operator to perform non-agricultural monitoring on the changed patch based on the target comparison image and the granularity. That is, in this embodiment, the changed patches will be presented to the operator for review in order from high to low according to the scores of the changed patches. For those with high scores, the operator can quickly confirm (because there is no doubt); for those with low scores, the system will prompt the operator to view in detail.

[0149] Refer to Figure 3 , Figure 3 is the schematic diagram of the device structure of the hardware operating environment involved in the solution of the embodiment of the present application.

[0150] Such as Figure 3As shown in the figure, the non-agriculturalization monitoring device may include: a processor 1001, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to implement the connection and communication between the processor 1001 and the memory 1005.

[0151] Optionally, the non-agriculturalization monitoring device may further include a user interface, a network interface, a camera, an RF (Radio Frequency) circuit, sensors, a WiFi module, etc. The user interface may include a display screen and an input sub-module such as a keyboard. Optionally, the user interface may further include a standard wired interface and a wireless interface. The network interface may include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0152] Those skilled in the art can understand that Figure 3 the structure of the non-agriculturalization monitoring device shown in

[0153] does not constitute a limitation on the non-agriculturalization monitoring device, and may include more or fewer components than shown in the figure, or combine some components, or have different component arrangements. Figure 3 As shown in

[0154] the memory 1005, as a storage medium, may include an operating system, a network communication module, and a non-agriculturalization monitoring program. The operating system is a program for managing and controlling the hardware and software resources of the non-agriculturalization monitoring device, and supports the operation of the non-agriculturalization monitoring program and other software and / or programs. The network communication module is used to implement the communication between the components inside the memory 1005, and the communication between the memory 1005 and other hardware and software in the non-agriculturalization monitoring system. Figure 3 In the non-agriculturalization monitoring device shown in

[0155] the processor 1001 is used to execute the non-agriculturalization monitoring program stored in the memory 1005 to implement the steps of the non-agriculturalization monitoring method described in any one of the above.

[0156] This application also provides a non-agriculturalization monitoring device, and the non-agriculturalization monitoring device includes:

[0157] A first determination module, configured to determine a remote sensing image set of the non-agriculturalization area and determine a reference image from the remote sensing image set when detecting a non-agriculturalization monitoring instruction for the non-agriculturalization area;

[0158] A second determination module, configured to extract the patch change information of each change patch between each other image and the reference image, and determine a target comparison image of the change patch based on the patch change information, where the other image is an image in the remote sensing image set except the reference image;

[0159] A display module for displaying a target comparison image of the changed patch on a preset monitoring interface for an operator to perform non-agriculturalization monitoring on the changed patch based on the target comparison image.

[0160] In a possible implementation manner of the present application, the non-agriculturalization monitoring device is further configured to: when detecting a non-agriculturalization monitoring instruction for a non-agriculturalization area, determine a multi-temporal remote sensing image set of the non-agriculturalization area, and determine a reference image from the remote sensing image set.

[0161] In a possible implementation manner of the present application, the non-agriculturalization monitoring device is further configured to: determine the patch area, change confidence level, and classification confidence level of each changed patch between the other image and the reference image; determine a change degree score of the changed patch based on the area proportion of the patch area in all changed patches and its first influence weight, the change confidence level and its second influence weight, and the classification confidence level and its third influence weight; and determine a target comparison image of the changed patch based on the change degree score.

[0162] In a possible implementation manner of the present application, the non-agriculturalization monitoring device is further configured to: input the other image and the reference image into a preset change detection model, perform comparison processing on the other image and the reference image based on the preset change detection model to obtain the patch area and change confidence level of each changed patch; and input each changed patch into a preset classification model, perform classification processing on the changed patch based on the preset classification model to obtain a classification confidence level.

[0163] In a possible implementation manner of the present application, the non-agriculturalization monitoring device is further configured to: determine the patch type of the changed patch; for the changed patches of the same type, display the changed patch and the target comparison image of the changed patch based on the high or low change degree score of the changed patch.

[0164] In a possible implementation manner of the present application, the non-agriculturalization monitoring device is further configured to: if it is determined that the change degree score of the changed patch is less than a preset score value, prompt the operator to compare the comparison granularity of the changed patch for the operator to perform non-agriculturalization monitoring on the changed patch based on the target comparison image and the granularity.

[0165] In a possible implementation manner of the present application, the non-agriculturalization monitoring device is further configured to: based on extraction models with different model structures, extract the patch change information of each change patch between each other image and the reference image to determine different change scores for the change patches; determine the average value of the different change scores as the target score of the change patch; and based on the target score, determine the target comparison image of the change patch.

[0166] The specific implementation manner of the non-agriculturalization monitoring device of the present application is basically the same as that of each embodiment of the above non-agriculturalization monitoring method, and will not be elaborated herein.

[0167] The embodiment of the present application provides a storage medium, and the storage medium stores one or more programs, and the one or more programs can also be executed by one or more processors to be used to implement the steps of the non-agriculturalization monitoring method described in any one of the above.

[0168] The specific implementation manner of the storage medium of the present application is basically the same as that of each embodiment of the above non-agriculturalization monitoring method, and will not be elaborated herein.

[0169] The embodiment of the present application provides a non-agriculturalization monitoring device, and the device includes:

[0170] A first determination module, configured to determine a remote sensing image set of the non-agriculturalization area and determine a reference image from the remote sensing image set when detecting a non-agriculturalization monitoring instruction for the non-agriculturalization area;

[0171] A second determination module, configured to extract the patch change information of each change patch between each other image and the reference image, and determine the target comparison image of the change patch based on the patch change information, where the other image is an image in the remote sensing image set except the reference image;

[0172] A display module, configured to display the target comparison image of the change patch on a preset monitoring interface for an operator to perform non-agriculturalization monitoring on the change patch based on the target comparison image.

[0173] In a possible implementation manner of the present application, the non-agriculturalization monitoring device is further configured to: when detecting a non-agriculturalization monitoring instruction for the non-agriculturalization area, determine a multi-temporal remote sensing image set of the non-agriculturalization area and determine a reference image from the remote sensing image set.

[0174] In a possible implementation manner of the present application, the non-agriculturalization monitoring device is further configured to: determine the patch area, change confidence level, and classification confidence level of each changed patch between the other image and the reference image; determine the change degree score of the changed patch based on the area proportion of the patch area among all the changed patches and its first influence weight, the change confidence level and its second influence weight, and the classification confidence level and its third influence weight; and determine the target comparison image of the changed patch based on the change degree score.

[0175] In a possible implementation manner of the present application, the non-agriculturalization monitoring device is further configured to: input the other image and the reference image into a preset change detection model, perform comparison processing on the other image and the reference image based on the preset change detection model to obtain the patch area and change confidence level of each changed patch; and input each changed patch into a preset classification model, and perform classification processing on the changed patch based on the preset classification model to obtain the classification confidence level.

[0176] In a possible implementation manner of the present application, the non-agriculturalization monitoring device is further configured to: determine the patch type of the changed patch; for the changed patches of the same type, display the changed patch and the target comparison image of the changed patch based on the high and low change degree scores of the changed patch.

[0177] In a possible implementation manner of the present application, the non-agriculturalization monitoring device is further configured to: if it is determined that the change degree score of the changed patch is less than a preset score value, prompt the operator to compare the comparison granularity of the changed patch, so that the operator can perform non-agriculturalization monitoring on the changed patch based on the target comparison image and the granularity.

[0178] In a possible implementation manner of the present application, the non-agriculturalization monitoring device is further configured to: extract the patch change information of each changed patch between each other image and the reference image based on an extraction model with different model structures to determine different change scores of the changed patch; determine the average value of the different change scores as the target score of the changed patch; and determine the target comparison image of the changed patch based on the target score.

[0179] The specific implementation manners of the device of the present application are basically the same as those of the above embodiments of the non-agriculturalization monitoring method, and will not be described in detail here.

[0180] It should be noted that in this text, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising that element.

[0181] The serial numbers of the embodiments of the present invention above are for description only and do not represent the superiority or inferiority of the embodiments.

[0182] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment methods can be implemented by means of software plus a hardware platform, or by hardware. However, in many cases, the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0183] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structural or equivalent process transformation made by using the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall equally be included in the patent protection scope of the present invention.

Claims

1. A non-agriculturalization monitoring device, characterized in that, The device includes: A first determination module, configured to determine a multi-temporal remote sensing image set of the non-agriculturalization area and determine a reference image from the remote sensing image set when detecting a non-agriculturalization monitoring instruction for the non-agriculturalization area; A second determination module, configured to extract patch change information of each change patch between each other image and the reference image, and determine a target comparison image of the change patch based on the patch change information, where the other image is an image in the remote sensing image set except the reference image; A display module, configured to display the target comparison image of the change patch on a preset monitoring interface for an operator to perform non-agriculturalization monitoring on the change patch based on the target comparison image.

2. The non-agriculturalization monitoring device according to claim 1, characterized in that The non-agriculturalization monitoring device is further configured to: determine the patch area, change confidence, and classification confidence of each change patch between the other image and the reference image; determine a change degree score of the change patch based on the area proportion of the patch area among all change patches and its first influence weight, the change confidence and its second influence weight, and the classification confidence and its third influence weight; and determine the target comparison image of the change patch based on the change degree score; And / or, the non-agriculturalization monitoring device is further configured to: input the other image and the reference image into a preset change detection model, perform comparison processing on the other image and the reference image based on the preset change detection model to obtain the patch area and change confidence of each change patch; input each change patch into a preset classification model, and perform classification processing on the change patch based on the preset classification model to obtain a classification confidence; And / or, the non-agriculturalization monitoring device is further configured to: determine the patch type of the change patch; For change patches of the same type, display the change patches and the target comparison images of the change patches based on the high or low change degree scores of the change patches; And / or, the non-agriculturalization monitoring device is further configured to: if it is determined that the change degree score of the change patch is less than a preset score value, prompt the operator to compare the comparison granularity of the change patch for the operator to perform non-agriculturalization monitoring on the change patch based on the target comparison image and the granularity; And / or, the non-agriculturalization monitoring device is further configured to: extract patch change information of each change patch between each other image and the reference image based on an extraction model with different model structures to determine different change scores of the change patch; determine the average value of the different change scores as the target score of the change patch; and determine the target comparison image of the change patch based on the target score.

3. A method for non-agriculturalization monitoring, characterized in that, The method includes: When detecting a non-agriculturalization monitoring instruction for a non-agriculturalization area, determining a multi-temporal remote sensing image set of the non-agriculturalization area and determining a reference image from the remote sensing image set; Extract the patch change information of each change patch between each other image and the reference image, and determine the target comparison image of the change patch based on the patch change information, where the other images are the images in the remote sensing image set except the reference image; Display the target comparison image of the change patch on a preset monitoring interface for the operator to perform non-agricultural monitoring on the change patch based on the target comparison image.

4. The non-agriculturalization monitoring method according to claim 3, characterized in that The step of extracting the patch change information of each change patch between each other image and the reference image and determining the target comparison image of the change patch based on the patch change information includes: Determine the patch area, change confidence, and classification confidence of each change patch between the other image and the reference image; Based on the area ratio of the patch area in all change patches and its first influence weight, the change confidence and its second influence weight, and the classification confidence and its third influence weight, determine the change degree score of the change patch; Based on the change degree score, determine the target comparison image of the change patch.

5. The non-agriculturalization monitoring method according to claim 4, characterized in that The step of determining the patch area of each change patch, the change confidence of each change patch, and the classification confidence of each change patch between the other image and the reference image includes: Input the other image and the reference image into a preset change detection model, and perform comparison processing on the other image and the reference image based on the preset change detection model to obtain the patch area and change confidence of each change patch; Input each change patch into a preset classification model, and perform classification processing on the change patch based on the preset classification model to obtain the classification confidence.

6. The non-agriculturalization monitoring method according to claim 4, characterized in that, The step of displaying the target comparison image of the change patch on a preset monitoring interface includes: Determine the patch type of the change patch; For change patches of the same type, display the change patch and the target comparison image of the change patch based on the high and low change degree scores of the change patch.

7. The non-agriculturalization monitoring method according to claim 4, wherein After the step of displaying the target comparison image of the change patch on a preset monitoring interface, the method includes: If it is determined that the change degree score of the change patch is less than the preset score value, prompt the operator to compare the comparison granularity of the change patch for the operator to perform non-agricultural monitoring on the change patch based on the target comparison image and the granularity.

8. The non-agriculturalization monitoring method according to claim 3, characterized in that, The step of extracting the patch change information of each change patch between each other image and the reference image and determining the target comparison image of the change patch based on the patch change information includes: Based on an extraction model with different model structures, extract the patch change information of each change patch between each other image and the reference image to determine different change scores of the change patch; Determine the average value of the different change scores as the target score of the change patch; Based on the target score, determine the target comparison image of the change patch.

9. A non-agriculturalization monitoring device, characterized in that, It includes a memory, a processor, and a non-agriculturalization monitoring program stored on the memory and executable on the processor. When the processor executes the non-agriculturalization monitoring program, it implements the steps of the non-agriculturalization monitoring method according to any one of claims 3 to 8.

10. A storage medium, characterized in that, A non-agriculturalization monitoring program is stored on the storage medium. When the non-agriculturalization monitoring program is executed by a processor, it implements the steps of the non-agriculturalization monitoring method according to any one of claims 3 to 8.

Images