Automatic law enforcement monitoring method and device based on remote sensing image and storage medium

Abstract

The invention relates to the technical field of territorial resource supervision, and provides an automatic law enforcement monitoring method and device based on a remote sensing image, and a storage medium. The automatic law enforcement monitoring method comprises the following steps: extracting at least one pattern spot area from a remote sensing image of a monitoring area; predicting based on pattern spot data corresponding to the pattern spot region to obtain a predicted use type; under the condition that the predicted use type is not matched with the planned use type and the predicted use type needs to be verified, obtaining a verification image corresponding to the pattern spot region; and under the condition that the pattern spot data is matched with the verification image, determining that land use abnormity exists in the pattern spot area. In the technical scheme, the pattern spot data can be further verified through the verification image under the condition that the predicted use type determined based on the pattern spot data is not matched with the planned use type, so that the influence of remote sensing image acquisition and identification errors on an abnormal judgment result can be reduced, and the accuracy of the abnormal judgment result is improved. And thus, the accuracy and reliability of automatic law enforcement monitoring can be improved.

Description

Technical Field

[0001] The present application relates to the field of land and resources supervision technology, and in particular to automatic law enforcement monitoring methods, equipment and storage media based on remote sensing images. Background Art

[0002] The development of satellite remote sensing technology has greatly promoted the effective monitoring and management of the earth's surface information, and has played an important role in land use, environmental monitoring, disaster warning, etc. With the acceleration of urbanization, higher requirements have been put forward for the standardization and rationality of land use, and effective regulatory measures are urgently needed to ensure legal and compliant land use behavior.

[0003] The common practice at present is to supervise and inspect land use through manual on-site surveys or regular collection of satellite remote sensing images. Specifically, technicians usually obtain preliminary information through remote sensing image analysis, and then conduct ground verification or other forms of review based on this information.

[0004] However, since the acquisition of remote sensing images will be affected by factors such as the environment, acquisition height, and acquisition angle, misjudgment may occur when using remote sensing images to analyze land use types, which in turn limits the accuracy and reliability of law enforcement monitoring and makes it difficult to meet the needs of efficient and accurate law enforcement. Summary of the invention

[0005] In order to help improve the accuracy and reliability of automatic law enforcement monitoring, the present application provides an automatic law enforcement monitoring method, device and storage medium based on remote sensing images.

[0006] In the first aspect, the present application provides an automatic law enforcement monitoring method based on remote sensing images, which adopts the following technical solutions: An automatic law enforcement monitoring method based on remote sensing images, the method comprising: Performing spot extraction on the remote sensing image of the monitoring area to obtain at least one spot area; For each of the image spot areas, inputting the image spot data corresponding to the image spot area in the remote sensing image into a pre-trained usage prediction model to obtain a predicted usage type corresponding to the image spot area; Determining whether the predicted usage type matches the planned usage type corresponding to the map area; In the case where the predicted usage type does not match the planned usage type, determining whether to verify the predicted usage type; In the case of determining to verify the predicted usage type, obtaining a verification image corresponding to the image spot area, wherein the verification image is acquired in a different manner from the remote sensing image; Determining whether the pattern data matches the verification image; When it is determined that the image spot data matches the verification image, it is determined that there is a land use anomaly in the image spot area.

[0007] By adopting the above technical scheme, when the predicted use type determined based on the patch data corresponding to the patch area in the remote sensing image does not match the planned use type, it can be further determined whether to verify the predicted use type, and when it is determined that the predicted use type is to be verified, the patch data is verified through the verification image. This can help reduce the impact of errors in remote sensing image acquisition and recognition on abnormal judgment results, and thus help provide accuracy and reliability of automatic law enforcement monitoring.

[0008] Optionally, the determining whether to verify the predicted usage type includes: Determining whether the prediction confidence corresponding to the predicted usage type is greater than or equal to a preset confidence threshold; When the prediction confidence is less than the confidence threshold, determine to verify the predicted usage type.

[0009] By adopting the above technical solution, it is possible to determine whether to verify the predicted usage type in combination with the confidence level of the predicted usage type, which can help improve the accuracy of the verification judgment, and further improve the monitoring efficiency while ensuring the accuracy of the monitoring results.

[0010] Optionally, after determining whether the confidence level corresponding to the predicted usage type is greater than or equal to a preset confidence level threshold, the method further includes: In the case where the prediction confidence is greater than the confidence threshold, determining whether there is a historical confidence corresponding to the image spot area; In the case where the historical confidence exists, determining whether a gap value between the prediction confidence and the historical confidence is less than or equal to a preset gap value threshold; When the gap value is greater than the gap value threshold, it is determined to verify the predicted usage type.

[0011] By adopting the above technical solution, when the prediction confidence is greater than the confidence threshold, it can be further determined whether to verify the predicted usage type based on the change in confidence, which can help improve the accuracy of the verification judgment result.

[0012] Optionally, before determining whether the prediction confidence corresponding to the predicted usage type is greater than or equal to a preset confidence threshold, the method further includes: Determining whether the predicted usage type is a preset high-risk type; In the case where the predicted usage type is not the high-risk type, performing the step of determining whether the prediction confidence is greater than or equal to a preset confidence threshold; In the case where the predicted usage type is the high-risk type, it is directly determined to verify the predicted usage type without performing the step of determining whether the prediction confidence is greater than or equal to a preset confidence threshold.

[0013] By adopting the above technical solution, before judging whether the predicted usage type needs to be verified based on the prediction confidence, it can be determined whether the predicted usage type is a high-risk type. If the predicted usage type is a high-risk type, it can be directly determined that the predicted usage type needs to be verified without executing the confidence judgment step. This can help strengthen the verification of high-risk types, and further help achieve a balance between accuracy and efficiency of automatic law enforcement monitoring.

[0014] Optionally, after determining whether the predicted usage type matches the planned usage type corresponding to the spot area, the method further includes: In the case where the predicted usage type matches the planned usage type, determining whether there is a historical confidence corresponding to the map area; In the case where the historical confidence exists, determining whether the difference between the prediction confidence and the historical confidence is less than or equal to a preset difference threshold; When the difference is greater than the difference threshold, an abnormal risk corresponding to the image spot area is recorded once; Determine whether the number of abnormal risks corresponding to the image spot area is greater than a preset risk number threshold; When the number of abnormal risks is greater than the risk number threshold, it is determined that there is a land use risk in the map area.

[0015] By adopting the above technical solution, when the predicted use type matches the planned use type, it is possible to further combine the difference between the predicted confidence and the historical confidence to determine whether there is an abnormal risk in the map area. This can help determine whether there is land use abnormality in the map area based on abnormal changes in confidence, which in turn can help improve the accuracy of automatic law enforcement monitoring.

[0016] Optionally, obtaining a verification image corresponding to the image spot area includes: Determine at least one target location from the image spot data based on a feature extraction method corresponding to the predicted usage type; Determine the spatial coordinates corresponding to the target position based on the coordinate mapping relationship between the pattern data and the pattern area; An image corresponding to the target position is acquired based on the spatial coordinates to obtain the verification image.

[0017] By adopting the above technical solution, the target location can be determined based on the feature extraction method corresponding to the predicted usage type, and the collection of verification images can be guided based on the spatial coordinates of the target location. This can help to achieve accurate collection of verification images, thereby helping to reduce the amount of data that needs to be collected and verified, and further help reduce the difficulty of automatic law enforcement monitoring.

[0018] Optionally, the target positions include more than two, and acquiring images corresponding to the target positions based on the spatial coordinates includes: determining image acquisition requirements based on the predicted usage type; Planning an image acquisition path based on the distribution of the spatial coordinates corresponding to each of the target positions and image acquisition requirements to obtain an image acquisition path; The image acquisition device is controlled to move along the image acquisition path to acquire images corresponding to each of the target positions.

[0019] By adopting the above-mentioned technical scheme, the image acquisition path can be planned based on the distribution of each target location and the image acquisition requirements corresponding to the predicted usage type, and the image acquisition device can be controlled to move along the image acquisition path to acquire the image of the target location. This can help improve the efficiency of image acquisition and also help ensure the quality of the acquired images.

[0020] Optionally, after determining whether the pattern data matches the verification image, the method further includes: In the case where the pattern data does not match the verification image, the predicted usage type is corrected based on the verification image to obtain a corrected usage type; Determining whether the revised use type matches the planned use type; In the case where the revised use type does not match the planned use type, determining that there is a land use anomaly in the map area; In the case where the corrected use type matches the planned use type, it is determined that there is no land use anomaly in the map area.

[0021] By adopting the above technical solution, when the map data does not match the verification image, the predicted use type can be corrected, and based on the relationship between the corrected use type and the planned use type, it can be determined whether there is a land use anomaly, which can help to further improve the accuracy of the land use anomaly identification results.

[0022] In a second aspect, the present application provides an electronic device, which adopts the following technical solution: An electronic device, comprising: at least one processor; Memory; At least one application, wherein at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute any one of the automatic law enforcement monitoring methods for remote sensing images provided in the first aspect.

[0023] In a third aspect, the present application provides a computer-readable storage medium, which adopts the following technical solution: A computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed in a computer, the computer is caused to execute any one of the automatic law enforcement monitoring methods based on remote sensing images provided in the first aspect.

[0024] In summary, the present application includes at least one of the following beneficial technical effects: 1. When the predicted use type determined based on the spot data corresponding to the spot area in the remote sensing image does not match the planned use type, the spot data can be further verified by verifying the image. This can help reduce the impact of errors in remote sensing image acquisition and recognition on abnormal judgment results, and thus help provide accuracy and reliability of automatic law enforcement monitoring.

[0025] 2. Whether to verify the predicted usage type can be determined in combination with the confidence level of the predicted usage type, which can help improve the accuracy of the verification judgment, and further improve the monitoring efficiency while ensuring the accuracy of the monitoring results. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 This is a flow chart of an automatic law enforcement monitoring method based on remote sensing images provided in an embodiment of the present application; Figure 2 It is a flowchart of a verification judgment method provided in an embodiment of the present application; Figure 3 It is a flowchart of another automatic law enforcement monitoring method based on remote sensing images provided in an embodiment of the present application; Figure 4 It is a flow chart of a verification image extraction method provided in an embodiment of the present application; Figure 5 It is a flowchart of an image acquisition path planning method provided in an embodiment of the present application; Figure 6 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solutions and advantages of this application more clear, the following Figure 1-6 It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0028] The present application embodiment discloses an automatic law enforcement monitoring method based on remote sensing images. Figure 1 , the automatic law enforcement monitoring method comprises the following steps: Step 101, extracting spots from a remote sensing image of a monitoring area to obtain at least one spot area.

[0029] The patch area is used to distinguish the land units or plots in the area to be detected. In actual implementation, the patch can be a single land type plot divided by various boundaries (administrative interfaces, land ownership boundaries) or linear features.

[0030] In one example, the remote sensing image of the monitoring area is segmented into a foreground and a background by threshold segmentation, and the independent areas in the foreground can be respectively determined as the spot areas.

[0031] In one example, the “Focal” tool in the “Object-Based Image Analysis” toolbox of ArcGIS is used to select appropriate segmentation parameters, such as segmentation scale, compactness, spectral index, etc., to segment the remote sensing image and obtain the patch area.

[0032] In another example, a number of seed points are determined in the remote sensing image of the monitoring area, and adjacent pixels are merged based on a similarity criterion to form a patch area. Specifically, the similarity criterion can be set based on factors such as color and texture of the image.

[0033] In actual implementation, the spot area can also be extracted from the remote sensing image through edge detection, machine learning and other methods. This embodiment does not limit the method of extracting the spot area.

[0034] Step 102: for each spot area, the spot data corresponding to the spot area in the remote sensing image is input into a pre-trained usage prediction model to obtain a predicted usage type corresponding to the spot area.

[0035] The use recognition model is pre-trained using training data, and the training data includes at least one set of sample data, each set of sample data includes spot data and use type labels corresponding to the spot data. Specifically, the use type can be set according to actual monitoring needs. In one example, the use type includes agricultural land, construction land, and unused land.

[0036] Spot data refers to data containing image data of a spot area. Specifically, since different spot areas have different sizes, and the usage prediction model generally has requirements on the size of input data, in the actual implementation process, the spot data may contain all image data in the spot area, or may only contain part of the image data corresponding to the spot area.

[0037] In one example, the spot data is an image of a preset shape and size obtained by cropping the remote sensing image with the geometric center of the spot area as the center.

[0038] Step 103, determining whether the predicted usage type matches the planned usage type corresponding to the patch area.

[0039] The planning use type corresponding to the map area is determined based on the planning, approval data and other materials corresponding to the monitoring area. Specifically, the planning use type corresponding to the map area is determined based on the planning, approval data and other materials corresponding to the location range corresponding to the map area.

[0040] The relationship between the predicted use type and the planned use type is preset. In actual implementation, the planned use type may be divided more thoroughly than the predicted use type. For example, the planned use type further describes agricultural land as cultivated land, forest land, grassland, farmland water utilization land, aquaculture water surface and other types, while the predicted use type is uniformly described as agricultural land. Therefore, the above further descriptions of agricultural land can be determined to match agricultural land, which can help resolve the differences caused by the different classification accuracy between the predicted use type and the planned use type.

[0041] In some embodiments, the classification method of the predicted usage type is the same as the classification method of the planned usage type. In this case, the predicted usage type and the planned usage type are matched only when they are the same. This can help improve the accuracy of the matching and improve the accuracy of the monitoring results.

[0042] Step 104: When the predicted usage type does not match the planned usage type, determine whether to verify the predicted usage type.

[0043] In one example, whether the predicted use type needs to be verified can be determined based on whether the predicted use type is a high-risk type. In one example, when the predicted use type is a high-risk type, it is determined that the predicted use type needs to be verified; when the predicted use type is not a high-risk type, it is determined that the predicted use type does not need to be verified, which can help improve the efficiency of automatic law enforcement monitoring. For example: the predicted use types include construction land, agricultural land, and unused land. At this time, the construction land is a high-risk type. Accordingly, when the predicted use type is construction land, it is determined that the predicted use type needs to be verified; when the predicted use type is agricultural land or unused land, it is determined that the predicted use type does not need to be verified.

[0044] In another example, the quality of the remote sensing image can be judged to determine whether to verify the predicted usage type. In one example, the quality of the remote sensing image is judged based on the environmental information, acquisition height, acquisition angle and other information when the remote sensing image is collected; if the remote sensing image quality is low, it may lead to inaccurate prediction results. At this time, the predicted usage information can be verified based on the determination need.

[0045] In actual implementation, it may also be determined whether the predicted usage type needs to be verified alone or in combination with other methods.

[0046] Step 105: When it is determined to verify the predicted usage type, a verification image corresponding to the image spot area is obtained.

[0047] The acquisition methods of the verification image and the remote sensing image are different. In actual implementation, the verification image can be acquired by mobile devices such as drones and vehicles, or by image acquisition devices (such as cameras) fixed in the monitoring area, without limiting the acquisition method of the verification image.

[0048] In actual implementation, the acquisition position of the verification image is close to the ground of the monitoring area, which can reduce the impact of environmental factors on the acquisition process, and thus help improve the accuracy of the acquired image.

[0049] It should be noted that compared with remote sensing images, the cost of collecting and analyzing verification images is very huge. Therefore, in this implementation, the verification image of the corresponding location will only be obtained when it is determined that the predicted usage type needs to be verified. This can help to reasonably control the monitoring cost while ensuring the prediction accuracy.

[0050] Step 106, determining whether the pattern data matches the verification image.

[0051] In one example, determining whether the spot data matches the verification image includes: aligning the coordinates of the spot data and the verification image; determining at least one target position from the spot data based on a feature extraction method corresponding to the predicted usage type; determining whether each target position in the verification image has a feature object that matches the predicted usage type; if so, determining that the spot data matches the verification data; if not, determining that the spot data does not match the verification data.

[0052] The target position is the position of the feature object. Specifically, since the feature objects corresponding to different usage types may be different, for example, the feature objects of construction land may be building corners, entrances, etc., and the feature objects of farmland may be crops, irrigation systems, etc., the feature extraction methods corresponding to different usage types may be different and need to be set separately.

[0053] In some embodiments, the position of the feature object in the pattern area is marked by the usage recognition model during the process of identifying the pattern data. In this way, during the matching process, only the position of the feature object that matches the predicted usage type among the marked feature objects needs to be determined as the target position.

[0054] Furthermore, since the data collection and processing process may cause image distortion, which may in turn cause the coordinates of the target position to shift, in the above example, when it is determined that there is no feature object matching the predicted usage type at a certain target position, it can be further determined whether there is a feature object matching the predicted usage type within a preset range centered on the target position; if so, it is determined that the pattern data matches the verification image.

[0055] In another example, determining whether the spot data matches the verification image includes: extracting verification data corresponding to the spot area from the verification image; performing a similarity comparison between the spot data and the verification data to obtain the similarity between the spot data and the verification data; when the similarity is greater than a preset similarity threshold, determining that the spot data matches the verification image; when the similarity is less than a preset similarity threshold, determining that the spot data does not match the verification data.

[0056] In actual implementation, the purpose identification can also be directly performed based on the verification image to obtain the verification usage type, and when the predicted usage type is the same as the verification usage type, it is determined that the pattern data matches the verification image.

[0057] Step 107: When it is determined that the image spot data matches the verification image, it is determined that there is a land use anomaly in the image spot area.

[0058] Optionally, after determining the land use anomaly, a land use anomaly evaluation report can be automatically generated based on the spot data and the verification image for use in supervision and law enforcement. Furthermore, the land use anomaly evaluation report can also record the planning information and / or corresponding planning documents corresponding to the spot area, which can help improve the compliance of the automatic supervision process.

[0059] Optionally, after determining whether the spot data matches the verification image, the method further includes: in the case where the spot data does not match the verification image, correcting the predicted use type based on the verification image to obtain a corrected use type; determining whether the corrected use type matches the planned use type; in the case where the corrected use type does not match the planned use type, determining that there is a land use anomaly in the spot area; in the case where the corrected use type matches the planned use type, determining that there is no land use anomaly in the spot area. In the above technical solution, in the case where the spot data does not match the verification image, the predicted use type can be corrected, and based on the relationship between the corrected use type and the planned use type, it can be determined whether there is a land use anomaly, which can help to further improve the accuracy of the land use anomaly identification results.

[0060] In one example, the predicted usage type is corrected based on the verification image to obtain a corrected usage type, including: correcting the data of the target position in the pattern data that does not match the verification image (i.e., the target position where there is no characteristic object matching the predicted usage type in the verification image) (the data of the target position in the verification image may be modified with reference to the data, or the data may be modified to a fixed value) to obtain the corrected data; and re-inputting the corrected data into the usage prediction model to obtain the corrected usage type.

[0061] In another example, the predicted use type is corrected based on the verification image to obtain the corrected use type, including: identifying the purpose based on the verification image to obtain the verification use method; when the predicted use type is different from the verification use type, determining the verification use method as the corrected use type; when the predicted use type is the same as the verification use type, directly determining that there is land use anomaly in the map area.

[0062] The implementation principle of an automatic law enforcement monitoring method based on remote sensing images in an embodiment of the present application is as follows: perform spot extraction on the remote sensing image of the monitoring area to obtain at least one spot area; for each spot area, input the spot data corresponding to the spot area in the remote sensing image into a pre-trained use prediction model to obtain a predicted use type corresponding to the spot area; determine whether the predicted use type matches the planned use type corresponding to the spot area; if the predicted use type does not match the planned use type, determine whether to verify the predicted use type; if it is determined that the predicted use type is to be verified, obtain a verification image corresponding to the spot area, the verification image and the remote sensing image are collected in a different manner; determine whether the spot data matches the verification image; if it is determined that the spot data matches the verification image, determine that there is a land use anomaly in the spot area. In the above technical scheme, when the predicted use type determined based on the spot data corresponding to the spot area in the remote sensing image does not match the planned use type, it can be further determined whether to verify the predicted use type, and when it is determined that the predicted use type is to be verified, the spot data is verified through the verification image, and when the spot data matches the verification image, it is determined that there is land use anomaly in the spot area. This can help reduce the impact of errors in remote sensing image acquisition and recognition on the anomaly judgment results, thereby helping to improve the accuracy of anomaly judgment, and further help to provide accuracy and reliability of automatic law enforcement monitoring.

[0063] In some embodiments, reference Figure 2 In step 104, determining whether to verify the predicted usage type includes the following steps: Step 201 : Determine whether the prediction confidence corresponding to the predicted usage type is greater than or equal to a preset confidence threshold.

[0064] Among them, the confidence of the predicted usage type is output by the usage prediction model. Specifically, the usage prediction model will determine the confidence of the image spot data for each usage type during the prediction of the image spot data, and determine the usage type with the highest confidence as the predicted usage type. In actual implementation, the softmax activation function can be used in the output layer of the usage recognition model to convert the original output of the model into a probability (i.e., confidence) distribution, so that the probability of each category is between 0 and 1, and the sum of the probabilities of each category is 1.

[0065] In one example, the confidence threshold is a value greater than 0.6, for example, the confidence threshold is 0.65.

[0066] Furthermore, before determining whether the prediction confidence corresponding to the predicted use type is greater than or equal to the preset confidence threshold, it also includes: determining whether the predicted use type is a preset high-risk type; if the predicted use type is not a high-risk type, executing the step of determining whether the prediction confidence is greater than or equal to the preset confidence threshold, that is, executing step 201; if the predicted use type is a high-risk type, directly determining to verify the predicted use type, and not executing the step of determining whether the prediction confidence is greater than or equal to the preset confidence threshold, that is, not executing step 201. In the above technical solution, before judging whether it is necessary to verify the predicted use type based on the prediction confidence, it is first determined whether the predicted use type is a high-risk type, and if the predicted use type is a high-risk type, it is directly determined that it is necessary to verify the predicted use type without executing the step of confidence judgment, which can help to strengthen the verification of high-risk types, and thus can help to achieve a balance between the accuracy and efficiency of automatic law enforcement monitoring.

[0067] In actual implementation, the confidence threshold can be set based on the risk level of the predicted usage type. The risk level is negatively correlated with the confidence threshold, that is, the greater the risk level, the lower the condition for judging whether the predicted usage type needs to be verified. In this way, the verification judgment standard can be dynamically adjusted according to the risk level of the predicted usage type, which can help achieve a balance between accuracy and efficiency of automatic law enforcement monitoring.

[0068] Step 202: When the prediction confidence is less than the confidence threshold, determine to verify the predicted usage type.

[0069] Optionally, when the predicted execution degree is greater than the confidence threshold, it may be directly determined that the predicted usage type does not need to be verified, or it may be determined based on other methods whether the predicted usage type needs to be verified.

[0070] In the above implementation, since it is possible to determine whether the predicted usage type needs to be verified based on the prediction confidence, it is possible to determine whether to verify the predicted usage type based on the confidence of the predicted usage type, which can help improve the accuracy of the verification judgment, and further improve the monitoring efficiency while ensuring the accuracy of the monitoring results.

[0071] For further reference, Figure 2 , step 201, after determining whether the prediction confidence corresponding to the predicted usage type is greater than or equal to a preset confidence threshold, the following steps are also included: Step 203: When the prediction confidence is greater than the confidence threshold, determine whether there is a historical confidence corresponding to the image spot area.

[0072] The historical confidence refers to the prediction confidence of the predicted usage type corresponding to the target image area determined historically.

[0073] In one example, determining whether there is a historical confidence corresponding to the map area includes: determining whether there is a historical prediction type; if there is a historical prediction type, determining whether the historical prediction type is the same as the prediction usage type; if the historical prediction type is the same as the prediction usage type, determining the prediction confidence corresponding to the historical prediction type as the historical confidence; if the historical prediction type is different from the prediction usage type, directly determining that the prediction usage type needs to be verified, and executing step 105.

[0074] Step 204: if there is a historical confidence, determine whether the difference between the predicted confidence and the historical confidence is less than or equal to a preset difference threshold.

[0075] The gap value is greater than or equal to zero. In actual implementation, the absolute value of the difference between the prediction confidence and the historical confidence can be determined as the gap value.

[0076] Step 205: When the difference value is greater than the difference value threshold, determine to verify the predicted usage type.

[0077] Optionally, when the gap value is smaller than the gap value threshold, it may be directly determined not to verify the predicted usage type, or it may be determined whether to verify the usage type based on other methods.

[0078] In the above technical solution, since when the prediction confidence is greater than the confidence threshold, it is further determined whether there is a historical confidence, and when there is a historical confidence, it is further determined whether to verify the preset usage type based on the difference between the prediction confidence and the historical confidence. In this way, it can be determined whether to verify the predicted usage type based on the change in confidence, which can help improve the accuracy of the verification judgment results.

[0079] In some embodiments, reference Figure 3 , step 103, after determining whether the predicted usage type matches the planned usage type corresponding to the spot area, further comprising: Step 301, when the predicted usage type matches the planned usage type, determine whether there is a historical confidence level corresponding to the patch area.

[0080] The historical confidence refers to the prediction confidence of the predicted usage type corresponding to the target image area determined historically.

[0081] In one example, determining whether there is a historical confidence corresponding to the map area includes: determining whether there is a historical prediction type; if there is a historical prediction type, determining whether the historical prediction type is the same as the prediction usage type; if the historical prediction type is the same as the prediction usage type, determining the prediction confidence corresponding to the historical prediction type as the historical confidence; if the historical prediction type is different from the prediction usage type, determining that there is no historical confidence.

[0082] Step 302: if there is a historical confidence, determine whether the difference between the predicted confidence and the historical confidence is less than or equal to a preset difference threshold.

[0083] Specifically, the difference between the predicted confidence and the historical confidence can be used to indicate changes in the map area. Generally speaking, if the state of the map area has not changed, the confidence will not drop significantly. Therefore, it is possible to determine whether there is an abnormal risk based on the difference between the predicted confidence and the historical confidence.

[0084] Step 303: When the difference is greater than the difference threshold, the abnormal risk corresponding to the image spot area is recorded.

[0085] Step 304, determining whether the number of abnormal risks corresponding to the image spot area is greater than a preset risk number threshold.

[0086] Optionally, when the planned use type of the map area changes or when it is determined that there is land use anomaly in the map area, the number of abnormal risks is reset to zero.

[0087] Step 305: When the number of abnormal risks is greater than the risk number threshold, it is determined that there is a land use risk in the map area.

[0088] Optionally, when the number of abnormal risks is less than or equal to the risk number threshold, risk warning information is generated based on the number of abnormal risks, which can help achieve automatic law enforcement monitoring.

[0089] In the above technical scheme, when the predicted use type matches the planned use type, the difference between the predicted confidence and the historical confidence is further combined to determine whether there is an abnormal risk in the map area, and when the number of times it is determined that there is an abnormal risk in the map area reaches a preset number of times, it is determined that there is a land use risk in the map area. This can help determine whether there is a land use anomaly in the map area based on abnormal changes in confidence, which can help improve the accuracy of automatic law enforcement monitoring.

[0090] In some embodiments, reference Figure 4 , step 105, obtaining a verification image corresponding to the spot area, including the following steps: Step 401 : determining at least one target location from the image patch data based on a feature extraction method corresponding to the predicted usage type.

[0091] The target position is the location of the feature object. In actual implementation, the feature objects corresponding to different usage types may be different. For example, the feature objects of construction land may be building corners, entrances, etc., and the feature objects of farmland may be crops, irrigation systems, etc. Therefore, the feature extraction methods corresponding to different usage types may be different and need to be set separately.

[0092] In some embodiments, the position of the feature object in the pattern area is marked by the usage recognition model during the process of identifying the pattern data. In this case, during the matching process, only the position corresponding to the feature object matching the predicted usage type in the marked feature object needs to be determined as the target position.

[0093] Step 402: determine the spatial coordinates corresponding to the target position based on the coordinate mapping relationship between the pattern data and the pattern area.

[0094] The coordinate mapping relationship may be determined based on the acquisition parameters of the remote sensing image, or may be determined based on the positions of known coordinate points in the image patch data. This embodiment does not limit the method for determining the coordinate mapping relationship.

[0095] Step 403: Acquire an image corresponding to the target position based on the spatial coordinates to obtain a verification image.

[0096] In one example, acquiring an image corresponding to a target position based on spatial coordinates includes: controlling a mobile image acquisition device to move to an image acquisition position corresponding to the target position to acquire a verification image.

[0097] In another example, acquiring an image corresponding to the target position based on the spatial coordinates includes: acquiring an original image acquired by an image acquisition device corresponding to the target position, and cropping the original image to obtain a verification image.

[0098] In the above implementation, the target location can be determined based on the feature extraction method corresponding to the predicted usage type, and the acquisition of the verification image can be guided based on the spatial coordinates of the target location. This can help to achieve accurate acquisition of the verification image, thereby helping to reduce the amount of data that needs to be collected and verified, and further help to reduce the difficulty of automatic law enforcement monitoring.

[0099] For further reference, Figure 5 In the case where there are more than two target positions, step 403, obtaining an image corresponding to the target position based on the spatial coordinates, includes the following steps: Step 501 : determining image acquisition requirements based on predicted usage types.

[0100] Specifically, the image acquisition requirements may include factors such as image acquisition angle and image color, which can be set according to actual needs. For example, if the predicted use type is construction land, the image acquisition requirements may include oblique photography, which can help determine the height of the building based on the verification image. Another example: if the predicted use type is farmland, the image acquisition requirements may include color images, which can help determine the coverage of crops based on the verification image.

[0101] Step 502 : Plan an image acquisition path based on the distribution of the spatial coordinates corresponding to each target position and the image acquisition requirement to obtain the image acquisition path.

[0102] Optionally, the image acquisition path is planned based on the distribution of spatial coordinates corresponding to each target position and the image acquisition requirements, including: determining at least two image acquisition positions based on the distribution of spatial coordinates corresponding to the target position and the image acquisition requirements; using a path planning algorithm to plan the path between each image acquisition position to obtain the image acquisition path.

[0103] The path planning algorithm may be a shortest path planning algorithm, such as Dijkstra's Algorithm.

[0104] Furthermore, the image acquisition path may include not only image acquisition positions, but also image acquisition angles corresponding to different positions of the image acquisition component, thereby ensuring that a suitable image is acquired.

[0105] In actual implementation, the image acquisition device can be an aircraft, such as a drone, or it can be a vehicle. Since different types of image acquisition devices have different moving methods, for example, for aircraft, flight altitude data must also be considered during path planning. Therefore, during the path planning process, the type of image acquisition device needs to be taken into account. This allows for targeted path planning, which can help to accurately acquire images.

[0106] Step 503: Control the image acquisition device to move along the image acquisition path to acquire images corresponding to each target position.

[0107] In the above technical scheme, since there may be more than two target locations, the image acquisition path can be planned based on the distribution of each target location and the image acquisition requirements corresponding to the predicted usage type, and the image acquisition device can be controlled to move along the image acquisition path to acquire images of the target locations. This can help improve the efficiency of image acquisition and also help ensure the quality of the acquired images.

[0108] In some embodiments, the loss function used in the usage prediction model training process includes a focal loss term L F and the consistency loss term L C Specifically, the focal loss term L F Used to deal with the problem of imbalanced categories; the consistency loss term L C It is used to ensure the consistency of the adjacent pixels predicted by the model in terms of categories, thereby reducing the impact of image acquisition errors on the model prediction results. This can help improve the training effect of the model, and in turn, can help improve the accuracy of the usage type prediction results.

[0109] In one example, the focal loss term L F It is expressed as follows:

[0110] Among them, L F is the focal loss term; α t is the balance coefficient, which is used to balance the weights of samples of different categories. Generally speaking, the balance coefficient is negatively correlated with the number of samples of the sample category, that is, the smaller the number of samples, the larger the balance coefficient. In this way, the model will pay more attention to the small number of samples during training; p t is the model’s predicted probability for the correct category, is the model’s output value; γ is an adjustment coefficient, which is used to reduce the relative loss of easily classified samples, so that the model can pay more attention to samples that are difficult to classify. When γ is a value greater than 0, the loss will decrease as the model’s confidence in the sample prediction increases, which means that the model’s loss for easily classified samples will be reduced, while the loss for samples that are difficult to classify will be relatively increased.

[0111] In another example, the consistency loss term L C It is expressed by the following formula:

[0112] Among them, L C is the consistency loss term; N is the total number of image pixels; i is the index of the current image pixel; N(i) is the neighborhood of pixel i, specifically, the neighborhood can be up, down, left, right, or eight neighborhoods, etc.; p i To simulate the predicted probability vector for pixel i, for example, the vector output by softmax; p j is the predicted probability vector for pixel j.

[0113] In one example, the loss function is expressed as follows:

[0114] Among them, Loss is the loss value; a is the cross entropy loss term L E The weight of; b is the focal loss term LF The weight of L C The weight of the loss term.

[0115] The present application also provides an electronic device. Figure 6 As shown, Figure 6 The electronic device 600 shown includes: a processor 601 and a memory 603. The processor 601 and the memory 603 are connected, such as through a bus 602. Optionally, the electronic device 600 may also include a transceiver 604. It should be noted that in actual applications, the transceiver 604 is not limited to one, and the structure of the electronic device 600 does not constitute a limitation on the embodiments of the present application.

[0116] Processor 601 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It may implement or execute various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 601 may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, and the like.

[0117] The bus 602 may include a path to transmit information between the above components. The bus 602 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus 602 may be divided into an address bus, a data bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0118] The memory 603 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, or an EEPROM (Electrically Erasable Programmable Read Only Memory), a disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store desired program codes in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto.

[0119] The memory 603 is used to store the application code for executing the solution of the present application, and the execution is controlled by the processor 601. The processor 601 is used to execute the application code stored in the memory 603 to implement the contents shown in the above method embodiment.

[0120] The electronic devices include, but are not limited to, mobile terminals such as mobile phones, laptop computers, PDAs (personal digital assistants), PADs (tablet computers), etc., and fixed terminals such as digital TVs, desktop computers, etc. It can also be a server terminal, etc. Figure 6 The electronic device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0121] An embodiment of the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed in a computer, it enables the computer to execute the automatic law enforcement monitoring method based on remote sensing images provided in the above embodiment.

[0122] It should be understood that although the steps in the flowchart of the accompanying drawings are shown in sequence according to the instructions of the arrows, these steps are not necessarily performed in the order indicated by the arrows. Unless otherwise clearly stated in this document, the execution of these steps is not strictly limited in order and can be performed in other orders.

[0123] The above are only some implementation methods of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. An automatic law enforcement monitoring method based on remote sensing images, characterized in that: The method comprises: Performing spot extraction on the remote sensing image of the monitoring area to obtain at least one spot area; For each of the spot areas, the spot data corresponding to the spot area in the remote sensing image is input into a pre-trained usage prediction model to obtain a predicted usage type corresponding to the spot area; Determining whether the predicted usage type matches the planned usage type corresponding to the map area; In the case where the predicted usage type does not match the planned usage type, determining whether to verify the predicted usage type; In the case of determining to verify the predicted usage type, obtaining a verification image corresponding to the image spot area, wherein the verification image is acquired in a different manner from the remote sensing image; Determining whether the pattern data matches the verification image; When it is determined that the image spot data matches the verification image, it is determined that there is a land use anomaly in the image spot area.

2. The method according to claim 1, characterized in that The determining whether to verify the predicted usage type includes: Determining whether the prediction confidence corresponding to the predicted usage type is greater than or equal to a preset confidence threshold; When the prediction confidence is less than the confidence threshold, determine to verify the predicted usage type.

3. The method according to claim 2, characterized in that After determining whether the confidence level corresponding to the predicted usage type is greater than or equal to a preset confidence level threshold, the method further includes: In the case where the prediction confidence is greater than the confidence threshold, determining whether there is a historical confidence corresponding to the image spot area; In the case where the historical confidence exists, determining whether a gap value between the prediction confidence and the historical confidence is less than or equal to a preset gap value threshold; When the gap value is greater than the gap value threshold, it is determined to verify the predicted usage type.

4. The method according to claim 2, characterized in that: Before determining whether the prediction confidence corresponding to the predicted usage type is greater than or equal to a preset confidence threshold, the method further includes: Determining whether the predicted usage type is a preset high-risk type; In the case where the predicted usage type is not the high-risk type, performing the step of determining whether the prediction confidence is greater than or equal to a preset confidence threshold; In the case where the predicted usage type is the high-risk type, it is directly determined to verify the predicted usage type without performing the step of determining whether the prediction confidence is greater than or equal to a preset confidence threshold.

5. The method according to claim 1, characterized in that: After determining whether the predicted usage type matches the planned usage type corresponding to the pattern area, the method further includes: In the case where the predicted usage type matches the planned usage type, determining whether there is a historical confidence corresponding to the map area; In the case where the historical confidence exists, determining whether the difference between the prediction confidence and the historical confidence is less than or equal to a preset difference threshold; When the difference is greater than the difference threshold, an abnormal risk corresponding to the image spot area is recorded once; Determine whether the number of abnormal risks corresponding to the image spot area is greater than a preset risk number threshold; When the number of abnormal risks is greater than the risk number threshold, it is determined that there is a land use risk in the map area.

6. The method according to claim 1, characterized in that The obtaining of the verification image corresponding to the pattern area includes: Determine at least one target location from the image spot data based on a feature extraction method corresponding to the predicted usage type; Determine the spatial coordinates corresponding to the target position based on the coordinate mapping relationship between the pattern data and the pattern area; An image corresponding to the target position is acquired based on the spatial coordinates to obtain the verification image.

7. The method according to claim 6, characterized in that The target positions include more than two, and acquiring images corresponding to the target positions based on the spatial coordinates includes: determining image acquisition requirements based on the predicted usage type; Planning an image acquisition path based on the distribution of the spatial coordinates corresponding to each of the target positions and the image acquisition requirements to obtain an image acquisition path; The image acquisition device is controlled to move along the image acquisition path to acquire images corresponding to each of the target positions.

8. The method according to claim 1, characterized in that After determining whether the pattern data matches the verification image, the method further includes: In the case where the pattern data does not match the verification image, the predicted usage type is corrected based on the verification image to obtain a corrected usage type; Determining whether the revised use type matches the planned use type; In the case where the revised use type does not match the planned use type, determining that there is a land use anomaly in the map area; In the case where the corrected use type matches the planned use type, it is determined that there is no land use anomaly in the map area.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; Memory; At least one application, wherein at least one application is stored in a memory and configured to be executed by at least one processor, and the at least one application is configured to: execute the automatic law enforcement monitoring method based on remote sensing images as described in any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed in a computer, the computer is caused to execute the automatic law enforcement monitoring method based on remote sensing images as described in any one of claims 1 to 8.

Images