Target detection land illegal land use identification method based on unmanned aerial vehicle image

Abstract

The application relates to the technical field of image processing, and discloses a target detection land illegal land use identification method based on unmanned aerial vehicle images. The method performs spectral difference processing and elevation change detection processing on historical legal land use data corresponding to a target area, a digital orthographic image map and a digital surface model to obtain a suspected change area set. Finally, the suspected change area set is input into a target illegal land use identification model to obtain an identification result. The target illegal land use identification model comprises a backbone network, a neck network and a detection head connected in sequence. The backbone network is obtained by retaining the first level to the fourth level of an original backbone network in a YOLOv8-P2 model and leading out a high-resolution branch at the second level. The neck network is obtained by replacing an original neck network in the YOLOv8-P2 model with a pruned one-way feature pyramid network. The detection head is obtained by replacing an original detection head in the YOLOv8-P2 model with multiple parallel sub-detection heads. In this way, the detection accuracy of illegal land use can be improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and relates to, but is not limited to, a method for identifying illegal land use based on target detection using UAV imagery. Background Technology

[0002] Urbanization and rural revitalization are progressing in parallel, leading to frequent changes in land use types. Currently, land can be monitored using satellite remote sensing, but this method is limited by time, spatial resolution, and accuracy, resulting in low accuracy in detecting illegal land use. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a target detection method for identifying illegal land use based on UAV imagery, which can improve the detection accuracy of illegal land use.

[0004] The specific technical solutions of this invention are as follows: This invention provides a method for target detection and illegal land use identification based on UAV imagery, the method comprising: Acquire digital orthophotos and digital surface models of the target area; The historical legal land use data corresponding to the target area are subjected to spectral difference processing and elevation change detection processing with digital orthophoto maps and digital surface models to obtain a set of suspected change areas; the set of suspected change areas includes at least one suspected change area. The set of suspected change areas is input into the target illegal land use identification model to obtain the identification results; the identification results include the category label, bounding box coordinates and confidence score of each suspected change area; The target illegal land use identification model consists of a backbone network, a neck network, and a detection head connected in sequence. The backbone network is obtained by retaining the first to fourth levels of the original backbone network in the YOLOv8-P2 model and drawing high-resolution branches at the second level. The neck network is obtained by replacing the original neck network in the YOLOv8-P2 model with a pruned unidirectional feature pyramid network. The detection head is obtained by replacing the original detection head in the YOLOv8-P2 model with multiple parallel sub-detection heads. The high-resolution branch is used to enhance the high-resolution detail features in the shallow feature map extracted by the backbone network, resulting in an enhanced shallow feature map; the pruned unidirectional feature pyramid network is used to fuse the enhanced shallow feature map with the deep feature map extracted by the backbone network from top to bottom.

[0005] In some embodiments, the historical legal land use data corresponding to the target area is subjected to spectral difference processing and elevation change detection processing with digital orthophoto maps and digital surface models to obtain a set of suspected change areas, including: The vector base map in the historical legal land use data is rasterized to obtain classified images; the classified images have the same resolution and coordinate system as the digital orthophoto map. The spectral difference map between the digital orthophoto map and the classification image is determined by weighted Euclidean distance method, and the digital surface model is processed by Gaussian filtering to generate a height variation mask. A pixel-level logical AND operation is performed between the spectral difference map and the height variation mask to obtain a binarized candidate region mask. If the area corresponding to the binarized candidate region mask is greater than the preset area threshold, the area corresponding to the binarized candidate region mask is determined as a suspected change region, and a set of suspected change regions is constructed based on the suspected change regions.

[0006] In some embodiments, the target illegal land use identification model is used to perform the following operations: Receive a set of suspected changed regions; Feature extraction is performed on each suspected change region in the suspected change region set through the first to fourth layers of the backbone network to obtain a multi-scale feature map; the multi-scale feature map includes at least a shallow feature map output by the second layer, a medium feature map output by the third layer, and a deep feature map output by the fourth layer. The high-resolution branch enhances the high-resolution detail features in the shallow feature map, resulting in an enhanced shallow feature map. The enhanced shallow, middle and deep feature maps are fused from top to bottom using the neck network to obtain the fused feature map. The detection head performs target detection on feature maps of different scales in the fused feature map to obtain the detection results.

[0007] In some embodiments, the high-resolution branch includes a first convolutional layer, a depth-separable convolutional layer, and a second convolutional layer; High-resolution branches are used to perform the following operations: The first convolutional layer compresses the number of channels in the shallow feature map to a preset dimension, resulting in a compressed feature map. Spatial features are extracted from the compressed feature map by depthwise separable convolution to obtain a feature map with enhanced details. The second convolutional layer restores the feature map after detail enhancement to the same number of channels as the shallow feature map, resulting in the transformed feature map. The transformed feature map and the shallow feature map are joined by a residual connection to obtain the enhanced shallow feature map.

[0008] In some embodiments, the branched unidirectional feature pyramid network includes multiple cascaded fusion modules; Pruned one-way feature pyramid networks are used to perform the following operations: It receives the enhanced shallow feature map from the high-resolution branch output, the mid-level feature map from the third-level output, and the deep feature map from the fourth-level output. By using multiple cascaded fusion modules, the enhanced shallow feature map, middle feature map and deep feature map are fused from top to bottom in a unidirectional manner to obtain the fused feature map. The fused feature map is output to the detection head for target detection.

[0009] In some embodiments, the plurality of sub-detection heads include a first sub-detection head, a second sub-detection head, and a third sub-detection head; the plurality of fusion modules include a first fusion module, a second fusion module, and a third fusion module; the first fusion module includes a first C2f module and a first upsampling module, the second fusion module includes a first stitching layer, a second C2f module, and a second upsampling module, and the third fusion module includes a second stitching layer and a third C2f module; The first fusion module is used to perform the following operations: The first C2f module receives the deep feature map output from the fourth level, extracts features from the deep feature map, obtains the first feature map, and outputs it to the third sub-detection head and the first upsampling module. The first feature map is upsampled by the first upsampling module to obtain the first upsampled feature map, which is then output to the first stitching layer in the second fusion module. The second fusion module is used to perform the following operations: The first splicing layer receives the first upsampled feature map and the middle layer feature map output from the third layer, and splices the first upsampled feature map with the middle layer feature map to obtain the first fused feature map. The second C2f module extracts features from the first fused feature map to obtain the second feature map, which is then output to the second upsampling module and the second sub-detection head. The second feature map is upsampled by the second upsampling module to obtain the second upsampled feature map, which is then output to the second stitching layer in the third fusion module. The third fusion module is used to perform the following operations: The second stitching layer receives the second upsampled feature map and the enhanced shallow feature map output by the high-resolution branch. The second upsampled feature map and the enhanced shallow feature map are then stitched together to obtain the second fused feature map. The third C2f module extracts features from the second fused feature map to obtain the third feature map, which is then output to the first sub-detection head.

[0010] In some embodiments, the method further includes: Obtain a training sample set; the training sample set includes target annotation information and multiple sample images; the multiple sample images include positive sample images and negative sample images, positive sample images refer to images that include illegal areas, and negative sample images refer to images that do not include illegal areas; Based on the training sample set, the initial illegal land use identification model is trained in stages by combining the loss function, the first weight, and the second weight until the initial illegal land use identification model converges, thus obtaining the target illegal land use identification model. The combined loss function includes the complete intersection-union loss function, the distribution focus loss function, and the binary cross-entropy loss function.

[0011] In some embodiments, based on a training sample set, the initial illegal land use identification model is trained in stages by combining a loss function, a first weight, and a second weight, including: Freeze the parameters of the backbone network and the target parameters of the detection head in the initial illegal land use identification model, and perform the first stage training of the initial illegal land use identification model based on the training sample set to obtain the first prediction annotation information; By combining the loss function, the first weight, and the second weight, the first total loss value between the first predicted annotation information and the target annotation information is determined until the first total loss value converges, thus obtaining the first illegal land use identification model. Unfreeze all parameters of the first illegal land use identification model, and conduct a second-stage training on the first illegal land use identification model based on the training sample set to obtain the second prediction annotation information; By combining the loss function, the first weight, and the second weight, the second total loss value between the second predicted annotation information and the target annotation information is determined until the second total loss value converges, thus obtaining the target illegal land use identification model.

[0012] In some embodiments, the total loss value is determined by the following formula; the total loss value includes a first total loss value and a second total loss value. in, The label for the detection head. For the first sub-detection head, For the second sub-detection head, For the third sub-detection head, For the first The complete intersection-union loss corresponding to each detection head For the first The distributed focal loss corresponding to each detection head; For the first The binary cross-entropy loss corresponding to each detector head. As the first weight, It is the second weight.

[0013] The beneficial effects of the technical solutions provided in the embodiments of the present invention include at least the following: In this embodiment of the invention, a digital orthophoto map and a digital surface model of the target area are acquired; the historical legal land use data corresponding to the target area are subjected to spectral difference processing and elevation change detection processing with the digital orthophoto map and the digital surface model to obtain a set of suspected change areas; the set of suspected change areas includes at least one suspected change area; the set of suspected change areas is input into the target illegal land use identification model to obtain the identification result; wherein, the identification result includes the category label, bounding box coordinates, and confidence score corresponding to each suspected change area; the target illegal land use identification model includes a backbone network, a neck network, and a detection head connected in sequence, and the backbone network retains YOLOv8. The YOLOv8-P2 model is derived from the first to fourth layers of the original backbone network, with a high-resolution branch introduced at the second layer. The neck network is obtained by replacing the original neck network in the YOLOv8-P2 model with a pruned unidirectional feature pyramid network. The detection head is obtained by replacing the original detection head in the YOLOv8-P2 model with multiple parallel sub-detection heads. The high-resolution branch is used to enhance the high-resolution detail features in the shallow feature maps extracted by the backbone network, resulting in enhanced shallow feature maps. The pruned unidirectional feature pyramid network is used to fuse the enhanced shallow feature maps with the deep feature maps extracted by the backbone network from top to bottom. In this way, shallow spatial information and deep semantic information complement each other, ensuring that the spatial information of extremely small targets is not diluted by deep semantics, enhancing the illegal land use identification model's ability to perceive targets at different scales, and thus improving the detection accuracy of illegal land use. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein: Figure 1 A flowchart illustrating the target detection and illegal land use identification method based on UAV imagery provided in this embodiment of the invention; Figure 2 This is a schematic diagram of the structure of an illegal land use identification model provided in an embodiment of the present invention; Figure 3 A schematic diagram of a high-resolution branch structure provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of a pruned unidirectional feature pyramid network provided in an embodiment of the present invention. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. The following embodiments are used to illustrate the present invention, but are not intended to limit the scope of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0017] It should be noted that the terms "first, second, and third" used in the embodiments of the present invention are only used to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, and third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of the present invention described herein can be implemented in an order other than that illustrated or described herein.

[0018] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which these embodiments of the invention pertain. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0019] Figure 1 A flowchart illustrating a method for identifying illegal land use based on target detection using UAV imagery is provided. This method can be executed via a control device, which may include at least one of a personal computer, laptop, smartphone, tablet, or portable wearable device; however, this embodiment does not limit the specific device used.

[0020] like Figure 1 As shown, the target detection and illegal land use identification method based on UAV imagery provided in this embodiment of the invention may include S101-S103.

[0021] S101. Obtain a digital orthophoto map and a digital surface model of the target area.

[0022] In some embodiments, the target area refers to the specific geographical area where illegal land use needs to be identified. A Digital Orthophoto Map (DOM) is a high-resolution aerial or satellite image that has undergone geometric and radiometric correction, eliminating displacement errors caused by sensor attitude and terrain undulations, and has a unified map projection coordinate system. A Digital Surface Model (DSM) is a continuous elevation model of the tops of surface features (including vegetation, buildings, etc.), reflecting the true three-dimensional morphology of the Earth's surface.

[0023] For example, a drone can be used to acquire digital orthophoto maps and digital surface models of the target area. In some application scenarios, a drone integrating Real-Time Kinematic (RTK) technology can be used, with the flight altitude set to 80 meters and both the forward and lateral overlap rates set to 85% to ensure that the captured images have sufficient stereo pairs for subsequent 3D reconstruction. During the drone aerial photography, RGB raw images and Position and Orientation System (POS) data, including Global Positioning System (GPS) position information and Inertial Measurement Unit (IMU) attitude information, can be acquired simultaneously. At the same time, a predetermined number (e.g., 6) of Ground Control Points (GCPs) are deployed within the target area, and the 3D coordinates of each GCP in the CGCS2000 coordinate system are measured using a Global Navigation Satellite System (GNSS) receiver to obtain the GCP coordinates. The acquired RGB raw images, POS data, and GCP coordinates are then imported into photogrammetry software (such as ContextCapture, Pix4Dmapper, etc.). Aerial triangulation is performed to recover the accurate exterior orientation elements of the images, followed by dense matching to generate a dense point cloud. Based on the dense point cloud and the aerial triangulation results, a digital orthophoto map with a resolution of 0.1 meters and the corresponding digital surface model are generated using digital differential correction technology.

[0024] In some embodiments, to improve identification accuracy, radiometric consistency correction can be applied to the digital orthophoto map of the target area.

[0025] For example, if the target area is a basic farmland protection area in Town A of a county in East China, a paddy field area with uniform illumination in the target area can be selected as the radiation reference. The histogram matching method is used to adjust the image block by image block so that the deviation of the mean pixel value of each image block from the reference area after correction does not exceed 5%, and the deviation of the standard deviation of pixels does not exceed 3%.

[0026] In some embodiments, to further improve recognition accuracy, the digital orthophoto map and digital surface model of the target area need to meet preset accuracy requirements. For example, the planar position accuracy of the digital orthophoto map, that is, the difference between the planar coordinates of the ground feature points on the digital orthophoto map and the actual ground coordinates is less than or equal to 0.05 meters; the elevation accuracy of the digital surface model, that is, the difference between the elevation value of a point on the digital surface model and its actual ground elevation value is less than or equal to 0.08 meters.

[0027] S102. Perform spectral difference processing and elevation change detection processing on the historical legal land use data corresponding to the target area, digital orthophoto map, and digital surface model to obtain a set of suspected change areas.

[0028] In some embodiments, historical legal land use data refers to vector data and attribute information of land use patches that have been legally approved and sourced from a land survey database (such as the previous year's land change survey database). Historical legal land use data includes at least the boundary vector, land use code, land use name, and patch area for each patch in the target area.

[0029] In some embodiments, the suspected change area set includes at least one suspected change area, which refers to an area in the target area that differs from historical legal land use data.

[0030] In some embodiments, the historical legal land use data corresponding to the target area is subjected to spectral difference processing and elevation change detection processing with digital orthophoto maps and digital surface models to obtain a set of suspected change areas. This includes: rasterizing the vector base map in the historical legal land use data to obtain a classified image; determining the spectral difference map between the digital orthophoto map and the classified image using a weighted Euclidean distance method, and performing Gaussian filtering on the digital surface model to generate an elevation change mask; performing a pixel-level logical AND operation on the spectral difference map and the elevation change mask to obtain a binarized candidate region mask; if the area corresponding to the binarized candidate region mask is greater than a preset area threshold, the area corresponding to the binarized candidate region mask is determined as a suspected change area, and a set of suspected change areas is constructed based on the suspected change areas. The classified image and the digital orthophoto map have the same resolution and coordinate system.

[0031] For example, historical legal land use data can be rasterized using Geographic Information System (GIS) software to convert it into a raster format, resulting in a categorized image. Then, for each pixel in the digital orthophoto map, based on its historical land use code within the categorized image, the reference spectral value corresponding to each pixel is looked up in a pre-defined lookup table. The difference between the measured spectral value and the reference spectral value for each pixel is determined using a weighted Euclidean distance method, generating a spectral difference map. The lookup table can be constructed based on the historical digital orthophoto map, by calculating the pixel mean values ​​of the red, green, and blue channels for all pixels corresponding to each land use code, thus recording the reference spectral values ​​corresponding to each land use code. Simultaneously with generating the spectral difference map, a Gaussian filter can be applied to the digital surface model to smooth minor surface undulations and estimate the background elevation of the target area. The difference between the digital surface model and the filtered background elevation is calculated pixel-by-pixel to obtain a height variation mask. After obtaining the spectral difference map and the height change mask, a pixel-level logical AND operation is performed between them. That is, only when the spectral difference value of a pixel in the spectral difference map is greater than a preset difference threshold and the height change mask is greater than its corresponding elevation threshold, is that pixel marked as a candidate change pixel, thus obtaining a binarized candidate region mask. The elevation threshold is a preset value and can be set according to the ground conditions of the target area. For example, if the target area is a forest area, the elevation change threshold can be set to 0.5 meters; if the target area is a hardened surface (such as an urban road), the elevation change threshold can be set to 0.3 meters. This application embodiment does not limit this. Then, the areas corresponding to the binarized candidate region masks are screened. If the area corresponding to the binarized candidate region mask is greater than a preset area threshold, it indicates that the area corresponding to the binarized candidate region mask has significant changes, which may involve illegal land use. In this case, the area corresponding to the binarized candidate region mask is determined as a suspected change area. The preset area threshold can be set according to actual needs and the characteristics of the target area. For example, it can be set to 50 square meters. This application embodiment does not limit this.

[0032] S103. Input the set of suspected change areas into the target illegal land use identification model to obtain the identification results.

[0033] In some embodiments, the identification results include a category label, bounding box coordinates, and a confidence score for the suspected area of ​​change. The category label indicates the type of illegal land use in the suspected area, such as excavation pits, makeshift guardhouses, or hardened ground. The bounding box coordinates indicate the location and extent of the suspected area of ​​change in the digital orthophoto map. The confidence score indicates the degree of confidence that the suspected area of ​​change belongs to a certain type of illegal land use, and its value ranges from 0 to 1. A confidence score closer to 1 indicates a higher degree of confidence that the suspected area of ​​change belongs to that type of illegal land use.

[0034] In some embodiments, the target illegal land use identification model includes a backbone network, a neck network, and a detection head connected in sequence. The backbone network is obtained by retaining the first to fourth levels of the original backbone network in the YOLOv8-P2 model and drawing a high-resolution branch at the second level. The neck network is obtained by replacing the original neck network in the YOLOv8-P2 model with a pruned unidirectional feature pyramid network. The detection head is obtained by replacing the original detection head in the YOLOv8-P2 model with multiple parallel sub-detection heads. The high-resolution branch is used to enhance the high-resolution detail features in the shallow feature map extracted by the backbone network to obtain an enhanced shallow feature map. The pruned unidirectional feature pyramid network is used to fuse the enhanced shallow feature map with the deep feature map extracted by the backbone network from top to bottom.

[0035] In some embodiments, Figure 2 A schematic diagram of a model for identifying illegal land use is shown. Figure 2 As shown, the illegal land use identification model is an improvement upon the YOLOv8-P2 model. This model comprises a backbone network, a neck network, and a head network connected in sequence. The backbone network, the front end of the model, extracts multi-scale feature information from the input image (such as a set of suspected change areas), including shallow feature maps, enhanced shallow feature maps, mid-level feature maps, and deep feature maps. The backbone network retains the first to fourth layers of the original YOLOv8-P2 model, ensuring sufficient capture of basic image features. A high-resolution branch is introduced at the second layer, designed to enhance the extraction of high-resolution detail features from the shallow feature maps, resulting in a more refined enhanced shallow feature map. The neck network, located after the backbone network, fuses the multi-scale feature information extracted by the backbone network to enhance feature representation. The neck network replaces the original neck network in the YOLOv8-P2 model with a pruned unidirectional feature pyramid network. This pruned unidirectional feature pyramid network can fuse the enhanced shallow feature maps with the deep feature maps extracted by the backbone network from top to bottom, achieving full interaction and utilization of multi-scale features and improving the illegal land use identification model's ability to detect targets of different sizes. The detection head, located after the neck network, is the output part of the illegal land use identification model. It is used to perform target detection on the fused feature maps and output the final detection result. The detection head replaces the original detection head in the YOLOv8-P2 model with multiple parallel sub-detection heads. Each detection head is used to detect features at different scales, thus enabling the illegal land use identification model to identify suspected illegal land use of different sizes, improving identification accuracy.

[0036] In some embodiments, the target illegal land use identification model is the trained illegal land use identification model.

[0037] In some embodiments, the target illegal land use identification model is used to perform the following operations: receiving a set of suspected change areas; extracting features from each suspected change area in the set of suspected change areas through the first to fourth layers of the backbone network to obtain a multi-scale feature map; the multi-scale feature map includes at least a shallow feature map output from the second layer, a medium feature map output from the third layer, and a deep feature map output from the fourth layer; enhancing the high-resolution detail features in the shallow feature map through a high-resolution branch to obtain an enhanced shallow feature map; fusing the enhanced shallow feature map, medium feature map, and deep feature map from top to bottom through a neck network to obtain a fused feature map; and performing target detection on the feature maps of different scales in the fused feature map through a detection head to obtain a detection result.

[0038] In some embodiments, the first layer includes a convolutional layer, and the second to fourth layers each include a convolutional layer and a C2-fusion (C2f) module. The convolutional layer is used for preliminary feature extraction of the input features, while the C2f module enhances the feature extraction capability. By combining with the convolutional layer, the fusion and extraction of features from different levels are achieved. In practical applications, after the backbone network receives a set of suspected change regions, it performs preliminary feature extraction on each suspected change region through convolutional layers in the first layer to obtain an initial feature map, which is then output to the second layer. The second layer then extracts features from the initial feature map through convolutional layers and a C2f module to obtain a shallow feature map, which is then output to the third layer. Simultaneously, the shallow feature map is output to a high-resolution branch for high-resolution detail feature enhancement, resulting in an enhanced shallow feature map. The third layer further extracts features from the shallow feature map through convolutional layers and a C2f module to obtain a mid-level feature map, which is then output to the fourth layer. The fourth layer performs deep feature extraction from the mid-level feature map output from the third layer through convolutional layers and a C2f module, resulting in a deep feature map.

[0039] In some embodiments, the high-resolution branch includes a first convolutional layer, a depthwise separable convolutional layer, and a second convolutional layer. The high-resolution branch is used to perform the following operations: compressing the number of channels of the shallow feature map to a preset dimension through the first convolutional layer to obtain a compressed feature map; extracting spatial features from the compressed feature map through a depthwise separable convolution to obtain a feature map with enhanced details; restoring the feature map with enhanced details to the same number of channels as the shallow feature map through the second convolutional layer to obtain a transformed feature map; and performing a residual connection between the transformed feature map and the shallow feature map to obtain an enhanced shallow feature map.

[0040] For example, Figure 3This is a schematic diagram of a high-resolution branch structure provided in an embodiment of this application. Figure 3 As shown, the high-resolution branch comprises a first convolutional layer, a depthwise separable convolutional layer, and a second convolutional layer connected in sequence. If the shallow feature map has 128 channels, the first convolutional layer can compress the number of channels to 64, resulting in a compressed feature map. The depthwise separable convolutional layer then processes the compressed feature map to extract the edges and texture details of small targets, resulting in a detail-enhanced feature map. Next, the second convolutional layer restores the number of channels in the detail-enhanced feature map to 128, resulting in a transformed feature map. Finally, the transformed feature map and the shallow feature map are joined element-wise using a residual concatenation to obtain the enhanced shallow feature map. In this way, the enhanced shallow feature map better preserves the detailed information in the input image, enabling the subsequent illegal land use identification model to more accurately detect targets and improve recognition accuracy.

[0041] In some embodiments, the pruned unidirectional feature pyramid network includes multiple cascaded fusion modules; the pruned unidirectional feature pyramid network is used to perform the following operations: receiving the enhanced shallow feature map output by the high-resolution branch, the middle feature map output by the third level, and the deep feature map output by the fourth level; performing top-down unidirectional feature fusion on the enhanced shallow feature map, middle feature map, and deep feature map through the multiple cascaded fusion modules to obtain the fused feature map; and outputting the fused feature map to the detection head for target detection.

[0042] In some embodiments, the number of fusion modules can be set according to actual needs. This application does not limit this.

[0043] For example, Figure 4 This is a schematic diagram of a pruned unidirectional feature pyramid network provided in an embodiment of this application. Figure 4 As shown, there are three fusion modules: the first fusion module, the second fusion module, and the third fusion module. The first fusion module is connected to the fourth layer of the backbone network, the second fusion module is connected to the third layer of the backbone network, and the third fusion module is connected to the second layer of the backbone network.

[0044] In some embodiments, the multiple fusion modules include a first fusion module, a second fusion module, and a third fusion module. The multiple sub-detection heads include a first sub-detection head, a second sub-detection head, and a third sub-detection head. The first sub-detection head performs target detection on the feature map output by the first fusion module to identify larger-scale suspected illegal land use targets; the second sub-detection head performs target detection on the feature map output by the second fusion module to identify medium-scale suspected illegal land use targets; and the third sub-detection head performs detection on the feature map output by the third fusion module to identify smaller-scale suspected illegal land use targets.

[0045] Understandably, by using multiple parallel sub-detection heads, the illegal land use identification model can cover illegal land use targets of different sizes, thereby improving the accuracy and robustness of identification.

[0046] In some embodiments, such as Figure 4 As shown, the first fusion module includes a first C2f module and a first upsampling module. The first fusion module is used to perform the following operations: receive the deep feature map output from the fourth layer through the first C2f module, extract features from the deep feature map to obtain a first feature map and output it to the third sub-detector head and the first upsampling module; upsample the first feature map through the first upsampling module to obtain a first upsampled feature map and output it to the first stitching layer in the second fusion module.

[0047] The second fusion module includes a first stitching layer, a second C2f module, and a second upsampling module. The second fusion module performs the following operations: receiving a first upsampled feature map and a mid-level feature map output from the third layer through the first stitching layer, and stitching the first upsampled feature map with the mid-level feature map to obtain a first fused feature map; extracting features from the first fused feature map through the second C2f module to obtain a second feature map and outputting it to the second upsampling module and the second sub-detection head; upsampling the second feature map through the second upsampling module to obtain a second upsampled feature map and outputting it to the second stitching layer in the third fusion module.

[0048] The third fusion module includes a second stitching layer and a third C2f module. The third fusion module performs the following operations: it receives the second upsampled feature map and the enhanced shallow feature map output by the high-resolution branch through the second stitching layer, stitches the second upsampled feature map with the enhanced shallow feature map to obtain the second fused feature map, and extracts features from the second fused feature map through the third C2f module to obtain the third feature map and outputs it to the first sub-detection head.

[0049] For example, after receiving the deep feature map output from the fourth layer, the first fusion module extracts deep features from it using the first C2f module to obtain a first feature map. Then, the first feature map is output to both the third sub-detection head and the first upsampling module. The third sub-detection head performs target detection on the first feature map, and the first upsampling module upsamples the first feature map to enlarge its size, resulting in a first upsampled feature map, which is then sent to the first stitching layer of the second fusion module. The first stitching layer of the second fusion module receives the first upsampled feature map and the mid-layer feature map output from the third layer of the backbone network, and stitches them together to obtain a first fused feature map. Then, the second C2f module extracts features from the first fused feature map to obtain a second feature map, which is output to the second sub-detection head for target detection. Simultaneously, the second feature map is output to the second upsampling module. The second upsampling module upsamples the second feature map to obtain a second upsampled feature map, which is then sent to the second stitching layer of the third fusion module. The second stitching layer of the third fusion module receives the second upsampled feature map and the enhanced shallow feature map output from the high-resolution branch, and stitches them together to obtain the second fused feature map. Finally, the third C2f module extracts features from the second fused feature map to obtain the third feature map, and outputs it to the first sub-detection head for target detection. The first, second, and third C2f modules can be the same, or they can be differentiated according to actual needs; this embodiment does not impose such limitations.

[0050] It is understood that, by introducing a high-resolution branch at the second level of the backbone network and combining it with a pruned unidirectional feature pyramid network, the embodiments of this application can enable shallow spatial information and deep semantic information to complement each other, ensuring that the spatial information of extremely small targets is not diluted by deep semantics, thereby enhancing the ability of the illegal land use identification model to perceive targets of different scales. Then, by combining multiple parallel sub-detection heads, the ability of the illegal land use identification model to identify small illegal land use can be improved, thereby improving the detection accuracy of illegal land use.

[0051] In some embodiments, the target detection method for identifying illegal land use based on UAV imagery provided in this application further includes: acquiring a training sample set; and, based on the training sample set, training the initial illegal land use identification model in stages by combining a loss function, a first weight, and a second weight until the initial illegal land use identification model converges, thereby obtaining the target illegal land use identification model.

[0052] In some embodiments, the training sample set includes target annotation information and multiple sample images; the multiple sample images include positive sample images and negative sample images, where positive sample images refer to images that include illegal areas and negative sample images refer to images that do not include illegal areas; the combined loss function includes the complete intersection-union loss function, the distributed focus loss function, and the binary cross-entropy loss function.

[0053] In some embodiments, a training sample set is used to train and optimize an initial illegal land use identification model, including multiple sample images and target annotation information corresponding to each sample image. The target annotation information includes at least a target category label and a target bounding box. The multiple sample images include positive sample images and negative sample images. Positive sample images refer to images containing at least one illegal area, and illegal targets include, but are not limited to, stockpiles of materials, borrow pits, makeshift guardhouses, and hardened ground. Negative sample images refer to images that do not contain any illegal areas and are used to simulate challenging background scenarios with high confusion.

[0054] In some embodiments, historical data on illegal land parcels can be obtained from local natural resources authorities, and corresponding image slices can be extracted from this data to obtain positive sample images. Legal land features that are visually similar to the illegal targets can be collected, such as regular hay bales in farmland, legal threshing grounds or farm roads, and dried-up natural ditches, to obtain negative sample images. Since the negative sample images do not include the illegal areas, only the positive sample images are labeled to obtain target annotation information.

[0055] In some embodiments, to focus on small targets that are difficult to detect, the number of positive sample images containing illegal areas of small targets can be increased.

[0056] In some embodiments, the full intersection-union loss function is used to determine the degree of overlap between the predicted bounding boxes and the ground truth bounding boxes output by the initial illegal land use identification model. By determining the ratio of the intersection to the union of the predicted and ground truth bounding boxes, the accuracy of the initial illegal land use identification model's predictions is determined. The distributed focus loss function is used to optimize the probability distribution of the category predictions to improve the adaptability of the initial illegal land use identification model to complex scenarios. The binary cross-entropy loss function is used to determine the difference between the predicted category labels and the ground truth labels output by the initial illegal land use identification model. By minimizing this loss, the accuracy of the initial illegal land use identification model in determining whether a target area belongs to the illegal land use category can be improved. The first weight and the second weight are both preset values ​​and can be set according to actual needs; this embodiment does not limit them.

[0057] In some embodiments, the phased training of the initial illegal land use identification model based on the training sample set by combining a loss function, a first weight, and a second weight includes: freezing the parameters of the backbone network and the target parameters of the detection head in the initial illegal land use identification model; performing a first-stage training on the initial illegal land use identification model based on the training sample set to obtain first predicted annotation information; determining a first total loss value between the first predicted annotation information and the target annotation information by combining a loss function, a first weight, and a second weight, until the first total loss value converges to obtain the first illegal land use identification model; unfreezing all parameters of the first illegal land use identification model; performing a second-stage training on the first illegal land use identification model based on the training sample set to obtain second predicted annotation information; and determining a second total loss value between the second predicted annotation information and the target annotation information by combining a loss function, a first weight, and a second weight, until the second total loss value converges to obtain the target illegal land use identification model.

[0058] In some embodiments, during the first stage of training, freezing some parameters of the backbone network and the detection head enables the initial illegal land use identification model to learn the basic features and target distribution in the sample images, avoiding premature entrapment in local optima and obtaining the first illegal land use identification model. During the second stage of training, all parameters of the first illegal land use identification model are unfrozen, allowing the model to comprehensively adjust and optimize all parameters based on training feedback.

[0059] In some embodiments, the total loss value is determined using the following formula 1. The total loss value includes a first total loss value and a second total loss value, which can be determined by the following formula 1.

[0060] (Formula 1) in, The label for the detection head. For the first sub-detection head, For the second sub-detection head, For the third sub-detection head, For the first The complete intersection-union loss corresponding to each detection head For the first The distributed focal loss corresponding to each detection head; For the first The binary cross-entropy loss corresponding to each detector head. As the first weight, It is the second weight.

[0061] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of the invention. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of the invention, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the invention. The sequence numbers of the above-described embodiments of the invention are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0062] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0063] In the several embodiments provided by this invention, it should be understood that the disclosed methods can be implemented in other ways. The methods disclosed in the several method embodiments provided by this invention can be arbitrarily combined without conflict to obtain new method embodiments. The features disclosed in the several method embodiments provided by this invention can be arbitrarily combined without conflict to obtain new method embodiments.

[0064] The above description is merely an embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for target detection and illegal land use identification based on UAV imagery, characterized in that, The method includes: Acquire digital orthophotos and digital surface models of the target area; The historical legal land use data corresponding to the target area are subjected to spectral difference processing and elevation change detection processing with digital orthophoto maps and digital surface models to obtain a set of suspected change areas; the set of suspected change areas includes at least one suspected change area. The set of suspected change areas is input into the target illegal land use identification model to obtain the identification results; the identification results include the category label, bounding box coordinates and confidence score of each suspected change area; The target illegal land use identification model consists of a backbone network, a neck network, and a detection head connected in sequence. The backbone network is obtained by retaining the first to fourth levels of the original backbone network in the YOLOv8-P2 model and drawing high-resolution branches at the second level. The neck network is obtained by replacing the original neck network in the YOLOv8-P2 model with a pruned unidirectional feature pyramid network. The detection head is obtained by replacing the original detection head in the YOLOv8-P2 model with multiple parallel sub-detection heads. The high-resolution branch is used to enhance the high-resolution detail features in the shallow feature map extracted by the backbone network, resulting in an enhanced shallow feature map; the pruned unidirectional feature pyramid network is used to fuse the enhanced shallow feature map with the deep feature map extracted by the backbone network from top to bottom.

2. The method according to claim 1, characterized in that, By performing spectral difference processing and elevation change detection on the historical legal land use data corresponding to the target area, digital orthophoto maps, and digital surface models, a set of suspected change areas is obtained, including: The vector base map in the historical legal land use data is rasterized to obtain classified images; the classified images have the same resolution and coordinate system as the digital orthophoto map. The spectral difference map between the digital orthophoto map and the classification image is determined by weighted Euclidean distance method, and the digital surface model is processed by Gaussian filtering to generate a height variation mask. A pixel-level logical AND operation is performed between the spectral difference map and the height variation mask to obtain a binarized candidate region mask. If the area corresponding to the binarized candidate region mask is greater than the preset area threshold, the area corresponding to the binarized candidate region mask is determined as a suspected change region, and a set of suspected change regions is constructed based on the suspected change regions.

3. The method according to claim 1, characterized in that, The target illegal land use identification model is used to perform the following operations: Receive a set of suspected changed regions; Feature extraction is performed on each suspected change region in the suspected change region set through the first to fourth layers of the backbone network to obtain a multi-scale feature map; the multi-scale feature map includes at least a shallow feature map output by the second layer, a medium feature map output by the third layer, and a deep feature map output by the fourth layer. The high-resolution branch enhances the high-resolution detail features in the shallow feature map, resulting in an enhanced shallow feature map. The enhanced shallow, middle and deep feature maps are fused from top to bottom using the neck network to obtain the fused feature map. The detection head performs target detection on feature maps of different scales in the fused feature map to obtain the detection results.

4. The method according to claim 1 or 3, characterized in that, The high-resolution branch includes a first convolutional layer, a depthwise separable convolutional layer, and a second convolutional layer; High-resolution branches are used to perform the following operations: The first convolutional layer compresses the number of channels in the shallow feature map to a preset dimension, resulting in a compressed feature map. Spatial features are extracted from the compressed feature map by depthwise separable convolution to obtain a feature map with enhanced details. The second convolutional layer restores the feature map after detail enhancement to the same number of channels as the shallow feature map, resulting in the transformed feature map. The transformed feature map and the shallow feature map are joined by a residual connection to obtain the enhanced shallow feature map.

5. The method according to claim 1 or 3, characterized in that, The pruned unidirectional feature pyramid network consists of multiple cascaded fusion modules; Pruned one-way feature pyramid networks are used to perform the following operations: It receives the enhanced shallow feature map from the high-resolution branch output, the mid-level feature map from the third-level output, and the deep feature map from the fourth-level output. By using multiple cascaded fusion modules, the enhanced shallow feature map, middle feature map and deep feature map are fused from top to bottom in a unidirectional manner to obtain the fused feature map. The fused feature map is output to the detection head for target detection.

6. The method according to claim 5, characterized in that, The multiple sub-detection heads include a first sub-detection head, a second sub-detection head, and a third sub-detection head; the multiple fusion modules include a first fusion module, a second fusion module, and a third fusion module; the first fusion module includes a first C2f module and a first upsampling module, the second fusion module includes a first stitching layer, a second C2f module, and a second upsampling module, and the third fusion module includes a second stitching layer and a third C2f module; The first fusion module is used to perform the following operations: The first C2f module receives the deep feature map output from the fourth level, extracts features from the deep feature map, obtains the first feature map, and outputs it to the third sub-detection head and the first upsampling module. The first feature map is upsampled by the first upsampling module to obtain the first upsampled feature map, which is then output to the first stitching layer in the second fusion module. The second fusion module is used to perform the following operations: The first splicing layer receives the first upsampled feature map and the middle layer feature map output from the third layer, and splices the first upsampled feature map with the middle layer feature map to obtain the first fused feature map. The second C2f module extracts features from the first fused feature map to obtain the second feature map, which is then output to the second upsampling module and the second sub-detection head. The second feature map is upsampled by the second upsampling module to obtain the second upsampled feature map, which is then output to the second stitching layer in the third fusion module. The third fusion module is used to perform the following operations: The second stitching layer receives the second upsampled feature map and the enhanced shallow feature map output by the high-resolution branch. The second upsampled feature map and the enhanced shallow feature map are then stitched together to obtain the second fused feature map. The third C2f module extracts features from the second fused feature map to obtain the third feature map, which is then output to the first sub-detection head.

7. The method according to claim 1, characterized in that, The method further includes: Obtain a training sample set; the training sample set includes target annotation information and multiple sample images; the multiple sample images include positive sample images and negative sample images, positive sample images refer to images that include illegal areas, and negative sample images refer to images that do not include illegal areas; Based on the training sample set, the initial illegal land use identification model is trained in stages by combining the loss function, the first weight, and the second weight until the initial illegal land use identification model converges, thus obtaining the target illegal land use identification model. The combined loss function includes the complete intersection-union loss function, the distribution focus loss function, and the binary cross-entropy loss function.

8. The method according to claim 7, characterized in that, Based on the training sample set, the initial illegal land use identification model is trained in stages by combining the loss function, the first weight, and the second weight, including: Freeze the parameters of the backbone network and the target parameters of the detection head in the initial illegal land use identification model, and perform the first stage training of the initial illegal land use identification model based on the training sample set to obtain the first prediction annotation information; By combining the loss function, the first weight, and the second weight, the first total loss value between the first predicted annotation information and the target annotation information is determined until the first total loss value converges, thus obtaining the first illegal land use identification model. Unfreeze all parameters of the first illegal land use identification model, and conduct a second-stage training on the first illegal land use identification model based on the training sample set to obtain the second prediction annotation information; By combining the loss function, the first weight, and the second weight, the second total loss value between the second predicted annotation information and the target annotation information is determined until the second total loss value converges, thus obtaining the target illegal land use identification model.

9. The method according to claim 8, characterized in that, The total loss value is determined by the following formula; the total loss value includes a first total loss value and a second total loss value. in, The label for the detection head. For the first sub-detection head, For the second sub-detection head, For the third sub-detection head, For the first The complete intersection-union loss corresponding to each detection head For the first The distributed focal loss corresponding to each detection head; For the first The binary cross-entropy loss corresponding to each detector head. As the first weight, It is the second weight.

Images