Remote sensing image cattle and sheep detection method and device based on edge enhancement and detail focusing

Abstract

The embodiment of the invention provides a remote sensing image cattle and sheep detection method and device based on edge enhancement and detail focusing, and is applied to the technical field of remote sensing images. The method comprises the steps of obtaining a to-be-processed natural pasture remote sensing image data set including cattle and sheep targets, processing the remote sensing image data set through a pre-trained remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focusing, and generating a natural pasture cattle and sheep identification data set; wherein a detail feature focusing diffusion pyramid network DFDPN and an edge enhancement module EEM are arranged in the NRSDet. In this way, the EEM enhances the edge information in the image, so that the model can more accurately capture the contour and details of the target when recognizing the target, the DFDPN further improves the sensitivity of the model to local details through multi-level feature focusing, and the EEM and the DFDPN cooperate with each other, so that not only is the target detection precision improved, but also the detection accuracy is improved. And the adaptive capacity of the model in a complex scene is also enhanced.

Description

Technical Field

[0001] This disclosure relates to the field of image processing technology, and more particularly to the field of remote sensing image technology, specifically to a method and apparatus for detecting cattle and sheep in remote sensing images based on edge enhancement and detail focusing. Background Technology

[0002] Natural pasture management is crucial for ecological protection and food resource supply. Traditional pasture management relied on manual ground monitoring, which was inefficient and difficult to implement large-scale real-time data analysis. In recent years, remote sensing technology, especially UAV remote sensing technology, has provided new tools for pasture management. Using UAV aerial imagery for cattle and sheep target detection can significantly reduce the input of manpower and resources, supporting applications such as cattle and sheep counting and intelligent grazing.

[0003] However, in natural pasture environments, the detection of cattle and sheep targets in remote sensing images also faces many technical challenges, including: dense target distribution: the large number and dense distribution of cattle and sheep in natural pastures cause mutual occlusion between targets, making them difficult to distinguish; complex background: the background of natural pastures is complex and variable, and factors such as lighting, vegetation, and terrain interfere with target recognition; small target size: in UAV remote sensing images, cattle and sheep targets are relatively small in size, lacking detailed information, making accurate detection difficult; limited computing resources: edge computing devices such as UAVs have limited computing power, requiring lightweight target detection models.

[0004] Deep learning-based object detection technology has been widely applied in fields such as medical image recognition, dense crowd detection, and remote sensing object recognition. However, directly applying existing object detection models to cattle and sheep detection in natural pastures often fails to achieve ideal results due to the aforementioned technical issues.

[0005] Currently, some studies have attempted to use deep learning for cattle and sheep target detection. These approaches are mainly based on the following types of target detection models: Two-stage object detection models (such as Faster R-CNN): These models first generate candidate regions, and then classify and regress the candidate regions. Although they have high accuracy, they are computationally intensive and cannot meet the real-time requirements of edge computing devices such as drones. Single-stage object detection models (such as SSD and YOLO): These models directly detect the object category and bounding box position, which is fast, but the detection accuracy for small and dense objects is relatively low. Transformer-based object detection models (such as DETR): These models utilize attention mechanisms to capture the relationships between elements within a sequence, giving them an advantage in handling long-range dependencies. However, they are computationally intensive and difficult to widely apply under current hardware conditions.

[0006] In addition, some studies have improved existing object detection models to address the problems of small and dense object detection, for example: Add a small target detection layer and construct a focus attention mechanism that fuses features: designed to focus on small targets and densely distributed targets; A multi-task joint learning object detection network is constructed, integrating a convolutional attention mechanism: while denoising, it utilizes local and global information of the target in the image.

[0007] Expanding the receptive field using dilated convolution: This aims to leverage long-range global contextual information.

[0008] However, existing two-stage target detection models suffer from high computational cost and slow speed due to the need to generate a large number of candidate regions and perform multiple calculations, making it difficult to meet the real-time requirements of edge computing devices such as drones. Existing single-stage target detection models suffer from low detection accuracy and are prone to false negatives and missed detections due to a lack of targeted optimization for small and dense targets. Existing Transformer-based target detection models suffer from high computational cost due to the exponential growth of computational complexity with sequence length, making them difficult to widely apply under current hardware conditions.

[0009] In summary, while existing improvement schemes have improved the detection accuracy of small and dense targets to some extent, they often ignore the local small features of the targets in the image, especially the boundary features. As a result, the detection effect is still not ideal when the background is complex, the target size is small and the distribution is dense. Summary of the Invention

[0010] This disclosure provides a method, apparatus, device, and storage medium for detecting cattle and sheep in remote sensing images based on edge enhancement and detail focusing.

[0011] According to a first aspect of this disclosure, a method for detecting cattle and sheep in remote sensing images based on edge enhancement and detail focusing is provided. The method includes: Obtain a dataset of remote sensing images of natural pastures containing two types of targets: cattle and sheep; The remote sensing image dataset is processed by the pre-trained remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focusing to generate a natural pasture cattle and sheep identification dataset. The remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focusing includes a detail feature focusing diffusion pyramid network DDPN and an edge enhancement module EEM.

[0012] In addition to the aspects and any possible implementations described above, a further implementation is provided, wherein the Detail Feature Focused Diffusion Pyramid Network (DFDPN) includes a multi-scale detail feature extraction module, a detail enhancement convolution (DEConv), and a feature diffusion mechanism; the multi-scale detail feature extraction module is used to fuse feature inputs of multiple different scales to generate a multi-scale feature map, the detail enhancement convolution (DEConv) is used to extract the multi-scale feature map to obtain a cross-scale detail feature map, and the feature diffusion mechanism is used to diffuse the cross-scale detail feature map to various detection scales.

[0013] In addition to the aspects described above and any possible implementations, a further implementation is provided in which the upsampling part of the multi-scale detail feature extraction module uses bilinear interpolation, and the downsampling part uses an ADown structure.

[0014] In addition to the aspects described above and any possible implementations, a further implementation is provided in which the edge enhancement module (EEM) includes an edge feature extraction module and a spatial feature extraction module; the edge feature extraction module uses SobelConv to extract image edge information, and the spatial feature extraction module is used to extract image spatial information.

[0015] In addition to the aspects and any possible implementations described above, a further implementation is provided in which the convolution kernel is reparameterized using edge detection operators in both the X and Y axes of SobelConv; The calculation process of the convolution kernel in the X and Y axes includes: The output of SobelConv includes: in, This represents the calculation of the convolution kernel in the X-axis direction. This indicates the calculation of the convolution kernel in the Y-axis direction. This means that the feature information obtained in the two directions will be added together at corresponding positions. This represents the final edge features of the output.

[0016] In addition to the aspects and any possible implementations described above, a further implementation is provided in which the remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focusing uses MobileNetV3 as the backbone network.

[0017] In addition to the aspects and any possible implementations described above, a further implementation is provided in which the remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focusing also includes a bounding box regression loss function MPDIoU based on minimum point distance; The bounding box regression loss function MPDIoU based on minimum point distance includes: Where H and W represent the image size, Represents the coordinates of the true bounding box. Indicates the coordinate values ​​of the predicted bounding box. , This represents the Euclidean distance between the coordinates of the ground truth bounding box and the predicted bounding box.

[0018] According to a second aspect of this disclosure, a remote sensing image cattle and sheep detection device based on edge enhancement and detail focusing is provided. The device includes: The acquisition module is used to acquire a dataset of remote sensing images of natural pastures that include two types of targets: cattle and sheep. The processing module is used to process the remote sensing image dataset using a pre-trained remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focusing, to generate a natural pasture cattle and sheep identification dataset; wherein, the remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focusing includes a detail feature focusing diffusion pyramid network DDPN and an edge enhancement module EEM.

[0019] According to a third aspect of this disclosure, an electronic device is provided. The electronic device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the program to implement the method described above.

[0020] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the method described above.

[0021] This application provides a remote sensing image cattle and sheep detection method based on edge enhancement and detail focusing. It acquires a dataset of remote sensing images of natural pastures containing both cattle and sheep, and then processes this dataset using a pre-trained remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focusing to generate a natural pasture cattle and sheep identification dataset. The NRSDet model includes a feature diffusion pyramid network (DFDPN) and an edge enhancement module (EEM). The EEM in NRSDet enhances edge information in the image, enabling the model to more accurately capture the contours and details of the target. Meanwhile, the DFDPN in NRSDet further improves the model's sensitivity to local details through multi-level feature focusing. The synergistic effect of these two components not only improves the accuracy of target detection but also enhances the model's adaptability in complex scenes.

[0022] It should be understood that the description in the Summary of the Invention is not intended to limit the key or essential features of the embodiments of this disclosure, nor is it intended to restrict the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0023] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. The drawings are provided for a better understanding of the invention and are not intended to limit the scope of this disclosure. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein: Figure 1 A flowchart of a remote sensing image cattle and sheep detection method based on edge enhancement and detail focusing according to embodiments of the present disclosure is provided; Figure 2 A schematic diagram of the overall structure of the NRSDet model according to an embodiment of the present disclosure is shown; Figure 3 A schematic diagram of the structure of the multi-scale detail feature extraction module according to an embodiment of the present disclosure is shown; Figure 4 A schematic diagram illustrating the calculation method of a special convolution kernel according to an embodiment of the present disclosure is provided; Figure 5 A schematic diagram of the edge enhancement module (EEM) structure according to an embodiment of the present disclosure is shown; Figure 6 A schematic diagram showing a comparison of heatmaps of the output features of the model before and after the improvement according to embodiments of the present disclosure is provided. Figure 7 A comparative schematic diagram of various detection methods for detecting the results of natural pasture cattle according to embodiments of the present disclosure is shown. Figure 8 A comparative schematic diagram of various detection methods for detecting sheep in natural pastures according to embodiments of the present disclosure is shown. Figure 9 A block diagram of a remote sensing image cattle and sheep detection device based on edge enhancement and detail focusing according to an embodiment of the present disclosure is shown; Figure 10 A block diagram of an exemplary electronic device capable of implementing embodiments of the present disclosure is shown. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0025] Furthermore, the term "and / or" in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0026] In this disclosure, EEM in NRSDet enhances edge information in the image, enabling the model to more accurately capture the contours and details of the target when recognizing it. Meanwhile, DDPPN in NRSDet further enhances the model's sensitivity to local details through multi-level feature focusing. The synergistic effect of these two not only improves the accuracy of target detection but also enhances the model's adaptability in complex scenes.

[0027] Figure 1 A flowchart of a remote sensing image cattle and sheep detection method 100 based on edge enhancement and detail focusing according to an embodiment of the present disclosure is shown.

[0028] In box 110, obtain the dataset of remote sensing images of natural pastures that include two types of targets: cattle and sheep.

[0029] In some embodiments, based on the complexity of the natural pasture environment, a dataset of remote sensing images of natural pastures containing two types of targets, cattle and sheep, can be selected as the detection targets to be processed.

[0030] In box 120, the remote sensing image dataset is processed by the pre-trained remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focus to generate a natural pasture cattle and sheep identification dataset; wherein, the remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focus includes a detail feature focus diffusion pyramid network DDPN and an edge enhancement module EEM.

[0031] In some embodiments, the above-mentioned remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focusing uses MobileNetV3 as the backbone network.

[0032] In some embodiments, considering the limited computing power of edge computing devices on UAVs, a lightweight target detection model is required. Therefore, MobileNetV3 can be selected as the backbone structure of the model. Based on this, a remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focusing is proposed. The improved model structure is as follows: Figure 2 As shown.

[0033] It is evident that using MobileNetV3 as the backbone network achieves a lightweight backbone network, reducing computational load and model parameter count while maintaining accuracy.

[0034] In some embodiments, NRSDet employs a Detail Feature Focusing Diffusion Pyramid Network (DFDPN) in its decoding section. This network ensures that features at each scale possess detailed information. Within the DFDPN structure, the detail feature focusing module can accept feature inputs from multiple scales, such as three scales, and captures rich, multi-scale detailed features through the Detail Enhancement Convolution (DEConv). Simultaneously, NRSDet also incorporates an EEM structure, which uses the Sobel edge operator to reparameterize the convolutional kernels, enhancing the model's ability to extract boundary information from image targets and improving the identification of cattle and sheep targets in densely distributed environments.

[0035] In some embodiments, the use of the Detail Feature Focused Diffusion Pyramid Network (DFDPN) can fuse contextual information from multiple scales, thereby improving the NRSDet model's ability to detect targets at different scales.

[0036] In some embodiments, the aforementioned Detail Feature Focused Diffusion Pyramid Network (DFDPN) includes a multi-scale detail feature extraction module, a detail enhancement convolution (DEConv), and a feature diffusion mechanism. The multi-scale detail feature extraction module is used to fuse feature inputs of multiple different scales to generate multi-scale feature maps. The detail enhancement convolution (DEConv) is used to extract cross-scale detail feature maps from the multi-scale feature maps. The feature diffusion mechanism is used to diffuse the cross-scale detail feature maps to various detection scales.

[0037] In some embodiments, to enhance the NRSDet model's ability to extract detailed information, especially the detailed information of small targets such as cattle and sheep in drone scenarios, the decoding part of the model designs a detailed feature aggregation and diffusion pyramid structure based on the multi-scale feature pyramid. This structure mainly fuses feature maps of three different scales through a multi-scale detailed feature extraction module to generate a multi-scale feature map, then obtains detailed information of local regions through detail enhancement convolution (DEConv) to obtain a cross-scale detailed feature map, and finally diffuses the obtained detailed information to various detection scales through a feature diffusion mechanism to better facilitate subsequent target detection and classification. The structure of the multi-scale detailed feature extraction module is as follows: Figure 3 As shown.

[0038] In some embodiments, the upsampling part of the multi-scale detail feature extraction module uses bilinear interpolation, and the downsampling part uses an ADown structure.

[0039] In some embodiments, the multi-scale detail feature extraction module receives feature maps of different scales, so scale transformation is required for the input feature maps of different sizes. Based on this, the upsampling part of the multi-scale detail feature extraction module uses bilinear interpolation, and the downsampling part uses an ADown structure, in order to reduce the number of parameters while preserving image information, thereby reducing model complexity and improving model efficiency.

[0040] In some embodiments, the ADown structure first performs average pooling on the input feature map, reducing the spatial dimension of the feature map while retaining important spatial features. Then, the pooled feature map is divided into two parts, x1 and x2. Convolution is performed on x1 to extract deeper features, and 3×3 max pooling is performed on x2 to further reduce the spatial dimension of the feature map while retaining the most significant features. After convolution is performed on x2, it is finally concatenated with x1 in the channel dimension to obtain the final output result.

[0041] In some embodiments, the above operations can be used to obtain feature maps of the same scale, which are then concatenated along the channel dimension and subjected to detail enhancement convolution (DEConv) to obtain multi-scale local detail information.

[0042] In some embodiments, detail-enhancing convolution combines ordinary convolution and special convolution. The special convolutions include four types: central difference convolution, corner difference convolution, horizontal difference convolution, and vertical difference convolution. These four types of convolutions are deployed in parallel to enhance the network's ability to represent and generalize features. In this way, the network can capture more comprehensive detail information in the image. This method not only improves the efficiency of feature extraction but also enhances the model's performance in object detection tasks. The calculation method of the special convolution kernel is as follows... Figure 4 As shown.

[0043] In some embodiments, the weights obtained during feature extraction are added to the weights obtained during convolution at corresponding positions. This addition operation not only preserves the information of the original features but also enhances the expression of detailed information, enabling the model to capture important details and features more comprehensively when processing images.

[0044] In some embodiments, the edge enhancement module (EEM) includes an edge feature extraction module and a spatial feature extraction module; the edge feature extraction module uses SobelConv to extract image edge information, and the spatial feature extraction module is used to extract image spatial information.

[0045] like Figure 5 As shown, the edge enhancement module EEM extracts edge information from the image through the SobelConv branch. In SobelConv, the convolution kernel is reparameterized using edge detection operators in the X and Y axes to effectively capture sudden changes in intensity in the image, thereby obtaining important boundary information.

[0046] In some embodiments, the SobelConv described above reparameterizes the convolution kernel using edge detection operators in the X and Y axes. The calculation process of the convolution kernel in the X and Y axes includes: ; ; The output of SobelConv above includes: ; in, This represents the calculation of the convolution kernel in the X-axis direction. This indicates the calculation of the convolution kernel in the Y-axis direction. This means that the feature information obtained from the two directions will be added together at corresponding positions to obtain the final edge features. , This represents the final edge features of the output.

[0047] In some embodiments, in addition to the extraction of image edge information as described above, the spatial information in the original image is also important. Therefore, while extracting the target edge information, a convolutional kernel is also used to extract the original image features to preserve rich spatial details. Finally, the edge features are concatenated with the original spatial features, and then a convolution is performed for feature fusion.

[0048] In some embodiments, a regular convolution can be defined as Then, the output features after feature concatenation can be obtained as follows: ; in, This indicates the output features after concatenation. H and W represent spatial features, and H and W represent image size.

[0049] In some embodiments, the output features can be processed by convolution and then residually connected with the original input feature map to obtain the final output features as shown below: .

[0050] In some embodiments, the above-described connection method can capture edge feature information while minimizing the loss of the original feature information.

[0051] In some embodiments, the edge enhancement module optimizes the convolution kernel parameters through a reparameterization method. This module reconstructs the convolution kernel to make it more sensitive to capturing edge features in the image while maintaining its original feature extraction capabilities.

[0052] It should be noted that the reparameterization process not only effectively focuses on edge information but also reduces the number of model parameters through parameter sharing, thereby lowering computational complexity. This design enables the edge enhancement module to operate efficiently in resource-constrained environments, making it suitable for real-time image processing applications.

[0053] In some embodiments, using the detail enhancement convolution (DEConv) and edge information enhancement module (EEM) can improve the model's target recognition accuracy by enhancing detail features and edge information, especially in complex backgrounds and dense target scenes.

[0054] In some embodiments, the above-mentioned remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focusing also includes a bounding box regression loss function MPDIoU based on minimum point distance; The bounding box regression loss function MPDIoU based on minimum point distance mentioned above includes: ； ; Where H and W represent the image size, Represents the coordinates of the true bounding box. Indicates the coordinate values ​​of the predicted bounding box. , This represents the Euclidean distance between the coordinates of the ground truth bounding box and the predicted bounding box.

[0055] In some embodiments, considering that the location of the detection box is more important for target recognition when the target is densely distributed, a bounding box regression loss function based on minimum point distance (MPDIoU) can be introduced. This function integrates the center distance between the predicted box and the ground truth box, the overlapping area, and the actual deviation of the width and height. It calculates the distance between the upper left and lower right points of the predicted box and the ground truth box, and continuously optimizes this value to achieve the goal of regressing the position of the predicted box.

[0056] In some embodiments, assuming image sizes H and W, and the coordinates of the ground truth bounding box and the predicted bounding box are respectively... First, the Euclidean distance between the ground truth bounding box and the predicted bounding box coordinates can be calculated, as shown below: ; ; The MDPIOU is then calculated by combining the areas where the ground truth bounding box and the predicted bounding box intersect and do not intersect.

[0057] In some embodiments, MPDIoU provides a more flexible and efficient way to evaluate the matching degree of bounding boxes by simplifying the similarity comparison between two bounding boxes. This approach can not only adapt to the regression of overlapping bounding boxes, but also effectively handle the case of non-overlapping bounding boxes, thereby improving the adaptability and accuracy of the model in various scenarios.

[0058] In some embodiments, by comprehensively considering the center distance of the bounding boxes, the overlapping area, and the actual deviations in width and height, MPDIoU can more comprehensively reflect the relationship between the predicted box and the ground truth box, making the bounding box regression process more robust and effective.

[0059] In some embodiments, using the minimum point distance bounding box regression loss MPDIoU can improve the detection capability for dense targets and reduce the problems of missed detections and overlapping detection boxes.

[0060] According to the embodiments of this disclosure, the following technical effects are achieved: By acquiring a dataset of remote sensing images of natural pastures containing two types of targets—cattle and sheep—and then processing it using a pre-trained remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focusing, a natural pasture cattle and sheep identification dataset is generated. The NRSDet model includes a feature diffusion pyramid network (DFDPN) and an edge enhancement module (EEM). The EEM in NRSDet enhances edge information in the images, enabling the model to more accurately capture the contours and details of targets. Meanwhile, the DFDPN in NRSDet further improves the model's sensitivity to local details through multi-level feature focusing. The synergistic effect of these two components not only improves the accuracy of target detection but also enhances the model's adaptability in complex scenes.

[0061] In some embodiments, to verify the effectiveness of the proposed algorithm, this disclosure also constructs a dataset of natural pasture images from the perspective of drone aerial photography, including two types of targets: cattle and sheep, and demonstrates the effectiveness of the algorithm through technical verification.

[0062] In some embodiments, the dataset contains 3124 orthophotos of size 640×640, divided into a training set of 2560 images, a test set of 360 images, and a validation set of 204 images.

[0063] In some embodiments, technical verification includes ablation experiments and comparative experiments.

[0064] In some embodiments, the experimental platform may use the Ubuntu 16.04 operating system, two Nvidia Tesla P100 GPUs with 16GB of video memory each, an Intel(R) Core i7-12700H CPU, and Python 3.7 and PyTorch 2.0.1 as the programming environment.

[0065] In some embodiments, the experiment can select SGD as the optimizer, with momentum set to 0.937, initial learning rate set to 0.0007, weight decay set to 0.2, batch size set to 4, training for 300 epochs, and training stopping when the accuracy no longer increases within 100 epochs during training.

[0066] In some embodiments, to verify the accuracy of the model disclosed herein in target detection of cattle and sheep in UAV aerial images, the mean average precision (mAP) can be used as an evaluation metric, and the average precision (AP) can be used as a single-class detection evaluation metric. The specific mAP calculation is as follows: ; Where c represents the number of categories of the detected target, and i is the category index.

[0067] The AP value is calculated using the PASCAL VOC12 method. For the curve formed by precision and recall, continuous recall values ​​are taken to calculate the mean precision. The calculation methods for precision (P) and recall (R) are as follows: ; ; Wherein, TP, FP, and FN represent the number of correctly detected samples, the number of incorrectly detected samples, and the number of missed samples, respectively.

[0068] In some embodiments, to verify the effectiveness of this disclosure, ablation experiments were conducted on a constructed natural pasture cattle and sheep identification dataset. The baseline model was Mobilenetv3, and the comparison results are shown in Table 1, where “√” indicates that the module was selected.

[0069] Table 1: Results of the ablation experiment In some embodiments, adding an edge enhancement module (EEM) to the model improves the average accuracy from 95.81% to 96.07%, while reducing the number of parameters and the model size compared to the original model structure, and increasing the FPS from 152.1 to 158.6. Replacing the original feature pyramid structure in the decoding part with a detail feature focusing diffusion pyramid network (DFDPN) improves the average accuracy from 95.81% to 96.43%, with only a small increase in model parameters and a slight sacrifice in computational speed, but a significant improvement in accuracy. When MPDIoU is introduced alone on top of the original model, the average accuracy improves to 96.23%. When both the self-developed edge enhancement module (EEM) and the detail feature focusing diffusion pyramid network (DFDPN) are used simultaneously, the model's average accuracy reaches 97.31%, without a significant increase in the number of parameters, fully demonstrating that both improvements enhance the model's detection performance.

[0070] At the same time, such as Figure 6 The heatmap comparison of the output features of the model before and after the improvement is shown. In the heatmaps before improvement (a) and after improvement (b), the temperature regions clearly indicate the degree of attention the model pays to different areas in the image. The comparison results show that the improved model significantly enhances its ability to extract target boundary features, demonstrating a stronger ability to focus on local target feature information. This phenomenon fully demonstrates that the combination of the Edge Enhancement Module (EEM) and the Detail Feature Focusing Diffusion Pyramid Network (DFDPN) provides the model with a more effective feature extraction method.

[0071] Specifically, the Edge Enhancement Module (EEM) enhances edge information in the image, enabling the NRSDet model to more accurately capture the contours and details of targets when recognizing them. Meanwhile, the Detail Feature Focusing Diffusion Pyramid Network (DFDPN) further improves the NRSDet model's sensitivity to local details through multi-layered feature focusing. The synergistic effect of these two components not only improves the accuracy of target detection but also enhances the model's adaptability in complex scenes.

[0072] In some embodiments, to verify the advancement of the proposed method, comparative experiments were conducted with various advanced methods on the constructed natural pasture cattle and sheep identification dataset. The experimental results are shown in Table 2.

[0073] Table 2: Results of the comparative experiment In some embodiments, the method of this disclosure achieves the highest average accuracy of 97.45% for cattle and sheep. For cattle alone, the method of this invention does not reach the highest average accuracy. This may be because cattle are large targets with less dense distribution, and the method of this disclosure does not demonstrate the highest superiority in the detection of such targets. However, except for YOLOv5, the method of this invention is still superior to other compared methods.

[0074] like Figure 7 The results presented here demonstrate the detection of cattle in natural pastures, and are compared with several state-of-the-art detection methods. The comparison images clearly show that Fast R-CNN and YOLOv7 exhibit a significant number of missed detections, indicating insufficient robustness in handling this type of target. Furthermore, Fast R-CNN and DETR models have significantly more parameters than other models, resulting in higher computational resource consumption. However, on the dataset constructed in this publication, the detection accuracy of these two models does not show a significant advantage over other methods, highlighting their limitations in specific application scenarios.

[0075] Similarly, Figure 8 The images shown here are partial comparisons of the detection results for sheep in natural pastures with those of other methods. Fast R-CNN exhibits many overlapping bounding boxes in scenes with densely distributed sheep flocks. This may be due to the fact that it does not use Non-Maximum Suppression (NMS) compared to other single-stage methods.

[0076] Compared to targets like cows, targets like sheep differ significantly from their surroundings and are often densely distributed. Various methods have achieved relatively high detection accuracy on the constructed datasets. Except for Fast R-CNN, the other methods and the NRSDet model do not differ much in the actual visual effect of the detection results, with only some deviations in the position of the predicted boxes.

[0077] In summary, the lightweight design of this disclosure is more suitable for real-time UAV detection. For example, MobileNetv3 reduces the number of parameters by 40% compared to YOLOv5 (see Table 2), meeting the requirements of edge computing. The multi-scale feature fusion in this disclosure can improve the accuracy of dense target detection; for example, DFDPN achieves an mAP of 97.49%, exceeding YOLOv5's 97.33% (see Table 2). The EEM edge enhancement module in this disclosure can extract the boundary information of image targets, thus reducing the number of model parameters and computational complexity. The MPDIoU loss in this disclosure can reduce detection box overlap; for example, compared to Fast R-CNN, the overlap rate of sheep detection is reduced by 60% (see Table 2). Figure 7 ).

[0078] Overall, this disclosure achieves lightweight model while ensuring detection accuracy, making it suitable for cattle and sheep target detection in remote sensing images of natural pastures and providing support for applications such as intelligent grazing.

[0079] Furthermore, the core idea of ​​this disclosure is an enhanced extraction method based on target edge and detail information. Existing alternatives include other Transformer-based target detection methods, such as DETR, or methods utilizing more complex multi-task learning to improve accuracy. However, these alternatives may not achieve the same level of computational efficiency or detection accuracy as this invention. Therefore, although other methods can accomplish certain objectives, the solution disclosed in this invention has unique advantages in both efficiency and accuracy.

[0080] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this disclosure.

[0081] The above is an introduction to the method embodiments. The following describes the solution described in this disclosure further through device embodiments.

[0082] Figure 9A block diagram of a remote sensing image cattle and sheep detection device 900 based on edge enhancement and detail focusing according to an embodiment of the present disclosure is shown. Figure 9 As shown, the device 900 includes: Module 910 is used to acquire a dataset of remote sensing images of natural pastures, including two types of targets: cattle and sheep. The processing module 920 is used to process the remote sensing image dataset using the pre-trained remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focus, to generate a natural pasture cattle and sheep identification dataset; wherein, the remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focus includes a detail feature focusing diffusion pyramid network DDPN and an edge enhancement module EEM.

[0083] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the described module can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0084] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0085] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0086] Figure 10 A block diagram of an exemplary electronic device 1000 capable of implementing embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0087] Electronic device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in ROM 1002 or a computer program loaded into RAM 1003 from storage unit 1008. RAM 1003 may also store various programs and data required for the operation of electronic device 1000. The computing unit 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. I / O interface 1005 is also connected to bus 1004.

[0088] Multiple components in electronic device 1000 are connected to I / O interface 1005, including: input unit 1006, such as keyboard, mouse, etc.; output unit 1007, such as various types of displays, speakers, etc.; storage unit 1008, such as disk, optical disk, etc.; and communication unit 1009, such as network card, modem, wireless transceiver, etc. Communication unit 1009 allows electronic device 1000 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0089] The computing unit 1001 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above, such as method 100. For example, in some embodiments, method 100 may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 1008.

[0090] In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1000 via ROM 1002 and / or communication unit 1009. When the computer program is loaded into RAM 1003 and executed by computing unit 1001, one or more steps of method 100 described above may be performed. Alternatively, in other embodiments, computing unit 1001 may be configured to perform method 100 by any other suitable means (e.g., by means of firmware).

[0091] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0092] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0093] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0094] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including voice input, speech input, or tactile input).

[0095] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0096] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0097] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0098] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for detecting cattle and sheep in remote sensing images based on edge enhancement and detail focusing, characterized in that, The method comprises: acquiring a natural pasture remote sensing image data set including two types of targets of cattle and sheep to be processed; processing the remote sensing image data set by using a pre-trained remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focusing to generate a natural pasture cattle and sheep recognition data set; wherein the remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focusing is provided with a detail feature focusing diffusion pyramid network DFDPN and an edge enhancement module EEM.

2. The method of claim 1, wherein, The detail feature focusing diffusion pyramid network DFDPN comprises a multi-scale detail feature extraction module, a detail enhancement convolution DEConv, and a feature diffusion mechanism; the multi-scale detail feature extraction module is used to fuse a plurality of different scale feature inputs to generate a multi-scale feature map, the detail enhancement convolution DEConv is used to extract the multi-scale feature map to obtain a cross-scale detail feature map, and the feature diffusion mechanism is used to diffuse the cross-scale detail feature map to each detection scale.

3. The method of claim 2, wherein, The up-sampling part of the multi-scale detail feature extraction module uses bilinear interpolation, and the down-sampling part uses an ADown structure.

4. The method of claim 1, wherein, The edge enhancement module EEM comprises an edge feature extraction module and a spatial feature extraction module; the edge feature extraction module uses a SobelConv to extract image edge information, and the spatial feature extraction module is used to extract image spatial information.

5. The method of claim 4, wherein, In the SobelConv, the convolution kernel is re-parameterized by using edge detection operators in X-axis and Y-axis directions; wherein the calculation process of the convolution kernel in the X-axis and Y-axis directions comprises: The output of the SobelConv comprises: wherein, represents a calculation of a convolution kernel in the X-axis direction, represents a calculation of a convolution kernel in the Y-axis direction, represents adding corresponding positions of feature information obtained in both directions, represents a final edge feature of an output.

6. The method of claim 1, wherein, The remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focusing adopts MobileNetV3 as a backbone network.

7. The method according to any one of claims 1 to 6, characterized in that, The remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focusing is also provided with a minimum point distance-based bounding box regression loss function MPDIoU. The minimum point distance-based bounding box regression loss function MPDIoU comprises: wherein H, W represent image size, denotes real box coordinate value, denotes predicted box coordinate value, , denotes Euclidean distance between real box and predicted box coordinate position.

8. A device for detecting cattle and sheep in remote sensing images based on edge enhancement and detail focusing, characterized in that, The method comprises: an acquisition module configured to acquire a natural pasture remote sensing image data set including two types of targets of cattle and sheep to be processed, a processing module configured to process the remote sensing image data set by using a pre-trained remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focusing to generate a natural pasture cattle and sheep recognition data set; wherein the remote sensing image cattle and sheep detection model NRSDet based on edge enhancement and detail focusing is provided with a detail feature focusing diffusion pyramid network DFDPN and an edge enhancement module EEM.

9. An electronic device, comprising: The method comprises: at least one processor; and a memory connected in communication with the at least one processor; the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of any one of claims 1-7.

10. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to execute the method of any one of claims 1-7.

Images