A dynamic detection and early warning method and system for pond reconstruction based on drone images

Abstract

The present invention discloses a dynamic detection and early warning method and system for pond reconstruction based on drone images. The method comprises: using a high-resolution camera carried by a drone to perform automated flight missions, obtaining orthophoto video of the pond area and performing preprocessing; using a deep learning model combining Mamba and Vision Transformer (ViT) to process the real-time video stream frame by frame, and realizing real-time segmentation and recognition of the pond area; calculating the number and area of ​​the pond through the real-time segmentation results, and comparing with the data at the previous moment, monitoring the changes in the pond area; according to the set early warning rules, when the pond reconstruction behavior is detected, the system automatically triggers an early warning, and pushes the early warning information to relevant personnel through text messages, emails, etc. The present invention realizes real-time monitoring and early warning of pond reconstruction behavior through the combination of drone imaging technology and deep learning models, improves monitoring efficiency and accuracy, and provides effective technical support for pond management.

Description

Technical Field

[0001] The present invention belongs to the technical field of sea area supervision, and specifically relates to a dynamic detection and early warning method and system for pond reconstruction based on drone images. Background Art

[0002] With the rapid development of the marine economy, pond aquaculture, as an important way to utilize marine resources, has been widely used in coastal areas of my country. Pond aquaculture not only provides abundant aquatic products for local residents, but also drives the development of related industries, becoming an important part of the economy of coastal areas. However, the rapid development of pond aquaculture has also brought some problems, such as illegal transformation of ponds and over-exploitation. These problems not only damage the marine ecological environment, but also affect the sustainable use of marine resources.

[0003] The traditional way of monitoring ponds mainly relies on manual inspections, which has many disadvantages. First, manual inspections are inefficient and difficult to cover large areas of sea. Second, manual inspections are greatly affected by weather, environment and other factors, making it difficult to ensure the continuity and stability of inspections. In addition, the accuracy of manual inspections is also difficult to guarantee, and missed inspections and false inspections are prone to occur. Therefore, the traditional way of monitoring can no longer meet the needs of modern marine resource management.

[0004] In recent years, with the rapid development of drone technology, the application of drones in the field of marine supervision has gradually increased. Drones have the advantages of strong flexibility, wide coverage, and low cost, and can efficiently obtain a wide range of marine image data. However, most of the existing drone image processing technologies focus on the analysis of static images, and the real-time processing and analysis capabilities of dynamic images are weak, which makes it difficult to meet the needs of dynamic detection of pond reconstruction. Although drone technology has shown great potential in marine supervision, its application still faces some challenges. First, the amount of image data obtained by drones is huge, and traditional image processing methods are difficult to process this data efficiently. Secondly, the dynamic changes in the pond area are complex, and traditional image recognition technology is difficult to accurately capture these changes. In addition, the existing drone image processing system lacks real-time and automation capabilities, and it is difficult to meet the needs of dynamic monitoring.

[0005] In order to meet these challenges, deep learning technology has gradually become an effective means to solve complex image processing problems. Deep learning models, such as convolutional neural networks (CNNs) and Transformers, have powerful feature extraction and pattern recognition capabilities and can extract useful information from complex image data. Summary of the invention

[0006] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and to provide a dynamic detection and early warning method and system for pond reconstruction based on drone images.

[0007] In order to achieve the above object, the present invention adopts the following technical solutions: One aspect of the present invention provides a dynamic detection and early warning method for pond reconstruction based on drone images, comprising the following steps: Use drones to obtain orthophoto video of the pond area and perform extraction, noise reduction, enhancement, classification and annotation; A dynamic detection and early warning model was built based on Mamba and ViT to process and segment the real-time video stream collected by the drone and identify the pond area. Monitor changes in the pond area based on the identification results; Set warning conditions and send warning information when pond reconstruction behavior is detected.

[0008] As a preferred technical solution, the use of drones to obtain orthophoto video of the pond area is specifically as follows: Plan the flight path of the drone, set the flight altitude, switch the observation pitch angle in some areas of the route, and obtain the environmental and structural characteristics of the pond.

[0009] As a preferred technical solution, the orthophoto video of the pond area obtained by the drone is extracted, denoised, enhanced, classified and annotated, specifically: Extract key frames of the orthophoto video of the pond area at equal time intervals and use the key frame extraction algorithm to intercept them; Convert the extracted key frame images into static pictures; Use Gaussian filtering algorithm to reduce image noise; Using an image enhancement algorithm to improve the contrast and clarity of the image; the image enhancement algorithm includes histogram equalization and CLAHE algorithm; The extracted static images are classified and labeled to create a sample library, wherein each sample is labeled with the scope and operation status of the enclosure and contains metadata; the metadata includes the shooting time, location and flight parameters.

[0010] As a preferred technical solution, the dynamic detection warning model includes a feature extraction module and a multi-scale feature fusion module connected in sequence; The feature extraction module includes a U-Mamba module and a multi-scale ViT module connected in sequence; The U-Mamba module includes several layers of Mamba-blocks connected in sequence to extract multi-level features. , and fuse upsampled and downsampled features through skip connections; The multi-scale ViT module enhances the multi-level features extracted by the U-Mamba module in a feature interaction manner. The ACA-Former module is introduced in each layer of the ViT module, so that the network can capture global semantic information while retaining local details. The multi-level features enhanced by the multi-scale ViT module are expressed as ,Finally, the extracted feature map is expressed as ; The multi-scale feature fusion module combines the multi-stage features Concatenated with the features obtained by upsampling via skip connection operations, we generate , as follows: ; Then, the final segmentation result is generated through a 1×1 convolution operation as follows: .

[0011] As a preferred technical solution, the training process of the dynamic detection warning model is: Use open source datasets to test the segmentation accuracy and real-time segmentation performance of the dynamic detection and warning model; The dataset is divided into training set and test set in a 1:1 ratio. Before training, all input images are normalized and the resolution is adjusted before being input into the network. The dynamic detection warning model is trained using the Soft-IoU loss function and optimized using the Adam method, while the CosineAnnealingLR scheduler is applied.

[0012] As the preferred technical solution, the accuracy evaluation indicators used are precision and recall, where the calculation formula for precision is as follows: ; in, TP Represents the number of true positive examples, that is, the number of samples that are actually positive and predicted correctly by the model; FP Represents the number of false positives, that is, the number of samples that are actually negative but are incorrectly predicted as positive by the model; The calculation formula for recall is as follows: ; in, FN Represents the number of false negatives, that is, the number of samples that are actually positive but are incorrectly predicted as negative by the model.

[0013] As a preferred technical solution, the changes in the enclosure area are monitored according to the identification results, specifically: Through the real-time segmentation results, the boundary information of the enclosure area is extracted, the area of ​​each enclosure is calculated using the boundary information, and the number of enclosures is counted; Compare the area and quantity of the pond at the current moment with the data at the previous moment, and calculate the change in area and quantity; Determine whether any transformation has occurred in the pond area based on the amount of change, and store the result.

[0014] As a preferred technical solution, the setting of the warning conditions to send a warning message when the pond reconstruction behavior is detected is specifically: According to actual needs, early warning rules are set, specifically: when the change in the area of ​​the pond exceeds a certain threshold or the change in the number of ponds exceeds a certain threshold, it is considered that the pond reconstruction behavior has occurred; When the system detects pond reconstruction activities, it will push warning information to the responsible personnel through the preset communication method.

[0015] Another aspect of the present invention further provides a dynamic detection and early warning system for pond reconstruction based on drone images, which is applied to the above-mentioned dynamic detection and early warning method for pond reconstruction based on drone images, including a data acquisition and preprocessing module, a real-time segmentation module, a change monitoring module and an early warning system module; The data acquisition and preprocessing module uses drones to obtain orthophoto video of the pond area and performs extraction, noise reduction, enhancement, classification and labeling; The real-time segmentation module builds a dynamic detection and early warning model based on Mamba and ViT, processes and segments the real-time video stream collected by the drone, and identifies the pond area; The change monitoring module monitors the changes in the enclosure area according to the identification results; The early warning system module is used to set early warning conditions and send early warning information when pond reconstruction behavior is detected.

[0016] Another aspect of the present invention provides a storage medium storing a program, which, when executed by a processor, implements the above-mentioned dynamic detection and early warning method for pond reconstruction based on drone images.

[0017] Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) The technical solution provided by the present invention realizes real-time monitoring and early warning of pond reconstruction behavior by combining drone imaging technology and deep learning models, bringing significant beneficial effects. First, automated flight tasks and high-resolution image acquisition improve monitoring efficiency and accuracy; second, the deep learning model combined with Mamba and Vision Transformer (ViT) realizes frame-by-frame processing of real-time video streams and real-time segmentation and recognition of pond areas, enhancing the real-time and dynamic nature of monitoring; in addition, through the production of sample libraries and result storage, it is convenient for subsequent data analysis and application, providing data support for the sustainable use of marine resources; finally, flexible warning rule settings and diversified communication methods ensure the flexibility and scalability of the warning mechanism, which can respond to and handle pond reconstruction behaviors in a timely manner.

[0018] (2) The present invention significantly improves the robustness and generalization ability of the model by introducing multi-scale feature fusion and feature interaction methods. It can operate stably in complex and changeable marine environments, effectively reduce false positives and false negatives, and improve the accuracy and reliability of dynamic detection of pond reconstruction.

[0019] (3) Through real-time monitoring and dynamic change recording, the present invention can generate a large amount of monitoring data to provide data support for marine resource management and decision-making. These data can be used to analyze the trend of pond reconstruction, evaluate the impact of reconstruction behavior, and provide a scientific basis for future regulatory strategies. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flow chart of a dynamic detection and early warning method for pond reconstruction based on drone images according to an embodiment of the present invention; Figure 2 is a network architecture diagram of a dynamic detection and early warning model according to an embodiment of the present invention; Figure 3 It is a structural schematic diagram of a dynamic detection and early warning system for pond reconstruction based on drone images according to an embodiment of the present invention; Figure 4 It is a schematic diagram of the structure of the storage medium of an embodiment of the present invention. DETAILED DESCRIPTION

[0021] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application. Example

[0022] like Figure 1 As shown, this embodiment provides a dynamic detection and early warning method for pond reconstruction based on drone images, comprising the following steps: S1: Use the high-resolution camera on the drone to perform automated flight missions, obtain orthophoto video of the pond area and perform preprocessing; S11: In order to obtain high-quality image data and environmental information, it is necessary to use drones to observe the pond from multiple angles. First, plan the flight path of the drone, set the flight altitude to 60 meters, and cover an area within 1 square kilometer. Switch the observation pitch angle in some areas of the route to ensure that the environmental and structural characteristics of the pond can be fully captured. In the video acquisition stage, a frame rate of 30 frames per second is used to ensure that dynamic information of the surrounding environment is recorded; S12: Data processing of multi-angle observation videos collected by drones, extraction of key frames based on equal time intervals, and interception and preprocessing of pond images using key frame extraction algorithms. Then, the extracted key frame images are converted into high-quality static images, including the use of Gaussian filtering algorithms to reduce image noise, and the use of image enhancement techniques such as histogram equalization and CLAHE algorithms to improve image contrast and clarity, making the structural features of the pond more obvious; S13: Combined with image processing technology, the extracted static images are classified and labeled to create a sample library. Each sample is labeled with the scope and operation status of the enclosure, including operating enclosures, closed enclosures, and poorly managed enclosures. Each image in the sample library has a resolution of 512x512 pixels and contains detailed metadata, such as shooting time, location, and flight parameters, to facilitate subsequent data analysis and application.

[0023] S2: Figure 1 As shown, a deep learning model combining Mamba and Vision Transformer (ViT) is used to process the real-time video stream collected by the drone frame by frame, perform real-time segmentation, and identify the pond area; S21: Input the real-time video stream collected by the drone frame by frame into the multi-scale U-shaped interaction network (MUI-Net); S22: In the feature extraction stage, the input image is first passed through the U-Mamba module to extract multi-level features. , by using jump connections to fuse upsampled and downsampled features, the feature representation can be gradually enhanced. Since there is a semantic gap between the feature levels in the U-Mamba module, the multi-scale ViT module is introduced to enhance these multi-level features in a feature interaction manner. Among them, the ACA-Former module is introduced in each layer of the ViT module, enabling the network to capture global semantic information while retaining local details. The multi-level feature representation after the multi-scale ViT module is enhanced is ,Finally, the extracted feature map is expressed as ; S23: After the feature extraction module, the network uses a multi-scale feature fusion module to Upsampling. Multi-stage features Concatenated with the features obtained by upsampling via skip connection operations, we generate The processing process can be expressed as: ; Then, the final segmentation result map is generated through a 1×1 convolution operation. The specific formula is as follows: .

[0024] In particular, the training process of the dynamic segmentation model is as follows: Model preparation: build the environment required for the MUI-Net model and use open source datasets to test the segmentation accuracy and real-time segmentation performance of the model; The dataset is divided into training and test sets in a 1:1 ratio. Before training, all input images are normalized and resized to a resolution of 256×256 before being input into the network. The model uses the Soft-IoU loss function to train the network and uses the Adam method for optimization, while applying the CosineAnnealingLR scheduler. During training, the learning rate is set to 0.001, the batch size is set to 16, and 500 rounds of training are performed. PyTorch is used on a computer equipped with an RTX 3090 GPU. The accuracy evaluation indicators are precision and recall. Precision measures the proportion of actual positive samples among the samples predicted by the model as positive. A high precision means that the model's prediction of the positive class is reliable. The calculation formula for precision is as follows: ; in, TP (True Positive) represents the number of true positive examples, that is, the number of samples that are actually positive and predicted correctly by the model. FP (False Positive) represents the number of false positive examples, that is, the number of samples that are actually negative but are mistakenly predicted as positive by the model.

[0025] The recall rate measures the proportion of actual positive samples that the model successfully captures. A high recall rate means that the model has a strong ability to identify positive samples. The calculation formula for the recall rate is as follows: ; in, FN(False Negative) represents the number of false negative examples, that is, the number of samples that are actually positive but are mistakenly predicted as negative by the model.

[0026] S3: Calculate the number of ponds and the area of ​​each pond through the real-time segmentation results and store them, compare the area and number of ponds at the current moment with those at the previous moment, and monitor the changes in the pond area; S31: extracting boundary information of the enclosure area through the real-time segmentation result, calculating the area of ​​each enclosure using the boundary information, and counting the number of enclosures; S32: Compare the area and quantity of the enclosure pond at the current moment with the data at the previous moment, and calculate the change in area and quantity; S33: Determine whether the pond area has been remodeled based on the change amount, and store the result for subsequent analysis.

[0027] S4: Set warning rules according to actual needs. When pond reconstruction is detected, the system automatically triggers a warning and pushes the warning information to relevant personnel via SMS, email, etc.

[0028] S41: Set warning rules according to actual needs, such as triggering a warning when the change in the area of ​​the pond exceeds a certain threshold or the number of ponds changes significantly; S42: When the system detects pond reconstruction, it automatically triggers the early warning mechanism; S43: Push warning information to relevant personnel through preset communication methods, such as SMS and email, to ensure timely response and processing.

[0029] like Figure 3 As shown, in another embodiment of the present application, a dynamic detection and early warning system for pond reconstruction based on drone images is provided, the system comprising a data acquisition and preprocessing module 10, a real-time segmentation module 20, a change monitoring module 30 and an early warning system module 40; The data acquisition and preprocessing module 10 uses a drone to obtain orthophoto video of the pond area and performs extraction, noise reduction, enhancement, classification and labeling; The real-time segmentation module 20 constructs a dynamic detection and early warning model based on Mamba and ViT, processes and segments the real-time video stream collected by the drone, and identifies the pond area; The change monitoring module 30 monitors the changes in the enclosure area according to the recognition result; The early warning system module 40 is used to set early warning conditions and send early warning information when pond reconstruction behavior is detected.

[0030] It should be noted here that the system provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure is divided into different functional modules to complete all or part of the functions described above. The system is a dynamic detection and early warning method for pond reconstruction based on drone images applied to the above embodiment.

[0031] like Figure 4 As shown, in another embodiment of the present application, a storage medium is further provided, storing a program, and when the program is executed by a processor, a dynamic detection and early warning method for pond reconstruction based on drone images is implemented, specifically: S1: Use drones to obtain orthophoto video of the pond area and perform extraction, noise reduction, enhancement, classification and annotation; S2: Build a dynamic detection and early warning model based on Mamba and ViT, process and segment the real-time video stream collected by the drone, and identify the pond area; S3: Monitor the changes in the pond area based on the identification results; S4: Set the warning conditions and send a warning message when the pond reconstruction behavior is detected.

[0032] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0033] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A dynamic detection and early warning method for pond reconstruction based on drone images, characterized in that: The steps include: Use drones to obtain orthophoto video of the pond area and perform extraction, noise reduction, enhancement, classification and annotation; A dynamic detection and early warning model was built based on Mamba and ViT to process and segment the real-time video stream collected by the drone and identify the pond area. Monitor changes in the pond area based on the identification results; Set warning conditions and send warning information when pond reconstruction behavior is detected.

2. According to claim 1, a dynamic detection and early warning method for pond reconstruction based on drone images is characterized in that: The method of using a drone to obtain an orthophoto video of the pond area is as follows: Plan the flight path of the drone, set the flight altitude, switch the observation pitch angle in some areas of the route, and obtain the environmental and structural characteristics of the pond.

3. According to claim 1, a dynamic detection and early warning method for pond reconstruction based on drone images is characterized in that: The orthophoto video of the pond area obtained by the drone is extracted, denoised, enhanced, classified and annotated, specifically: Extract key frames of the orthophoto video of the pond area at equal time intervals and use the key frame extraction algorithm to intercept them; Convert the extracted key frame images into static pictures; Use Gaussian filtering algorithm to reduce image noise; Using an image enhancement algorithm to improve the contrast and clarity of the image; the image enhancement algorithm includes histogram equalization and CLAHE algorithm; The extracted static images are classified and labeled to create a sample library, wherein each sample is labeled with the scope and operation status of the enclosure and contains metadata; the metadata includes the shooting time, location and flight parameters.

4. The method for dynamic detection and early warning of pond reconstruction based on drone images according to claim 1 is characterized in that: The dynamic detection warning model includes a feature extraction module and a multi-scale feature fusion module connected in sequence; The feature extraction module includes a U-Mamba module and a multi-scale ViT module connected in sequence; The U-Mamba module includes several layers of Mamba-blocks connected in sequence to extract multi-level features. , and fuse upsampled and downsampled features through skip connections; The multi-scale ViT module enhances the multi-level features extracted by the U-Mamba module in a feature interaction manner. The ACA-Former module is introduced in each layer of the ViT module, so that the network can capture global semantic information while retaining local details. The multi-level features enhanced by the multi-scale ViT module are expressed as ,Finally, the extracted feature map is expressed as ; The multi-scale feature fusion module combines the multi-stage features Concatenated with the features obtained by upsampling via skip connection operations, we generate , as follows: ； Then, the final segmentation result is generated through a 1×1 convolution operation as follows: 。 5. The method for dynamic detection and early warning of pond reconstruction based on drone images according to claim 4 is characterized in that: The training process of the dynamic detection and early warning model is as follows: Use open source datasets to test the segmentation accuracy and real-time segmentation performance of the dynamic detection and warning model; The dataset is divided into training set and test set in a 1:1 ratio. Before training, all input images are normalized and the resolution is adjusted before being input into the network. The dynamic detection warning model is trained using the Soft-IoU loss function and optimized using the Adam method, while the CosineAnnealingLR scheduler is applied.

6. The method for dynamic detection and early warning of pond reconstruction based on drone images according to claim 5 is characterized in that: The accuracy evaluation indicators used are precision and recall, where the calculation formula for precision is as follows: ； in, TP Represents the number of true positive examples, that is, the number of samples that are actually positive and predicted correctly by the model; FP Represents the number of false positives, that is, the number of samples that are actually negative but are incorrectly predicted as positive by the model; The calculation formula for recall is as follows: ； in, FN Represents the number of false negatives, that is, the number of samples that are actually positive but are incorrectly predicted as negative by the model.

7. The method for dynamic detection and early warning of pond reconstruction based on drone images according to claim 1 is characterized in that: The monitoring of changes in the enclosure area according to the identification results is specifically as follows: Through the real-time segmentation results, the boundary information of the enclosure area is extracted, the area of ​​each enclosure is calculated using the boundary information, and the number of enclosures is counted; Compare the area and quantity of the pond at the current moment with the data at the previous moment, and calculate the change in area and quantity; Determine whether any transformation has occurred in the pond area based on the amount of change, and store the result.

8. The method for dynamic detection and early warning of pond reconstruction based on drone images according to claim 1 is characterized in that: The setting of the warning conditions to send a warning message when the pond reconstruction behavior is detected is specifically: According to actual needs, early warning rules are set, specifically: when the change in the area of ​​the pond exceeds a certain threshold or the change in the number of ponds exceeds a certain threshold, it is considered that the pond reconstruction behavior has occurred; When the system detects pond reconstruction activities, it will push warning information to the responsible personnel through the preset communication method.

9. A dynamic detection and early warning system for pond reconstruction based on drone images, characterized in that: A dynamic detection and early warning method for pond reconstruction based on drone images, applied to any one of claims 1 to 8, comprising a data acquisition and preprocessing module, a real-time segmentation module, a change monitoring module, and an early warning system module; The data acquisition and preprocessing module uses drones to obtain orthophoto video of the pond area and performs extraction, noise reduction, enhancement, classification and labeling; The real-time segmentation module builds a dynamic detection and early warning model based on Mamba and ViT, processes and segments the real-time video stream collected by the drone, and identifies the pond area; The change monitoring module monitors the changes in the enclosure area according to the identification results; The early warning system module is used to set early warning conditions and send early warning information when pond reconstruction behavior is detected.

10. A storage medium storing a program, characterized in that: When the program is executed by the processor, a dynamic detection and early warning method for pond reconstruction based on drone images as described in any one of claims 1 to 8 is implemented.

Images