Natural shoreline and mangrove forest supervision method and system based on video identification

Abstract

The invention provides a natural shoreline and mangrove forest supervision method and system based on video recognition, and the method comprises the steps: S1, installing and deploying information collection equipment, and setting a preset position and a cruise plan according to different monitoring targets; s2, accessing original sample data acquired by information acquisition equipment, reinforcing an original sample data set through a three-level data enhancement framework, and constructing a shoreline target sample data set; s3, optimizing by taking the YOLOv8n model as a basic model, and constructing a shoreline target recognition model; s4, constructing a target entity-spatial constraint-ownership state triple verification rule fusion judgment model as a behavior recognition model; and S5, inputting a recognition result of the shoreline target recognition model into a behavior recognition model, judging an illegal behavior through a triple verification rule, and generating early warning information. According to the invention, the working efficiency of natural shoreline supervision can be improved, and a replicable solution is provided for fine management of the coastal zone and the mangrove forest.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of remote sensing monitoring, in particular to a natural shoreline and mangrove monitoring method and system based on video recognition. BACKGROUND

[0002] As a sensitive interface of land-sea interaction, natural shoreline is not only the core ecological corridor for maintaining the integrity of marine ecosystems, but also the key carrier for maintaining biogeochemical cycles and ecological services. Its unique ecological barrier effect is irreplaceable in disaster prevention and reduction, carbon sink storage, and biodiversity conservation in the coastal zone.

[0003] Mangrove is a wetland woody plant community dominated by mangrove plants in the intertidal zone of tropical and subtropical coasts (growing in a unique land-sea interaction zone), which is flooded by periodic tidal water. It is a special ecological system that transitions from land to sea. Mangrove is an ecological key area at the junction of land and sea, a wetland ecosystem composed of woody plants with special adaptation mechanisms, and has multiple functions such as ecological protection, biodiversity maintenance, and climate regulation, which is very important for maintaining the ecological balance of the coastal zone.

[0004] The shoreline monitoring technology system has formed a multi-dimensional research framework, covering shoreline temporal change and driving factor analysis, shoreline intelligent monitoring, natural shoreline restoration technology design and benefit evaluation, etc. Among them, mangrove, as a typical ecological system along the coast, has great value, but its survival is threatened by human activities, and it is inevitable to gradually degenerate. Therefore, it is imperative to monitor human activities around the shoreline. In terms of shoreline monitoring technology methods, the fusion of satellite remote sensing and ground investigation has become the mainstream paradigm, and a "sky-ground" three-dimensional monitoring network has been initially established: Landsat series satellites are used to acquire and classify shoreline information, which can realize large-scale shoreline change monitoring; high-resolution, resource series satellites and WorldView sub-meter satellite images with spatial resolution better than 2 meters can stably improve the accuracy of artificial shoreline identification and subdivide shoreline types; further fusion of high-resolution images, interest points, roads and other multi-source heterogeneous data, combined with ground investigation, can effectively quantify the encroachment effect of urban expansion on the coastal zone. However, the existing satellite remote sensing technology, although plays an important role in sea island monitoring, its monitoring capability is limited by sensor type, resolution and environmental factors. For example, optical remote sensing relies on visible and near-infrared bands, and cloud cover will completely block the ground reflection signal; although radar remote sensing can penetrate clouds, its resolution and cost-effectiveness ratio may limit its application. In addition, satellite remote sensing technology also has difficulty in identifying small and hidden targets such as sewage discharge into the sea and mechanical construction, as well as rapid change events.

[0005] The breakthrough development of video target detection technology provides a new technical path for the supervision of coastal zones. By integrating computer vision and edge computing, it exhibits significant advantages in terms of monitoring timeliness, spatial resolution, and target recognition accuracy, effectively supplementing existing satellite remote sensing monitoring technologies. In practice, manual patrols are the primary means of coastal supervision, leading to delayed clue discovery and low efficiency. Some advanced regions have established coastal monitoring networks, including fixed intelligent cameras and unmanned aerial vehicle patrols, to improve the timeliness and accuracy of supervision in key areas. However, existing technical systems are mostly inherited from urban security scenarios and lack adaptability to the special physical environment of coastal zones.

[0006] Wu W, Cheng Q, Ren Y. Ship detection algorithm based on YOLOv2 and its application in Hengqin island monitoring system[J]. Geospatial Information, 2020, 18(1): 86-89+118. A dataset for ship detection was constructed based on surveillance video data, and the YOLOv2 network architecture was applied to ship detection. The initial candidate boxes were selected by clustering based on the aspect ratio of ships. The model was integrated with GIS maps and unmanned aerial vehicle images to realize real-time ship monitoring in the island waters. Based on the position, size, direction, and speed of ships, the Yolov2 model monitors abnormal behaviors of ships in military and civilian aspects. However, the model version used is relatively old, which limits the performance of the model to some extent, and the application scenario has certain limitations.

[0007] Kong J, Zhang X, Zhao Z, et al. Real-time target detection method for airport birds based on improved YOLOv8[J]. Science Technology and Engineering, 2024, 24(32): 13944-13952. DOI:10.12404 / j.issn.1671-1815.2309930. A real-time detection method for airport birds based on improved YOLOv8 was proposed. The method uses video monitoring equipment to detect birds and avoid threats to the safe operation of aircraft. It also achieves high detection accuracy and speed at a low cost. The method includes designing a feature fusion module to achieve multi-scale feature fusion at a finer granularity and designing a CSPPF (CBAM-spatial pyramid pooling fast) module to further improve detection accuracy and speed. It provides a new approach for bird detection in complex environments. Distributed monitoring devices are deployed near the airport runway to monitor the runway and its surrounding airspace in real time. The improved YOLOv8s algorithm is used to perform target detection tasks for airport birds. However, it cannot improve the model's performance in detecting small and dispersed targets on natural coastlines, where the targets have high similarity to the background (such as garbage). SUMMARY

[0008] In view of the defects in the prior art, the application provides a natural shoreline and mangrove supervision method and system based on video recognition.

[0009] According to the natural shoreline and mangrove supervision method and system based on video recognition provided by the application, the scheme is as follows: In a first aspect, a natural shoreline and mangrove supervision method based on video recognition is provided, and the method comprises the following steps: Step S1: install and deploy an information collection device, set a preset position and a cruise plan according to different monitoring targets; Step S2: access the original sample data collected by the information collection device, strengthen the original sample data set through a three-level data enhancement framework, and construct a shoreline target sample data set; Step S3: optimize a YOLOv8n model as a basic model to construct a shoreline target recognition model; Step S4: construct a target entity-space constraint-ownership state triple verification rule fusion judgment model containing target entity rules, spatial constraint rules and ownership state rules, as a behavior recognition model of the natural shoreline and the mangrove; Step S5: input the recognition result of the shoreline target recognition model into the behavior recognition model, determine the illegal behavior through the triple verification rule, and generate an early warning information.

[0010] Preferably, the information collection device in step S1 comprises a sea area camera and a drone; The video access of the sea area camera relies on the GB / T 28181 standard, adopts the SIP protocol for signaling docking through a signaling gateway, and adopts the RTP / RTCP protocol for media stream cascading; The collected camera data is preprocessed, including format and coding detection, integrity check, denoising and color correction; storage adopts RAID 6 or RAID 10 technology, and combines storage virtualization technology and a distributed file system; The cruise route of the drone is a linear route, the flight speed, flight height and fixed camera pitch angle are set to ensure that the route coverage range matches the predetermined area; the data collection means of the drone includes ordinary photos, high-definition videos and thermal imaging images, the collected drone data is stored in the drone storage device in real time, and is downloaded, backed up and managed after the task is completed.

[0011] Preferably, the three-level data enhancement framework in step S2 comprises a geometric transformation layer; The geometric transformation layer simulates the spatial posture change of the target on the natural shoreline, and adopts random rotation, horizontal flip and multi-scale scaling method to process the image: 1) Random rotation of the image: Rotate around a center point, where the center point is the image center (w / 2, h / 2), and rotate any point (x, y) in the image counterclockwise around the center point. The new coordinates of the angle are (x', y'), where The value range is [-15°, 15°], expressed as follows:

[0012] Where w is the image width and h is the image height; 2) Horizontal image flip: For any point (x, y) in the image coordinate system, the new coordinates after horizontal flipping are (x', y'), as shown in the following data representation:

[0013] Where w is the image width in pixels; 3) Multi-scale image scaling: Gaussian pyramid analysis was used for multi-scale image scaling, with values ​​ranging from 0.5. 1.5 times, the specific data is expressed as follows:

[0014] in, Indicates the first Gaussian pyramid l Layer image in coordinates (x, y) Pixel value at that location, l Indicates the pyramid level number; w(m, n) It is a Gaussian kernel weight function; (m, n) This is the spatial index of the convolution kernel, with a value range of [ [k,k], where k is the kernel half-width parameter; express l-1 Layer image in coordinates (2x + m, 2y + n) The pixel value at that location reflects the downsampling correspondence.

[0015] Preferably, the three-level data augmentation framework in step S2 includes a photometric adjustment layer; The photometric adjustment layer enhances the model's robustness to changes in illumination by processing the image with random brightness and contrast adjustments. 1) Random brightness adjustment: Brightness adjustment is performed randomly using a linear brightness adjustment strategy and a gamma correction strategy in the RGB color space, with values ​​ranging from [-25%, +25%], as expressed below: Linear brightness adjustment in, Contrast coefficient, This is the brightness offset. is the red band of the input image; is the red band of the output image; is the green band of the input image; is the green band of the output image; is the blue band of the input image; is the blue band of the output image; Gamma correction wherein, is the input pixel value, is the output pixel value, is the maximum pixel value; is the correction parameter, which determines the degree of nonlinearity; 2) Contrast adjustment: A linear contrast adjustment method is adopted to enhance or weaken the visibility of image details by changing the distribution range of image pixel values, with a value range of [-20%, +20%], expressed as follows:

[0016] wherein, is the input pixel value, is the output pixel value, is the contrast gain coefficient, is the brightness offset.

[0017] Preferably, the tertiary data enhancement framework in step S2 includes an environment simulation layer; 1) Fog effect simulation and rain effect simulation: Based on the Unreal Engine tool, the original sample data set is adjusted to generate a fog effect with a visibility range of [0.5-5km] and a rain effect of [5-50mm / h]; 2) Complex occlusion scene simulation: Based on the generative adversarial network GAN, a complex occlusion scene is generated, and its core components, the generator G and the discriminator D, are dynamically optimized through adversarial training; The generator G accepts the original sample data set and the occlusion ratio parameter o, where o takes a value range of [0.1, 0.3], and the differentiable rendering layer in the generator G generates an occlusion mask M that conforms to the physical law, which is expressed as:

[0018] wherein, is the sigmoid function, denotes the weight matrix; denotes the input feature vector; denotes the bias vector; Simultaneously, the discriminator D is used to evaluate the realism of the synthesized image and the accuracy of the occlusion ratio. The adversarial objective function adopts a minimax game approach, and its expression is:

[0019] Where x represents real data and z represents random noise; Pdata represents the discriminator's score on the real data; Pdata represents the distribution of the real data. This indicates that x is expected to be obtained from Pdata; This indicates the score given by the discriminator to the generated data; Through adversarial training, the generator G and discriminator D are optimized alternately to ultimately synthesize images of complex occluded scenes.

[0020] Preferably, step S3 includes: improving the YOLOv8n model, optimizing the YOLOv8n model, and using the shoreline target sample dataset as the model training samples; This includes: optimizing the ability to extract texture features from small targets. The YOLOv8n model includes a feature extraction network backbone, a feature fusion network neck, and a detection head. The backbone part of the YOLOv8n model uses the C2f module as the basic building unit. The CFP_C2f module is introduced into the C2f module of the backbone part. The C2f module is reconstructed through a dynamic weighting mechanism to form the CFP_C2f structure. The CFPConv module introduces a multi-scale feature pyramid structure, capturing different features through dilated convolutional groups. Feature fusion employs adaptive weights Its expression is:

[0021]

[0022] in, This represents the feature output obtained after performing the k-th dilated convolution operation on the input X; Indicates the activation function; Represents the learnable weight parameter matrix; Indicates the feature map Output the global feature vector; The C2f module adopts a channel separation strategy to split the input feature map into a static path and a dynamic path, and embeds the context feature pyramid convolution CFPConv module in the dynamic path of the C2f module to form a double feature enhancement module. The double feature enhancement module combines cross-stage feature fusion and a spatial attention mechanism, thereby optimizing the extraction ability of small target texture features while maintaining the lightweight of the model.

[0023] Preferably, the optimizing the YOLOv8n model further comprises embedding a parameter-free attention mechanism; A parameter-free attention mechanism SimAM module is embedded between the backbone and the neck. The SimAM module combines spatial information and channel information, assigns a unique attention weight to each pixel by using Euclidean distance as a similarity measurement method and based on the similarity, and quantifies the importance of the pixel-level features through an energy function to achieve target region saliency enhancement, and its expression is:

[0024] wherein, and is a linear transformation of the target neuron and other neurons of the attribute feature in a single channel, i is an index in the spatial dimension, and M is the number of neurons in the corresponding channel, and are the weights and biases of the transformation; represents an energy function; represents the true label value of the current target region; represents a reference target value; represents the target value of the pixel-level feature; represents the input pixel-level feature sample.

[0025] Preferably, the optimizing the YOLOv8n model further comprises optimizing a loss function; The loss function of YOLOv8n includes a classification loss Class Loss, i.e. , and a positioning loss Box Loss, wherein the Box Loss includes a CloU loss, i.e. , and a DFL loss, and their expressions are respectively:

[0026] wherein, is a true label, is a predicted probability, and N is the number of positive samples;

[0027] wherein, to predict the distance between the bounding box and the center point of the real box, c is the diagonal length of the minimum enclosing rectangle, to the aspect ratio consistency coefficient, is a weight parameter;

[0028] where y is the actual offset, and is the adjacent discrete value, and is the corresponding probability; The total loss is the weighted sum:

[0029] where, represents the bounding box regression loss weight; represents the distribution focus loss weight; represents the class loss weight; To further improve the monitoring accuracy of small targets, the normalized Wasserstein distance NWD is introduced into the regression branch of the loss function, and the NWD-CIoU joint loss function is adopted; where NWD is the Wasserstein distance between the predicted box and the real box distribution, and its expression is:

[0030] where, represents the 2nd order Wasserstein distance; represents the real box; represents the predicted box; represents the normalization constant; The NWD-CIoU joint loss function adopts adaptive weight to balance the two kinds of measurement, and its expression is:

[0031] where, and respectively represent the dynamic weight coefficient of CIoU loss and the dynamic weight coefficient of loss, which is adaptively adjusted according to the target size, and the weight will be increased for small targets, enhancing the optimization effect of NWD; and respectively represent the loss values of and , the smaller the value, the higher the matching degree of the predicted box and the real box.

[0032] Preferably, the target entity-space constraint-ownership state triple verification rule fusion judgment model (Violation=Rule (T, S, P)) in step S4 identifies the violation behavior through multi-dimensional cross verification, including: 1) Target entity rule T: Based on natural shoreline target monitoring sample analysis, a list of core supervision entities is established, and targets not in the supervision entity list are not subject to violation behavior judgment; 2) Space constraint rule S: Fusion of collected data to establish a three-level geographic fence, each level must meet the next rule judgment: Sea-facing verification: the target must be located on the seaward side of the coastline, and the GIS spatial analysis tool is used to determine whether the target position is located on the seaward side of the coastline. If it is, it meets the seaward requirement; Avoidance verification: the target must not be in the designated area, according to the coastal zone and marine space planning data of each level, using GIS spatial analysis tool to determine whether the target position is located within the designated area range. If it is not within the above range, it meets the avoidance requirement; Range verification: the target must not exceed the set boundary, all types of monitoring targets on the coastline must apply for use or construction in advance, and must not use or construct outside the approved range. Combined with various survey data, the GIS analysis spatial tool is used to determine whether the target position is located within the set range. If it is within the above range, it meets the range requirement; 3) Ownership state rule P: After meeting the space constraint rule, a three-level verification system for the target ownership compliance is established: Basic ownership verification: interface with the sea area use right certificate database for verification of qualification; Special permit check: cross-contrast related special permit records; Special case verification: identify exemption cases to avoid misjudgment of compliance projects; Finally, rule fusion judgment: the sufficient and necessary condition for natural shoreline violation behavior is Violation=Rule (T∩S∩P), and T, S, and P must be met simultaneously to determine the violation behavior.

[0033] In the second aspect, a natural shoreline and mangrove supervision system based on video recognition is provided, which includes: Unmanned aerial vehicle management module: for deploying unmanned aerial vehicle equipment, setting up patrol plan and flight parameters, collecting and storing video stream data of unmanned aerial vehicle, the flight parameters including flight speed, height and camera pitch angle; Video management module: for deploying sea area camera, accessing camera video stream, realizing video live broadcast, playback, cloud control and patrol plan setting function; Algorithm warehouse module: for storing the shoreline target sample data set and the shoreline target recognition model, supporting model dynamic loading and version control, the shoreline target sample data set is obtained by strengthening the three-level data enhancement framework, and the shoreline target recognition model is an improved YOLOv8n model; Verification and early warning module: integrate the behavior recognition model, call the shoreline target recognition model in the algorithm warehouse module to process video data, label target position and category, determine the violation behavior and generate early warning information; Cockpit module: integrate the data collected by the unmanned aerial vehicle management module and the video management module and the recognition results of the verification and early warning module, display data distribution and attribute content.

[0034] Compared with the prior art, the present application has the following advantages: 1. Satellite remote sensing means is used to supervise the activities around the coastline, and the monitoring capability is limited by the sensor type, resolution and environmental factors, which has the disadvantages of low monitoring frequency, lagging reaction and the like; on the basis of this, the present application additionally introduces camera, unmanned aerial vehicle and ground sensor and the like equipment, and through reasonable deployment point and setting of cruise plan, the timeliness of data collection and processing of the coastline supervision is improved; 2. Although the YOLOv8n model has excellent overall performance and can meet the real-time and high efficiency requirements of shoreline intelligent supervision to a certain extent, it has obvious deficiencies in small target detection in complex scenes; on the basis of the model, the present application introduces the contextual feature pyramid convolution CFPConv module, embeds the parameter-free attention mechanism SimAM (Simple Attention Module) module, optimizes the loss function of YOLOv8n and the like, strengthens the extraction ability of small target texture features, realizes the enhancement of target region saliency, and improves the precision of the model in the difficulties of small target size and relatively dispersed distribution, high similarity interference between target and background (such as garbage) and the like; 3. On the basis of identifying the target on the natural coastline, it is further determined whether the target has a violation behavior, and in view of the fuzziness problem of violation behavior determination in natural coastline supervision, the method constructs a rule fusion determination system of target entity-space constraint-ownership state triple verification (Violation=Rule (T, S, P)), which significantly improves the recognition accuracy of violation behavior through multi-dimensional cross verification, and is the basis of intelligent identification of natural coastline violation behavior. The natural coastline violation behavior and clues are quickly and effectively detected, which can effectively replace the traditional manual patrol mode, and improve the problem of untimely discovery of natural coastline supervision clues and low supervision efficiency.

[0035] Other beneficial effects of the present application will be illustrated in the specific embodiments by the introduction of specific technical features and technical solutions, and those skilled in the art should be able to understand the beneficial technical effects brought by the technical features and technical solutions through the introduction of the technical features and technical solutions. BRIEF DESCRIPTION OF DRAWINGS

[0036] Other features, objects, and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments thereof, read in conjunction with the accompanying drawings: Figure 1 It is a schematic diagram of the method of the present application as a whole; Figure 2 It is a schematic diagram of a three-level data enhancement framework; Figure 3 It is a schematic diagram of the improved model network structure; Figure 4 It is a schematic diagram of the rules of triple verification; Figure 5 It is a schematic diagram of the system of the present application as a whole. DETAILED DESCRIPTION

[0037] The present application will be described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present application, but do not limit the present application in any form. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present application. These all belong to the protection scope of the present application.

[0038] The embodiment of the present application provides a natural shoreline and mangrove supervision method based on video recognition. Referring to FIG. 1, the method specifically comprises the following steps: Figure 1 Step S1: Install and deploy information collection equipment, i.e. sea area camera and unmanned aerial vehicle, and set preset position and cruise plan according to different monitoring targets.

[0039] The data collection and storage of this step S1 are specifically as follows: Camera: Access first relies on GB / T 28181 standard to realize interconnection and intercommunication between modules, and each module is connected through signaling gateway. The signaling flow adopts SIP protocol cascade, and the media flow adopts RTP / RTCP protocol cascade. In the data receiving stage, first, the format and encoding of the video stream are detected, the format and encoding mode of the video file are identified through automatic script, and the integrity is checked to ensure that the video data is lossless. At the same time, the video data is preprocessed such as denoising and color correction to improve the accuracy of subsequent analysis.

[0040] ​At the same time, the RAID technology is adopted to improve the redundancy and reliability of data, including RAID 6 and RAID 10, so as to effectively improve the data protection level and prevent data loss. In combination with the storage virtualization technology (including VMware vSAN, Ceph, etc.), the unified management, dynamic resource allocation and scheduling of the storage resources are realized, and the flexibility and efficient utilization of the storage resources are improved. In addition, the distributed file system (including HDFS, GlusterFS, etc.) is adopted to process large-scale video data, so as to ensure efficient data access and load balancing, and improve the scalability and fault tolerance of the storage architecture.

[0041] The unmanned aerial vehicle: According to the task requirements and monitoring priorities, the specific suspicious event response cruise is carried out to ensure the full coverage inspection of the supervision area. Usually, the areas with potential environmental problems or supervision requirements are selected, such as the seaside bathing beach and the sewage outlet near Shapa Bay. The selection of the flight area combines the flight radius of the unmanned aerial vehicle and the topographic features of the surrounding area. The flight area is planned to automatically fly the route, set the flight speed (such as 5 m / s), the flight height (such as 100 m), and fix the camera pitch angle (such as 90° vertical downward), to ensure that the route coverage range matches the predetermined area. The flight task record adopts a linear route, which is suitable for long-distance coastal inspection. During the flight process, multiple data collection methods are adopted, including ordinary photos, high-definition videos and thermal imaging maps. Ordinary photos and videos are used for visual inspection, and thermal imaging maps are used to detect abnormal temperature changes to identify potential pollution sources or illegal activities.

[0042] All collected data are stored in real time in the storage device of the unmanned aerial vehicle, and immediately downloaded and backed up after the task is completed to prevent data loss. At the same time, according to different task requirements, the data are classified and managed.

[0043] Step S2: Access the original sample data collected by the information collection device, strengthen the original sample data set through a three-level data enhancement framework, and construct a coastal target sample data set.

[0044] The collected video data are subjected to frame extraction processing, the image data collected by the network are subjected to size preprocessing, the images with clear natural coastal targets and diverse scenes are selected for sample labeling, and an original natural coastal target sample data set is obtained. In the complex and variable natural scenes of the coastal zone, the imaging quality of the video images is significantly reduced due to the drastic change of light intensity, dynamic interference of water background, and severe weather conditions such as strong wind, heavy waves, and haze, which causes the weakening of target feature information or false detection, etc. The three-level data enhancement framework is proposed to strengthen the above sample data set, as shown in the framework. Figure 2

[0045] The three-level data enhancement framework in this step includes a geometric transformation layer, a photometric adjustment layer and an environment simulation layer.​ 1、Geometry transformation layer: To simulate the spatial pose changes of the target on the natural coastline, random rotation, horizontal flipping and multi-scale scaling are used to process the image: 1) Random rotation of the image: The present application mainly adopts rotation around the center point, the center point is the center of the image (w / 2, h / 2), and the new coordinates (x', y') of the angle of any point (x, y) in the image are rotated counterclockwise around the center point , wherein, The value range is [-15°, 15°], and the expression is as follows:

[0046] Wherein, w is the image width; h is the image height; 2) Image horizontal flipping: The new coordinates (x', y') of any point (x, y) in the image coordinate system after horizontal flipping are as follows:

[0047] Wherein, w is the image width, in pixels; 3) Image multi-scale scaling: In order to ensure the features of the image as much as possible, the present application adopts Gaussian pyramid analysis method for image multi-scale scaling, and the value range is 0.5 1.5 times, and the specific data expression is as follows:

[0048] Wherein, Gaussian pyramid l-th layer image at coordinates (x, y) pixel value, l represents the pyramid level number; w(m, n) is the Gaussian kernel weight function; (m, n) is the convolution kernel space index, the value range is [ k, k] k is the convolution kernel half-width parameter; Gaussian pyramid l-th layer image at coordinates (2x+m, 2y+n) pixel value, which embodies the down-sampling corresponding relationship.

[0049] 2、Lightness adjustment layer; In order to enhance the robustness of the model to illumination changes, random brightness and contrast adjustment is used to process the image; 1) Random brightness adjustment: The core of image brightness adjustment is to perform mathematical transformation on pixel values, and the present application randomly uses linear brightness adjustment strategy and Gamma correction strategy of RGB color space for brightness adjustment, and the value range is [-25%, +25%], which is expressed as follows: Linear brightness adjustment wherein, is a contrast coefficient, is a brightness offset, is the red band of the input image; is the red band of the output image; is the green band of the input image; is the green band of the output image; is the blue band of the input image; is the blue band of the output image; Gamma correction wherein, is the input pixel value, is the output pixel value, is the maximum pixel value; is the correction parameter, which determines the degree of nonlinearity (commonly 2.2 or 2.4); 2) Contrast adjustment: A linear contrast adjustment method is adopted to enhance or weaken the visibility of image details by changing the distribution range of image pixel values, with a value range of [-20%, +20%], expressed as follows:

[0050] wherein, is the input pixel value, is the output pixel value, is the contrast gain coefficient, is the brightness offset.

[0051] 3, Environment simulation layer; 1) Fog effect simulation and rain effect simulation: Based on the Unreal Engine tool, the original sample data set is adjusted to generate a fog effect with a visibility range of [0.5-5km] and a rain effect of [5-50mm / h]; 2) Complex occlusion scene simulation: Based on the generative adversarial network (GAN), a complex occlusion scene is generated. The core components, generator (G) and discriminator (D), are dynamically optimized through adversarial training. The principle is that the generator G accepts the original sample data set and the occlusion ratio parameter o, where o takes a value range of [0.1, 0.3], and the differentiable rendering layer in the generator G generates an occlusion mask M that conforms to the physical law, which is expressed as:

[0052] wherein, is a sigmoid function, denotes a weight matrix; represents an input feature vector; represents a bias vector; At the same time, the authenticity of the synthesized image and the accuracy of the occlusion ratio are evaluated by using the discriminator D, and the adversarial objective function adopts the form of minimax game, and its expression is:

[0053] Wherein, x is real data, z is random noise; represents the score of the discriminator on the real data; Pdata represents the distribution of the real data, represents the expectation that x is obtained from Pdata; represents the score of the discriminator on the generated data; Through adversarial training, the generator (G) and the discriminator (D) are alternately optimized, and finally the image of the complex occlusion scene is synthesized.

[0054] Step S3: taking the YOLOv8n model as a basic model for optimization to construct a shoreline target recognition model.

[0055] YOLOv8n model: The application selects a lightweight version YOLOv8n as a basic model to construct a natural shoreline target recognition model. Mainly based on the characteristics of low hardware resource occupation and fast inference speed, it can meet the real-time and high efficiency requirements of natural shoreline target rapid detection. YOLOv8n is mainly composed of three parts, including a feature extraction network (backbone), a feature fusion network (neck) and a detection head (head).

[0056] Wherein, the backbone part is responsible for feature extraction, adopts a series of convolution and deconvolution layers, and uses residual connection and bottleneck structure to reduce the size of the network and improve performance. This part adopts C2f module as the basic constituent unit, compared with C3 module in YOLOv5 model, C2f module reduces redundant parameters and improves calculation efficiency through more effective structure design. The neck network is located between the main network and the head network, and its function is to perform feature fusion and enhancement. The head network is the decision part of the target detection model, which is responsible for producing the final detection result.

[0057] Improved YOLOv8n model: Although the YOLOv8n model performs outstandingly in overall performance, it has obvious deficiencies in small target detection in complex scenes. The target size on the natural coastline is small and the distribution is relatively discrete, and there is high similarity interference between the target and the background (such as garbage, etc.). These problems make it difficult for the standard model to distinguish effective target areas in the feature extraction stage, and the boundary box regression accuracy decreases, which easily leads to missed detection and false detection, thereby affecting the application effect of the model in the actual scene. Therefore, in order to improve the overall detection performance of the model, the YOLOv8n is optimized from three aspects respectively, and the model training sample uses the coastline target sample data set.

[0058] (1) Optimize the extraction ability of small target texture features: The C2f module of the backbone part is introduced into the context feature pyramid convolution CFPConv module, the C2f module is reconstructed through a dynamic weighting mechanism to form a CFP_C2f structure. The C2f module adopts a channel separation strategy to split the input feature map into a static path and a dynamic path, and by splicing different outputs and original feature maps together, the context feature pyramid convolution CFPConv module can better aggregate multi-scale information.

[0059] The context feature pyramid convolution CFPConv module introduces a multi-scale feature pyramid structure, which captures different feature The feature fusion adopts adaptive weights , and its expression is:

[0060]

[0061] Among them, represents the feature output obtained after the kth hole convolution operation on the input X; represents an activation function; represents a learnable weight parameter matrix; represents the feature map Output global feature vector; The context feature pyramid convolution CFPConv module is embedded in the dynamic path of the C2f module to form a double feature enhancement module. The double feature enhancement module combines cross-stage feature fusion and spatial attention mechanism, while maintaining the lightweight of the model, and strengthens the extraction ability of small target texture features.

[0062] (2) Embedding parameter-free attention mechanism: A parameter-free attention mechanism SimAM (Simple Attention Module) module is embedded between the backbone and the neck. The SimAM module combines spatial information and channel information, which can help the network focus on important information in the input features without increasing the network parameters, and improve the feature expression ability of the network. The core idea of SimAM is based on the local self-similarity of images. In an image, adjacent pixels usually have strong similarity, while distant pixels have weak similarity. SimAM uses the Euclidean distance as a similarity measurement method, and assigns a unique attention weight to each pixel based on the similarity. The importance of the pixel-level feature is quantified through an energy function, which realizes the enhancement of the target region saliency, and its expression is as follows:

[0063] wherein, and are linear transformations of the target neuron and other neurons of the attribute feature in a single channel, i is the index in the spatial dimension, and M is the number of neurons in the corresponding channel, and are the weights and biases of the transformation; represents the energy function; represents the true label value of the current target region; represents the reference target value; represents the target value of the pixel-level feature; represents the input pixel-level feature sample.

[0064] (3) Optimization of loss function: The loss function of YOLOv8n consists of two parts, including the classification loss Class Loss ( ) and the positioning loss Box Loss, wherein Box Loss contains CloU loss ( ) and DFL loss, and their expressions are as follows:

[0065] wherein, is the true label, is the predicted probability, and N is the number of positive samples;

[0066] wherein, is the distance between the center points of the predicted box and the real box, c is the diagonal length of the minimum bounding rectangle, is the width-height ratio consistency coefficient, is the weight parameter;

[0067] where y is the actual offset, and are adjacent discrete values, and are corresponding probabilities; the total loss is a weighted sum:

[0068] where, represents the bounding box regression loss weight; represents the distribution focus loss weight; represents the category loss weight.

[0069] To further improve the monitoring accuracy of small targets, the normalized Wasserstein distance (NWD) is introduced into the regression branch of the loss function, and the NWD-CIoU joint loss function is adopted; where NWD is the Wasserstein distance between the predicted box and the real box distribution, and is normalized, and its expression is:

[0070] where, represents the 2nd order Wasserstein distance; represents the real box; represents the predicted box; represents the normalization constant; The NWD-CIoU joint loss function uses adaptive weights to balance the two metrics, and its expression is:

[0071] where, and respectively represent the dynamic weight coefficient of CIoU loss and the dynamic weight coefficient of loss, which is dynamically adjusted according to the target size. When the target is small, the weight will be increased, enhancing the optimization effect of NWD; and respectively represent the loss values of and , the smaller the value, the higher the matching degree of the predicted box and the real box.

[0072] In summary, the improved model is as follows Figure 3As shown, in addition to the above-mentioned parameters, Conv in the figure is a convolutional layer; Upsample is an up-sampling module; Concat is a connection layer; MaxPool is a maximum pooling layer; Detect is a detection output; SPPF is a SPPF (Spatial Pyramid Pooling Fast) module, which is an important component in the YOLOv5 target detection model, reduces the amount of calculation through a serial structure, and at the same time maintains the function of SPP, and is mainly used for fusing features at different scales and enhancing the semantic information of the feature map.

[0073] Step S4: A target entity-space constraint-ownership state triple verification rule fusion judgment model containing target entity rules, space constraint rules and ownership state rules is constructed as a behavior recognition model of natural shoreline and mangrove forest.

[0074] Only targets on the natural shoreline are identified, and it cannot be determined whether the target has a violation behavior. If these targets appear in the collected video stream, it will inevitably produce many false clues, such as excavators parked on the shoreline without work, recorded sea discharge outlets, and aquaculture for protection and repair of mangroves, etc. These are all reasonable existences. In view of the ambiguity of the violation behavior judgment in the natural shoreline supervision, the rule fusion judgment model of the target entity-space constraint-ownership state triple verification (Violation=Rule (T, S, P)) proposed by the application is shown in Figure 4 The rule system significantly improves the identification accuracy of the violation behavior through multi-dimensional cross verification, and is the basis for intelligent identification of natural shoreline violation behaviors.

[0075] The model system architecture includes: 1) Target entity rule T: Based on the analysis of natural shoreline target monitoring samples, a list of core supervision entities is established, such as sea discharge outlets (including hidden illegal discharge facilities), large-scale fishing nets, oyster shell storage yards, small-scale illegal structures (temporary board houses, cofferdams), shoreline construction machinery (excavators, bulldozers), shoreline destruction behaviors (illegal mangrove cutting, reclamation, pond digging), etc. The targets not in the list of supervision entities are not subject to violation behavior judgment.

[0076] 2) Space constraint rule S: Fusion of data collected from shoreline survey, coastal zone and marine space planning, aquaculture investigation, and sea use approval, etc. to establish three levels of geographic fences, each level must meet the requirements to enter the next rule judgment: ① Verification of sea-facing nature: the target must be located on the seaward side of the coastline. The GIS spatial analysis tool is used to determine whether the target position is located on the seaward side of the coastline. If it is located, it meets the requirement of sea-facing nature.

[0077] ②Avoidance verification: the target must not be in the ecological protection red line, strictly protected coastline and other areas. According to the data of each level of coastal zone and marine space planning, the spatial analysis tool of GIS is used to judge whether the target position is located in the range of ecological protection red line, strictly protected coastline and other areas. If it is not located in the above range, it meets the avoidance requirement.

[0078] ③Scope verification: the target must not exceed the approved coordinate range of the project or the boundary of the recorded aquaculture plot. Various monitoring targets on the coastline need to be applied for use or construction in advance, and cannot be used or constructed outside the approved range. Combined with various sea use approval investigation data, the spatial analysis tool of GIS is used to judge whether the target position is located in the approved or recorded range. If it is in the above range, it meets the scope requirement.

[0079] 3) Ownership state rule P: After meeting the spatial constraint rules, a three-level verification system of ownership compliance is established for the target: ①Basic ownership verification: connect the sea area use right certificate database for qualification verification; ②Special license check: cross comparison of agricultural and rural department aquaculture record and ecological environment department pollution / construction license record; ③Special case verification: identify the exemption cases such as national defense and flood control to avoid misjudgment of compliant projects; Finally, rule fusion judgment is carried out: the sufficient and necessary condition for the establishment of natural coastline violation behavior is Violation=Rule(T∩S∩P), and T, S and P must be met at the same time to determine the violation behavior.

[0080] Step S5: input the identification result of the coastline target identification model into the behavior identification model, determine the violation behavior through the three verification rules, and generate warning information.

[0081] The application also provides a natural coastline and mangrove supervision system based on video identification, as shown in Figure 5 , which comprises: Unmanned aerial vehicle management module: used for deploying unmanned aerial vehicle equipment, setting cruise plan and flight parameters, collecting and storing video stream data of unmanned aerial vehicle, and its functions include gimbal control, cruise route and time plan setting, video management and download, etc., used for obtaining video or pictures of natural coastline target area; flight parameters include flight speed, height and camera pitch angle; Video management module: used for deploying sea area camera, accessing camera video stream, realizing video live broadcast, playback, gimbal control and cruise plan setting functions, used for obtaining video or pictures of natural coastline target area; An algorithm warehouse module: used for storing a shore line target sample data set and a shore line target recognition model, supporting model dynamic loading and version control, and providing target detection capability through a standardized interface, the shore line target sample data set being obtained through a three-level data enhancement framework, and the shore line target recognition model being an improved YOLOv8n model; A verification and early warning module: integrating the behavior recognition model, calling the shore line target recognition model in the algorithm warehouse module to process and analyze monitoring video data, marking the position and category of the recognized target in the video picture, and judging whether the target is a suspected violation event according to the violation behavior recognition model, and if so, generating early warning information. The system user can push the verified suspected violation event to the front-line supervision personnel for on-site verification in real time through visual interpretation, information verification, etc. A cockpit module: integrating the data collected by the unmanned aerial vehicle management module and the video management module and the violation behavior data recognized by the verification and early warning module, and displaying data distribution and attribute content.

[0082] The embodiment of the present application provides a natural shore line and mangrove supervision method and system based on video recognition, integrates video services collected by sea area cameras and unmanned aerial vehicle equipment, designs a natural shore line intelligent supervision system, effectively improves the work efficiency of natural shore line supervision, and provides a replicable solution for fine management of coastal zones and mangroves.

[0083] Those skilled in the art know that, in addition to implementing the system provided by the present application and each device, module and unit thereof in a pure computer readable program code manner, the same functions can also be achieved by logically programming the method steps in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers. Therefore, the system provided by the present application and each device, module and unit thereof can be considered as a hardware component, and the devices, modules and units included therein for achieving various functions can also be considered as structures within the hardware component; the devices, modules and units for achieving various functions can also be considered as both software modules for implementing methods and structures within hardware components.

[0084] The specific embodiments of the present application are described above. It should be understood that the present application is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essential content of the present application. In the case of no conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A natural shoreline and mangrove supervision method based on video recognition, characterized in that, The method comprises the following steps: Step S1: Install the deployment information collection device, set the preset position and the cruise plan according to different monitoring targets; Step S2: Access the original sample data collected by the information collection device, strengthen the original sample data set through a three-level data enhancement framework, and construct a natural shoreline target sample data set; Step S3: Optimize the YOLOv8n model as a basic model to construct a shoreline target recognition model; Step S4: Construct a target entity-space constraint-ownership state triple verification rule fusion judgment model as a behavior recognition model of the natural shoreline and the mangrove forest; Step S5: Input the recognition result of the shoreline target recognition model into the behavior recognition model, determine the illegal behavior through the triple verification rule, and generate an early warning information.

2. The method of claim 1, wherein the method further comprises: The information collection device in step S1 comprises a sea area camera and a UAV; The video access of the sea area camera relies on the GB / T 28181 standard, adopts the SIP protocol for signaling docking through a signaling gateway, and adopts the RTP / RTCP protocol for media stream cascading; The collected camera data is preprocessed, including format and coding detection, integrity check, denoising and color correction; storage adopts RAID 6 or RAID 10 technology, and combines storage virtualization technology and a distributed file system; The cruise route of the UAV is a linear route, ensuring that the route coverage range matches the predetermined area, and the collected UAV data is stored in the UAV storage device in real time, and is downloaded, backed up and classified managed after the task is completed.

3. The method of claim 1, wherein the method further comprises: The three-level data enhancement framework in step S2 comprises a geometric transformation layer; The geometric transformation layer simulates the spatial posture change of the target on the natural shoreline, and processes the image by using random rotation, horizontal flip and multi-scale scaling method.

4. The method of claim 3, wherein the method further comprises: The three-level data enhancement framework in step S2 comprises a photometric adjustment layer; The photometric adjustment layer enhances the robustness of the model to illumination changes, and processes the image by using random brightness and contrast adjustment.

5. The method of claim 4, wherein the method further comprises: The three-level data enhancement framework in step S2 comprises an environment simulation layer; 1) Fog effect simulation and rain effect simulation: Based on the Unreal Engine tool, the original sample data set is adjusted to generate a fog effect with a visibility range of [0.5-5km] and a rain effect of [5-50mm / h]; 2) Complex occlusion scene simulation: Based on the generative adversarial network GAN, a complex occlusion scene is generated, and the core components generator G and discriminator D are dynamically optimized through adversarial training to synthesize an image of the complex occlusion scene.

6. The method of claim 1, wherein the method further comprises: Step S3 comprises improving the YOLOv8n model, optimizing the YOLOv8n model, and using the shoreline target sample data set as the model training sample; It comprises optimizing the extraction ability of small target texture features: The backbone part of the YOLOv8n model adopts the C2f module as the basic constituting unit, introduces the context feature pyramid convolution CFPConv module into the backbone part C2f module, reconstructs the C2f module through a dynamic weighting mechanism, and forms a CFP_C2f structure. The context feature pyramid convolution (CFPConv) module introduces a multi-scale feature pyramid structure, and different features are captured by a group of dilated convolutions , and the feature fusion adopts adaptive weights , and the expression is wherein, represents the feature output obtained after the kth hole convolution operation on the input X; represents an activation function; represents a learnable weight parameter matrix; represents the feature map outputs a global feature vector; The C2f module adopts a channel separation strategy to split the input feature map into a static path and a dynamic path, and embeds the context feature pyramid convolution (CFPConv) module into the dynamic path of the C2f module to form a double feature enhancement module.

7. The method of claim 6, wherein the method further comprises: The optimization of the YOLOv8n model further includes embedding a parameter-free attention mechanism; A parameter-free attention mechanism SimAM module is embedded between the backbone and the neck in the YOLOv8n model, the SimAM module combines spatial information and channel information, assigns a unique attention weight to each pixel by using Euclidean distance as a similarity measurement method and based on the similarity, and quantifies the importance of the pixel-level features through an energy function to achieve target region saliency enhancement, and the expression is: wherein, and are linear transformations of the attribute feature on the target neuron and other neurons in a single channel, i is an index in the spatial dimension, and M is the number of neurons in the corresponding channel, and are weights and biases of the transformation; denotes an energy function; denotes a true label value of the current target region; denotes a reference target value; denotes a target value of the pixel-level feature; denotes an input pixel-level feature sample.

8. The method of claim 7, wherein the method further comprises: The optimization of the YOLOv8n model further includes optimizing the loss function; The loss function of YOLOv8n includes a classification loss Class Loss, i.e. , and a positioning loss Box Loss, wherein BoxLoss includes a CloU loss, i.e. , and a DFL loss, the expressions of which are as follows, respectively: wherein, is the true label, is the predicted probability, N is the number of positive samples; wherein, is the distance between the predicted box and the real box center point, c is the minimum circumscribed rectangle diagonal length, is the width-height ratio consistency coefficient, is the weight parameter; where y is the actual offset, and is the adjacent discrete value, and is the corresponding probability; The total loss is a weighted sum: wherein, denotes a bounding box regression loss weight; denotes a distribution focus loss weight; denotes a class loss weight; In order to further improve the monitoring accuracy of small targets, the normalized Wasserstein distance (NWD) is introduced into the regression branch of the loss function, and the NWD-CIoU joint loss function is used; Wherein, NWD is the Wassertein distance between the predicted box and the real box distribution, and the expression is: wherein, denotes the 2nd order Wasserstein distance; denotes the true box; denotes the predicted box; denotes a normalization constant; The NWD-CIoU joint loss function uses adaptive weights to balance the two measurements, and the expression is: Wherein, and respectively represent the dynamic weight coefficient of CIoU loss and the dynamic weight coefficient of loss, which is dynamically adjusted according to the target size; and respectively represent and loss values.

9. The method of claim 1, wherein the method further comprises: The target entity-space constraint-ownership state triple verification rule fusion judgment model (Violation=Rule(T,S,P)) in step S4 identifies the violation behavior through multi-dimensional cross verification, including: 1) Target entity rule T: Based on the analysis of natural shoreline target monitoring samples, a list of core supervision entities is established, and targets not in the supervision entity list are not subject to violation behavior judgment; 2) Space constraint rule S: Fusion of collected data, establishment of three-level geographic fence, each level must meet the requirements to enter the next rule judgment: Seawards verification: the target must be located on the seaward side of the coastline, and the GIS spatial analysis tool is used to determine whether the target position is located on the seaward side of the coastline, if it is, it meets the seawards requirement; Avoidance verification: the target must not be in the set area, according to the data of each level of coastal zone and marine space planning, the GIS spatial analysis tool is used to determine whether the target position is located within the set area range, if it is not located within the above range, it meets the avoidance requirement; Range verification: the target must not exceed the set boundary, each type of monitoring target on the coastline must apply for use or construction in advance, and must not use or construct outside the approved range, combined with various types of investigation data, the GIS analysis spatial tool is used to determine whether the target position is located within the set range, if it is within the above range, it meets the range requirement; 3) Ownership state rule P: After meeting the space constraint rule, a three-level verification system for the ownership compliance of the target is established: Basic ownership verification: interface with the sea area use right certificate database for verification of qualification; Special permit check: cross comparison of related special permits; Special case verification: identify exemption cases to avoid misjudgment of compliance projects; Finally, the rule fusion judgment is carried out: the sufficient and necessary condition for the natural coastline violation behavior to be established is Violation = Rule (T∩S∩P), and T, S and P must be satisfied at the same time to determine the violation behavior.

10. A natural coastline and mangrove monitoring system based on video recognition, based on the natural coastline and mangrove monitoring method based on video recognition according to any one of claims 1-9, characterized in that, Comprise: The unmanned aerial vehicle management module is used for deploying unmanned aerial vehicle equipment, setting a cruise plan and flight parameters, collecting and storing video stream data of the unmanned aerial vehicle, and the flight parameters include flight speed, height and camera pitch angle. The video management module is used for deploying a sea area camera, accessing the camera video stream, and realizing video live broadcast, playback, pan-tilt control and cruise plan setting functions. The algorithm warehouse module is used for storing coastline target sample data set and coastline target recognition model, supporting model dynamic loading and version control, the coastline target sample data set is obtained through a three-level data enhancement framework, and the coastline target recognition model is an improved YOLOv8n model. The verification and early warning module integrates the behavior recognition model, calls the coastline target recognition model in the algorithm warehouse module to process the video data, labels the target position and category, determines the violation behavior and generates the early warning information. The cockpit module integrates the data collected by the unmanned aerial vehicle management module and the video management module and the recognition result of the verification and early warning module, and displays the data distribution and attribute content.

Images