An environment monitoring method and system based on a lightweight visual perception framework and application

Abstract

The application discloses an environment monitoring method based on a lightweight visual perception framework and belongs to the technical fields of computer vision and environment monitoring.The method mainly comprises the following steps: S1, normalizing and fusing environment index features of multi-source images through an environment perception module; S2, extracting candidate instance masks through a lightweight backbone network and a shared mask prototype mechanism by means of an ecological structure extraction module; S3, scoring and screening the candidate masks by means of a context target refinement module through a prompt guiding mechanism to obtain target masks; and S4, finally mapping the perception results to a geographic coordinate system through an environment insight visualization module.The application can realize real-time monitoring on edge computing devices while ensuring high precision and improving the inference speed by tens of times compared with large visual models.The application solves the problems of high inference delay of existing models, difficulty in deploying the models on resource-limited devices and low multi-scale environment target recognition precision.

Description

Technical Field

[0001] This invention relates to the fields of computer vision, remote sensing image processing and embedded AI technology, and specifically to a lightweight visual perception framework for real-time environmental monitoring. Background Technology

[0002] With the rapid development of remote sensing imaging equipment, urban monitoring systems, and unmanned platforms, environmental monitoring is gradually shifting from traditional manual surveys and statistical analysis to an intelligent monitoring model centered on image perception. [1] Automatic identification of land cover, vegetation distribution, water body changes, and traces of human activity can provide important support for ecological protection, urban management, and disaster early warning.

[0003] Current environmental monitoring methods mainly rely on deep learning models for semantic segmentation or instance segmentation of images. However, current mainstream high-precision models typically have large parameter scales and high inference complexity. While they can achieve good results on server-side applications, they have significant limitations in edge computing devices or real-time monitoring scenarios. [2,3] On the one hand, large-scale visual models suffer from high inference latency, making it difficult to meet the demands of real-time environmental monitoring; on the other hand, they are highly dependent on computing and storage resources, making them difficult to deploy on resource-constrained devices such as drones and mobile terminals. [4] Furthermore, traditional models often require separate training for different tasks when facing multi-scale and multi-category environmental targets, lacking a unified target selection and perception mechanism.

[0004] Therefore, how to reduce model complexity, improve inference efficiency, and enhance the understanding of complex environmental semantics while ensuring the accuracy of environmental target recognition has become a pressing technical problem in the field of intelligent environmental perception.

[0005] The references are as follows: [1] He, K., Gkioxari, G., Doll´ar, P., Girshick, R.: Mask r-cnn. In: Proceedings of the IEEE International Conference on Computer Vision (ICCV), pp. 2980–2988 (2017). [2] Redmon, J., Farhadi, A.: Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767 (2018). [3] Howard, A., Sandler, M., Chu, G., Chen, L.-C., Chen, B., Tan, M.,Wang, W., Zhu, Y., Pang, R., Vasudevan, V., Le, QV, Adam, H.: Searching formobilenetv3. In: Proceedings of the IEEE / CVF International Conference on ComputerVision (ICCV), pp. 1314–1324 (2019). [4] Lin, T.-Y., Doll´ar, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.2117–2125 (2017). Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to provide an environmental monitoring method and system based on a lightweight visual perception framework.

[0007] To achieve the above objectives, the present invention provides an environmental monitoring method based on a lightweight visual perception framework, comprising the following steps: S1. The environment perception module normalizes the multi-source images and fuses the environmental index features. S2. The ecological structure extraction module extracts candidate instance masks through a lightweight backbone network and a shared mask prototype mechanism. S3. The candidate mask is scored and filtered using a prompting and guidance mechanism through the context target refinement module.

[0008] S4. Map the perception results to the geographic coordinate system through the environmental insight visualization module.

[0009] Preferably, the specific method of step S1 includes the following sub-steps: S1-1. Acquire the original image of the area to be monitored and perform geometric and radiometric normalization to align with heterogeneous data sources; According to the formula: ; in, R represents the original image, H represents the set of real numbers, W represents the height, and 3 represents the number of channels. S1-2, According to the formula: ; in, This represents the preprocessed image. Both transformations preserve spatial proportions while adapting the image scale to the network input size. S × S ; S1-3. For environmentally oriented tasks, this invention supports the inclusion of environmental indicators; According to the formula: ; in, The image represents the enhanced image; `concat()` represents the stitching function; `NDVI` represents the Normalized Difference Vegetation Index; and `NDWI` represents the Normalized Difference Water Index. This serves as the visual observation basis for subsequent feature extraction.

[0010] Preferably, the specific method of step S2 includes the following sub-steps: S2-1, Feature encoding: Backbone network extracts multi-level features; According to the formula: ; Where F represents the feature map, Let C represent the input image, C represent the number of channels, and (h, w) represent the spatial resolution after downsampling. S2-2, Instance-level prediction; According to the formula: ; Where B represents the bounding box, p represents location information, and C represents category information. F represents the detection head, and F represents the input feature map; S2-3, Prototype-based mask reconstruction; According to the formula: ; Where P represents the prototype set, K The number of representative prototypes; Each instance mask is reconstructed through a linear combination; According to the formula: ; in, Representing the i Predicted mask for each target, Represents the activation function. right K Summing the prototypes, Representing the i The target corresponds to the first k The coefficients of each prototype, Representing the k One prototype feature map; Upsample to the original image size; According to the formula: ; in, This represents the upsampled mask. Represents the upsampling function. This represents the mask before upsampling. H Represents the mask height of the target output. W Represents the width of the target output mask; Non-maximum suppression methods based on IoU, using thresholds Redundancy detection is performed on (B, p) to obtain the retained set; According to the formula: ; in, S Represents the final set of tests. The bounding box representing the instance. The confidence score represents the instance. N’ This represents the predicted number of target instances. S2-4. Training Objectives; Multi-task loss function for instance segmentation models; ; in, Represents the total loss. This represents the loss used for classification. Represents the mask segmentation loss, and These represent the weighting coefficients that balance the different loss terms.

[0011] Preferably, the specific method of step S3 includes the following sub-steps: S3-1, Contextual target refinement; This stage allows for the refinement of the candidate set S through manual or algorithmic hints, and the selection of the target for a specific task; a general selection function Ψ is defined. The function is provided by a set of prompts Parameterization; S3-2, Point prompt; According to the formula: ; in, Represents an indicator function. pt Represents spatial location, ( x,y ) represents the first i The mask prediction map of each instance is located at coordinates ( x,y The probability value at position ) Represents the mask threshold; For location (x, y) If the first i The mask prediction probability of each instance is greater than the threshold. If so, then the location is considered to belong to that instance. = 1), otherwise it does not belong to ( = 0); S3-3, Box Prompt; According to the formula: ; in, Representing the i One prediction box, This represents a prompt box. Represents area, , degree of overlap (value range [0,1]); S3-4, Text prompts; According to the formula: ; in, Represents the extracted first i A visual feature vector, Represents an image encoder. Represented by a mask For images Select a region; According to the formula: ; Where t represents the semantic feature vector of the text prompt, Representative vector The inner product with t, Represents the L2 norm. Representing the i Visual feature vectors of each instance The similarity between the text feature vector t and the text feature vector t; S3-5, Combined prompts for selection; According to the formula: ; in, Representing the i The overall score of each candidate instance. Represents the weighting coefficient. This represents the mask confidence score. Represents the box matching degree. Represents the degree of text matching; According to the formula: ; in, Represents the optimal mask. This represents the parameter that maximizes the function's value. Representative candidate mask set S The first in i A mask, S Represents the set of candidate masks.

[0012] Preferably, the specific method of step S4 includes the following sub-steps: S4-1, Environmental Insight Visualization; According to the formula: ; in, This represents the final optimized mask. ( ) represents the category mapping function. Represents the mask processing function. Represents the original best prediction mask; The entire formula can be expressed as: Final visualization mask = Category mapping function, which is a two-stage processing pipeline; S4-2. Map the refined mask onto the geospatial coordinate system for environmental interpretation; According to the formula: ; in, This represents the final output result. Represents the fusion function. Represents a geographic reference system; S4-3, Coverage Calculation; According to the formula: ; in, Represents coverage, | | represents the total number of pixels. This represents the total number of pixels in the study area; S4-4, Calculation of rate of change; According to the formula: ; in, Represents the rate of change. Representative moment t Fine-grained mask, Representative moment t- A fine-grained mask of 1, | | Represents the total number of pixels in the area of ​​change, | | Representative Moment t- The total number of mask pixels of 1.

[0013] Preferably, the lightweight backbone network is a lightweight convolutional encoder-decoder structure derived from YOLOv8-seg.

[0014] Preferably, the shared mask prototype mechanism generates multiple shared mask prototypes through parallel segmentation heads, and reconstructs the instance mask through linear combination based on the mask coefficients.

[0015] An environmental monitoring system based on a lightweight visual perception framework includes: The environment perception module is used to normalize multi-source images and fuse environmental index features; The ecological structure extraction module is used to extract candidate instance masks through a lightweight backbone network and a shared mask prototype mechanism. The context target refinement module is used to score and filter candidate masks using a prompting and guidance mechanism to obtain the target mask; The environmental insight visualization module is used to map the perception results to a geographic coordinate system; The ecological structure extraction module includes: a lightweight backbone network unit for extracting multi-level features; a shared mask prototype unit for generating K shared mask prototypes and reconstructing instance masks through linear combination; and a non-maximum suppression unit for removing redundant detection.

[0016] Preferably, the environmental perception module further includes a geometric correction unit and a radiometric calibration unit.

[0017] An electronic device includes a processor and a memory storing a computer program, wherein the processor executes the program to implement the aforementioned environmental monitoring method based on a lightweight visual perception framework, or to control the system as described in any one of claims 8-9.

[0018] The specific principle of the environmental monitoring method based on a lightweight visual perception framework provided by this invention is as follows: In traditional instance segmentation networks, predicting a high-resolution mask for each target in an image individually consumes enormous computational resources and memory. This invention breaks this computational bottleneck by introducing a shared mask prototyping mechanism. Specifically, firstly, the method provided in this invention employs global feature reuse: the parallel segmentation head does not directly output the mask for each target, but instead generates... K The method uses a globally shared mask prototype; these prototypes are independent of the specific number of instances and capture common features such as basic edges, textures, and spatial distributions in the image, thus significantly limiting the number of channels and computational complexity of the feature map. Secondly, the method provides low-dimensional linear reconstruction: the detection head outputs a set of low-dimensional mask coefficients for each detected instance; therefore, the final instance mask can be reconstructed simply by linearly combining these coefficients with the shared mask prototypes. Thirdly, the method provides reduced edge memory usage: this decoupled "prototype + coefficient" operation compresses the computational load from being proportional to the "number of targets and resolution" to requiring only one calculation of the prototype map and a simple matrix multiplication, greatly reducing floating-point operations and memory usage, thus ensuring high frame rate real-time inference on resource-constrained devices. Fourth, due to the vast differences in target scales in environmental monitoring, traditional networks, employing fixed sampling levels and feature extraction schemes, are prone to missing small targets or blurring the boundaries of large targets. Therefore, the method provided in this invention addresses this issue through a prompting and guidance mechanism in the contextual target refinement module: a dynamic perception strategy of "extract first, then filter." This invention breaks away from the traditional "end-to-end multi-class hard classification" model. First, the backbone network, acting as a category-independent feature extractor, extracts all potential candidate ecological structures. Then, guided by prompts such as points, boxes, or text, the candidate set is dynamically filtered. To overcome scale sensitivity in cross-modal semantic alignment and address text prompts, this invention introduces a CLIP-based image-text alignment module. This module calculates the cosine similarity between the mask features of an instance and the text features. This means the model does not need to forcibly identify targets at all scales through a single fixed receptive field, but rather flexibly matches corresponding mask regions at different scale levels based on the semantic focus of text or spatial prompts. Fifth, the method provided by this invention uses joint scoring to eliminate multi-scale imbalance: when there is overlap of multi-scale targets, the joint scoring function accurately locates and extracts the mask of the target of the specific scale that the user is concerned about by assigning different cue domain weights, thereby effectively overcoming the imbalance of large and small targets in the feature space.

[0019] The method provided by this invention utilizes the data flow relationship between modules: the four core modules constitute a unidirectional closed-loop data flow from the original input to the geospatial conclusion, as detailed below: Data fusion (environmental perception module): Input: Original environment image .

[0020] Processing: After geometric and radiative transformations, the data is spliced ​​together with the introduced spectral prior features along the channel dimension.

[0021] Output: Enhanced multi-channel image matrix .

[0022] Feature Decoding (Ecological Structure Extraction Module): Input: Enhanced image .

[0023] Processing: Multi-level feature maps are extracted using a lightweight backbone network. The detection head outputs bounding boxes, classification probabilities, and mask coefficients; the segmentation head outputs in parallel. K A shared mask prototype. The two are reconstructed through linear combination, and then filtered by nonmaximum suppression.

[0024] Output: A set of candidate instances containing masks, bounding boxes, and confidence scores. .

[0025] Semantic matching (context target refinement module): Input: a set of candidate instances and external input prompts .

[0026] Processing: The matching degree between each mask and the cue feature in the set is calculated using a joint scoring function.

[0027] Output: The highest-scoring refined target mask .

[0028] Spatial Mapping (Environmental Insight Visualization Module): Input: target mask .

[0029] Processing: Spatial consistency is enhanced by sequentially applying morphological filtering and connected component pruning to obtain the final mask. Then the pixel coordinates are mapped back to the actual geographic coordinate system.

[0030] Output: Monitoring results with practical geographical significance .

[0031] The beneficial effects of this invention are as follows: (1) The environmental monitoring method based on the lightweight visual perception framework provided by the present invention maintains the ability to identify environmental targets with high precision, while significantly reducing the model parameter scale and computational complexity. The inference speed is more than 50 times faster than that of large visual models, which can meet the needs of real-time environmental monitoring.

[0032] (2) The environmental monitoring method based on the lightweight visual perception framework provided by the present invention effectively improves the expressive ability of multi-scale environmental structure through the shared mask prototype mechanism, and enhances the generalization performance of the model in complex scenarios. (3) The environmental monitoring method based on the lightweight visual perception framework provided by the present invention can support flexible environmental target selection methods by introducing a target refinement strategy guided by prompts, which effectively improves the intelligence and interpretability of environmental perception.

[0033] (4) The environmental monitoring method based on the lightweight visual perception framework provided by the present invention can be successfully applied to scenarios such as long-term monitoring of urban green spaces and water bodies by fixed cameras and real-time vegetation monitoring by drones. Attached Figure Description

[0034] Figure 1 This is a flowchart illustrating an environmental monitoring method and system based on a lightweight visual perception framework, according to an exemplary embodiment. Detailed Implementation

[0035] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0036] Example 1 Figure 1 This is a flowchart illustrating an environmental monitoring method and system based on a lightweight visual perception framework, according to an exemplary embodiment.

[0037] like Figure 1 As shown, an environmental monitoring method and system based on a lightweight visual perception framework includes the following steps: S1, normalizes multi-source images and fuses environmental index features through the environmental perception module; S2 utilizes the ecological structure extraction module to extract candidate instance masks through a lightweight backbone network and a shared mask prototype mechanism; S3, through the context target refinement module, uses a prompting and guidance mechanism to score and filter candidate masks, obtaining... ; S4 maps the perception results to the geographic coordinate system through the environmental insight visualization module.

[0038] The specific method of step S1 includes the following sub-steps: S1-1、 Given raw images of the multi-source environment to be monitored, perform geometric and radiometric normalization to align heterogeneous features. Data source; According to the formula:

[0039] in, R represents the original image, H represents the set of real numbers, W represents the height, and 3 represents the number of channels.

[0040] S1-2, According to the formula:

[0041] in, This represents the preprocessed image. Both transformations preserve spatial proportions while adapting the image scale to the network input size. S × S .

[0042] S1-3. For environmentally oriented tasks, this invention supports the inclusion of environmental indicators; According to the formula:

[0043] in, This represents the enhanced image. `concat()` represents the stitching function, `NDVI` represents the Normalized Difference Vegetation Index, and `NDWI` represents the Normalized Difference Water Index. (Image of the enhanced image) This serves as the visual observation basis for subsequent feature extraction.

[0044] Furthermore, the specific method of step S2 includes the following sub-steps: S2-1, Feature encoding: Backbone network extracts multi-level features; According to the formula:

[0045] Where F represents the feature map, Let C represent the input image, C represent the number of channels, and (h, w) represent the spatial resolution after downsampling.

[0046] S2-2, Instance-level Prediction According to the formula:

[0047] Where B represents the bounding box, p represents location information, and C represents category information. 'F' represents the detection head, and 'F' represents the input feature map.

[0048] S2-3, Prototype-based Mask Reconstruction A parallel split head generation K A shared mask prototype; According to the formula:

[0049] Where P represents the prototype set, K The number of prototypes.

[0050] Each instance mask is reconstructed through a linear combination; According to the formula:

[0051] in, Representing the i Predicted mask for each target, Represents the activation function. right K Summing the prototypes, Representing the i The target corresponds to the first k The coefficients of each prototype, Representing the k A prototype feature map.

[0052] Upsample to the original image size; According to the formula:

[0053] in, This represents the upsampled mask. Represents the upsampling function. This represents the mask before upsampling. H Represents the mask height of the target output. W This represents the width of the mask for the target output.

[0054] Non-maximum suppression methods based on IoU, using thresholds Redundancy detection is performed on (B, p) to obtain the retained set; According to the formula:

[0055] in, S Represents the final set of tests. The bounding box representing the instance. The confidence score represents the instance. N’ This represents the predicted number of target instances.

[0056] S2-4 Training Objectives Multi-task loss function for instance segmentation models;

[0057] in, Represents the total loss. This represents the loss used for classification. Represents the mask segmentation loss, and These represent the weighting coefficients that balance the different loss terms.

[0058] Furthermore, the specific method of step S3 includes the following sub-steps: S3-1, Contextual Target Refinement This stage allows for the refinement of the candidate set S through manual or algorithmic hints, and the selection of the target for a specific task. We define a general selection function Ψ. The function is provided by a set of prompts Parameterization.

[0059] S3-2, Point prompt; According to the formula:

[0060] in, Represents an indicator function. pt Represents spatial location, ( x,y ) represents the first i The mask prediction map of each instance is located at coordinates ( x,y The probability value at position ) This represents the mask threshold.

[0061] For location (x, y) If the first i The mask prediction probability of each instance is greater than the threshold. If so, then the location is considered to belong to that instance. = 1), otherwise it does not belong to ( = 0).

[0062] S3-3, Box Prompt; According to the formula:

[0063] in, Representing the i One prediction box, This represents a prompt box. Represents area, , degree of overlap (value range [0,1]).

[0064] S3-4, Text prompts; According to the formula:

[0065] in, Represents the extracted first i A visual feature vector, Represents an image encoder. Represented by a mask For images Select a region.

[0066] According to the formula:

[0067] Where t represents the semantic feature vector of the text prompt, Representative vector The inner product with t, Represents the L2 norm. Representing the i Visual feature vectors of each instance The similarity between the text feature vector t and the text feature vector t.

[0068] S3-5, Combined prompts for selection; According to the formula:

[0069] in, Representing the i The overall score of each candidate instance. Represents the weighting coefficient. This represents the mask confidence score. Represents the box matching degree. This represents the degree of text matching.

[0070] According to the formula:

[0071] in, Represents the optimal mask. This represents the parameter that maximizes the function's value. Representative candidate mask set S The first in i A mask, S Represents the set of candidate masks.

[0072] Furthermore, the specific method of step S4 includes the following sub-steps: S4-1, Environmental Insight Visualization; According to the formula:

[0073] in, This represents the final optimized mask. ( ) represents the category mapping function. Represents the mask processing function. This represents the original best prediction mask.

[0074] The entire formula can be expressed as: Final visualization mask = Category mapping function. This is a two-stage processing pipeline.

[0075] S4-2. Map the refined mask onto the geospatial coordinate system for environmental interpretation.

[0076] According to the formula:

[0077] in, This represents the final output result. Represents the fusion function. Represents a geographic reference system.

[0078] S4-3, Coverage Calculation; According to the formula:

[0079] in, Represents coverage, | | represents the total number of pixels. This represents the total number of pixels in the study area.

[0080] S4-4, Calculation of rate of change; According to the formula:

[0081] in, Represents the rate of change. Representative moment t Fine-grained mask, Representative moment t- A fine-grained mask of 1, | | Represents the total number of pixels in the area of ​​change, | | Representative Moment t- The total number of mask pixels of 1.

[0082] The lightweight backbone network is a lightweight convolutional encoder-decoder structure derived from YOLOv8-seg.

[0083] Furthermore, the shared mask prototype mechanism generates multiple shared mask prototypes through parallel segmentation heads, and reconstructs the instance mask based on the mask coefficients through linear combination.

[0084] This invention also provides an environmental monitoring system based on a lightweight visual perception framework, comprising: The environment perception module is used to normalize multi-source images and fuse environmental index features; The ecological structure extraction module is used to extract candidate instance masks through a lightweight backbone network and a shared mask prototype mechanism. The context target refinement module is used to score and filter candidate masks using a prompting and guidance mechanism to obtain the target mask; The environmental insight visualization module is used to map the perception results to a geographic coordinate system; The ecological structure extraction module includes: a lightweight backbone network unit for extracting multi-level features; a shared mask prototype unit for generating K shared mask prototypes and reconstructing instance masks through linear combination; and a non-maximum suppression unit for removing redundant detection.

[0085] Furthermore, the environmental perception module also includes a geometric correction unit and a radiometric calibration unit.

[0086] The present invention also provides an electronic device, including a processor and a memory storing a computer program, wherein the processor executes the program to implement the aforementioned environmental monitoring method based on a lightweight visual perception framework, or to control the aforementioned environmental monitoring system based on a lightweight visual perception framework.

[0087] In one possible embodiment, as shown in Table 1, the EnvLightning algorithm of the present invention is compared with the CLIP, MaskCLIP, SCLIP, GEM, ClearCLIP, and SegEarth-OV algorithms on various datasets. The experimental results in Table 1 are the average values ​​of multiple experiments.

[0088]

[0089] As shown in Table 1, the EnvLightning algorithm of this invention has a higher average intersection-union ratio than other existing algorithms, indicating a greater overlap between the predicted segmented region and the actual segmented region. Therefore, the environmental monitoring method based on the lightweight visual perception framework of this invention has excellent performance.

[0090] Example 2 The EnvLightning framework provided by this invention can be applied to vegetation monitoring on drone platforms to verify its real-time monitoring capabilities on resource-constrained edge devices; the specific steps are as follows: S1. Environmental perception module processing flow: Acquire raw images collected by the sensors on the UAV. First, geometric and radiometric normalization is performed to align heterogeneous data sources and adapt to the network input size of 512×512. Environmental index calculation and fusion: Normalized Difference Vegetation Index (NDVI) and Normalized Difference Water Index (NDWI) were extracted.

[0091] The original image and the environmental index are concatenated through channels to form an enhanced image, which is then used as subsequent input. The formula is as follows: .

[0092] S2, Ecological Structure Extraction Module Configuration: Lightweight backbone network: It adopts a lightweight convolutional encoder-decoder structure derived from YOLOv8-seg to extract multi-level features.

[0093] Detection head: Outputs bounding box, classification probability, and mask coefficient.

[0094] Shared mask prototype mechanism: parallel segmentation head generation A shared mask prototype P. Instance masks are reconstructed through a linear combination of coefficients and the prototype:

[0095] in This is the Sigmoid activation function.

[0096] Non-maximum suppression (NMS): Applying IoU-based NMS to remove redundant detection.

[0097] S3, Contextual Target Refinement Module: This embodiment uses text prompts for target filtering; the preset prompt text covers descriptions of specific vegetation or land features.

[0098] The image-text alignment module based on CLIP is used to extract visual and text features, and cosine similarity is calculated to measure semantic correspondence.

[0099] The joint scoring function integrates multiple cue weights to select the optimal target mask. .

[0100] S4, Environmental Insight Visualization Module: Morphological post-processing: for the selected mask Morphological filtering and connected component pruning are applied to enhance spatial consistency.

[0101] Geographic coordinate mapping: Maps the segmentation results back to the Earth coordinate system.

[0102] Metric Calculation: Calculate Coverage Rate and rate of change It is used to identify areas where vegetation has increased or decreased.

[0103] Hardware deployment and performance testing: Test platform: NVIDIA Jetson Orin Nano (8 GB).

[0104] Inference framework: Employs the TensorRT inference engine with FP16 precision, dynamically adjusting the input size.

[0105] Performance metrics: Single-frame inference time is approximately 45ms, and memory usage remains below 2.1 GB.

[0106] Comparative experiment: Deploying the Segment Anything Model (SAM, ViT-H encoder) with the same hardware configuration, SAM consumes over 6 GB of GPU memory and only achieves 1.5–2.0 FPS. The method provided by this invention significantly improves inference speed and greatly reduces memory usage while maintaining high accuracy, fully meeting the real-time monitoring needs of UAVs.

[0107] Example 3 This embodiment demonstrates the application of the present invention in long-term environmental sensing in urban fixed monitoring scenarios, focusing on verifying its continuous monitoring capabilities under complex lighting conditions and dynamic changes. The specific steps are as follows: S1. Environmental perception module processing flow: The data comes from video streams continuously collected by surveillance cameras at high points in the city.

[0108] Because urban environments contain a large number of man-made features and natural landscapes, the input images undergo geometric and radiometric transformation preprocessing and are strictly adjusted to a network input size of 512×512.

[0109] NDVI and NDWI are introduced as additional spectral prior knowledge to enhance the model's ability to understand scenes in cloud shadows or water surface reflections.

[0110] S2, Ecological Structure Extraction Module Configuration: To achieve extremely low latency, the model continues to reuse a lightweight backbone network based on the YOLOv8-seg architecture for multi-level feature extraction.

[0111] Through a shared mask prototype representation mechanism, the network can simultaneously process large-scale urban structures and small-scale targets, outputting a set of candidate instances.

[0112] S3, Contextual Target Refinement Module: This embodiment mainly uses dot hints and box hints mechanisms.

[0113] The system supports automatic sampling point grid or manual selection of regions of interest. By calculating the overlap or binary indicator function, it can dynamically and finely filter out specific environmental targets such as "encroached grassland" or "abnormally expanding water bodies".

[0114] S4, Environmental Insight Visualization Module: The morphologically processed masking results are continuously mapped to the urban geographic coordinate system.

[0115] Time series analysis: utilizing The formula compares mask changes across consecutive frames or across dates.

[0116] All-weather monitoring capability: Experiments show that even in nighttime scenes with significantly reduced light and contrast, EnvLightning can still reliably identify target structures such as buildings and trees illuminated by artificial lighting, demonstrating its temporal predictive consistency and all-weather deployment potential.

[0117] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention. The above preferred features can be used in any combination without conflict.

Claims

1. An environmental monitoring method based on a lightweight visual perception framework, characterized in that, Includes the following steps: S1. The environment perception module normalizes the multi-source images and fuses the environmental index features. S2. The ecological structure extraction module extracts candidate instance masks through a lightweight backbone network and a shared mask prototype mechanism. S3. The candidate mask is scored and filtered by the context target refinement module using a prompting and guidance mechanism to obtain the target mask; S4. Map the perception results to the geographic coordinate system through the environmental insight visualization module.

2. The environmental monitoring method based on a lightweight visual perception framework according to claim 1, characterized in that, The specific method of step S1 includes the following sub-steps: S1-1. Acquire the original image of the area to be monitored and perform geometric and radiometric normalization to align with heterogeneous data sources; According to the formula: ； in, R represents the original image, H represents the set of real numbers, W represents the height, and 3 represents the number of channels. S1-2, According to the formula: ； in, This represents the preprocessed image. Both transformations preserve spatial proportions while adapting the image scale to the network input size. S × S ; S1-3. For environmentally oriented tasks, this invention supports the inclusion of environmental indicators; According to the formula: ； in, The image represents the enhanced image; `concat()` represents the stitching function; `NDVI` represents the Normalized Difference Vegetation Index; and `NDWI` represents the Normalized Difference Water Index. This serves as the visual observation basis for subsequent feature extraction.

3. The environmental monitoring method based on a lightweight visual perception framework according to claim 1, characterized in that, The specific method of step S2 includes the following sub-steps: S2-1, Feature encoding: Backbone network extracts multi-level features; According to the formula: ； Where F represents the feature map, Let C represent the input image, C represent the number of channels, and (h, w) represent the spatial resolution after downsampling. S2-2, Instance-level prediction; According to the formula: ； Where B represents the bounding box, p represents location information, and C represents category information. F represents the detection head, and F represents the input feature map; S2-3, Prototype-based mask reconstruction; According to the formula: ； Where P represents the prototype set, K The number of representative prototypes; Each instance mask is reconstructed through a linear combination; According to the formula: ； in, Representing the i Predicted mask for each target, Represents the activation function. right K Summing the prototypes, Representing the i The target corresponds to the first k The coefficients of each prototype, Representing the k One prototype feature map; Upsample to the original image size; According to the formula: ； in, This represents the upsampled mask. Represents the upsampling function. This represents the mask before upsampling. H Represents the mask height of the target output. W Represents the width of the target output mask; Non-maximum suppression methods based on IoU, using thresholds Redundancy detection is performed on (B, p) to obtain the retained set; According to the formula: ； in, S Represents the final set of tests. The bounding box representing the instance. The confidence score represents the instance. N’ This represents the predicted number of target instances; S2-4. Training Objectives; Multi-task loss function for instance segmentation models; ； in, Represents the total loss. This represents the loss used for classification. Represents the mask segmentation loss, and These represent the weighting coefficients that balance the different loss terms.

4. The environmental monitoring method based on a lightweight visual perception framework according to claim 1, characterized in that, The specific method of step S3 includes the following sub-steps: S3-1, Contextual target refinement; This stage allows for the refinement of the candidate set S through manual or algorithmic hints, and the selection of the target for a specific task; a general selection function Ψ is defined. The function is provided by a set of prompts Parameterization; S3-2, Point prompt; According to the formula: ； in, Represents an indicator function. pt Represents spatial location, ( x,y ) represents the first i The mask prediction map of each instance is located at coordinates ( x,y The probability value at position ) Represents the mask threshold; For location (x, y) If the first i The mask prediction probability of each instance is greater than the threshold. If so, then the location is considered to belong to that instance. = 1), otherwise it does not belong to ( = 0); S3-3, Box Prompt; According to the formula: ； in, Representing the i One prediction box, This represents a prompt box. Represents area, , degree of overlap (value range [0,1]); S3-4, Text prompts; According to the formula: ； in, Represents the extracted first i A visual feature vector, Represents an image encoder. Represented by a mask For images Select a region; According to the formula: ； Where t represents the semantic feature vector of the text prompt, Representative vector The inner product with t, Represents the L2 norm. Representing the i Visual feature vectors of each instance The similarity between the text feature vector t and the text feature vector t; S3-5, Combined prompts for selection; According to the formula: ； in, Representing the i The overall score of each candidate instance. Represents the weighting coefficient. This represents the mask confidence score. Represents the box matching degree. Represents the degree of text matching; According to the formula: ； in, Represents the optimal mask. This represents the parameter that maximizes the function's value. Representative candidate mask set S The first in i A mask, S Represents the set of candidate masks.

5. The environmental monitoring method based on a lightweight visual perception framework according to claim 1, characterized in that, The specific method of step S4 includes the following sub-steps: S4-1, Environmental Insight Visualization; According to the formula: ； in, This represents the final optimized mask. ( ) represents the category mapping function. Represents the mask processing function. Represents the original best prediction mask; The entire formula can be expressed as: Final visualization mask = Category mapping function, which is a two-stage processing pipeline; S4-2. Map the refined mask onto the geospatial coordinate system for environmental interpretation; According to the formula: ； in, This represents the final output result. Represents the fusion function. Represents a geographic reference system; S4-3, Coverage Calculation; According to the formula: ； in, Represents coverage, | | represents the total number of pixels. This represents the total number of pixels in the study area; S4-4, Calculation of rate of change; According to the formula: ； in, Represents the rate of change. Representative moment t Fine-grained mask, Representative moment t- A fine-grained mask of 1, | | Represents the total number of pixels in the area of ​​change, | | Representative Moment t- The total number of mask pixels of 1.

6. The environmental monitoring method based on a lightweight visual perception framework according to claim 1, characterized in that, The lightweight backbone network is a lightweight convolutional encoder-decoder structure derived from YOLOv8-seg.

7. The environmental monitoring method based on a lightweight visual perception framework according to claim 1, characterized in that, The shared mask prototype mechanism generates multiple shared mask prototypes through parallel segmentation heads, and reconstructs the instance mask based on the mask coefficients through linear combination.

8. An environmental monitoring system based on a lightweight visual perception framework, characterized in that, include: The environment perception module is used to normalize multi-source images and fuse environmental index features; The ecological structure extraction module is used to extract candidate instance masks through a lightweight backbone network and a shared mask prototype mechanism. The context target refinement module is used to score and filter candidate masks using a prompting and guidance mechanism to obtain the target mask; The environmental insight visualization module is used to map the perception results to a geographic coordinate system; The ecological structure extraction module includes: a lightweight backbone network unit for extracting multi-level features; a shared mask prototype unit for generating K shared mask prototypes and reconstructing instance masks through linear combination; and a non-maximum suppression unit for removing redundant detection.

9. The environmental monitoring system based on a lightweight visual perception framework according to claim 8, characterized in that, The environmental perception module also includes a geometric correction unit and a radiometric calibration unit.

10. An electronic device, characterized in that, It includes a processor and a memory storing a computer program, wherein the processor executes the program to implement the method as described in any one of claims 1 to 7, or to control the system as described in any one of claims 8 to 9.

Images