Environment monitoring method and monitoring system based on unmanned aerial vehicle

Abstract

The invention provides an environment monitoring method and monitoring system based on an unmanned aerial vehicle, and the method comprises the steps: obtaining an initial environment monitoring image and target monitoring labeling information in a target monitoring environment region, loading the initial environment monitoring image to a target environment monitoring labeling neural network, and carrying out the generation of an unoptimized image; outputting an unoptimized monitoring annotation image, loading image block features corresponding to each image block in the unoptimized monitoring annotation image to a target generation result evaluation component, determining an error index of the target generation result evaluation component, and optimizing a context cache matrix based on the error index, the initial environment monitoring image is loaded to the target environment monitoring annotation neural network for image generation with optimization, an optimized monitoring annotation image is output, and image blocks in the optimized monitoring annotation image all meet the target monitoring annotation information. According to the invention, the efficiency and accuracy of monitoring and labeling the environment monitoring image can be improved.

Description

Technical Field

[0001] The present application relates to the field of machine learning, and more specifically, to an environmental monitoring method and monitoring system based on a drone. Background Art

[0002] In the field of environmental monitoring, traditional remote sensing data analysis methods mainly rely on manual visual interpretation and semi-automatic processing tools to identify specific ecological indicators through satellite or aerial images and manually annotate them. However, such methods face significant efficiency bottlenecks when processing high spatiotemporal resolution data, especially when dealing with sudden environmental pollution events or large-scale ecological monitoring tasks. The lag and subjective bias of manual annotation often lead to missed detection of key information. In the existing technology, although the automatic annotation system based on the rule engine has improved the processing speed, its rigid feature extraction algorithm is difficult to adapt to the spectral variation of complex surface cover types, and it is easy to introduce registration errors in the process of multi-sensor data fusion, which limits the application accuracy in scenarios such as vegetation dynamic monitoring and water pollution tracking. In recent years, deep learning technology has shown potential in the field of remote sensing image classification. Models such as convolutional neural networks (CNNs) are used to automatically extract ground features and automatically annotate images according to set rules according to user needs, but they require a large number of templates, which are manually designed or a large number of training samples with annotation constraints are collected for model training. The effect of the model is greatly affected by the sample size. In other words, existing methods are not efficient and accurate enough in monitoring and labeling images. Summary of the invention

[0003] In view of this, an embodiment of the present application provides an environmental monitoring method and a monitoring system based on a drone.

[0004] According to one aspect of an embodiment of the present application, a method for environmental monitoring based on a drone is provided, comprising: Acquire an initial environmental monitoring image of a target monitoring environment area photographed by a drone, and acquire target monitoring annotation information, wherein the target monitoring annotation information is used to indicate monitoring annotation requirements for the target monitoring environment area; Loading the initial environment monitoring image into the target environment monitoring annotation neural network to generate an unoptimized image, and outputting an unoptimized monitoring annotation image; The image block features corresponding to each image block in the unoptimized monitoring annotation image are loaded into the target generation result evaluation component respectively, the error index of the target generation result evaluation component is determined respectively, and the context cache matrix is ​​iterated based on the error index, wherein the error index is used to indicate whether the image block corresponding to the image block feature satisfies the consistency of the target monitoring annotation information; The initial environmental monitoring image is loaded into the target environmental monitoring annotation neural network for optimized image generation, and an optimized monitoring annotation image is output, wherein the optimized monitoring annotation image represents the environmental monitoring image output by the target environmental monitoring annotation neural network after the context cache matrix is ​​iterated based on the error index, and each image block in the optimized monitoring annotation image satisfies the target monitoring annotation information.

[0005] According to another aspect of an embodiment of the present application, a monitoring system is provided, comprising: one or more processors; and one or more memories, wherein computer readable codes are stored in the memories, and when the computer readable codes are executed by the one or more processors, the one or more processors execute the method described above.

[0006] The technical effects of the present application at least include: the present application obtains an initial environmental monitoring image and target monitoring annotation information in a target monitoring environment area, wherein the target monitoring annotation information is used to indicate the monitoring annotation requirements of the target monitoring environment area; the initial environmental monitoring image is loaded into a target environmental monitoring annotation neural network for unoptimized image generation, and an unoptimized monitoring annotation image is output, wherein the unoptimized monitoring annotation image includes image blocks that do not satisfy the target monitoring annotation information, and the unoptimized monitoring annotation image is constructed from image blocks output by the target environmental monitoring annotation neural network, the target environmental monitoring annotation neural network is used to determine the output image blocks one by one according to the loaded image blocks and the context cache matrix according to the image block scanning order, and determine the corresponding monitoring annotation image based on the output image blocks, the target environmental monitoring annotation neural network includes multiple feature weight focusing components, and the context cache matrix includes each feature weight focusing component in the multiple feature weight focusing components The invention discloses a method for optimizing the environmental monitoring annotation image and the method for optimizing the environmental monitoring annotation image. The method comprises the steps of: loading the image block features corresponding to each image block in the unoptimized monitoring annotation image into the target generation result evaluation component, determining the error index of the target generation result evaluation component, and optimizing the context cache matrix based on the error index, wherein the error index is used to indicate that the image block corresponding to the image block features satisfies the consistency of the target monitoring annotation information; loading the initial environmental monitoring image into the target environmental monitoring annotation neural network for optimized image generation, and outputting the optimized monitoring annotation image, wherein the optimized monitoring annotation image represents the environmental monitoring image output by the target environmental monitoring annotation neural network after the context cache matrix is ​​iterated based on the error index, and each image block in the optimized monitoring annotation image satisfies the target monitoring annotation information. Compared with directly debugging the environmental monitoring annotation neural network in order to meet a labeling method, there is no need to modify the pre-debugged environmental monitoring annotation neural network and perform additional repeated debugging. During the content generation process of the environmental monitoring annotation neural network, the present application adopts a generation result evaluation component to evaluate whether the generated image meets the preset annotation requirements, and back-propagates the generation error of the generation result evaluation component to optimize the state of the environmental monitoring annotation neural network, so that the probability of meeting the annotation requirements is greater, thereby guiding the environmental monitoring annotation neural network after pre-debugging to generate monitoring annotation images that meet the annotation requirements.

[0007] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0009] Figure 1 It is a schematic diagram of the architecture of an application scenario provided by this application; Figure 2 It is a flowchart of a drone-based environmental monitoring method provided by the present application; Figure 3 It is a structural diagram of a monitoring system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0010] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0011] In order to facilitate a clearer understanding of the present application, the application scenarios of the media data processing method of the present application are first introduced, such as Figure 1 As shown, the application scenario includes a monitoring system 10 and a drone cluster. The drone cluster may include one or more drones, and the number of drones is not limited. Figure 1 As shown, the drone cluster may specifically include drone 1, drone 2, ..., drone n; it can be understood that drone 1, drone 2, drone 3, ..., drone n can all be connected to the monitoring system 10 through a network, so that each drone can exchange data with the monitoring system 10 through the network connection.

[0012] It is understandable that the monitoring system 10 may refer to a device for executing the drone-based environmental monitoring method provided in the embodiment of the present application, and the monitoring system 10 may also be used to store images or videos taken by the drone. Among them, the monitoring system 10 may be a server, for example, an independent physical server, or a server cluster or distributed system composed of at least two physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms. The drone may specifically refer to an industrial drone, an agricultural drone, a special drone, etc., but is not limited thereto. Each drone and the monitoring system 10 may be directly or indirectly connected by wired or wireless communication. At the same time, the number of drones and monitoring systems 10 may be one or at least two, and this application does not limit this.

[0013] For further information, see Figure 2 , is a flow chart of a method for environmental monitoring based on a drone provided in an embodiment of the present application. Figure 2 As shown, this method can be Figure 1 The monitoring system 10 is used to perform, wherein the drone-based environmental monitoring method may include the following steps: Step S100: Acquire an initial environmental monitoring image of a target monitoring environment area photographed by a drone, and acquire target monitoring annotation information, wherein the target monitoring annotation information is used to indicate monitoring annotation requirements for the target monitoring environment area.

[0014] In step S100, the initial environmental monitoring image of the target monitoring environment area is collected by the drone, and the target monitoring annotation information associated with the area is obtained simultaneously. In this process, the sensor module carried by the drone can be configured with parameters, including adjusting the resolution, focal length, exposure time of the optical sensor and the band range of the multispectral sensor to ensure that the collected image data can cover the key monitoring elements of the target environmental area. For example, if the target monitoring environment is an industrial pollution area, the drone may be configured with a high-resolution visible light sensor (resolution ≥ 4096 × 2160 pixels) and a short-wave infrared sensor (band range 1400-3000nm) to simultaneously capture the visible features and thermal radiation features of the surface. The acquisition of the initial environmental monitoring image must follow the principle of spatiotemporal consistency: the flight path of the drone is planned through the geographic information system (GIS) module to ensure that the spatial coverage of the aerial image meets the monitoring requirements, and the time stamp alignment mechanism is used to ensure that the continuously shot frame sequence has temporal consistency. During the image acquisition process, data preprocessing operations are performed in real time, including a dehazing algorithm based on dark channel prior to eliminate the influence of atmospheric scattering, image contrast enhancement based on adaptive histogram equalization, and image stitching based on feature matching to generate a regional panorama.

[0015] The acquisition of target monitoring annotation information involves interaction with the annotation rule database, which stores preset annotation specifications for different environmental monitoring tasks, such as the type of area to be annotated, the style of annotation (such as color, line, pattern, annotation content), etc. For example, in a forest fire monitoring scenario, the annotation information may include the infrared feature threshold of the fire area (such as the HSV color space range of the area with a temperature ≥ 300°C is defined as H∈[0,15], S∈[80,100], V∈[90,100]), the vector annotation rule for the direction of smoke diffusion (using the HOG descriptor, the histogram of oriented gradients), and the unit standard for the affected area. By parsing the structured description file of the annotation information (such as JSON or XML format), it is converted into an operable tensor form, such as encoding the annotation style constraint as a conditional vector of the style transfer network. , mapping content detection requirements into a semantic segmentation template matrix For dynamic monitoring scenarios, a labeling information update mechanism can also be established. For example, when an abnormal pH value of the water body is detected, the detection item of the heavy metal pollutant ion concentration in the labeling rule is automatically triggered. At the data association level, the initial environmental monitoring image is bound to the geographic coordinates through the spatial registration algorithm, and the affine transformation matrix is ​​used. The pixel coordinates are converted to the UTM coordinate system, and the R-tree index structure is used to establish the correspondence between the image blocks and the annotation information. To cope with the data acquisition challenges in complex environments, a multimodal data fusion strategy can be implemented: the visible light image , infrared images and synthetic aperture radar images Through the weighted fusion algorithm ( ,in ) generates multi-channel input data, and sets the confidence weight of each modal data in the annotation information. In the quality control stage, abnormal data detection is performed, and a texture analysis algorithm based on local binary pattern (LBP) is used to identify image defect areas, and the drone re-flight mechanism is triggered to collect data. The final generated initial environmental monitoring image set And target monitoring annotation information set It will meet technical indicators such as spatial resolution δ≤0.1m, number of spectral channels k≥5, and time synchronization error Δt≤50ms, providing input data that meets the format requirements for subsequent neural network processing.

[0016] Step S200: Load the initial environment monitoring image into the target environment monitoring annotation neural network to generate an unoptimized image, and output an unoptimized monitoring annotation image, wherein the unoptimized monitoring annotation image includes image blocks that do not meet the target monitoring annotation information, and the unoptimized monitoring annotation image is constructed by the image blocks output by the target environment monitoring annotation neural network. The target environment monitoring annotation neural network is used to determine the output image blocks one by one according to the loaded image blocks and the context cache matrix according to the image block scanning order, and determine the corresponding monitoring annotation image based on the output image blocks. The target environment monitoring annotation neural network includes multiple feature weight focusing components, and the context cache matrix includes feature mapping tuples corresponding to each feature weight focusing component in the multiple feature weight focusing components.

[0017] In step S200, the acquired initial environment monitoring image is input into the target environment monitoring annotation neural network for unoptimized image generation, that is, the forward propagation of the complete network to generate an unoptimized monitoring annotation image. For example, based on the Transformer architecture, the serialization processing of image blocks and context association modeling can be realized through the feature weight focusing component. First, perform image block preprocessing: input image Divide into blocks of size p×p, where H and W are the image height and width, C is the number of channels (for example, C=3 for RGB images), and the number of blocks For a visible light image with a resolution of 2048×2048, if p=16 is set, 128×128=16384 image blocks are generated, and each block is converted into an embedding vector of d=512 dimensions through a linear projection layer. , and add position code To preserve spatial information, the calculation follows the formula , where the position code is generated using a sinusoidal function: In the feature weight focusing component, the global dependency between blocks is established through the multi-head attention mechanism. The calculation process of each attention head is , where the query matrix , key matrix , value matrix The input embedding is obtained by linear transformation, , h is the number of attention heads. For example, when h=8, the dimension of each head is d k =64, attention score matrix Characterizes the correlation strength between all image patches. Context cache matrix (l is the sequence length) remains fixed at this stage, which stores the key-value pair set generated by each feature weight focusing component when processing the first t-1 image blocks For example, when processing the tth image block, the current block With the existing cache Splicing to form an extended key value matrix , which is used to calculate the attention output of the current block. The generation of the unoptimized monitoring annotation image follows an autoregressive pattern: the image blocks are predicted one by one in raster scan order (from left to right, from top to bottom), and the sth block B s The generation of depends on the first s-1 blocks And its corresponding cache matrix Specifically, when processing the sth block, the following operations are performed: (1) The current block B s The embedding vector of is input into the Transformer encoder layer and is normalized by layer ( ,in is the mean and standard deviation of the feature, is a learnable parameter) and then enters the multi-head attention module; (2) each attention head calculates the attention weights of the current block and the historical blocks in the cache in parallel to generate context-aware features ,in represents the attention degree of the sth block to the jth block in the i-th attention head, is the value vector of the jth block of the ith attention head; (3) concatenate the outputs of all heads and generate a prediction block through a feed-forward neural network (FFN) , FFN usually contains two fully connected layers and GELU activation function: ,in is the weight matrix. In this process, the immutability of the context cache matrix is ​​strictly maintained, that is, after each image block is processed, its corresponding key-value pair is only added to the cache for use by subsequent blocks, but its value will not be updated through gradient backpropagation. For example, in the river pollution monitoring scenario, when the fifth image block containing the pollution source is processed, the cache information of the first four blocks (such as upstream water color characteristics, flow velocity vector, etc.) is used to calculate the attention weight, but since the cache is not optimized, it may lead to a prediction deviation of the pollution diffusion range, such as mistaking the actual pollution area R p Wrongly marked as normal area R n Unoptimized monitoring annotation image I unop The construction of t is achieved through the reverse block operation: the predicted image is reassembled into a complete image in the original scanning order. If the block size is 16×16, the gaps between blocks are eliminated by bilinear interpolation during the reassembly. The interpolation weight is calculated as ,in( )) is the coordinate of the center of the nearest neighbor block. Quality checks are performed simultaneously at this stage, such as by comparing the structural similarity index (SSIM) Global consistency with I: ,in, and Represent the original image I and the unoptimized labeled image I respectively unopt The pixel mean, and Represent the original image I and the unoptimized labeled image I respectively unopt The pixel value variance, c1 and c2 are constants to avoid the denominator being 0. When the SSIM value is lower than the threshold, the block regeneration mechanism is triggered. To improve processing efficiency, a memory optimization strategy is adopted: for large-scale images exceeding 4096×4096 pixels, tiling is implemented to divide the image into overlapping sub-regions (overlapping pixel number = 32), respectively input into the neural network and then fused by weight (fusion function , The fully labeled image is synthesized by linearly decaying with the distance from the edge. x and y represent the pixel values ​​or feature vectors of the overlapping parts of adjacent sub-regions. is the weight coefficient, which controls the fusion ratio of the two sub-regions in the overlapping area.

[0018] Step S300: The image block features corresponding to each image block in the unoptimized monitoring and annotation image are loaded into the target generation result evaluation component respectively, the error index of the target generation result evaluation component is determined respectively, and the context cache matrix is ​​iterated based on the error index, wherein the error index is used to indicate whether the image block corresponding to the image block feature satisfies the consistency of the target monitoring and annotation information.

[0019] In step S300, the target generation result evaluation component performs error analysis on the unoptimized monitoring annotation image, and optimizes the context cache matrix based on the error index to improve the annotation quality. First, each image in the unoptimized monitoring annotation image is divided into blocks. (p is the block size, c is the number of feature channels) Input target generation result evaluation component, which consists of a fully connected layer , non-linear activation layers (such as GELU function ,in is the standard normal distribution cumulative function) and the output layer The forward propagation formula is: ,in, Indicates dividing the image into blocks B s Operations that convert from multi-dimensional tensors to one-dimensional vectors, and is the bias of the fully connected layer, and the output dimension k corresponds to the number of labeled information categories (for example, in soil pollution monitoring, k=5 can represent the concentration levels of heavy metals such as cadmium, lead, arsenic, mercury, and chromium). The error index is calculated using the contrast loss function ,in, is the mean square error index of the sth block, k is the total number of labeled categories, is the true annotation vector of the sth block in the target monitoring annotation information. For example, when =0.9 indicates that the lead pollution concentration has reached the threshold of 90%. After calculating the loss value for each block independently, the error gradient is calculated through the back propagation algorithm. , focusing on the context cache matrix The specific optimization process uses the projected gradient descent method: Let the current cache matrix be , the learning rate is , the gradient of the error function with respect to the cache matrix is , then the update formula is , where the projection operation Constrain the matrix elements to the range [-1,1] to prevent numerical overflow.

[0020] In a river pollution diffusion prediction case, if the error index corresponding to the 15th image block is =0.76Significantly higher than the average error = 0.23, calculate the key-value pair gradient of the block in the third attention head of Transformer, for example and , and propagates layer by layer to the corresponding position of the cache matrix through the chain rule for correction. For example, when the pollution boundary labeling is deviated, the corrected key matrix The weight of the water flow direction feature will be increased, so that the pollution diffusion range can be predicted more accurately when the subsequent blocks are generated. To improve the optimization efficiency, a layered optimization strategy is implemented: a smaller learning rate is used for the underlying feature weight focusing component (the layer close to the input). To retain the basic features, a larger learning rate is used for high-level components To quickly adapt to the needs of semantic-level annotation. Dynamic learning rate adjustment follows the cosine annealing rule , where T is the total number of iterations and t is the current iteration index.

[0021] In an industrial waste gas emission monitoring scenario, if the labeling rule requires simultaneous detection of SO2 concentration (continuous value regression task) and plume morphology (discrete classification task), a multi-task loss function will be constructed. , where the regression loss , classification loss , weight coefficient Determined by grid search on the validation set. During the optimization process, an asynchronous parameter update mechanism is used: when processing the sth block, only the cache matrix entries before its corresponding scan order position x are updated , while leaving subsequent entries unchanged, which is done by masking the matrix Implementation, where If and only if and , to ensure that the temporal dependency is not destroyed. In the case of crop pest monitoring, if the rust area in the 23rd block is missing (the error index exceeds the threshold = 0.5), it will locate the 2nd, 4th, and 7th attention heads used to generate the block, calculate the gradient direction of its key value matrix, and update the parameters through the momentum acceleration method: , where the momentum coefficient =0.9 can accelerate convergence. The quality monitoring module runs synchronously and calculates the average error change rate after every 10 iterations. ,when The early stopping mechanism is triggered.

[0022] Step S400: Load the initial environmental monitoring image into the target environmental monitoring annotation neural network for optimized image generation, and output the optimized monitoring annotation image, wherein the optimized monitoring annotation image represents the environmental monitoring image output by the target environmental monitoring annotation neural network after the context cache matrix is ​​iterated based on the error index, and each image block in the optimized monitoring annotation image satisfies the target monitoring annotation information.

[0023] Step S400 re-inputs the initial environment monitoring image into the target environment monitoring annotation neural network with optimized context cache matrix to generate an optimized monitoring annotation image that meets the annotation requirements. This process reuses the block scanning mechanism in step S200, but uses the gradient-optimized context cache matrix in the key image block processing stage. , whose element values ​​are updated by error back propagation in step S300. First, cache matrix injection is performed: the optimized key-value pairs are Write the feature weights of each layer of the neural network into the focus component according to the attention head number. For example, in the third attention head, the updated key matrix ,in It is the parameter increment calculated in step S300.

[0024] In a marine oil pollution monitoring case, the optimized bond matrix increases the attention weight of the oil film edge texture features, so that the thin oil film area (thickness < 0.1mm) that was originally unlabeled can be accurately labeled in the optimized image. In the image generation stage, each block B is processed in raster order. s When , an improved attention calculation is performed: for the s-th block, its query vector With the optimized key matrix To calculate the similarity, the formula is: ,in, is the attention weight of the sth block to the jth historical block, reflecting the feature relevance, is the key vector of the jth historical block of the ith attention head in the optimized cache matrix, d k is the dimension of the key vector, For example, when processing the 25th block containing the oil diffusion front, the optimized cache matrix makes the attention weight of this block and the 7th, 12th, and 18th blocks upstream The FFN was improved from 0.08 to 0.23, effectively capturing the trajectory characteristics of oil pollution moving with ocean currents. The feedforward neural network (FFN) adopted a dynamic width adjustment strategy at this stage: for high-complexity areas (such as the building-green space interlaced zone in urban heat island effect monitoring), the FFN hidden layer dimension was automatically expanded to ,in , d is the original dimension of the FFN hidden layer, is the block complexity factor, which is calculated by the block texture energy (G is the gray gradient amplitude of each block) and spectral heterogeneity index ( is the variance of the kth band, c is the total number of bands) (e.g. ,in, , is the maximum value of texture energy and spectral heterogeneity index of all blocks in the training set, which is used for normalization. , is the weight coefficient), u and v are spatial frequency domain coordinates, representing the position index of the block in the two-dimensional discrete Fourier transform (DFT) or gradient amplitude space, Indicates the gradient magnitude of the block at position (u,v). Optimize monitoring and annotation images The generation of introduces the residual connection mechanism: each block prediction result With initial block Perform weighted fusion, the formula is , where the weight coefficient By block confidence score Dynamic decision, when >0.9 =0.8 to maintain the stability of the annotation. <0.5 = 0.3 to enhance the original feature retention. After generating the complete image, the Laplacian pyramid decomposition is used to Decompose into Three frequency bands, where L0 is the original resolution layer, L1 is the mid-frequency layer downsampled by 2 times, and L2 is the low-frequency layer downsampled by 4 times. Each layer independently calculates the consistency index with the target annotation information. For example, the high-frequency layer L0 focuses on edge sharpness verification (using Canny edge matching ,in, To optimize monitoring annotation images, E A The low-frequency layer L2 verifies the consistency of regional distribution (through Wasserstein distance measure, To optimize the probability distribution of labeled images, is the probability distribution of the true annotation, is the set of all possible joint distributions, is the transfer plan, which represents the cost of moving a unit mass from position x (labeled result) to position y (true label), is the Euclidean distance between the source point x and the target point y, which measures the spatial offset cost). When any layer indicator exceeds the threshold, the local regeneration process is triggered: locate the abnormal block position , before keeping If the blocks remain unchanged, Start re-executing a limited number of generation iterations (usually 3-5 times), using the Partial Cache Reset mechanism to The key-value pairs are replaced with the initial values, and then the correction area is generated by combining incremental optimization. In the final image output stage, cross-modal alignment is performed: the optimized annotated image Multispectral Data Cube (b is the number of bands) for spatial registration, and the sub-pixel offset caused by the change of the drone's posture is eliminated by thin-plate spline interpolation. Its energy function Minimize to achieve non-rigid alignment, where is the thin plate spline interpolation function, which represents the spatial deformation field of the image. is the coordinate of the ith control point, is the feature matching point between the multispectral data and the visible light image, is the target displacement vector, indicating the offset of the control point to be corrected. is the regularization coefficient, balancing the data fitting term and the deformation smoothing term. The quality assessment module runs synchronously to calculate the comprehensive score of the optimized image. , where the intersection and union ratio Measures the accuracy of region annotation, where is the overlapping area, The joint area, F1 score evaluates the classification accuracy, and Hausdorff distance Quantify the boundary matching degree, where X is the set of edge points in the predicted annotation area, and Y is the set of edge points in the actual annotation area. is the Euclidean distance between points x and y, sup and inf represent the supremum (maximum lower bound) and infimum (minimum upper bound), respectively, and the weight coefficient When S>0.9, the optimization is considered successful and I opt Stored in the database; otherwise, the whole process will be re-optimized, and the backup cache matrix will be enabled Perform rolling iterations to avoid error accumulation.

[0025] As an embodiment, the method further includes: Step 10: Load the initial environment monitoring image into the target environment monitoring annotation neural network to obtain a first image block features, wherein the target environment monitoring annotation neural network includes r feature weight focusing components, the first image block feature is the output result of the rth feature weight focusing component, the target environment monitoring annotation neural network outputs a image blocks one by one in the a image block scanning order through repeated iterations, the s+1th image block is determined by the sth image block output by the target environment monitoring annotation neural network in the s image block scanning order and the sth context cache matrix, which are loaded into the target environment monitoring annotation neural network in the s+1 image block scanning order, the sth context cache matrix is ​​used to represent the r feature map binary tuples corresponding to the s image block scanning order, the r feature map binary tuples are matched one by one with the r feature weight focusing components, and the a first image block features are used to determine the unoptimized monitoring annotation image constructed by the a image blocks, wherein r≥2, a≥s+1, s≥1; Step 20: Load a first image block features into the target generation result evaluation component respectively, obtain a evaluation results, and optimize the target context cache matrix, wherein the target context cache matrix represents the context cache matrix corresponding to the target image block output by the target environment monitoring annotation neural network after being loaded into the target environment monitoring annotation neural network, and the target image block includes the image block that does not meet the target monitoring annotation information determined by the a evaluation results; Step 30: Load the initial environment monitoring image into the target environment monitoring annotation neural network, determine a second image block features based on the optimized target context cache matrix, and generate an optimized monitoring annotation image, wherein the acquisition process of the second image block features is consistent with the acquisition process of the first image block features.

[0026] In step S10, the initial environment monitoring image is loaded into the target environment monitoring annotation neural network to obtain a first image block features, the target environment monitoring annotation neural network includes r feature weight focusing components, the first image block feature is the output result of the rth feature weight focusing component, the target environment monitoring annotation neural network will output a image blocks one by one in the a image block scanning order through repeated iterations, the s+1th image block is determined by the sth image block output by the target environment monitoring annotation neural network in the s image block scanning order and the sth context cache matrix loaded into the target environment monitoring annotation neural network in the s+1 image block scanning order, the sth context cache matrix is ​​used to represent the r feature mapping tuples corresponding to the s image block scanning order, the r feature mapping tuples are matched one by one with the r feature weight focusing components, and the a first image block features are used to determine the unoptimized monitoring annotation image constructed by the a image blocks, where r≥2, a≥s+1, and s≥1.

[0027] The initial environmental monitoring image is the original image data obtained by shooting the target monitoring environment area with a drone, which covers various environmental information of the target area. For example, when monitoring a forest environment, the image will contain information about elements such as trees, grass, and streams. The target environment monitoring annotation neural network is a pre-trained model, such as the transformer network, which has the ability to process and analyze input images to generate corresponding monitoring annotation images. When loading the initial environmental monitoring image into the neural network, it is necessary to ensure that the format and size of the image meet the input requirements of the neural network. Image preprocessing technology can be used, such as using the OpenCV library to resize and normalize the image to convert the image into a format suitable for neural network processing.

[0028] The r feature weight focusing components in the target environment monitoring annotation neural network are very critical parts, such as the attention mechanism components. These components enable the neural network to focus on important features in the image, improving the performance and accuracy of the model. In the process of processing image blocks, each feature weight focusing component will weight the features of the current image block according to the feature mapping binary in the context cache matrix, thereby highlighting important feature information. Taking the monitoring of the urban environment as an example, the attention mechanism component can focus more on important features such as buildings and roads, while ignoring some irrelevant background information.

[0029] In terms of the image block scanning order, the initial environmental monitoring image is usually divided into blocks and processed in sequence from left to right and from top to bottom. The determination of the s+1th image block depends on the sth image block and the sth context cache matrix. The context cache matrix contains the feature mapping tuples corresponding to each feature weight focusing component in multiple feature weight focusing components. These feature mapping tuples record the feature information of the previously processed image blocks, which helps the target environment monitoring annotation neural network to better understand the context information of the current image block. Through repeated iterations, the target environment monitoring annotation neural network outputs a image blocks in sequence in the order of a image block scanning, and these a first image block features are the output results of the rth feature weight focusing component, which are used together to construct the unoptimized monitoring annotation image.

[0030] In step S20, a first image block features are respectively loaded into the target generation result evaluation component to obtain a evaluation results, and the target context cache matrix is ​​optimized. The target context cache matrix represents the context cache matrix corresponding to the target image block output by the target environment monitoring and annotation neural network after being loaded into the target environment monitoring and annotation neural network. The target image block includes the image block that does not meet the target monitoring annotation information determined by the a evaluation results.

[0031] The structure of the target generation result evaluation component includes a fully connected layer, a nonlinear activation function layer, and a fully connected layer connected in sequence. Its function is to evaluate the image blocks output by the target environment monitoring annotation neural network. After a first image block features are loaded into the component, the component will evaluate the corresponding image blocks according to these features and obtain a evaluation results. These evaluation results reflect the situation that each image block meets the target monitoring annotation information. The target monitoring annotation information specifies the monitoring annotation requirements for the target monitoring environment area, such as the annotation style, the image content to be annotated, etc. If the evaluation result of an image block shows that it does not meet the target monitoring annotation information, then the image block belongs to the target image block.

[0032] The target context cache matrix is ​​optimized based on a evaluation results. The purpose of the optimization is to allow the target environment monitoring annotation neural network to refer to more accurate context information when processing image blocks in the subsequent process, thereby improving the accuracy of the output results. For example, when the annotation color of an image block does not meet the target monitoring annotation information requirements, the feature map binary associated with the image block in the target context cache matrix will be adjusted to change the decision of the target environment monitoring annotation neural network when processing subsequent image blocks.

[0033] In step S30, the initial environment monitoring image is loaded into the target environment monitoring annotation neural network, a second image block features are determined based on the optimized target context cache matrix, and an optimized monitoring annotation image is generated. The acquisition process of the second image block features is consistent with the acquisition process of the first image block features.

[0034] The initial environment monitoring image is loaded into the target environment monitoring annotation neural network again. At this time, the target environment monitoring annotation neural network uses the optimized target context cache matrix. Since the target context cache matrix has been optimized, the target environment monitoring annotation neural network can refer to more accurate context information when processing image blocks, so that the generated a second image block features are more in line with the requirements of the target monitoring annotation information. Similar to the acquisition process of the first image block feature, the target environment monitoring annotation neural network outputs a image blocks in sequence in the a image block scanning order based on the image block scanning order and the optimized context cache matrix, and then obtains a second image block features.

[0035] Generate an optimized monitoring annotation image based on the features of a second image blocks. During the generation process, the image blocks corresponding to the features of these image blocks are spliced ​​and combined according to certain rules to construct a complete optimized monitoring annotation image. Due to the use of the optimized context cache matrix, each image block in the optimized monitoring annotation image meets the requirements of the target monitoring annotation information and can more accurately reflect the actual situation of the target monitoring environment area.

[0036] In the implementation of the feature weight focusing component, the relevant algorithms of the attention mechanism can be used, such as the multi-head attention mechanism, and its calculation formula is: ,in, , , where Q, K, and V are query matrix, key matrix, and value matrix, respectively. and is the learnable weight matrix, d k is the dimension of the key vector.

[0037] In the implementation of the target generation result evaluation component, a fully connected layer and a nonlinear activation function layer are used to build a model, and the evaluation result is obtained through feedforward calculation. For the optimization of the context cache matrix, an optimization algorithm such as gradient descent can be used, and its update formula is: ,in, is the element value in the context cache matrix at the tth iteration, is the learning rate, is the error indicator about gradient.

[0038] As an implementation manner, step S20, loading a first image block features into a target generation result evaluation component respectively, obtaining a evaluation results, and optimizing the target context cache matrix, includes: Step S21: loading a first image block features into the target generation result evaluation component respectively, determining a error indicators, and matching a evaluation results with the a error indicators one by one; Step S22: determining a target error index corresponding to the x-th image block scanning order from a error indexes, wherein the target error index is an error index that does not meet the set error index requirement, x≤a; Step S23: Optimizing the target context cache matrix based on the target error index.

[0039] In step S21, a first image block features are loaded into the target generation result evaluation component respectively, a error index is determined, and a evaluation results are matched one by one with a error index. The first image block feature is the result output by the rth feature weight focusing component in the target environment monitoring annotation neural network after the initial environment monitoring image is loaded into the target environment monitoring annotation neural network. These features can reflect the key information of each image block. The target generation result evaluation component is a component for evaluating the output results of the target environment monitoring annotation neural network, and its structure includes a fully connected layer, a nonlinear activation function layer, and a fully connected layer connected in sequence. The fully connected layer is used to linearly transform the input feature vector, and the nonlinear activation function layer introduces nonlinear factors to increase the expression ability of the model.

[0040] The features of a first image block are input into the target generation result evaluation component respectively. The component processes each feature according to the internal calculation rules to obtain the corresponding error index. The error index is used to indicate whether the image block corresponding to the image block feature satisfies the consistency of the target monitoring annotation information. The target monitoring annotation information specifies the monitoring annotation requirements for the target monitoring environment area, such as the annotation style, the image content to be annotated, etc. The feasible error index calculation methods include mean square error (MSE), cross entropy loss, etc.

[0041] Taking the monitoring of urban environment as an example, assuming that the target monitoring annotation information requires that buildings be marked with specific colors, and the marking color of the buildings in a certain image block does not meet the requirements, then the error index corresponding to the image block will be larger. By calculating a error index, a evaluation results can be obtained, each evaluation result corresponds to an error index, which intuitively reflects the degree to which each image block meets the target monitoring annotation information.

[0042] In step S22, a target error index corresponding to the x-th image block scanning order is determined from the a error indexes, where the target error index is an error index that does not meet the set error index requirement, x≤a. The set error index requirement is a pre-set standard used to determine whether the error index is within an acceptable range. The a error indexes are checked and compared one by one to find out the error index that does not meet the set error index requirement.

[0043] For example, when setting the error index requirement, the threshold of the error index is set to 0.1. When a certain error index is greater than 0.1, it is considered that the error index does not meet the requirements. Among the a error indexes, those that are greater than 0.1 will be found. Assuming that the error index corresponding to the x-th image block scanning sequence is greater than 0.1, then this error index is the target error index. The image block corresponding to the target error index is the image block that does not meet the target monitoring annotation information and needs to be further processed and optimized.

[0044] In step S23, the target context cache matrix is ​​optimized based on the target error index. The target context cache matrix is ​​the context cache matrix corresponding to the target image block output by the target environment monitoring and annotation neural network after being loaded into the target environment monitoring and annotation neural network. The context cache matrix contains feature mapping tuples corresponding to each feature weight focusing component in multiple feature weight focusing components. These feature mapping tuples record the feature information of the previously processed image blocks, which are used to help the target environment monitoring and annotation neural network better understand the context information of the current image block.

[0045] By optimizing the target context cache matrix, the context information that the target environment monitoring and annotation neural network refers to when processing image blocks can be adjusted, thereby improving the accuracy of the output results. The optimization process is actually to adjust the element values ​​in the context cache matrix according to the target error index, so that the target environment monitoring and annotation neural network can make more accurate decisions when processing image blocks in the future.

[0046] In order to optimize the target context cache matrix, an optimization algorithm such as gradient descent can be used. The error index is gradually reduced by calculating the gradient of the error index with respect to the elements in the context cache matrix and then updating the element values ​​in the context cache matrix in the opposite direction of the gradient.

[0047] In practical applications, the target error index is first obtained, and then the parameter change rate of the target generation result evaluation component is calculated based on the error index. The parameter change rate reflects the rate at which the error index changes with the change of the element value in the context cache matrix. The original parameter matrix is ​​debugged according to the parameter change rate and the preset debugging learning rate. The size of the debugging learning rate is positively correlated with the size of the parameter update amount of the iterative debugging of the original parameter matrix, that is, the larger the learning rate, the larger the parameter update amount in each iteration.

[0048] Through continuous iterative debugging, each generation optimizes the target context cache matrix through the debugged parameter matrix. At the same time, after each optimization of the target context cache matrix, the error index and parameter change rate corresponding to the x-th image block scanning order of the corresponding generation will be obtained again until the error index corresponding to the x-th image block scanning order meets the set error index requirements. At this time, the parameter matrix after debugging of the corresponding generation is determined as the target parameter matrix, and the element values ​​in the target parameter matrix are the values ​​for optimizing the target context cache matrix.

[0049] The target context cache matrix is ​​added to the target parameter matrix, and the added matrix is ​​determined as the optimization result of the target context cache matrix. In this way, the optimization of the target context cache matrix is ​​completed, so that the target environment monitoring annotation neural network can refer to more accurate context information when processing image blocks in the subsequent process, thereby improving the quality of the output optimized monitoring annotation image.

[0050] By executing steps S21-S23, the image blocks output by the target environment monitoring annotation neural network can be evaluated, the image blocks that do not meet the target monitoring annotation information can be found, and the target context cache matrix can be optimized. In practical applications, the effective implementation of these steps can improve the accuracy and reliability of environmental monitoring annotation.

[0051] As an implementation mode, step S23, optimizing the target context cache matrix based on the target error index, includes: Step S231: obtaining an adjustable original parameter matrix; Step S232: determining the parameter change rate of the target generation result evaluation component according to the target error index, and debugging the original parameter matrix based on the parameter change rate until a target parameter matrix is ​​obtained, and the element values ​​in the target parameter matrix are values ​​for optimizing the target context cache matrix; Step S233: Add the target context cache matrix and the target parameter matrix, and determine the added matrix as the optimization result of the target context cache matrix.

[0052] In step S231, an adjustable original parameter matrix is ​​obtained. The original parameter matrix is ​​a basic matrix for debugging and optimizing the target context cache matrix, which contains a series of adjustable parameter values. These parameter values ​​will be adjusted according to the target error index in the subsequent optimization process, so as to optimize the target context cache matrix. The original parameter matrix can be obtained in a variety of ways, one of which is random initialization. For example, when using a deep learning framework for model training, the element values ​​in the original parameter matrix can be randomly generated according to a certain distribution (such as a Gaussian distribution). Another way is to initialize the original parameter matrix based on experience or prior knowledge. Assuming that in the field of environmental monitoring, based on previous monitoring experience, certain features have specific importance weights in the context cache matrix, the element values ​​in the original parameter matrix can be set according to these experiences, so that the initial parameter matrix is ​​more targeted. The dimensions and structure of the original parameter matrix usually match the target context cache matrix for subsequent effective calculation and adjustment.

[0053] In step S232, the parameter change rate of the target generation result evaluation component is determined according to the target error index, and the original parameter matrix is ​​debugged based on the parameter change rate until the target parameter matrix is ​​obtained, and the element values ​​in the target parameter matrix are the values ​​for optimizing the target context cache matrix. The target error index is determined in step S22, which represents the error index corresponding to the scanning order of the x-th image block, and the error index does not meet the set error index requirement. The target generation result evaluation component is a component used to evaluate the output results of the target environment monitoring annotation neural network, and the error index output by it reflects the consistency of the image block in meeting the target monitoring annotation information.

[0054] The parameter change rate of the target generation result evaluation component is calculated based on the target error index. The parameter change rate can be obtained by finding the gradient of the error index with respect to the elements in the original parameter matrix. In deep learning, the gradient represents the rate of change of a function at a certain point. By calculating the gradient of the error index with respect to the parameters, we can know how to adjust the parameters to reduce the error index. For example, using the mean square error (MSE), the chain rule can be used to find the partial derivative of the MSE with respect to the elements in the original parameter matrix to obtain the parameter change rate.

[0055] Based on the obtained parameter change rate, the original parameter matrix is ​​debugged. During the debugging process, an important parameter is required, namely the debugging learning rate. The size of the debugging learning rate is positively correlated with the size of the parameter update amount of the iterative debugging of the original parameter matrix. If the debugging learning rate is set too large, the parameter update amount will be large, which may cause the algorithm to fail to converge, or even cause the parameter value to diverge; if the debugging learning rate is set too small, the parameter update amount will be small, the algorithm convergence speed will be very slow, and more iterations will be required to achieve the ideal optimization effect.

[0056] The original parameter matrix is ​​iteratively debugged according to the debugging learning rate. Each generation optimizes the target context cache matrix through the debugged parameter matrix. At the same time, after each optimization of the target context cache matrix, the error index and parameter change rate corresponding to the x-th image block scanning order of the corresponding generation are obtained again. This process is repeated until the error index corresponding to the x-th image block scanning order meets the set error index requirements. At this time, the parameter matrix after debugging of the corresponding generation is determined as the target parameter matrix.

[0057] In step S233, the target context cache matrix is ​​added to the target parameter matrix, and the added matrix is ​​determined as the optimization result of the target context cache matrix. After the iterative debugging of step S232, the target parameter matrix is ​​obtained, and the element values ​​in the matrix are obtained after optimizing the original parameter matrix according to the target error index. The corresponding elements of the target context cache matrix and the target parameter matrix are added to obtain a new matrix. This new matrix combines the information of the original target context cache matrix and the optimized and adjusted target parameter matrix, which enables the target environment monitoring and annotation neural network to refer to more accurate context information when processing image blocks. For example, assuming that the target context cache matrix is ​​C and the target parameter matrix is ​​P, the optimized target context cache matrix C' is C' = C+P.

[0058] By executing steps S231-S233, the target context cache matrix can be effectively optimized based on the target error index. This optimization can significantly improve the quality of the monitoring annotation image generated by the target environment monitoring annotation neural network.

[0059] As an implementation manner, step S232, determining the parameter change rate of the target generation result evaluation component according to the target error index, and debugging the original parameter matrix based on the parameter change rate to obtain the target parameter matrix, includes: Step S2321: obtaining a predetermined debugging learning rate, wherein the magnitude of the debugging learning rate is positively correlated with the magnitude of the parameter update amount of the iterative debugging of the original parameter matrix; Step S2322: iteratively debug the original parameter matrix according to the debugging learning rate, and optimize the target context cache matrix through the debugged parameter matrix in each generation. At the same time, after each optimization of the target context cache matrix, obtain the error index and parameter change rate corresponding to the x-th image block scanning order of the corresponding generation again, until the error index corresponding to the x-th image block scanning order meets the set error index requirement, and determine the parameter matrix after debugging of the corresponding generation as the target parameter matrix, wherein the parameter matrix after debugging of each generation is determined by the parameter matrix before debugging of each generation, the parameter change rate corresponding to each generation and the debugging learning rate.

[0060] In step S232, through these two steps, the original parameter matrix is ​​gradually debugged to obtain the target parameter matrix that can optimize the target context cache matrix, thereby improving the performance of the target environment monitoring and annotation neural network and making the generated monitoring and annotation image more consistent with the target monitoring and annotation information.

[0061] In step S2321, a predetermined debugging learning rate is obtained, and the size of the debugging learning rate is positively correlated with the size of the parameter update amount of the iterative debugging of the original parameter matrix. The debugging learning rate is an important hyperparameter, which controls the update amplitude of the original parameter matrix during the iterative debugging process. In the optimization algorithm, such as the gradient descent algorithm, the update of the parameters is based on the current parameter change rate and the debugging learning rate. If the debugging learning rate is set too large, the update amount of the parameters will be very large during each iteration, which may cause the algorithm to fail to converge, and even the parameter value divergence will occur, making the optimization process unable to achieve the desired effect. On the contrary, if the debugging learning rate is set too small, the update amount of the parameters will be very small during each iteration, the algorithm convergence speed will be very slow, and more iterations will be required to achieve the desired optimization effect, which will increase the calculation time and resource consumption.

[0062] The debugging learning rate can be predetermined in a variety of ways. One way is to use a fixed learning rate, that is, to select a suitable constant as the debugging learning rate based on experience or experimental results. For example, in many deep learning tasks, commonly used fixed learning rate values ​​are 0.01, 0.001, etc. Another way is to adopt a learning rate decay strategy, that is, to gradually reduce the debugging learning rate as the number of iterations increases. This strategy can use a larger learning rate in the early stage of the optimization process so that the parameters can quickly approach the optimal value, and then use a smaller learning rate in the later stage for more refined adjustments to avoid missing the optimal solution. Feasible learning rate decay methods include step decay, exponential decay, etc. Taking exponential decay as an example, its calculation formula is: ,in, is the debugging learning rate at the tth iteration, is the initial debugging learning rate, is the decay coefficient, and t is the number of iterations.

[0063] In step S2322, the original parameter matrix is ​​iteratively debugged according to the debugging learning rate, and the target context cache matrix is ​​optimized by the debugged parameter matrix in each generation. At the same time, after each optimization of the target context cache matrix, the error index and parameter change rate corresponding to the x-th image block scanning order of the corresponding generation are obtained again, until the error index corresponding to the x-th image block scanning order meets the set error index requirement, and the parameter matrix after debugging of the corresponding generation is determined as the target parameter matrix. The parameter matrix after debugging of each generation is determined by the parameter matrix before debugging of each generation, the parameter change rate corresponding to each generation and the debugging learning rate.

[0064] After obtaining the debugging learning rate, start iterative debugging of the original parameter matrix. In each generation of iteration, first determine the parameter change rate of the target generation result evaluation component based on the current original parameter matrix and the target error index. The parameter change rate can be obtained by finding the gradient of the error index with respect to the elements in the original parameter matrix. In deep learning, the gradient represents the rate of change of a function at a certain point. By calculating the gradient of the error index with respect to the parameters, we can know how to adjust the parameters to reduce the error index. For example, using the mean square error (MSE), use the chain rule to find the partial derivatives of the MSE with respect to the elements in the original parameter matrix to obtain the parameter change rate. Then, update the original parameter matrix according to the debugging learning rate and the parameter change rate. Assume that the current original parameter matrix is , the debugging learning rate is , the parameter change rate of the tth generation is , then the parameter matrix update formula for the t+1th generation is: , this formula indicates that in each iteration, the original parameter matrix is ​​updated in the opposite direction of the parameter change rate, and the update amplitude is controlled by the debugging learning rate.

[0065] Next, the target context cache matrix is ​​optimized using the debugged parameter matrix. Specifically, the debugged parameter matrix is ​​added to the target context cache matrix to obtain the optimized target context cache matrix. Then, the initial environment monitoring image is loaded into the target environment monitoring annotation neural network, a new image block is generated based on the optimized target context cache matrix, and the error index and parameter change rate corresponding to the scanning order of the xth image block of the corresponding generation are obtained again.

[0066] This iterative process is repeated until the error index corresponding to the x-th image block scanning order meets the set error index requirement. The set error index requirement is a pre-set standard used to determine whether the error index is within an acceptable range. For example, the threshold of the error index is set to 0.1. When the error index corresponding to the x-th image block scanning order is less than 0.1, it is considered that the error index meets the requirement. At this time, the parameter matrix after the corresponding generation of debugging is determined as the target parameter matrix, and the element value in the target parameter matrix is ​​the value for optimizing the target context cache matrix. During the iterative debugging process, loop structures and optimization algorithms (such as gradient descent algorithms) are used to update parameters. At the same time, the entire iterative process can be monitored and recorded, and the changes in the error index can be analyzed so as to adjust the debugging learning rate or optimization algorithm in time to ensure that the iterative process can converge efficiently.

[0067] By executing steps S2321-S2322, the original parameter matrix can be effectively iterated and debugged according to the target error index and the debugging learning rate, and finally the target parameter matrix can be obtained. The quality of the monitoring annotation image generated by the target environment monitoring annotation neural network can be significantly improved.

[0068] As an implementation method, step S400, loading the initial environment monitoring image into the target environment monitoring annotation neural network to generate an optimized image, and outputting the optimized monitoring annotation image, includes: Step S410: loading the initial environment monitoring image into the target environment monitoring annotation neural network, generating a second image block features, and the context cache matrix corresponding to the x-th image block scanning order has been optimized; Step S420: loading a second image block features into the target generation result evaluation component respectively, and obtaining a evaluation results again, wherein the a evaluation results obtained again all represent that the corresponding image blocks meet the target monitoring annotation information; Step S430: performing sampling processing on the features of a second image blocks to obtain an optimized monitoring and annotated image.

[0069] In step S400, through these steps, the previously optimized context cache matrix is ​​fully utilized to generate a high-quality optimized monitoring annotation image that meets the target monitoring annotation information.

[0070] In step S410, the initial environment monitoring image is loaded into the target environment monitoring annotation neural network to generate a second image block features, and the context cache matrix corresponding to the xth image block scanning order has been optimized. The initial environment monitoring image is the original image data obtained by shooting the target monitoring environment area by an unmanned aerial vehicle, which contains rich environmental information of the target area. For example, when monitoring an urban area, the image may contain information on elements such as buildings, roads, and green areas. The target environment monitoring annotation neural network is a pre-trained and optimized model, such as a transformer network, which has the ability to process and analyze the input image and generate the corresponding monitoring annotation image. When loading the initial environment monitoring image, it is necessary to ensure that the format and size of the image meet the input requirements of the target environment monitoring annotation neural network. Image preprocessing technology can be used, such as using the OpenCV library to resize and normalize the image to convert the image into a format suitable for neural network processing.

[0071] In the previous step, the context cache matrix corresponding to the scanning order of the x-th image block has been optimized based on the target error index. The context cache matrix contains feature mapping pairs corresponding to each feature weight focusing component in multiple feature weight focusing components. These feature mapping pairs record the feature information of the previously processed image blocks, which helps the target environment monitoring and annotation neural network to better understand the context information of the current image block. The optimized context cache matrix can provide more accurate reference information for the neural network, allowing the neural network to make more reasonable decisions when processing image blocks.

[0072] The target environment monitoring annotation neural network processes the image blocks one by one according to the image block scanning order. The image block scanning order usually uses a left-to-right and top-to-bottom method to block and process the initial environment monitoring image in sequence. When processing each image block, the neural network combines the optimized context cache matrix and generates the corresponding image block features through internal calculations and transformations. After a series of processing, a second image block features are finally obtained. These features can more accurately reflect the content and characteristics of the image blocks, laying the foundation for the subsequent generation of high-quality optimized monitoring annotation images.

[0073] In step S420, the features of a second image blocks are loaded into the target generation result evaluation component respectively, and a evaluation results are obtained again, wherein the a evaluation results obtained again all represent that the corresponding image blocks meet the target monitoring annotation information. The target generation result evaluation component is a component used to evaluate the output results of the target environment monitoring annotation neural network, and its structure includes a fully connected layer, a nonlinear activation function layer, and a fully connected layer connected in sequence. The fully connected layer is used to linearly transform the input feature vector, and the nonlinear activation function layer introduces nonlinear factors to increase the expression ability of the model.

[0074] The a second image block features are respectively input into the target generation result evaluation component, and the component processes each feature according to the internal calculation rules to obtain the corresponding evaluation result. The evaluation result reflects the degree to which each image block meets the target monitoring annotation information. The target monitoring annotation information specifies the monitoring annotation requirements for the target monitoring environment area, such as the annotation style, the image content to be annotated, etc. Since the optimized context cache matrix is ​​used in step S410, the generated second image block features are more in line with the requirements of the target monitoring annotation information, so the a evaluation results obtained again indicate that the corresponding image blocks meet the target monitoring annotation information.

[0075] For example, when monitoring a forest environment, the target monitoring annotation information requires that tree areas be annotated with a specific color and grass areas be annotated with another color. After the optimized context cache matrix, the image blocks corresponding to the second image block features generated by the target environment monitoring annotation neural network can be accurately annotated as required, and the evaluation results given by the target generation result evaluation component show that these image blocks meet the annotation requirements.

[0076] In step S430, the features of the a second image blocks are sampled to obtain an optimized monitoring and annotated image. The sampling process is to select appropriate features from the features of the a second image blocks to construct the final optimized monitoring and annotated image. A variety of sampling methods can be used, such as random sampling, stratified sampling, etc. Random sampling is to randomly select a certain number of features from the features of the a second image blocks; stratified sampling is to divide the image blocks into different layers according to certain features (such as position, category, etc.), and then extract a certain number of features from each layer.

[0077] In the sampling process, the rationality and representativeness of the sampling need to be considered to ensure that the selected features can accurately reflect the situation of the entire target monitoring environment area. For example, when monitoring a large urban area, the environmental characteristics of different areas may be different. A stratified sampling method can be used to stratify the city according to different functional areas (commercial areas, residential areas, industrial areas, etc.), and then extract an appropriate amount of second image block features from each layer.

[0078] After sampling, the image blocks corresponding to the selected second image block features are spliced ​​and combined to construct a complete optimized monitoring and annotation image. During the splicing process, it is necessary to ensure that the position and order of the image blocks are correct to form a coherent and accurate image. The final optimized monitoring and annotation image is the result generated by the optimized target environment monitoring and annotation neural network, in which each image block meets the requirements of the target monitoring and annotation information and can accurately reflect the actual situation of the target monitoring environment area.

[0079] By executing steps S410 - S430, the optimized context cache matrix can be fully utilized to generate high-quality optimized monitoring annotation images, improve the accuracy and reliability of environmental monitoring, and provide strong support for environmental monitoring work.

[0080] As an embodiment, the method further includes: Step S24: performing standardization processing on the a first image block features and the a second image block features respectively to obtain a first probability density and a second probability density; Step S25: determining a target distribution deviation based on the first probability density and the second probability density, where the target distribution deviation is used to evaluate the deviation between the first probability density and the second probability density; Step S26: Optimize the target context cache matrix along the direction that minimizes the target distribution deviation.

[0081] Steps S24 - S26 are further supplements to the image generation and optimization process in the UAV-based environmental monitoring method. Through these steps, the first image block features and the second image block features are deeply analyzed to evaluate the optimization effect and further optimize the target context cache matrix, thereby improving the accuracy and reliability of the entire environmental monitoring annotation.

[0082] In step S24, a first image block features and a second image block features are respectively standardized to obtain a first probability density and a second probability density. The first image block feature is the result of loading the initial environment monitoring image into the target environment monitoring annotation neural network, and the rth feature weight focusing component in the target environment monitoring annotation neural network is output when the context cache matrix is ​​not optimized; the second image block feature is the corresponding feature obtained by loading the initial environment monitoring image into the target environment monitoring annotation neural network again after the context cache matrix is ​​iteratively optimized based on the error index. Standardization is a feasible method for data preprocessing, and its purpose is to convert data of different ranges and scales into data with the same scale and distribution for subsequent analysis and comparison. Feasible standardization methods include Z-score standardization and Min-Max standardization.

[0083] After standardizing a first image block features and a second image block features respectively, standardized feature data are obtained. In order to further analyze the distribution of these feature data, their probability density is calculated. The probability density function describes the probability distribution of random variables at each value point. For continuous random variables, the probability density function can be estimated by methods such as kernel density estimation. The basic idea of ​​kernel density estimation is to place a kernel function around each data point, and then perform weighted summation of these kernel functions to obtain the probability density function of the entire data set. For example, using the Gaussian kernel function, the first probability density of the first image block feature and the second probability density of the second image block feature are obtained through kernel density estimation. For example, when monitoring a forest environment, the first image block feature may contain feature information of trees, grass and other areas before optimization, and the second image block feature is the feature information of the corresponding area after optimization. Through standardization and probability density estimation, the distribution changes of these features before and after optimization can be clearly seen.

[0084] In step S25, the target distribution deviation is determined based on the first probability density and the second probability density, and the target distribution deviation is used to evaluate the deviation between the first probability density and the second probability density. The target distribution deviation can measure how much the distribution of image block features has changed after the target context cache matrix is ​​optimized, thereby reflecting the degree of influence of the optimization operation on the output results of the target environment monitoring and annotation neural network. Feasible indicators for measuring distribution deviation are KL divergence (Kullback-Leibler divergence) and JS divergence (Jensen-Shannon divergence). The calculation formula of KL divergence is: , where P(x) is the first probability density and Q(x) is the second probability density. KL divergence represents the amount of information loss from probability distribution P to probability distribution Q. The larger its value, the greater the difference between the two distributions. It should be noted that KL divergence is asymmetric, that is, .

[0085] JS divergence is an improvement on KL divergence. It is symmetrical and its calculation formula is: ,in The JS divergence value range is between [0, 1], where a value of 0 means that the two distributions are exactly the same, and a value of 1 means that the two distributions are completely different.

[0086] By calculating the target distribution deviation, the difference in distribution between the first image block feature and the second image block feature can be intuitively understood. For example, if the target distribution deviation is small, it means that the optimization operation of the target context cache matrix has not changed the distribution of the image block feature much, which may mean that the optimization effect is not obvious; on the contrary, if the target distribution deviation is large, it means that the optimization operation has a significant impact on the distribution of the image block feature, which may mean that the optimization has achieved a good effect.

[0087] In step S26, the target context cache matrix is ​​optimized along the direction that minimizes the size of the target distribution deviation. The target context cache matrix is ​​a cache mechanism used by the target environment monitoring and annotation neural network when processing image blocks. It contains feature mapping binaries corresponding to each feature weight focusing component in multiple feature weight focusing components. These feature mapping binaries record the feature information of the previously processed image blocks and are used to help the target environment monitoring and annotation neural network better understand the context information of the current image block. The goal is to adjust the element values ​​in the target context cache matrix so that the target distribution deviation between the first probability density and the second probability density is minimized. This can be achieved through an optimization algorithm, such as a gradient descent algorithm. The basic idea of ​​the gradient descent algorithm is to update the parameters along the negative gradient direction of the objective function so that the value of the objective function gradually decreases. In this problem, the objective function is the target distribution deviation, and the parameters are the element values ​​in the target context cache matrix.

[0088] Assume that the target distribution deviation is ,in Represents the element value in the target context cache matrix. The update formula of the gradient descent algorithm is: ,in is the element value in the target context cache matrix at the tth iteration, is the learning rate, is the target distribution deviation about gradient.

[0089] By continuously iteratively updating the element values ​​in the target context cache matrix, the target distribution deviation is gradually reduced. When the target distribution deviation reaches a smaller value or meets the preset convergence condition, the iteration process stops, and the target context cache matrix obtained at this time is the optimized result.

[0090] Through the execution of steps S24-S26, the distribution difference between the first image block feature and the second image block feature can be deeply analyzed, the effect of the target context cache matrix optimization can be evaluated, and the target context cache matrix can be further optimized to improve the quality of the monitoring and annotation image generated by the target environment monitoring and annotation neural network. In practical applications, these steps can help to monitor and annotate the target environment area more accurately, providing a more reliable basis for environmental monitoring work.

[0091] As an implementation mode, the method further includes a pre-training process of an environmental monitoring annotation neural network, specifically including: Step S1: Obtain image training samples without labels; Step S2: pre-debugging the initial environment monitoring annotation neural network based on the image training sample to obtain the target environment monitoring annotation neural network, wherein the initial environment monitoring annotation neural network and the target environment monitoring annotation neural network execute a global feature weight focusing strategy on the loaded image and execute a local feature weight focusing strategy on the output image.

[0092] Steps S1-S2 in the pre-training process of the environmental monitoring annotation neural network are an important basis for building an effective neural network model. Through these two steps, from data collection to model training, the pre-debugging of the initial environmental monitoring annotation neural network is gradually completed, and finally the target environmental monitoring annotation neural network is obtained.

[0093] In step S1, image training samples without labels are obtained. Image training samples are data used to train neural networks. Without labels, these images do not have pre-labeled monitoring information, such as the specific locations and categories of various environmental elements in the image (such as trees in the forest, buildings in the city, etc.) are not clearly marked. These image training samples can be obtained in a variety of ways, such as from public image data sets, satellite image databases, or through large-scale photography of drones in different environmental areas. Taking the monitoring of urban environment as an example, a large number of images of urban areas can be downloaded from the satellite image database. These images contain various elements such as streets, buildings, and greenery in the city, but these elements are not labeled in detail.

[0094] Obtaining unlabeled image training samples. On the one hand, a large amount of unlabeled image data is relatively easy to obtain, which can provide rich information for neural networks to learn the common features of images; on the other hand, unsupervised learning methods can allow neural networks to automatically discover potential patterns and structures in images without relying on a large amount of manual annotation.

[0095] In step S2, the initial environment monitoring annotation neural network is pre-debugged based on the image training sample to obtain the target environment monitoring annotation neural network. The initial environment monitoring annotation neural network and the target environment monitoring annotation neural network perform a global feature weight focusing strategy on the loaded image and a local feature weight focusing strategy on the output image. The initial environment monitoring annotation neural network is a pre-built neural network model with certain structure and parameters, but has not been fully trained for the environment monitoring annotation task.

[0096] The global feature weighted focusing strategy, also known as global attention, means that when a neural network processes an input image, it can pay attention to the overall features and global information of the image. For example, when processing an image of a forest environment, the global attention mechanism allows the neural network to take into account information such as the overall layout of the forest and the distribution range of vegetation. The global attention mechanism is usually implemented by calculating the degree of correlation between each position in the input image. A feasible calculation method is to use the attention score. Assume that the feature representation of the input image is , where N is the number of features in the image and D is the dimension of the features. The global attention mechanism calculates the attention score To measure the degree of correlation between different features, the attention score can be calculated using the dot product attention formula: , where Q and K are the query matrix and key matrix respectively, d k is the dimension of the key vector.

[0097] The local feature weight focusing strategy, also known as local attention, means that the neural network can focus on the local area and details of the image when outputting the image. For example, when annotating trees in a forest, the local attention mechanism allows the neural network to more accurately identify the boundaries, types and other details of the trees. The local attention mechanism is usually achieved by limiting the scope of attention and only focusing on the local area around the current position.

[0098] When pre-debugging the initial environment monitoring annotation neural network, unsupervised learning algorithms such as autoencoders and generative adversarial networks are used. Taking the autoencoder as an example, the goal of the autoencoder is to encode the input image into a low-dimensional representation, and then decode the output image from this low-dimensional representation as similar as possible to the input image. The loss function of the autoencoder can use the mean square error (MSE), which is calculated as follows: ,in is the input image, is the decoded output image and n is the number of samples.

[0099] During the training process, the parameters of the initial environmental monitoring annotation neural network are continuously adjusted so that the value of the loss function gradually decreases. As the training progresses, the neural network will learn the feature representation and potential structure of the image. After enough training iterations, the target environmental monitoring annotation neural network is obtained. This network can better process environmental monitoring images and perform better in subsequent environmental monitoring annotation tasks.

[0100] When implementing these steps, a variety of technical means can be used. In terms of obtaining image training samples, web crawler technology can be used to download images from public image data sets and databases, or historical images can be used as samples, subject to the permission of laws and regulations.

[0101] See also Figure 3 , is a schematic diagram of the structure of a monitoring system 10 provided in an embodiment of the present application. Figure 3 As shown, the monitoring system 10 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, the above-mentioned monitoring system 10 may also include: a user interface 1003, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, the user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and the user interface 1003 may optionally include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may optionally also be at least one storage device away from the aforementioned processor 1001. As Figure 3 As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

[0102] exist Figure 3 In the monitoring system 10 shown, the network interface 1004 can provide network communication functions; the user interface 1003 is mainly used to provide an input interface; and the processor 1001 can be used to call the device control application stored in the memory 1005 to implement the method provided in the above embodiment.

[0103] It should be understood that the monitoring system 10 described in the embodiment of the present application can execute the above Figure 2 The description of the environmental monitoring method based on drone in the corresponding embodiment. In addition, the description of the beneficial effects of the same method will not be repeated.

[0104] As an example, the above-mentioned program instructions may be deployed and executed on a monitoring system, or deployed and executed on at least two monitoring systems at one location, or executed on at least two monitoring systems distributed at at least two locations and interconnected through a communication network. At least two monitoring systems distributed at at least two locations and interconnected through a communication network may constitute a blockchain network.

[0105] The computer-readable storage medium may be the central storage unit of the monitoring system provided in any of the aforementioned embodiments, such as a hard disk or a memory of the monitoring system. The computer-readable storage medium may also be an external storage device of the monitoring system, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the monitoring system. Furthermore, the computer-readable storage medium may also include both the central storage unit of the monitoring system and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the monitoring system. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0106] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish between contents in different media, rather than to describe a specific order. In addition, the terms "including" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, device, product, or equipment that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules that are not listed, or may optionally include other step units inherent to these processes, methods, devices, products, or equipment.

[0107] The present application also provides a computer program product, including a computer program / instruction, which implements the above-mentioned Figure 2 The description of the above-mentioned drone-based environmental monitoring method in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated. For technical details not disclosed in the embodiments of the computer program product involved in this application, please refer to the description of the method embodiment of this application.

[0108] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0109] The method and related apparatus provided by the embodiment of the present application are described with reference to the method flow chart and / or structural diagram provided by the embodiment of the present application. Specifically, each process and / or box in the method flow chart and / or structural diagram, as well as the combination of the processes and / or boxes in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable network connection device to generate a machine, so that the instructions executed by the processor of the computer or other programmable network connection device generate instructions for implementing the process in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable network-connected device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable network connected device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process. Figure 1 A process or multiple processes and / or structures illustrate the steps of the functions specified in a box or multiple boxes. The above disclosure is only the preferred embodiment of the present application, and it is certainly not used to limit the scope of the rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope of the present application.

Claims

1. A method for environmental monitoring based on drones, characterized in that: include: Acquire an initial environmental monitoring image of a target monitoring environment area photographed by a drone, and acquire target monitoring annotation information, wherein the target monitoring annotation information is used to indicate monitoring annotation requirements for the target monitoring environment area; Loading the initial environment monitoring image into the target environment monitoring annotation neural network to generate an unoptimized image, and outputting an unoptimized monitoring annotation image; The image block features corresponding to each image block in the unoptimized monitoring annotation image are loaded into the target generation result evaluation component respectively, the error index of the target generation result evaluation component is determined respectively, and the context cache matrix is ​​iterated based on the error index, wherein the error index is used to indicate whether the image block corresponding to the image block feature satisfies the consistency of the target monitoring annotation information; The initial environmental monitoring image is loaded into the target environmental monitoring annotation neural network for optimized image generation, and an optimized monitoring annotation image is output, wherein the optimized monitoring annotation image represents the environmental monitoring image output by the target environmental monitoring annotation neural network after the context cache matrix is ​​iterated based on the error index, and each image block in the optimized monitoring annotation image satisfies the target monitoring annotation information.

2. The method according to claim 1, characterized in that in, The unoptimized monitoring and annotated image includes image blocks that do not meet the target monitoring and annotated information. The unoptimized monitoring and annotated image is constructed by image blocks output by the target environment monitoring and annotated neural network. The target environment monitoring and annotated neural network is used to determine the output image blocks one by one according to the loaded image blocks and the context cache matrix according to the image block scanning order, and determine the corresponding monitoring and annotated image based on the output image blocks. The target environment monitoring and annotated neural network includes multiple feature weight focusing components, and the context cache matrix includes feature mapping binaries corresponding to each feature weight focusing component in the multiple feature weight focusing components.

3. The method according to claim 2, characterized in that The method further comprises: The initial environment monitoring image is loaded into the target environment monitoring annotation neural network to obtain a first image block features, wherein the target environment monitoring annotation neural network includes r feature weight focusing components, the first image block feature is the output result of the rth feature weight focusing component, the target environment monitoring annotation neural network outputs a image blocks one by one in the a image block scanning order through repeated iterations, the s+1th image block is determined by the sth image block output by the target environment monitoring annotation neural network in the s image block scanning order and the sth context cache matrix, which are loaded into the target environment monitoring annotation neural network in the s+1 image block scanning order, the sth context cache matrix is ​​used to represent the r feature mapping binaries corresponding to the s image block scanning order, the r feature mapping binaries are matched one by one with the r feature weight focusing components, and the a first image block features are used to determine the unoptimized monitoring annotation image constructed by the a image blocks, wherein r≥2, a≥s+1, s≥1; The a first image block features are loaded into the target generation result evaluation component respectively to obtain a evaluation results, and the target context cache matrix is ​​optimized, wherein the target context cache matrix represents the context cache matrix corresponding to the target image block output by the target environment monitoring annotation neural network after being loaded into the target environment monitoring annotation neural network, and the target image block includes the image block determined by the a evaluation results that does not meet the target monitoring annotation information; The initial environment monitoring image is loaded into the target environment monitoring annotation neural network, a second image block features are determined based on the optimized target context cache matrix, and the optimized monitoring annotation image is generated, wherein the acquisition process of the second image block features is consistent with the acquisition process of the first image block features.

4. The method according to claim 3, characterized in that The step of loading the a first image block features into the target generation result evaluation component respectively, obtaining a evaluation results, and optimizing the target context cache matrix comprises: Loading the a first image block features into the target generation result evaluation component respectively, determining a error indicators, and matching the a evaluation results with the a error indicators one by one; Determine a target error index corresponding to the x-th image block scanning order from the a error indexes, wherein the target error index is an error index that does not meet the set error index requirement, x≤a; The target context cache matrix is ​​optimized based on the target error indicator.

5. The method according to claim 4, characterized in that The optimizing the target context cache matrix based on the target error index includes: Get the adjustable original parameter matrix; Determining a parameter change rate of the target generation result evaluation component according to the target error index, and debugging the original parameter matrix based on the parameter change rate until a target parameter matrix is ​​obtained, wherein element values ​​in the target parameter matrix are values ​​for optimizing the target context cache matrix; The target context cache matrix and the target parameter matrix are added, and the added matrix is ​​determined as the optimization result of the target context cache matrix.

6. The method according to claim 5, characterized in that Determining the parameter change rate of the target generation result evaluation component according to the target error index, and debugging the original parameter matrix based on the parameter change rate to obtain the target parameter matrix, includes: Obtaining a predetermined debugging learning rate, wherein the magnitude of the debugging learning rate is positively correlated with the magnitude of the parameter update amount of the iterative debugging of the original parameter matrix; The original parameter matrix is ​​iteratively debugged according to the debugging learning rate, and the target context cache matrix is ​​optimized by the debugged parameter matrix in each generation. Meanwhile, after each optimization of the target context cache matrix, the error index and parameter change rate corresponding to the x-th image block scanning order of the corresponding generation are obtained again, until the error index corresponding to the x-th image block scanning order meets the set error index requirement, and the debugged parameter matrix of the corresponding generation is determined as the target parameter matrix, wherein the debugged parameter matrix of each generation is determined by the parameter matrix before debugging of each generation, the parameter change rate corresponding to each generation and the debugging learning rate.

7. The method according to claim 4, characterized in that The step of loading the initial environment monitoring image into the target environment monitoring annotation neural network to generate an optimized image and outputting an optimized monitoring annotation image comprises: Loading the initial environment monitoring image into the target environment monitoring annotation neural network to generate the a second image block features, wherein the context cache matrix corresponding to the x-th image block scanning order has been optimized; The a second image block features are loaded into the target generation result evaluation component respectively, and a evaluation results are obtained again, wherein the a evaluation results obtained again all represent that the corresponding image blocks meet the target monitoring annotation information; The a second image block features are sampled to obtain the optimized monitoring annotated image.

8. The method according to claim 3, characterized in that The method further comprises: Performing standardization processing on the a first image block features and the a second image block features respectively to obtain a first probability density and a second probability density; determining a target distribution deviation based on the first probability density and the second probability density, the target distribution deviation being used to evaluate a deviation between the first probability density and the second probability density; The target context cache matrix is ​​optimized along a direction that minimizes the size of the target distribution deviation.

9. The method according to claim 1, characterized in that: The method further comprises: Obtain image training samples without labels; The initial environment monitoring annotation neural network is pre-debugged based on the image training samples to obtain the target environment monitoring annotation neural network, wherein the initial environment monitoring annotation neural network and the target environment monitoring annotation neural network execute a global feature weight focusing strategy on the loaded image and execute a local feature weight focusing strategy on the output image.

10. A monitoring system, characterized in that: include: one or more processors; and one or more memories, wherein the memories store computer-readable codes, which, when executed by the one or more processors, enable the one or more processors to perform the method as claimed in any one of claims 1 to 9.

Images