Blue carbon resource remote sensing identification and achievement intelligent review method based on multi-modal large model

Abstract

The invention provides a multi-modal large model-based blue carbon resource remote sensing recognition and intelligent result review method. The method comprises the following steps of: segmenting a remote sensing image to generate a blue carbon vegetation pattern spot unit; performing multi-modal collaborative reasoning and classification on each pattern spot; extracting global visual feature vectors of the pattern spots through an image vectorization model; extracting texture features based on a gray-level co-occurrence matrix, HSV color space statistical features and pattern spot morphological features to form remote sensing physical feature vectors; calling the multi-modal large model, inputting cue words and image feature values fusing domain knowledge, and generating semantic interpretation feature vectors; splicing the three types of feature vectors into a joint vector, performing nonlinear transformation through a multi-layer perceptron, mapping the joint vector to a classification space through a full connection layer, and outputting a category probability and confidence; when the confidence coefficient is lower than a dynamic threshold value, an error type is analyzed based on a homologous multi-modal large model, and a diagnosis result is output; and generating a pattern spot correction instruction containing an error positioning and correction strategy.

Description

Technical Field

[0001] This invention belongs to the interdisciplinary fields of remote sensing data processing, artificial intelligence and ecology. Specifically, it relates to a method for remote sensing identification and intelligent review of blue carbon resources based on a multimodal large model, which is particularly suitable for the refined identification and interpretation quality control of typical blue carbon ecosystems such as mangroves and salt marshes. Background Technology

[0002] Currently, blue carbon ecosystems play a crucial role in global carbon cycling and ecological protection, especially typical coastal vegetation such as mangroves and salt marshes, which have become important carriers for achieving strategic goals. However, remote sensing monitoring of these ecological resources has long faced technical bottlenecks. The mainstream method still relies mainly on manual visual interpretation, requiring interpreters to interpret vegetation community characteristics in high-resolution images based on their personal experience. This approach is not only time-consuming and labor-intensive, but also suffers from inconsistent results due to differences in subjective perception. Different personnel may reach significantly different interpretations of the same area, making the standardization of large-scale monitoring data extremely difficult.

[0003] With the development of artificial intelligence technology, automated classification methods based on machine learning are gradually being applied to blue carbon resource identification. While these methods have improved efficiency, they have also revealed new limitations. Traditional models often rely on end-to-end deep learning architectures, whose internal decision-making processes are like "black boxes," lacking traceable reasoning logic. When a model misclassifies a salt marsh as a mudflat or incorrectly segments a mangrove community, technicians find it difficult to trace the root cause of the error—whether it stems from spectral confusion, texture interference, or a misjudgment due to blurred boundaries. This lack of interpretability severely restricts the reliability and operational value of the results.

[0004] In recent years, the emergence of multimodal large models has provided new possibilities for remote sensing interpretation. These models demonstrate strong cross-modal correlation capabilities in natural image understanding, but their application in highly specialized remote sensing scenarios remains in the exploratory stage. Existing attempts often simply apply general architectures, failing to address the core contradictions in the remote sensing field: on the one hand, there is a semantic gap between natural language descriptions (such as "discontinuous community distribution") and pixel-level image features; on the other hand, models struggle to effectively integrate domain knowledge (such as the influence of tides on vegetation boundaries), leading to discrepancies between interpretation results and professional understanding. Furthermore, existing technical systems generally lack closed-loop review mechanisms, relying heavily on fixed thresholds to assess the confidence level of recognition results, failing to dynamically adjust standards based on patch complexity, and unable to automatically locate error types and provide correction paths.

[0005] These technological shortcomings collectively lead to a predicament in which blue carbon resource monitoring suffers from insufficient accuracy, low efficiency, and questionable reliability of results. Existing methods are often inadequate, especially when dealing with complex scenarios such as mangrove-aquaculture ecotones and mixed salt marshes and tidal creeks. Summary of the Invention

[0006] To address the shortcomings of existing technologies, such as low efficiency of manual interpretation, lack of traceability in traditional model decisions, and difficulty in integrating professional knowledge into multimodal technologies, this invention provides a scheme for remote sensing identification and intelligent review of blue carbon resources. This method achieves high-precision identification through a multimodal collaborative reasoning architecture, combines a dynamic autonomous review mechanism to ensure the traceability of results, and ultimately constructs a closed-loop optimization system for knowledge sharing. Its main innovations include:

[0007] 1. Multimodal feature deep fusion mechanism

[0008] Breaking through the limitations of traditional single-modality or simple feature stitching, a three-layer coupled remote sensing interpretation framework is established:

[0009] The visual feature layer uses a convolutional neural network to extract global information of the patches and capture the spatial distribution patterns of the vegetation community.

[0010] The physical feature layer integrates grayscale co-occurrence matrix texture, HSV color space statistics and morphological indicators (such as aspect ratio and boundary morphology) to quantify the ecological characteristics of mangroves and salt marshes.

[0011] The semantic interpretation layer innovatively designs structured cue word templates to drive multimodal large models to generate descriptions that conform to professional standards (such as "community homogeneity reaches 80%" and "boundaries are constrained by tidal lines"), and transforms them into machine-readable vectors. After nonlinear fusion of the three types of features, the classification robustness is significantly improved in complex intertidal scenarios.

[0012] 2. Confidence-driven autonomous review closed loop

[0013] Solving the "black box" problem by building a dynamically adjustable review system:

[0014] The confidence threshold is dynamically set based on Bayesian posterior or quantile statistics. When the model's judgment probability is lower than the threshold, the large multimodal model with the same source is reused for error tracing.

[0015] The diagnostic results accurately pinpoint the error type (classification confusion / vegetation mixing / boundary distortion) and its patch coordinates, and simultaneously generate actionable correction instructions—for example, outputting sub-patch segmentation coordinates for mixed communities and providing redraw vectors for boundary distortion.

[0016] Once the review is completed, the map patch update process is automatically triggered, forming an autonomous loop of "identification-diagnosis-correction".

[0017] 3. A sustainable and optimized knowledge-sharing ecosystem

[0018] Improve system adaptability through a homogeneous model architecture and expert feedback mechanism:

[0019] The semantic interpretation feature vector is stored in a shared knowledge base, ensuring the consistency of error diagnosis and feature generation knowledge;

[0020] The KL divergence of the predicted distribution and the real distribution is calculated using expert correction data, the model parameters are optimized by weighting, and the dependence on artificial labeling is continuously reduced.

[0021] This scheme is verified in mangrove forest, salt marsh and other blue carbon resource monitoring:

[0022] The misjudgment rate of easily confused vegetation such as short leaf rush is significantly reduced;

[0023] Support real-time processing of high-resolution images above 0.5 meters, and the review efficiency is several times higher than manual;

[0024] Through boundary redrawing, subgraph spot segmentation and other instruction outputs, a verifiable and traceable business closed loop is realized.

[0025] The technical scheme adopted by the present application to solve its technical problems is:

[0026] A blue carbon resource remote sensing identification and achievement intelligent review method based on a multi-modal large model, comprising:

[0027] Segmenting remote sensing images to generate blue carbon vegetation map spot units;

[0028] Performing multi-modal collaborative reasoning and classification on each spot:

[0029] Extracting global visual feature vectors of the spot through an image vectorization model;

[0030] Extracting texture features based on a gray level co-occurrence matrix, HSV color space statistical features, and spot morphological features to form remote sensing physical feature vectors;

[0031] Calling a multi-modal large model, inputting prompt words and image feature values fused with domain knowledge, and generating semantic interpretation feature vectors describing color distribution, texture regularity and boundary rationality;

[0032] Concatenating the three types of feature vectors into a joint vector, performing nonlinear transformation through a multilayer perceptron, and then mapping to a classification space through a fully connected layer and outputting class probability and confidence;

[0033] When the confidence is lower than the dynamic threshold, analyze the error type based on the homologous multi-modal large model and output the diagnosis result;

[0034] Generating spot correction instructions containing error positioning and correction strategies.

[0035] Further, the semantic interpretation feature vector is generated by: the prompt word contains the coastal zone blue carbon ecosystem type, the remote sensing interpretation sign generation task, and at least one of the image feature values of the texture feature value, the HSV value, and the patch area; and the domain knowledge contains the community composition characteristics of mangroves or salt marshes.

[0036] Further, the dynamic threshold is realized by: calculating the category-related threshold based on the Bayesian posterior probability, or dynamically adjusting the threshold based on the test set confidence quantile.

[0037] The adjustment of the dynamic threshold includes: starting the Bayesian threshold recalculation when the confidence distribution has a preset significant deviation; and updating the quantile threshold when the newly added test samples reach a preset size.

[0038] Further, the error type includes: classification error, mixed vegetation community within the patch, or distorted patch boundary delineation; and the diagnosis result contains the error positioning patch coordinates; and after the diagnosis result is output, a patch update process based on the correction instruction is triggered.

[0039] Further, in the multi-modal collaborative reasoning and classification:

[0040] The image vectorization model is a convolutional neural network.

[0041] The patch morphological features include area, aspect ratio, and boundary morphology.

[0042] The multi-layer perceptron contains two or more hidden layers.

[0043] Further, the prompt word is constructed by a structured template of ecosystem type + task instruction + feature label:

[0044] Wherein:

[0045] The ecosystem type is mangrove or salt marsh wetland.

[0046] The task instruction is “generate a description that meets the remote sensing interpretation specification”.

[0047] The feature label includes at least two of the texture feature value, the HSV value, and the patch area.

[0048] Further, the generation of the patch correction instruction includes:

[0049] When the error type is classification error, output the corrected category label.

[0050] When the error type is mixed vegetation, output the sub-patch segmentation coordinates.

[0051] When the error type is boundary distortion, output the boundary redrawing vector coordinates.

[0052] Further, the homologous multimodal large model realizes knowledge sharing by calling the same multimodal large model.

[0053] Further, the model training method comprises:

[0054] receiving the expert's corrected labeling data;

[0055] calculating the KL divergence between the model prediction distribution and the expert labeling distribution;

[0056] fine-tuning with the KL divergence weighted cross-entropy loss function.

[0057] And a blue carbon resource remote sensing identification and achievement intelligent review system, comprising:

[0058] a graph segmentation module for segmenting a remote sensing image to generate a blue carbon vegetation graph unit;

[0059] a multimodal collaborative reasoning module, which performs the following operations:

[0060] an image feature extraction unit for extracting a graph global visual feature vector through a convolutional neural network;

[0061] a physical feature extraction unit for calculating a gray level co-occurrence matrix texture feature, an HSV statistical feature, and a morphological feature;

[0062] a semantic interpretation unit for calling a multimodal large model to generate a semantic feature vector describing color distribution and boundary rationality;

[0063] a classification unit for fusing the three types of feature vectors and outputting a class probability and a confidence;

[0064] an intelligent review module for outputting an error diagnosis result based on the homologous multimodal large model when the confidence is lower than a dynamic threshold;

[0065] a correction instruction generation module for generating a graph instruction containing error positioning and correction strategies.

[0066] And a computer device comprising a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the method as described above.

[0067] A non-transitory computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the method as described above.

[0068] Compared with the prior art, the present application and its preferred schemes have at least the following beneficial effects:

[0069] 1. Solving the problem of interpretability

[0070] Through the multi-modal collaborative architecture (visual features + remote sensing physical features + semantic interpretation features), the blue carbon recognition decision-making process is transparent for the first time. The model can clearly trace the error source (such as misjudging reed as short-leaf sawgrass due to texture feature misreading), and overcome the defects of traditional "black box" model that cannot be audited:

[0071] 2. Quality of review efficiency is improved

[0072] The confidence-driven dynamic review mechanism realizes autonomous quality control:

[0073] The threshold strategy combined with Bayesian posterior and quantile statistics adapts to the complexity of the scene;

[0074] The homologous model error diagnosis function quickly locates the problem type (classification confusion / vegetation mixture / boundary distortion);

[0075] The polygon-level correction instruction (sub-polygon segmentation coordinates, etc.) significantly reduces the time consumption of manual intervention.

[0076] 3. Optimization of resources throughout the chain

[0077] Training phase: Expert feedback mechanism reduces dependence on manual annotation;

[0078] Deployment phase: Knowledge sharing design improves processing efficiency;

[0079] Application phase: Compatible with 0.5-meter or higher resolution images, suitable for coastal monitoring needs.

[0080] 4. Breakthrough in business landing ability

[0081] Through the polygon update triggering mechanism, a traceable achievement management system is established to support mangrove carbon sink accounting, salt marsh protection and other strategic needs. BRIEF DESCRIPTION OF DRAWINGS

[0082] The present application will be further described in detail below in combination with the drawings and specific embodiments:

[0083] Figure 1 The polygon image map in a specific embodiment of the present application;

[0084] Figure 2 The overall flowchart of the embodiment of the present application;

[0085] Figure 3 The RCA-Net framework diagram of the embodiment of the present application;

[0086] Figure 4 The intelligent review implementation flowchart in a specific embodiment of the present application. DETAILED DESCRIPTION

[0087] In order to make the features and advantages of the present application more apparent, the following detailed examples are given, described in detail as follows.

[0088] It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the application. Unless otherwise defined, all technical and scientific terms used in this description have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs.

[0089] It should be noted that the terms used herein are only for the purpose of describing specific embodiments, and are not intended to limit the exemplary embodiments according to the present application. As used herein, the singular form is intended to include the plural form unless the context clearly indicates otherwise, and it should also be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of the features, steps, operations, devices, components and / or combinations thereof.

[0090] In order to solve the core problems of "low automation" and "black box identification difficult to review" in blue carbon remote sensing identification, the present application proposes a blue carbon resource remote sensing identification and achievement intelligent review method based on multi-modal large model, RCA-Net (Remote-sensing Classification & Assessment Network). The scheme combines the semantic understanding ability of multi-modal large model, the professional extraction ability of remote sensing feature engineering and the decision ability of neural network classifier, and innovatively constructs a system architecture for blue carbon remote sensing identification and achievement quality evaluation. Through the deep fusion of remote sensing object data, remote sensing features and human visual interpretation signs, the organic connection of key links such as patch identification, quality judgment, problem analysis and auxiliary correction is realized, and a closed-loop working mechanism covering remote sensing achievements from output to review is constructed.

[0091] The core technical advantage of the present application is that not only the coupling application of image understanding and remote sensing knowledge is realized, but also the explanatory analysis and expert feedback mechanism are introduced, so that the system can not only accurately identify the patch category, but also intelligently analyze the error causes of the patch, assist manual accurate correction of the interpretation results, and significantly improve the transparency, review efficiency and use trust of the achievements. The system has good adaptability and scalability, and is especially suitable for fine remote sensing identification and achievement quality control of typical coastal blue carbon ecosystems such as mangrove forests and salt marshes.

[0092] The purpose of the present application is to provide a blue carbon resource remote sensing identification and achievement intelligent review method based on multi-modal large model, RCA-Net, which can realize the whole process closed-loop processing from remote sensing image input, interpretation mapping, achievement quality evaluation, error attribution analysis to man-machine collaborative correction, and overcome the technical difficulties of low efficiency, strong subjectivity and lack of explainability in traditional manual review.

[0093] To achieve the above object, the embodiment of the present application adopts the following technical solutions:

[0094] As shown in Figure 2 , Figure 3 , the embodiment of the present application provides a method for remote sensing identification and achievement intelligent review of blue carbon resources based on a multi-modal large model, RCA-Net, comprising the following steps:

[0095] Step S1: Collecting high-resolution remote sensing images of the coastal zone, pre-processing and patch segmentation of the data based on a deep learning segmentation model (U-Net, SAM, etc.), extracting blue carbon vegetation patches, and forming a patch unit set to be classified .

[0096] Step S2: Using the patch unit set , inputting the remote sensing identification and review model RCA-Net, classifying the patches j (j = 1, 2, 3, … n), obtaining a classification result set , and outputting the confidence of the patch belonging to category j as the basis for achievement quality review.

[0097] Step S3: When ≥ 0.5, it can be determined that the patch meets the quality requirements, and when <0.5, it is considered that the patch does not meet the quality requirements.

[0098] Step S4: When encountering a patch that does not meet the quality requirements, use a multi-modal large model to analyze the reasons, including classification errors, patch boundary blurring, and non-single community.

[0099] Step S5: According to the error reason, perform manual patch correction.

[0100] As a preferred scheme of the embodiment, in step S1, due to the small area of marine blue carbon vegetation types, the resolution is high, and the present application suggests using remote sensing image data with a resolution of 0.5m or above, or using orthophotos taken by a drone. In addition, the present application can adapt to any number of image bands, so as to flexibly adapt to the needs of various experimental / production data.

[0101] As a preferred scheme of the embodiment, in step S2, the embodiment uses a multi-modal vector model, a multi-modal large language model, and combines a remote sensing feature extraction method to develop RCA-Net, and uses 80% of the data in the study area for training and 20% for verification. The model vectorizes the data from multiple dimensions by 3 vector extraction layers (picture data vectorization, remote sensing feature extraction, and human visual interpretation feature vectorization), and classifies using a neural network to output the category , confidence value :

[0102] (1) Picture data vectorization: using a pre-trained multi-modal vectorization model (such as qwen-multimodal-embedding-v1, etc.) to convert the standardized image patch data into a 1024-dimensional vector;

[0103] (2) Remote sensing feature extraction: using remote sensing feature extraction methods to extract image patch texture features (GLCM, etc.), color features (R, G, B, HSV, etc.), shape features (length, width, aspect ratio, etc.) into feature vectors, geographic area description, band synthesis, sign image, field photo, etc. The specific features can be selected according to the research area and research object requirements.

[0104] (3) Human-computer visual interpretation feature vectorization: using a pre-trained multi-modal large language model (such as qwen-vl-max, Doubao-vision-pro-32k, etc.) combined with prompt words (domain knowledge: remote sensing, vegetation, coastal zone, carbon sink, mangrove, salt marsh. Task purpose: generate remote sensing visual interpretation signs, including color, texture, image patch morphology, distribution state, boundary rationality, community singleness, etc. Image features: texture features are <float32>, area is <folat32>, the aspect ratio of the map patch is <float32>etc.), human-computer visual interpretation feature extraction is carried out, and the interpreted feature of natural language description data can be automatically output.

[0105] On this basis, a pre-trained text vector model (such as qwen-multimodal-embedding-v1, Doubao-embedding, etc.) is used to convert the interpreted feature into a 1024-dimensional vector.

[0106] (4) Neural network: merge the three vector layers, use a feedforward neural network to map the vector to seven categories (the number of categories is determined according to the number of regional marine blue carbon resource categories), and use gradient descent method when the model accuracy meets the requirements, then complete the training.

[0107]

[0108] In the formula, is the picture data feature vector, is the remote sensing feature vector, is the human-computer visual interpretation feature vectorization, represents the splicing operation of the feature vector, which connects the feature vectors of different modalities into a long vector. MLP is a multilayer perceptron (Multilayer Perceptron) used for nonlinear transformation of the spliced feature vector to extract higher-level feature representation. is a fully connected layer (Fully Connected Layer) used to map the feature vector output by the MLP to a classification space for final classification training, The function converts the output of the fully connected layer into a probability distribution, and each element represents the probability that the input belongs to the corresponding category.

[0109] In addition, in order to avoid the influence of the accuracy of sample labels on the precision of the model, the application designs a model fine tuner based on multi-expert feedback driving, selects multiple experts to judge the test data, so that the model is more robust, and the formula is as follows:

[0110]

[0111] In the formula, represents the update amount of the matrix , is the learning rate, represents the gradient of the weight matrix , that is, the partial derivative of the loss function with respect to W, is the cross-entropy loss, is the regularization coefficient, is the divergence calculation, which is used to measure the difference between the human behavior distribution and the model prediction distribution.

[0112] After the training is completed, the model can be put into use, and the input image patch The model can automatically determine the confidence of belonging to the category The confidence of belonging to the category , automatically identify the classification, and review the quality of remote sensing interpretation results.

[0113] As a preferred scheme of the embodiment, in steps S3 and S4, the quality requirement can be adjusted according to the experimental / production requirement (for example, when ≥ 0.9, etc., it can be determined that the image patch meets the quality requirement. The threshold value can be a fixed value, or it can be dynamically calculated based on the Bayesian posterior or quantile.

[0114] As a preferred scheme of the embodiment, in step S4, the present application provides a deep analysis function for remote sensing extraction results, mainly using a pre-trained multi-modal large language model (such as qwen-vl-max, Doubao-vision-pro-32k, etc.), combined with a prompt word (domain knowledge: remote sensing, vegetation, coastal zone, carbon sink, mangrove, salt marsh. Task purpose: this image patch is identified by the model as having poor extraction quality, please determine what the main problems are, error types include classification error, community is not single, boundary is unreasonable, etc. Image features: texture features are <float32>, area is <folat32>, the aspect ratio of the map patch is <float32>, data results were originally classified as <int>, the review model classifies the category as <int>, the model determines the probability as <float32>The system will output suggestions for modifying the map features and provide them to technical personnel for targeted modifications.

[0115] The advantages of the solutions provided in the embodiments of the present invention include:

[0116] (1) Reduce data dependence and improve training efficiency

[0117] By leveraging the semantic reasoning capabilities of Large Language Models (MLLM) and combining it with a domain knowledge injection mechanism, this invention significantly reduces the reliance on large-scale manually labeled data, shortening the model development cycle and reducing labor costs while maintaining accuracy.

[0118] (2) Multimodal collaborative reasoning enhances the objectivity and consistency of review.

[0119] By deeply integrating image information, remote sensing physical features, and human-computer visual interpretation features, the model can simultaneously consider spatial, spectral, and semantic multidimensional cues, making consistent and objective judgments on blue carbon patches, thus overcoming the problem that traditional manual interpretation and review are easily influenced by subjective experience.

[0120] (3) Explainable review mechanism to support accurate error correction and rapid review

[0121] The classification module has an embedded interpreter that can simultaneously provide error type labeling, confidence distribution and regional feature diagnosis, and generate targeted modification suggestions. This makes it easier for technicians to quickly identify the source of error and implement corrections, thereby improving the efficiency and reliability of results review and overcoming the difficulties of the "black box" mode of remote sensing identification.

[0122] (4) Outstanding comprehensive benefits

[0123] This invention not only excels in marine carbon sink identification, but also reduces manpower and time costs and improves the accuracy and efficiency of results review in application scenarios such as land use classification, providing a new technical path and industry benchmark for the commercial production and application of remote sensing data.

[0124] The following is a more detailed demonstration and description of the above-mentioned solution of the present invention through a more specific embodiment:

[0125] 1. Using UAV imagery, a multi-scale segmentation algorithm was employed to extract patch S1, whose category was determined by the remote sensing identification model to be reeds, denoted as... ,like Figure 1 As shown, however, this patch is actually *Cynanchum amblyceum*, which is a patch with an incorrect category labeling.

[0126] 2. Input the image patches into the intelligent review model. The intelligent review model first transforms them into a three-dimensional vector (image dimension, remote sensing feature dimension, and interpretation feature dimension).

[0127] (1) Picture dimension: converted into a 1024*1 vector, such as (-0.025735985, 0.00882494, 0.033350062, 0.012395961, 0.024915062, 0.011082483, -0.075319834, -0.062061898, 0.032016058, 0.027870391, …)

[0128] (2) Remote sensing feature dimension: extract remote sensing features, including 0° direction texture, 135° direction texture, 45° direction texture, 90° direction texture, all direction texture, HSV, maximum variance, R, G, B, R variance, G variance, B variance, area, etc., such as (0.010469387, 0.008800405, 0.008793239, 0.012760485, 0.16034022, 0.209793069, 1.206469547, 114.9826239, 126.9281302, 80.763424, 32.66933584, 24.8852285, 32.61150529, 27737.43002)

[0129] (3) Interpretation feature dimension: use multi-modal large model to extract human-computer visual interpretation features, prompt words and results are as follows:

[0130] Table 1 Human-computer visual interpretation sign generation table

[0131] Prompt words Results Domain knowledge: remote sensing, vegetation, coastal zone, carbon sink, mangrove, salt marsh. Task purpose: I hope you can help me review the remote sensing interpretation results. Please look at the following picture, which is a picture spot interpreted by remote sensing. Please describe it from the following aspects: color, texture, spot shape, area size, distribution state, rationality of spot boundary sketch, whether there is only a single community in the picture, and the proportion of other communities. Image features: you can refer to the following remote sensing feature values: 0° direction texture is 0.010469387, 135° direction texture is 0.008800405, 45° direction texture is 0.008793239, 90° direction texture is 0.012760485, all direction texture is 0.16034022, HSV value is 0.209793069, maximum variance of HSV value is 1.206469547, R value is 114.9826239, G value is 126.9281302, B value is 80.763424, R variance value is 32.66933584, G variance value is 24.8852285, B variance value is 32.61150529, area is 27737.43002 square meters. Color aspects: ① The main color feature of the patch is mainly green, reflecting that the area is mainly covered by vegetation, which is consistent with the typical color characteristics of vegetation communities (such as mangroves, salt marshes, etc.). The variation in the depth of green may reflect the different growth states of vegetation or the differences in vegetation types. The areas with some yellowish or light brown color may represent bare land, mudflats, or areas with sparse vegetation. ② Color consistency and variation: The overall color is relatively uniform, but there are some changes. These changes can help to some extent to distinguish different vegetation growth conditions or community types. Texture aspects: ① Different direction texture features: The 0° direction texture is 0.010469387, the 135° direction texture is 0.008800405, the 45° direction texture is 0.008793239, the 90° direction texture is 0.012760485, and the texture of all directions is 0.16034022. These texture values are relatively small, indicating that the texture of the vegetation within the patch does not change dramatically in different directions, which may imply that the vegetation grows relatively uniformly, but there are some subtle differences. ② Texture and vegetation relationship: From a visual perspective, the texture features within the patch are consistent with the natural growth patterns of vegetation, showing a certain regularity and continuity, which is consistent with the characteristics of coastal vegetation communities (such as mangroves or salt marshes). Patch morphology, area size, and distribution state aspects ① Patch morphology: The patch morphology is irregular, which is consistent with the distribution morphology of natural vegetation communities, especially the coastal vegetation, which is usually not regular in shape due to the influence of topography, tides, and other factors. ② Area size: The area is 27737.43002 square meters, which is moderate in size, and is reasonable for a coastal vegetation community, which can support a certain scale of ecological system. ③ Distribution state: The vegetation within the patch is relatively continuous, but there are some small gaps or patches, which may be related to the natural growth of vegetation, soil conditions, hydrological conditions, and other factors, which is consistent with the actual situation. Patch boundary drawing rationality: ① Boundary features: The drawing of the patch boundary basically follows the obvious division between vegetation and non-vegetation areas, and the boundary is relatively clear, which can accurately distinguish the vegetation community from other surrounding features. The irregularity of the boundary also reflects the level of detail in remote sensing interpretation, avoiding excessive regularization and information distortion. Community composition: ① Single community judgment: From the figure, it can be judged as a single community (possibly mangrove or salt marsh vegetation), but there are also a small amount of other communities or bare land, etc. Through the subtle changes in color and texture, it can be seen that the proportion of other communities or bare land is relatively small, estimated to be within 20%.

[0132] The above is the human-computer interpretation feature, which is converted into a 1*1024 vector using a vector model, such as (-0.048007041, -0.006232674, -0.05256933, -0.012818556, 0.020383123, -0.030169968, -0.015099701, 0.001030194, …)

[0133] 3. And through the neural network, the probability of judging as short-leaf common rush is 0.52, and the probability of judging as reed is 0.15, so it is considered that the original attribute of the picture patch is wrong, and the correct one should be short-leaf common rush.

[0134] 4. If the correct class is known, a multi-modal large model is needed to help guide the correction, and the prompt words and results are as follows:

[0135] Table 2 Correction suggestion generation table

[0136] Prompt words Results Domain knowledge: remote sensing, vegetation, coastal zone, carbon sink, mangrove, salt marsh. Task purpose: This map patch is identified by the model as having poor extraction quality. Please determine what the main issues are. Error types include classification error, non-single community, unreasonable boundary, etc. Image features: You can refer to the following remote sensing feature values: 0° direction texture is 0.010469387, 135° direction texture is 0.008800405, 45° direction texture is 0.008793239, 90° direction texture is 0.012760485, all direction texture is 0.16034022, HSV value is 0.209793069, maximum variance of HSV value is 1.206469547, R value is 114.9826239, G value is 126.9281302, B value is 80.763424, R variance value is 32.66933584, G variance value is 24.8852285, B variance value is 32.61150529, area is 27737.43002 square meters. The patch review model classifies it as short-leaf sawgrass, with a model judgment probability of 0.52. The classification error model classifies the patch as short-leaf sawgrass. According to the information provided, the model judgment probability is only 0.52, indicating that the model's classification result for this patch is not highly reliable, and there is a high possibility of classification error. The community is not single. From the image features, the vegetation distribution within the patch is complex, with different texture characteristics and color changes (such as different direction texture values and RGB value variance), which may mean that the vegetation community within the patch is not single reed or short-leaf sawgrass, but may contain multiple vegetation types growing together, making it difficult for the model to accurately classify. Other potential problems: Unreasonable boundary The boundary of the patch may not be accurate enough. Although there is no direct information indicating that the boundary has problems, due to the complexity of the vegetation community, the boundary may not accurately reflect the actual distribution of vegetation.

[0137] 5、The implementation process is as shown in the above Figure 4 The result can quickly guide manual correction.

[0138] Based on the same inventive concept, the present application further provides a computer device, comprising one or more processors, and a memory for storing one or more computer programs; the program comprises program instructions, and the processor is configured to execute the program instructions stored in the memory. The processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., which are the computing core and control core of the terminal, and are configured to implement one or more instructions, and are specifically configured to load and execute one or more instructions in the computer storage medium to implement the above method.

[0139] It needs to be further explained that, based on the same inventive concept, the present application further provides a computer storage medium, which stores a computer program, and the computer program is executed by the processor to perform the above method. The storage medium can adopt any combination of one or more computer readable media. The computer readable medium can be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example but not limited to, an electrical, magnetic, optical, infrared, or semiconductor system, device or component, or any combination of the above. More specific examples (non-exhaustive list) of the computer readable storage medium include: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer readable storage medium can be any tangible medium containing or storing a program, which can be used by or in conjunction with an instruction execution system, device or component.

[0140] It should be noted that, unless otherwise defined, technical terms or scientific terms used in the present application shall have the common meaning understood by one of ordinary skill in the art to which the present application pertains. The terms "first", "second", and the like used in the present application do not denote any order, number, or importance, but are only used to distinguish different components. The terms "include" or "contain" and the like mean that the elements or objects before the terms encompass the elements or objects listed after the terms and their equivalents, and do not exclude other elements or objects. The terms "connect" or "connected" and the like are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. The terms "upper", "lower", "left", "right", and the like are only used to represent relative positional relationships, and when the absolute positions of the described objects are changed, the relative positional relationships can also be changed accordingly.

[0141] The above description is only the preferred embodiments of the present application, and does not limit the other forms of the present application. Any person skilled in the art can modify or change the above-mentioned disclosed technology content to obtain equivalent embodiments. However, any simple modification, equivalent change, and modification of the above-mentioned embodiments without departing from the technical solution of the present application, according to the technical essence of the present application, still belongs to the protection scope of the present application.

[0142] The present application is not limited to the above-mentioned best mode, and anyone can derive other various forms of a blue carbon resource remote sensing identification and achievement intelligent review method based on a multi-modal large model under the inspiration of the present application. Any equivalent change and modification made within the scope of the present application shall be covered by the present application. < / int> < / int>

Claims

1. A method for blue carbon resource remote sensing identification and achievement intelligent review based on a multi-modal large model, characterized in that, The method comprises the following steps: Segmenting remote sensing images to generate a blue carbon vegetation map patch unit; Performing multi-modal collaborative reasoning and classification on each patch: Extracting the global visual feature vector of the patch through an image vectorization model; Extracting texture features based on a gray level co-occurrence matrix, HSV color space statistical features, and patch morphological features to form a remote sensing physical feature vector; Calling a multi-modal large model, inputting prompt words fused with domain knowledge and image feature values, and generating semantic interpretation feature vectors describing color distribution, texture regularity, and boundary rationality; Concatenating the three types of feature vectors into a joint vector, performing nonlinear transformation through a multilayer perceptron, mapping to a classification space through a fully connected layer, and outputting class probability and confidence; When the confidence is lower than a dynamic threshold, analyzing the error type based on a homologous multi-modal large model and outputting a diagnosis result; Generating a patch correction instruction containing error positioning and correction strategies.

2. The method according to claim 1, wherein the method is characterized in that: The generation of the semantic interpretation feature vector satisfies: the prompt words contain the type of coastal blue carbon ecosystem, the task of generating remote sensing interpretation signs, and at least one of the image feature values of texture feature value, HSV value, and patch area; the domain knowledge contains the community composition characteristics of mangrove or salt marsh.

3. The method according to claim 1, characterized in that: The dynamic threshold is realized by calculating the class-related threshold based on the Bayesian posterior probability or dynamically adjusting the threshold based on the confidence quantile of the test set; The adjustment of the dynamic threshold includes: starting the Bayesian threshold recalculation when the confidence distribution deviates significantly; updating the quantile threshold when the number of new test samples reaches a preset size.

4. The method according to claim 1, characterized in that: The error type includes: classification error, mixed vegetation community within the patch, or distorted patch boundary; the diagnosis result contains the error positioning patch coordinates; after the diagnosis result is output, the patch update process based on the correction instruction is triggered.

5. The blue carbon resource remote sensing identification and achievement intelligent review method based on a multi-modal large model according to claim 1, characterized in that: In the multi-modal collaborative reasoning and classification: The image vectorization model is a convolutional neural network; The patch morphological features include area, aspect ratio, and boundary morphology; The multilayer perceptron contains two or more hidden layers.

6. The blue carbon resource remote sensing identification and achievement intelligent review method based on a multi-modal large model according to claim 1, characterized in that: The prompt words are constructed through a structured template of ecosystem type + task instruction + feature label: Among them: The ecosystem type is mangrove or salt marsh wetland; The task instruction is "generate a description that meets the remote sensing interpretation specification"; The feature label includes at least two of texture feature value, HSV value, and patch area.

7. The blue carbon resource remote sensing identification and achievement intelligent review method based on a multi-modal large model according to claim 1, characterized in that: The generation of the patch correction instruction includes: When the error type is classification error, output the corrected class label; When the error type is mixed vegetation, output the sub-patch segmentation coordinates; When the error type is boundary distortion, output the boundary redrawing vector coordinates.

8. The method according to claim 1, characterized in that: The homologous multi-modal large model realizes knowledge sharing by calling the same multi-modal large model.

9. The method according to claim 1, characterized in that: The model training method comprises: Receiving the expert-corrected labeled data; Calculating the KL divergence between the model prediction distribution and the expert labeling distribution; Fine-tuning with the KL divergence weighted cross-entropy loss function.

10. A blue carbon resource remote sensing identification and achievement intelligent review system, characterized in that, Including: A graph patch segmentation module for segmenting remote sensing images to generate blue carbon vegetation graph patch units; A multi-modal collaborative reasoning module that performs the following operations: An image feature extraction unit that extracts global visual feature vectors from the graph patch using a convolutional neural network; A physical feature extraction unit that calculates gray level co-occurrence matrix texture features, HSV statistical features, and morphological features; A semantic interpretation unit that calls a multi-modal large model to generate a semantic feature vector describing color distribution and boundary rationality; A classification unit that fuses the three types of feature vectors and outputs class probability and confidence; An intelligent review module that, when the confidence is lower than the dynamic threshold, outputs an error diagnosis result based on a homologous multi-modal large model; A correction instruction generation module that generates patch instructions containing error positioning and correction strategies.

Images