End-to-end multi-source visual model and large language model collaborative meteorological intelligent analysis method

Abstract

The application discloses an end-to-end multi-source visual model and large language model cooperative meteorological intelligent analysis method, which is different from a simple cloud top height inversion model and an ordinary LLM question and answer system, and is an end-to-end meteorological intelligent analysis method that cooperatively combines a multi-source meteorological visual model, a tool calling link, a structured physical feature and LLM inference capability, so that the system can automatically complete meteorological data grabbing, cloud top height inversion, index statistics, knowledge enhanced explanation, multi-round question and answer and report generation from a natural language instruction, and constrain LLM output through a structured physical anchor point, thereby reducing numerical illusion and physical inconsistency problems in meteorological analysis.

Description

Technical Field

[0001] This invention relates to the field of meteorological analysis technology, and in particular to a method and system for intelligent meteorological analysis that combines an end-to-end multi-source visual model with a large language model. Background Technology

[0002] Cloud top height (CTH) is a key physical parameter in weather monitoring, severe convection identification, aviation safety, radiation budget estimation, and numerical weather prediction data assimilation. High cloud top height areas are often associated with deep convective clouds, cumulonimbus clouds, and severe convective weather processes. Therefore, rapid inversion and interpretation of regional CTH is an important aspect of intelligent meteorological operations.

[0003] The Advanced Geostationary Radiometric Imager (ARI) on the Fengyun-4B satellite can provide multi-channel, high spatiotemporal resolution remote sensing data, suitable for cloud top height area inversion in regions such as the Yellow Sea and Bohai Sea. CALIPSO (Cloud-AerosolLidar and Infrared Pathfinder Satellite Observation), equipped with a lidar system, can provide high-precision cloud height observations along its orbit. Although its spatial coverage is sparse, it can serve as a high-precision training or validation reference.

[0004] With the development of deep learning, multimodal visual models based on architectures such as U-Net, Transformer, and Cross-Attention have been increasingly used for cloud top height inversion. These models can learn nonlinear mappings from multi-channel remote sensing data and output pixel-level cloud top height fields. However, visual models are typically only responsible for "computing physical quantities" and do not possess the capabilities for natural language task understanding, multi-step planning, knowledge retrieval, and interactive question answering.

[0005] Meanwhile, the development of large language models and agent technologies enables models to perform a thinking-action loop based on the ReAct (Reasoning and Acting) mechanism and to schedule external tools through Function Calling. RAG (Retrieval-Augmented Generation) technology can introduce professional meteorological knowledge from external knowledge bases into the LLM context, improving the professionalism and interpretability of the answers.

[0006] However, while existing Large Language Models (LLMs) possess strong capabilities in natural language understanding, task planning, and report generation in meteorological analysis scenarios, they lack the direct ability to read multi-channel remote sensing tensors, perform cloud top height inversion, and verify the consistency of meteorological physical quantities. When users raise complex questions such as "whether there are strong convective clouds in a certain area at a certain time," ordinary LLMs often can only make inferences based on general textual knowledge, which easily leads to problems such as unclear data sources, numerical fabrication, inconsistent physical interpretations, and difficulty in reproducing data.

[0007] On the other hand, existing cloud top height (CTH) retrieval models (such as the infrared window method, CO2 slicing method, and binary star geometric parallax method) typically only output pixel-level dense prediction results, which are mostly in the form of two-dimensional or three-dimensional arrays, making them difficult for LLMs to reliably understand and utilize. If remote sensing images or high-dimensional tensors are directly given to LLMs for interpretation, not only is the context length insufficient to carry millions of pixels of information, but LLMs also lack strong constraints on numerical physical quantities, which can easily lead to numerical illusions in key indicators such as average cloud top height, maximum cloud top height, high cloud coverage, and strong convection risk rate. Summary of the Invention

[0008] This invention addresses the problems of existing technologies by proposing an end-to-end collaborative meteorological intelligent analysis method and system that integrates multi-source meteorological visual models, tool call chains, structured physical features, and LLM inference capabilities with a large language model. This system can automatically complete meteorological data acquisition, cloud top height inversion, indicator statistics, knowledge enhancement interpretation, multi-round question answering, and report generation starting from natural language commands. Furthermore, by constraining LLM output through structured physical anchors, it reduces numerical illusions and physical inconsistencies in meteorological analysis.

[0009] To achieve the above objectives, the present invention provides the following technical solution:

[0010] In a first aspect, the present invention provides an end-to-end intelligent meteorological analysis method that combines a multi-source visual model with a large language model, comprising the following steps:

[0011] Receive natural language meteorological tasks from users, and perform intent parsing, parameter extraction, task planning, and tool scheduling on the cognitive center of the large language model;

[0012] Based on the scheduling results, the satellite data reading tool in the multi-source meteorological data input module is invoked to acquire and preprocess satellite remote sensing data;

[0013] The dual-stream vision model is invoked to perform cloud top height inversion on the preprocessed satellite remote sensing data, and the dense cloud top height prediction results are output.

[0014] The cloud top height dense prediction results are statistically calculated by the structured expression module to generate structured semantic features that include regional cloud top height statistical indicators and strong convection risk criteria.

[0015] The structured semantic features are injected into the context of the large language model in a controlled prompt template manner through the physical reality anchor injection module, and knowledge enhancement is performed by retrieving meteorological knowledge base and generating knowledge through retrieval enhancement module.

[0016] Meteorological analysis reports and / or risk assessment results are generated by a large language model based on injected structured physical anchors and enhanced knowledge;

[0017] During the generation of the meteorological analysis report, the system checks the consistency between the numerical values ​​appearing in the report and the corresponding numerical values ​​in the structured semantic features. If the error exceeds a preset threshold, it triggers regeneration or tool re-call.

[0018] Furthermore, the dual-stream vision model adopts a dual-stream U-Net network structure, in which one branch extracts multi-channel spectral radiation features and the other branch extracts spatial texture or auxiliary physical features. The two branches are fused through a cross-attention module and the decoder outputs the cloud top height prediction field.

[0019] The total training loss of the dual-stream vision model is expressed as:

[0020] ,

[0021] in, For monitoring loss of the isometric cloud top height, For CALIPSO orbital monitoring loss, For spatial smoothing constraint loss, , , These are the weighting coefficients;

[0022] The area-based supervised loss is expressed as:

[0023] ,

[0024] in, This represents the set of valid pixels in the training samples. Indicates the effective pixel mask. Indicates the referenced cloud top height label;

[0025] The CALIPSO orbital monitoring loss is expressed as:

[0026] ,

[0027] in, This represents the set of pixels traversed by the CALIPSO orbit. This represents the high-precision cloud height observation value from CALIPSO.

[0028] The spatial smoothing constraint loss is expressed as:

[0029] ,

[0030] in, This represents the predicted cloud top height at the (i,j)th pixel location.

[0031] Furthermore, the structured semantic features generated by the structured expression module include at least: task parameters, data source, model inference state, cloud top height statistics, cloud height distribution, spatial location features, strong convection risk criteria, physical anchor constraints, retrieval enhancement generation retrieval context, and credibility assessment results.

[0032] The cloud top height statistics include the regional average cloud top height, the regional highest cloud top height, the proportion of high clouds, and the proportion of deep convective clouds.

[0033] The severe convection risk criterion is based on the severe convection risk rate. express:

[0034] ,

[0035] in, For the proportion of high cloud, For the proportion of deep convective clouds, This is the highest cloud top altitude in the region. For reference to normalized height, , , The weighting coefficients are satisfied. Furthermore, the step of the physical anchor injection module performing numerical consistency checks includes:

[0036] Let the actual physical anchor point value calculated by the tool in the structured semantic features be . The corresponding value extracted from the large language model generation report is Then the relative error of the kth indicator Represented as:

[0037] ,

[0038] in To prevent extremely small constants with a denominator of zero;

[0039] Overall numerical consistency score Represented as:

[0040] ,

[0041] When any indicator satisfies At that time, it was determined that there was a risk of numerical inconsistency in the output of the large language model, among which If a preset error threshold is set, the system will trigger a regeneration or tool re-call.

[0042] Furthermore, the aforementioned end-to-end multi-source visual model and large language model collaborative meteorological intelligent analysis method also includes evaluation and optimization steps:

[0043] The overall credibility score is calculated through the evaluation and optimization module. :

[0044] ,

[0045] in, For numerical consistency score, For tool call accuracy, For physical consistency score, , , The weighting coefficients are and satisfy the following conditions: ;

[0046] The accuracy of the tool call , Indicates the correct number of tool calls. Indicates the total number of tool calls;

[0047] The physical consistency score , This indicates the number of judgments that satisfy the meteorological and physical rules. Indicates the total number of rules involved in the evaluation;

[0048] The overall credibility score is fed back to task planning prompts, tool parameter constraints, and report generation templates for continuous optimization.

[0049] Secondly, this invention proposes an end-to-end intelligent meteorological analysis system that combines a multi-source visual model with a large language model, comprising:

[0050] The user interaction module is used to receive meteorological analysis tasks from users' natural language input.

[0051] The large language model cognitive center is used for user task intent parsing, parameter extraction, task planning, and tool scheduling;

[0052] The tool layer includes satellite data reading tools, latitude and longitude positioning tools, cloud height inversion tools, statistical analysis tools, visualization and plotting tools, and report generation tools, which are used to perform specific computational tasks for scheduling the cognitive center of the large language model.

[0053] The multi-source meteorological data input module is used to provide FY-4B multi-channel satellite data and CALIPSO high-precision orbit data;

[0054] A dual-stream visual model is used to map satellite remote sensing observations into dense cloud top height predictions, and is encapsulated as the cloud height inversion tool.

[0055] The structured representation module is used to perform statistical analysis and semantic compression on the cloud top height density prediction results to generate structured semantic features;

[0056] The physical reality anchor injection module is used to inject the structured semantic features into the context of the large language model in a controlled prompt template manner, and to perform consistency checks on the values ​​in the large language model's generated report.

[0057] The retrieval enhancement and knowledge generation enhancement module is used to retrieve relevant knowledge from the meteorological knowledge base and introduce it into the large language model for reasoning.

[0058] The output results module is used to output meteorological analysis reports, multi-round Q&A, risk assessments, and visualization results.

[0059] The evaluation and optimization module is used to comprehensively score the system output based on numerical consistency, tool invocation accuracy, and analysis reliability, and provide feedback for optimization.

[0060] Furthermore, the cognitive center of the large language model includes:

[0061] The intent parsing unit is used to identify whether a user request belongs to one or more combinations of data query, cloud top height inversion, strong convection risk assessment, spatial positioning, visualization drawing, or report generation.

[0062] The parameter extraction unit is used to extract time, region, satellite type, spatial range, indicator type, output format, and follow-up context from natural language.

[0063] The task planning unit uses the ReAct mechanism to break down complex meteorological analysis tasks into multiple executable sub-tasks;

[0064] The tool scheduling unit uses a Function Calling mechanism to generate standardized tool call requests. Each tool call request includes the tool name, input parameters, output field constraints, and exception handling strategy.

[0065] Furthermore, the dual-stream vision model uses FY-4B multi-channel satellite data as its primary input, and the input tensor is represented as follows: Where C represents the number of satellite channels, and H and W represent spatial resolution; the model outputs a cloud top height prediction map. ;

[0066] The dual-stream vision model is represented as follows: ,in This is a branch for encoding spectral radiation features. Encoding branches for spatial textures or auxiliary physical features. For cross-attention modules, For decoders;

[0067] The dual-stream vision model uses FY-4B cloud top height labels as surface supervision during the training phase, and constructs an orbital mask in the CALIPSO orbital passage area to apply orbital supervision.

[0068] Furthermore, the steps by which the structured representation module processes the cloud top height density prediction results include:

[0069] Let the effective cell set be... , If we represent the effective pixel mask, then:

[0070] Regional average cloud top height ;

[0071] Highest cloud top height in the region ;

[0072] High cloud share ,in The threshold for determining high cloud cover;

[0073] percentage of deep convective clouds ,in The threshold for determining deep convective clouds;

[0074] This is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise.

[0075] Furthermore, the numerical consistency check rules set by the physical real anchor injection module include: when the error between the numerical value appearing in the final report and the corresponding numerical value in the structured semantic features exceeds a preset threshold, or when a numerical value that does not exist in the structured semantic features appears, the system will trigger regeneration or prompt the large language model to call the tool to supplement the calculation.

[0076] The meteorological knowledge base of the enhanced knowledge generation module includes the physical meaning of cloud top height, rules for identifying convective clouds, the relationship between cumulonimbus clouds and high cloud top height, the physical meaning of FY-4B satellite channels, empirical rules for cloud top height thresholds, and remote sensing meteorological analysis specifications.

[0077] The evaluation and optimization module feeds back the comprehensive credibility score to the task planning prompts, tool parameter constraints, and report generation templates, forming a closed-loop optimization.

[0078] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0079] This invention differs from simple cloud top height inversion models and ordinary LLM question-answering systems. Instead, it proposes a meteorological intelligent analysis method and system that combines multi-source visual models, structured physical anchors, RAG meteorological knowledge enhancement, and tool call closed loops. Starting from natural language questions, it automatically completes task parsing, data reading, model inference, statistical analysis, knowledge retrieval, and report generation, reducing manual operation steps. It converts the cloud top height physical quantity output by the visual model into structured semantic features and injects them as physical reality anchors into the LLM context, ensuring that the analysis conclusions are supported by real data and reducing numerical illusions and physical inconsistencies in meteorological analysis. Through JSON structured field constraints, data... With its value consistency check and tool re-call mechanism, this invention can reduce the problem of fabrication in key values ​​such as cloud height, proportion, location, and risk level in LLM. Through the RAG knowledge enhancement module, the system can combine cloud top height knowledge, strong convection identification rules, and the physical meaning of satellite channels to interpret the model output, not only providing conclusions but also explaining the basis for judgment. The tool layer can be further expanded to include tools such as radar data reading, precipitation estimation, typhoon identification, external weather interfaces, automatic mapping, and business report templates, making it suitable for building larger-scale meteorological intelligent agent systems. LLM can be deployed through inference frameworks such as vLLM, and the visual model can be resident on GPUs or batch-processed for inference. The overall link is suitable for low-latency, reusable business applications. Attached Figure Description

[0080] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0081] Figure 1 This is an architecture diagram of an end-to-end multi-source visual model and large language model collaborative meteorological intelligent analysis system provided in an embodiment of the present invention.

[0082] Figure 2The flowchart illustrates the end-to-end multi-source visual model and large language model collaborative meteorological intelligent analysis method provided in this embodiment of the invention. Detailed Implementation

[0083] To better understand this technical solution, the method of the present invention will be described in detail below with reference to the accompanying drawings.

[0084] 1) To address the problem that LLM cannot directly process multi-source meteorological remote sensing data, this invention provides a meteorological intelligent agent with LLM as the cognitive center and multi-source meteorological visual model as the core computing tool, which enables natural language tasks to be automatically transformed into executable data reading, model reasoning, statistical analysis and report generation processes.

[0085] 2) To address the problem that visual model outputs are difficult for LLM to reliably understand, this invention converts the dense prediction results of cloud top height into structured semantic features in the form of JavaScript Object Notation (JSON), making indicators such as average cloud top height, highest cloud top height, high cloud percentage, and strong convection risk rate physical anchors that can be referenced by LLM.

[0086] 3) To address the numerical illusion problem in LLM meteorological analysis, this invention constructs a "structured physical anchor injection" mechanism, which constrains the actual calculation results when generating LLM reports and conducting multi-round question and answer, thereby improving numerical consistency and physical credibility.

[0087] 4) To address the lack of knowledge enhancement and interpretation capabilities in existing meteorological systems, this invention constructs a meteorological knowledge base and a RAG knowledge enhancement module, enabling the Agent to combine cloud top height knowledge, convective cloud identification rules, and the physical meaning of satellite channels to interpret model outputs.

[0088] 5) To address the difficulty in quantifying and evaluating Agent output, this invention establishes a closed-loop evaluation index that includes numerical consistency, tool invocation accuracy, and analysis reliability, for continuous optimization of task planning and tool invocation links.

[0089] Example 1

[0090] like Figure 1 As shown, this invention proposes an end-to-end collaborative meteorological intelligent analysis system based on structured physical anchor injection and multi-source visual model and large language model. The system includes a user interaction module, an LLM cognitive hub, a tool layer, a multi-source meteorological data input module, a dual-flow visual model, a structured expression module, a physical reality anchor injection module, a RAG knowledge enhancement module, an output results module, and an evaluation and optimization module.

[0091] Users input meteorological analysis tasks via natural language, such as "Analyze whether there are strong convective cloud clusters in the Yellow and Bohai Sea regions on June 30, 2023." The LLM cognitive center performs intent parsing, parameter extraction, ReAct task planning, and Function Calling tool scheduling for this task. Based on the scheduling results, the tool layer calls satellite data reading tools, latitude and longitude positioning tools, cloud height inversion tools, statistical analysis tools, visualization and plotting tools, and report generation tools.

[0092] The cloud top height inversion tool uses a dual-stream visual model to invert cloud top height from FY-4B multi-channel satellite data, and combines it with CALIPSO high-precision orbital data as a training or validation reference. After the visual model outputs the CTH dense prediction results, the structured expression module calculates indicators such as average cloud top height, maximum cloud top height, high cloud percentage, and strong convection risk rate, and organizes them into JSON structured semantic features. These structured physical features are injected into the LLM context through the physical reality anchor injection module, and together with the meteorological knowledge obtained from RAG retrieval, participate in the final inference and report generation.

[0093] 1. LLM Cognitive Center

[0094] The LLM cognitive center is used to convert user natural language questions into actionable tasks. First, the intent parsing unit identifies whether the user request belongs to one or more combinations of data query, cloud top height inversion, strong convection risk assessment, spatial positioning, visualization drawing, or report generation. Second, the parameter extraction unit extracts time, region, satellite type, spatial range, indicator type, output format, and follow-up context from the natural language.

[0095] The task planning unit employs the ReAct mechanism to break down complex meteorological analysis tasks into multiple executable sub-tasks. For example, for the task of "analyzing whether there are strong convective cloud clusters in the Yellow and Bohai Seas on a certain day," the planning unit generates the following steps: read FY-4B data for the specified date; determine the latitude and longitude range of the Yellow and Bohai Seas region; call the cloud height inversion model; calculate the proportion of high clouds and the highest cloud top height; retrieve the strong convection identification rules; and generate a risk assessment and text report.

[0096] The Function Calling tool scheduling unit generates standardized tool call requests based on the mission planning results. Each tool call request includes the tool name, input parameters, output field constraints, and exception handling strategies. For example, the parameters for the satellite data reading tool include date, time, area boundaries, and channel list; the parameters for the cloud height inversion tool include preprocessed multi-channel tensors, model version, and inference mode; and the parameters for the report generation tool include JSON structured features, RAG retrieval results, and output format requirements.

[0097] 2. Tool layer

[0098] The tool layer serves as the execution interface between the LLM cognitive center and the actual computational process. The satellite data reading tool reads FY-4B multi-channel satellite data and performs channel selection, missing data processing, and data format conversion. The latitude and longitude positioning tool crops remote sensing data according to user-specified areas or latitude and longitude ranges. The statistical analysis tool performs regional statistics on the model output results. The visualization and plotting tool generates high cloud distribution maps, cloud top height heat maps, or risk area maps. The report generation tool organizes the structured results and LLM-generated content into a readable analysis report.

[0099] 3. Multi-source meteorological data input module

[0100] Multi-source data inputs include FY-4B multi-channel satellite data and CALIPSO high-precision orbital data. FY-4B data is used for areal cloud top height inversion during the online inference phase; CALIPSO data is primarily used for sparse high-precision supervision and offline validation during the training phase to improve the visual model's ability to fit real cloud top heights. During online operational inference, even without a CALIPSO orbit passing through, the system can still complete regional CTH inversion based on FY-4B multi-channel data.

[0101] 4. Dual-stream visual model

[0102] A dual-stream vision model is used to map multi-source remote sensing observations to dense cloud top height predictions. The model primarily uses FY-4B multi-channel satellite data as input, and the input tensor can be represented as follows: Where C represents the number of satellite channels, and H and W represent the spatial resolution. The model outputs a cloud top height prediction map. .

[0103] Specifically, let the FY-4B multi-channel satellite observation data be:

[0104]

[0105] Where C represents the number of satellite channels, and H and W represent the spatial height and width, respectively. The two-stream visual model is denoted as... Its output cloud top height prediction field is:

[0106]

[0107] in, This represents the predicted cloud top height at the (i,j)th pixel location.

[0108] In terms of model structure, this invention employs a dual-stream U-Net cloud top height inversion network. One branch extracts multi-channel spectral radiation features, while the other branch extracts spatial texture or auxiliary physical features. The two branches interact through a Cross-Attention feature fusion module, enabling the model to establish correlations between different modalities or feature scales.

[0109] Furthermore, let the spectral radiation feature coding branch be... Spatial texture or auxiliary physical feature encoding branch is Then the two branches are obtained as follows:

[0110]

[0111] Through cross attention modules The two types of features are fused and then processed by the decoder. Output cloud top height prediction field:

[0112]

[0113]

[0114] Through the aforementioned dual-flow structure, the model can simultaneously utilize multi-channel spectral radiation information and spatial texture information, thereby improving the ability of cloud top height inversion results to characterize deep convective clouds, high cloud region boundaries, and local strong cloud top structures.

[0115] During training, the model uses FY-4B cloud top height labels as planar supervision and constructs an orbital mask in the CALIPSO orbital pass-through region to apply additional supervision to high-precision orbital cloud height samples. The total loss can be composed of a weighted average of the planar supervision loss and the orbital supervision loss, thus balancing spatial continuity and high-precision physical calibration. After the model training is completed, it is packaged into a cloud height inversion tool, which can be invoked by the agent through Function Calling.

[0116] During the training phase, the total loss of the model can be further expressed as:

[0117]

[0118] in, For monitoring loss of the isometric cloud top height, For CALIPSO orbital monitoring loss, For spatial smoothing constraint loss, , , These are the weighting coefficients;

[0119] The area-based supervised loss is expressed as:

[0120]

[0121] in, This represents the set of valid pixels in the training samples. Indicates the effective pixel mask. Indicates the referenced cloud top height label;

[0122] The CALIPSO orbital monitoring loss is expressed as:

[0123]

[0124] in, This represents the set of pixels traversed by the CALIPSO orbit. This represents the high-precision cloud height observation value from CALIPSO.

[0125] The spatial smoothing constraint loss is expressed as:

[0126]

[0127] in, Let represent the predicted cloud top height at the (i,j)th pixel location. This loss is used to constrain the cloud top height prediction field to maintain reasonable spatial continuity and reduce isolated abrupt changes caused by local noise, missing interpolation, or channel anomalies.

[0128] 5. Structured Expression Module

[0129] The CTH dense prediction results output by the visual model contain millions of pixels of information, which cannot be directly and completely incorporated into the LLM context. To bridge the semantic gap between high-dimensional visual results and LLM language inference, this invention sets up a structured expression module to perform index statistics, region analysis, and semantic compression on the prediction results.

[0130] In the structured representation process, let the effective pixel set be:

[0131]

[0132] The regional average cloud top height and the regional highest cloud top height can be expressed as follows:

[0133]

[0134]

[0135] The proportions of high clouds and deep convective clouds can be expressed as follows:

[0136]

[0137]

[0138] in, The threshold for determining high clouds is, for example, 7 km. The threshold for determining deep convective clouds is, for example, 10 km. This is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise.

[0139] To convert cloud top height statistics into risk anchors that can be referenced by LLM, a strong convection risk rate can be defined as:

[0140]

[0141] in, , , Let be the weight coefficient, and satisfy... ; This indicates the reference normalized height, which can be taken as the region's historical maximum cloud top height, the experience upper limit height, or a preset height threshold. The higher the value, the greater the likelihood that the region will have high cloud tops, deep cloud bodies, and strong convective development characteristics.

[0142] The structured representation module does not simply output a single cloud top height statistic; instead, it transforms the dense cloud top height prediction field into a multi-level JSON semantic object. This object includes at least task parameters, data sources, model inference state, cloud top height statistics, cloud height distribution, spatial location characteristics, strong convection risk criteria, physical anchor constraints, RAG retrieval context, and credibility assessment results. Through this structured semantic object, LLM can explicitly reference real physical quantities when generating meteorological analysis reports, thereby reducing numerical illusions and enhancing the interpretability of results. Finally, these indicators are organized into JSON structured semantic features.

[0143] The following is an example of JSON generated by the Structured Expression module:

[0144] {

[0145] "task_info": {

[0146] "task_id": "CTH_20230630_HBH_001",

[0147] "task_type": "Analysis of strong convective cloud clusters",

[0148] "region_name": "Yellow and Bohai Sea Region",

[0149] "date": "2023-06-30",

[0150] "time_range": "00:00-23:59 UTC",

[0151] "spatial_extent": {

[0152] "lon_min": 117.0,

[0153] "lon_max": 126.0,

[0154] "lat_min": 34.0,

[0155] "lat_max": 42.0

[0156] },

[0157] "output_requirements": [

[0158] "Genting height statistics"

[0159] "Assessment of Severe Convection Risk"

[0160] "Distribution of high cloud cover areas"

[0161] "Text Analysis Report" ]

[0163] },

[0164] "data_source": {

[0165] "satellite": "FY-4B",

[0166] "sensor": "AGRI",

[0167] "channels_used": [

[0168] "Ch1",

[0169] "Ch2",

[0170] "Ch3",

[0171] "Ch4",

[0172] "Ch5",

[0173] "Ch6",

[0174] "Ch7",

[0175] "Ch8",

[0176] "Ch9",

[0177] "Ch10",

[0178] "Ch11",

[0179] "Ch12",

[0180] "Ch13",

[0181] "Ch14",

[0182] "Ch15"

[0183] ],

[0184] "auxiliary_reference": {

[0185] "name": "CALIPSO",

[0186] "usage": "High-precision orbit supervision and offline verification reference during the training phase"

[0187] },

[0188] "preprocessing": [

[0189] "Crop area",

[0190] "Latitude and longitude registration"

[0191] Outlier removal

[0192] "Channel normalization",

[0193] "Missing pixel mask processing" ]

[0195] },

[0196] "model_inference": {

[0197] "model_name": "Dual-Stream U-Net CTH Inversion Model",

[0198] "fusion_method": "Cross-Attention",

[0199] "input_shape": [15, 1024, 1024],

[0200] "output_shape": [1, 1024, 1024],

[0201] "output_variable": "cloud_top_height",

[0202] "unit": "km",

[0203] "valid_pixel_ratio": 0.96,

[0204] "inference_status": "success"

[0205] },

[0206] "cth_statistics": {

[0207] "mean_cth_km": 8.30,

[0208] "median_cth_km": 7.85,

[0209] "max_cth_km": 13.70,

[0210] "min_cth_km": 0.85,

[0211] "std_cth_km": 2.15,

[0212] "p90_cth_km": 11.20,

[0213] "p95_cth_km": 12.40,

[0214] "p99_cth_km": 13.20

[0215] },

[0216] "cloud_layer_distribution": {

[0217] "low_cloud_ratio": 0.18,

[0218] "middle_cloud_ratio": 0.26,

[0219] "high_cloud_ratio": 0.42,

[0220] "deep_convective_cloud_ratio": 0.14,

[0221] "threshold_definition": {

[0222] "low_cloud": "CTH < 3 km",

[0223] "middle_cloud": "3 km <= CTH < 7 km",

[0224] "high_cloud": "7 km <= CTH < 10 km",

[0225] "deep_convective_cloud": "CTH >= 10 km"

[0226] }

[0227] },

[0228] "spatial_features": {

[0229] "high_cloud_area_count": 4,

[0230] "main_high_cloud_regions":

[0231] {

[0232] "region_id": "HC_001",

[0233] "center_lon": 121.50,

[0234] "center_lat": 38.20,

[0235] "area_ratio": 0.18,

[0236] "mean_cth_km": 10.60,

[0237] "max_cth_km": 13.70,

[0238] "risk_level": "High Risk"

[0239] },

[0240] {

[0241] "region_id": "HC_002",

[0242] "center_lon": 123.10,

[0243] "center_lat": 36.90,

[0244] "area_ratio": 0.11,

[0245] "mean_cth_km": 9.80,

[0246] "max_cth_km": 12.30,

[0247] "risk_level": "Medium to high risk"

[0248] }

[0249] ],

[0250] "max_cth_location": {

[0251] "longitude": 121.80,

[0252] "latitude": 38.45,

[0253] "cth_km": 13.70

[0254] }

[0255] },

[0256] "convective_risk_analysis": {

[0257] "convective_risk_rate": 0.31,

[0258] "risk_level": "Medium to high risk",

[0259] "risk_basis": [

[0260] "The highest cloud top in the region reaches an altitude of 13.70 km."

[0261] "The percentage of deep cloud areas with a CTH greater than 10 km is 0.14%".

[0262] "Gao Yun's share reached 0.42",

[0263] "Continuous high cloud top structures exist in localized areas."

[0264] ],

[0265] "decision_rules": {

[0266] "high_cth_threshold_km": 10.0,

[0267] "severe_cth_threshold_km": 12.0,

[0268] "high_cloud_ratio_threshold": 0.30,

[0269] "risk_rate_threshold": 0.25

[0270] },

[0271] "final_judgement": "A distinct high-topped cloud cluster exists in the Yellow and Bohai Seas region, exhibiting characteristics of strong convective development; the overall assessment classifies it as medium-to-high risk."

[0272] },

[0273] "physical_anchor_for_llm": {

[0274] "anchor_type": "Visual model physical quantity constraints",

[0275] "anchor_variables": [

[0276] "mean_cth_km",

[0277] "max_cth_km",

[0278] "high_cloud_ratio",

[0279] "deep_convective_cloud_ratio",

[0280] "convective_risk_rate",

[0281] "max_cth_location

[0282] ],

[0283] "llm_constraint_instruction": "When generating meteorological analyses, structured physical quantities must be used as the basis; values ​​not present in this JSON must not be fabricated."

[0284] "hallucination_suppression": true

[0285] },

[0286] "rag_context": {

[0287] "knowledge_base_used": true,

[0288] "retrieved_topics": [

[0289] "Relationship between cloud top height and severe convection",

[0290] "Identification Rules for Deep Convection Clouds"

[0291] "Physical meaning of the FY-4B AGRI channel",

[0292] "High cloud cover and characteristics of convection development"

[0293] ],

[0294] "retrieval_purpose": "Used to explain the physical relationship between cloud top height statistics and the risk of severe convective weather."

[0295] },

[0296] "confidence_evaluation": {

[0297] "numerical_consistency_score": 0.97,

[0298] "tool_call_accuracy_score": 0.94,

[0299] "physical_consistency_score": 0.91,

[0300] "overall_confidence": 0.93,

[0301] "uncertainty_factors": [

[0302] "CALIPSO orbital data serves as a sparse spatial reference."

[0303] "Assessing the risk of severe convective weather still requires further verification using radar, precipitation, and reanalysis data."

[0304] "Nighttime or missing data areas may reduce the reliability of local inversion." ]

[0306] },

[0307] "output_artifacts": {

[0308] "report_available": true,

[0309] "visualization_available": true,

[0310] "generated_outputs": [

[0311] {

[0312] "type": "Meteorological Analysis Report",

[0313] "name": "HBH_CTH_Analysis_Report_20230630"

[0314] },

[0315] {

[0316] "type": "Gaoyun Area Distribution Map",

[0317] "name": "High_Cloud_Distribution_20230630"

[0318] },

[0319] {

[0320] "type": "Heat map of Genting altitude",

[0321] "name": "CTH_Heatmap_20230630"

[0322] } ]

[0324] }

[0325] }

[0326] The aforementioned JSON structured semantic features are not text freely generated by the LLM, but rather physical quantities obtained through statistical calculations from the output of the visual model. Therefore, they can serve as the real physical anchor points for subsequent LLM inference and report generation.

[0327] 6. Physically Real Anchor Point Injection Module

[0328] The Physical Reality Anchor Injection module injects structured semantic features into the LLM context using a controlled prompt template. This template requires the LLM to reference numerical fields from the JSON when generating meteorological analysis conclusions, and prohibits the creation of indicators not output by the tool. When users request information such as maximum cloud top height, risk area location, or high cloud percentage, the LLM prioritizes reading the results from the structured fields and, if necessary, calls statistical analysis tools for supplementary calculations.

[0329] During the generation process, the system sets up numerical consistency check rules. For example, when fields such as "maximum cloud top height," "average cloud top height," and "severe convective weather risk rate" appear in the final report, the system compares them with the JSON structured features. If the error exceeds a preset threshold or a value not present in the JSON is found, the system triggers regeneration or prompts the LLM to call tools for supplementary calculations. This mechanism can significantly reduce numerical illusions in LLM meteorological analysis.

[0330] Specifically, let the actual physical anchor point value calculated by the tool in the structured JSON be... The corresponding value extracted from the LLM report is Then the relative error of the k-th indicator is:

[0331]

[0332] in, To prevent extremely small constants with a denominator of zero, the overall numerical consistency score can be expressed as:

[0333]

[0334] When any indicator satisfies At that time, it was determined that there was a risk of numerical inconsistency in the LLM output, among which A preset error threshold is set. The system can trigger output acceptance, regeneration, or tool recall according to the following rules:

[0335]

[0336] Through the above mechanism, key values ​​used by LLM in its reports, such as the highest cloud top height, average cloud top height, high cloud percentage, strong convection risk rate, and location coordinates, are all constrained by structured physical anchors.

[0337] 7. RAG Knowledge Enhancement Module

[0338] The RAG knowledge enhancement module is used to introduce meteorological expertise into LLM reasoning. The meteorological knowledge base includes the physical meaning of cloud top height, rules for identifying convective clouds, the relationship between cumulonimbus clouds and high cloud top height, the physical meaning of FY-4B satellite channels, empirical rules for cloud top height thresholds, and remote sensing meteorological analysis specifications. The text in the knowledge base is segmented into knowledge fragments and stored quantitatively. During retrieval, relevant knowledge is recalled based on the user task, structured indicators, and the current dialogue context.

[0339] For example, when the JSON structured features show a high maximum cloud top height and a large proportion of high clouds, the RAG module retrieves knowledge fragments such as "deep convective clouds typically have high cloud top heights, strong vertical development, and distinct cold cloud top characteristics." LLM, based on real numerical anchors and retrieved knowledge, jointly generates an analytical conclusion regarding "whether there is a risk of strong convection." Structured physical features ensure numerical accuracy, while RAG knowledge ensures professional interpretation.

[0340] 8. Output Result Module

[0341] The output module includes meteorological analysis reports, multi-round Q&A, severe convective weather risk assessment, and high cloud area distribution maps or visualizations. For the initial analysis request, the system outputs regional cloud top height statistics, severe convective weather risk levels, key assessment criteria, and visualizations. For follow-up user inquiries, such as "Where is the highest cloud top height?", "Is there a possibility of severe convection in this area?", or "Draw a high cloud area distribution map," the LLM cognitive center continues to utilize latitude and longitude positioning tools, statistical analysis tools, or visualization tools based on the context.

[0342] The report generation tool combines structured metrics, RAG search results, and LLM language-generated results into a readable report. Key numerical values ​​in the report are derived from JSON fields output by the tool or from its statistical analysis module, avoiding arbitrary generation by the LLM language.

[0343] 9. Evaluation and Optimization Module

[0344] The evaluation and optimization module is used to form a closed-loop optimization. The numerical consistency index is used to measure whether the numerical values ​​in the final text report are consistent with the structured physical characteristics; the tool call accuracy is used to measure whether the LLM has selected the correct tools, passed in the correct time and regional parameters, and completed the necessary calls according to the process; the analysis credibility is used to evaluate whether the final conclusions conform to the physical rules in the meteorological knowledge base.

[0345] The evaluation and optimization module can integrate numerical consistency, tool invocation accuracy, and physical consistency to obtain a comprehensive reliability score.

[0346]

[0347] in, Indicates the numerical consistency score. Indicates the accuracy of tool calls. Indicates the physical consistency score. , , Let be the weight coefficient, and satisfy... The tool invocation accuracy and physical consistency score can be expressed as follows:

[0348]

[0349]

[0350] in, Indicates the correct number of tool calls. This indicates the total number of tool calls. This indicates the number of judgments that satisfy the meteorological and physical rules. This indicates the total number of rules involved in the evaluation. This comprehensive score can be fed back to task planning prompts, tool parameter constraints, and report generation templates to continuously optimize the analysis process of the meteorological intelligent agent.

[0351] In the experimental verification, three sets of comparisons can be set up: direct response from ordinary LLM, response enhanced only by RAG, and response enhanced by CV-JSON physical anchors. By statistically analyzing numerical error rate, physical consistency, and analytical credibility, the effect of structured physical anchor injection on suppressing numerical illusion in LLM is verified. The evaluation results can be fed back to task planning prompts, tool parameter constraints, and report generation templates to achieve continuous optimization.

[0352] Example 2

[0353] like Figure 2 As shown, this invention provides an end-to-end intelligent meteorological analysis method that combines a multi-source visual model with a large language model. The process includes the following steps:

[0354] S101 receives natural language meteorological tasks from users;

[0355] S102, parse the time, region, task type and output requirements in the task;

[0356] S103, invokes satellite data reading tools and latitude / longitude positioning tools;

[0357] S104, Acquire and preprocess FY-4B multi-channel satellite data;

[0358] S105, call the dual-stream visual model to perform cloud top height inversion;

[0359] S106, outputs CTH dense prediction results;

[0360] S107, Statistical analysis of regional indicators and generation of JSON structured semantic features;

[0361] S108, injects structured physical reality anchors into the LLM context;

[0362] S109, Search the meteorological knowledge base and perform RAG knowledge enhancement;

[0363] S110 generates meteorological analysis reports, multi-round Q&A, risk assessment, and visualization results;

[0364] S111, evaluated based on numerical consistency, tool call accuracy, and analysis reliability;

[0365] S112 feeds back the evaluation results to the LLM planning module and the tool calling module.

[0366] Input Example: The user inputs "Analyze whether there are strong convective clouds in the Yellow and Bohai Sea regions on June 30, 2023, and provide the main evidence." The system parses the data to show the date as June 30, 2023, the region as the Yellow and Bohai Seas, the task type as strong convective risk assessment, and the output requirements as a text report and visualization results.

[0367] Processing example: The system reads the corresponding FY-4B multi-channel satellite data, crops the Yellow Sea and Bohai Sea regions, calls the dual-stream visual model to output the CTH dense forecast field, calculates the highest cloud top height, high cloud ratio and strong convection risk rate, and retrieves the convective cloud identification rules from the meteorological knowledge base.

[0368] Output Example: The system generates a meteorological analysis report containing "regional cloud top height distribution characteristics, location of the highest cloud top height, proportion of high clouds, severe convection risk level, judgment criteria, and distribution map of high cloud areas." Key values ​​in the report are derived from structured JSON fields or tool calculations.

[0369] The difference between this invention and the prior art is that:

[0370] Comparison Dimensions Traditional cloud high inversion or ordinary LLM solution Technical solution of the present invention Core Principles Traditional cloud hyperspectral inversion focuses on physical models or single deep learning predictions; ordinary LLM focuses on text generation, and the two are disconnected from each other. Inject multi-source visual models, LLM cognitive centers, RAG knowledge enhancement, and structured physical anchors into a unified end-to-end meteorological intelligent agent. Data dependency Cloud-based high-altitude models rely on satellite data but lack natural language task understanding; LLM relies on textual knowledge but lacks real remote sensing computation. FY-4B multichannel data is used for online inference, CALIPSO is used for training / validation reference, and LLM accesses real data for computation via tool calls. Output format Visual models output high-dimensional arrays, while LLMs output natural language, making direct alignment between the two difficult. The CTH dense prediction field is transformed into JSON structured semantic features, and then an interpretable report is generated by LLM. Hallucination Control Ordinary LLMs are prone to fabricating meteorological data, and RAGs cannot guarantee that the data comes from real calculations. Numerical illusions are reduced by constraining LLM outputs through physical reality anchor injection and numerical consistency checks. Automation capabilities Traditional processes require manual operation of models, statistical indicators, graphing, and report writing. The agent automatically completes data reading, model inference, statistical analysis, plotting, multi-turn question answering, and report generation.

[0371] The key points and inventive aspects of this invention mainly include the following aspects:

[0372] 1) A meteorological intelligent agent architecture with LLM as the cognitive center and multi-source meteorological visual model as the core tool is proposed to realize the end-to-end link from natural language input to data reading, cloud top height inversion, statistical analysis, knowledge interpretation and report generation.

[0373] 2) A method is proposed to encapsulate the dual-stream U-Net cloud top height inversion model into an Agent core tool, enabling LLM to automatically call the visual model through Function Calling to complete the calculation of real remote sensing data.

[0374] 3) A structured representation method for dense CTH prediction results oriented towards LLM is proposed, which converts the million-pixel cloud top height field into JSON structured semantic features such as average cloud top height, highest cloud top height, high cloud ratio, and strong convection risk rate.

[0375] 4) A structured physical anchor injection mechanism is proposed, which uses the physical real values ​​calculated by the visual model as LLM context constraints to reduce the problems of numerical illusion and physical inconsistency in meteorological analysis.

[0376] 5) A meteorological interpretation method combining the RAG meteorological knowledge base and structured physical anchors is proposed, enabling the system to simultaneously possess real numerical evidence and professional knowledge interpretation capabilities.

[0377] 6) A closed-loop evaluation mechanism for meteorological agent output is proposed to evaluate and optimize system output from three dimensions: numerical consistency, tool call accuracy, and analysis credibility.

[0378] Compared with the prior art, the present invention has the following advantages:

[0379] 1) High degree of automation. This invention can automatically complete task parsing, data reading, model reasoning, statistical analysis, knowledge retrieval, and report generation starting from natural language problems, reducing manual operation steps.

[0380] 2) High physical reliability. This invention converts the physical quantity of cloud top height output by the visual model into structured semantic features, and injects them into the LLM context as physical reality anchors, so that the analysis conclusions are supported by real data.

[0381] 3) It can suppress numerical illusions. Through JSON structured field constraints, numerical consistency checks, and tool re-call mechanisms, this invention can reduce the fabrication of key values ​​such as cloud height, proportion, location, and risk level in LLM.

[0382] 4) Strong professional explanatory capabilities. Through the RAG knowledge enhancement module, the system can combine cloud top height knowledge, strong convection identification rules, and the physical meaning of satellite channels to interpret the model output, not only providing conclusions but also explaining the basis for the judgments.

[0383] 5) High scalability. The tool layer can be further expanded to include tools for radar data reading, precipitation estimation, typhoon identification, external weather interfaces, automatic mapping, and operational report templates, making it suitable for building larger-scale meteorological intelligent systems.

[0384] 6) Suitable for business deployment. LLM can be deployed through inference frameworks such as vLLM (large model inference deployment framework), and the visual model can be resident on the GPU or batch processed for inference. The overall pipeline is suitable for low-latency and reusable business applications.

[0385] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. However, these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for intelligent meteorological analysis using an end-to-end multi-source visual model and a large language model, characterized in that, Includes the following steps: Receive natural language meteorological tasks from users, and perform intent parsing, parameter extraction, task planning, and tool scheduling on the cognitive center of the large language model; Based on the scheduling results, the satellite data reading tool in the multi-source meteorological data input module is invoked to acquire and preprocess satellite remote sensing data; The dual-stream vision model is invoked to perform cloud top height inversion on the preprocessed satellite remote sensing data, and the dense cloud top height prediction results are output. The cloud top height dense prediction results are statistically calculated by the structured expression module to generate structured semantic features that include regional cloud top height statistical indicators and strong convection risk criteria. The structured semantic features are injected into the context of the large language model in a controlled prompt template manner through the physical reality anchor injection module, and knowledge enhancement is performed by retrieving meteorological knowledge base and generating knowledge through retrieval enhancement module. Meteorological analysis reports and / or risk assessment results are generated by a large language model based on injected structured physical anchors and enhanced knowledge; During the generation of the meteorological analysis report, the system checks the consistency between the numerical values ​​appearing in the report and the corresponding numerical values ​​in the structured semantic features. If the error exceeds a preset threshold, it triggers regeneration or tool re-call.

2. The end-to-end multi-source visual model and large language model collaborative meteorological intelligent analysis method according to claim 1, characterized in that, The dual-stream vision model adopts a dual-stream U-Net network structure, in which one branch extracts multi-channel spectral radiation features and the other branch extracts spatial texture or auxiliary physical features. The two branches are fused through a cross-attention module and the decoder outputs the cloud top height prediction field. The total training loss of the dual-stream vision model is expressed as: in, For monitoring loss of the isometric cloud top height, For CALIPSO orbital monitoring loss, For spatial smoothing constraint loss, , , These are the weighting coefficients; The area-based supervised loss is expressed as: , Where Ω represents the set of valid pixels in the training samples, Indicates the effective pixel mask. Indicates the referenced cloud top height label; The CALIPSO orbital monitoring loss is expressed as: , in, This represents the set of pixels traversed by the CALIPSO orbit. This represents the high-precision cloud height observation value from CALIPSO. The spatial smoothing constraint loss is expressed as: in, This represents the predicted cloud top height at the (i,j)th pixel location.

3. The end-to-end multi-source visual model and large language model collaborative meteorological intelligent analysis method according to claim 1, characterized in that, The structured semantic features generated by the structured expression module include at least: task parameters, data source, model inference state, cloud top height statistics, cloud height distribution, spatial location features, strong convection risk criteria, physical anchor constraints, retrieval enhancement to generate retrieval context, and credibility assessment results. The cloud top height statistics include the regional average cloud top height, the regional highest cloud top height, the proportion of high clouds, and the proportion of deep convective clouds. The severe convection risk criterion is based on the severe convection risk rate. express: in, For the proportion of high cloud, For the proportion of deep convective clouds, This is the highest cloud top altitude in the region. For reference to normalized height, , , The weighting coefficients are and satisfy the following conditions: .

4. The end-to-end multi-source visual model and large language model collaborative meteorological intelligent analysis method according to claim 1, characterized in that, The steps for the physical anchor point injection module to perform numerical consistency checks include: Let the actual physical anchor point value calculated by the tool in the structured semantic features be . The corresponding value extracted from the large language model generation report is Then the relative error of the kth indicator Represented as: , in To prevent extremely small constants with a denominator of zero; Overall numerical consistency score Represented as: , When any indicator satisfies At that time, it was determined that there was a risk of numerical inconsistency in the output of the large language model, among which If a preset error threshold is set, the system will trigger a regeneration or tool re-call.

5. The end-to-end multi-source visual model and large language model collaborative meteorological intelligent analysis method according to claim 1, characterized in that, It also includes evaluation and optimization steps: The overall credibility score is calculated through the evaluation and optimization module. : , in, For numerical consistency score, For tool call accuracy, For physical consistency score, , , The weighting coefficients are and satisfy the following conditions: ; The accuracy of the tool call , Indicates the correct number of tool calls. Indicates the total number of tool calls; The physical consistency score , This indicates the number of judgments that satisfy the meteorological and physical rules. Indicates the total number of rules involved in the evaluation; The overall credibility score is fed back to task planning prompts, tool parameter constraints, and report generation templates for continuous optimization.

6. An end-to-end intelligent meteorological analysis system that integrates multi-source visual models and large language models, characterized in that, include: The user interaction module is used to receive meteorological analysis tasks from users' natural language input. The large language model cognitive center is used for user task intent parsing, parameter extraction, task planning, and tool scheduling; The tool layer includes satellite data reading tools, latitude and longitude positioning tools, cloud height inversion tools, statistical analysis tools, visualization and plotting tools, and report generation tools, which are used to perform specific computational tasks for scheduling the cognitive center of the large language model. The multi-source meteorological data input module is used to provide FY-4B multi-channel satellite data and CALIPSO high-precision orbit data; A dual-stream visual model is used to map satellite remote sensing observations into dense cloud top height prediction results, and is encapsulated as the cloud height inversion tool. The structured representation module is used to perform statistical analysis and semantic compression on the dense prediction results of cloud top height, generating structured semantic features; The physical reality anchor injection module is used to inject the structured semantic features into the context of the large language model in a controlled prompt template manner, and to perform consistency checks on the values ​​in the large language model's generated report. The retrieval enhancement and knowledge generation enhancement module is used to retrieve relevant knowledge from the meteorological knowledge base and introduce it into the large language model for reasoning. The output results module is used to output meteorological analysis reports, multi-round Q&A, risk assessments, and visualization results. The evaluation and optimization module is used to comprehensively score the system output based on numerical consistency, tool invocation accuracy, and analysis reliability, and provide feedback for optimization.

7. The end-to-end multi-source visual model and large language model collaborative meteorological intelligent analysis system according to claim 6, characterized in that, The cognitive center of the large language model includes: The intent parsing unit is used to identify whether a user request belongs to one or more combinations of data query, cloud top height inversion, strong convection risk assessment, spatial positioning, visualization drawing, or report generation. The parameter extraction unit is used to extract time, region, satellite type, spatial range, indicator type, output format, and follow-up context from natural language. The task planning unit uses the ReAct mechanism to break down complex meteorological analysis tasks into multiple executable sub-tasks; The tool scheduling unit uses a Function Calling mechanism to generate standardized tool call requests. Each tool call request includes the tool name, input parameters, output field constraints, and exception handling strategy.

8. The end-to-end multi-source visual model and large language model collaborative meteorological intelligent analysis system according to claim 6, characterized in that, The dual-stream vision model uses FY-4B multi-channel satellite data as its primary input, and the input tensor is represented as follows: Where C represents the number of satellite channels, and H and W represent the spatial resolution; Model outputs cloud top height prediction map ; The dual-stream vision model is represented as follows: ,in This is a branch for encoding spectral radiation features. Encoding branches for spatial textures or auxiliary physical features. For cross-attention modules, For decoders; The dual-stream vision model uses FY-4B cloud top height labels as surface supervision during the training phase, and constructs an orbital mask in the CALIPSO orbital passage area to apply orbital supervision.

9. The end-to-end multi-source visual model and large language model collaborative meteorological intelligent analysis system according to claim 6, characterized in that, The steps for the structured representation module to process the cloud top height density prediction results include: Let the effective cell set be... , If we represent the effective pixel mask, then: Regional average cloud top height ; Highest cloud top height in the region ; High cloud share ,in The threshold for determining high cloud cover; percentage of deep convective clouds ,in The threshold for determining deep convective clouds; This is an indicator function that takes the value 1 when the condition inside the parentheses is true, and 0 otherwise.

10. The end-to-end multi-source visual model and large language model collaborative meteorological intelligent analysis system according to claim 6, characterized in that, The numerical consistency check rules set by the physical real anchor injection module include: when the error between the numerical value appearing in the final report and the corresponding numerical value in the structured semantic features exceeds a preset threshold, or when a numerical value that does not exist in the structured semantic features appears, the system will trigger regeneration or prompt the large language model to call the tool to supplement the calculation. The meteorological knowledge base of the enhanced knowledge generation module includes the physical meaning of cloud top height, rules for identifying convective clouds, the relationship between cumulonimbus clouds and high cloud top height, the physical meaning of FY-4B satellite channels, empirical rules for cloud top height thresholds, and remote sensing meteorological analysis specifications. The evaluation and optimization module feeds back the comprehensive credibility score to the task planning prompts, tool parameter constraints, and report generation templates, forming a closed-loop optimization.

Images