Meteorological knowledge question-answering method and system and storage medium

Abstract

The invention provides a meteorological knowledge question answering method and system, and the method comprises the following steps: a data preprocessing step, an index construction step, an initial retrieval step, a reordering step, a dynamic fusion weighting step and an answer generation step, the candidate content corresponding to each candidate index in the preliminary candidate index set obtained in the initial retrieval step is input into a pre-trained multi-modal reordering model, the correlation score of each candidate content relative to the query problem is obtained, and normalization processing and ordering are conducted on the scores; in the dynamic fusion weighting step, a text modal weight and an image modal weight of a modal layer are calculated based on a normalized reordering score, the weight of the modal layer obtained through fusion calculation, a preset index layer weight and a normalized correlation score are fused, and a final comprehensive score of each candidate content is calculated. And screening out a plurality of candidate contents with high comprehensive scores according to the scores. The meteorological knowledge question-answering method and system have good accuracy, specialty and usability.

Description

Technical Field

[0002] This invention relates to the fields of meteorology and machine learning, specifically to a meteorological knowledge question-answering method, system, and storage medium based on multimodal hierarchical indexing and dynamic fusion weighting. Background Technology

[0003] With the development of meteorological science, a large amount of heterogeneous knowledge resources have been generated and accumulated in the field of meteorology. These resources include both standardized text materials (such as observation reports, technical manuals, scientific research papers and policy regulations) and a large amount of data with mixed text and graphics (such as statistical charts, cross-sectional diagrams, manuscripts and formula tables).

[0004] Currently, most weather Q&A systems rely primarily on text retrieval or use simple OCR linearization to process images before storing them in the database. This results in the flattening of the spatial structure of charts, captions, and mathematical expressions, and the omission or misinterpretation of key values, coordinate relationships, and graphic semantics, affecting the professionalism of retrieval and answers.

[0005] Another common problem is the lack of granular indexing and retrieval strategies: text is typically indexed at the paragraph or full-text level, making it difficult to cater to the different needs of conceptual queries (requiring quick topic location), mechanism queries (requiring precise matching of causal relationships and parameters), and comprehensive queries (combining both). Regarding modality fusion, differences exist between image-text vector representations and scoring scales; direct merging often leads to weight bias or information loss. Furthermore, static weighting strategies cannot dynamically adjust based on query intent and candidate distribution, resulting in slow response times for simple questions or failure to retrieve key evidence for complex questions. Summary of the Invention

[0006] The purpose of this invention is to provide a meteorological knowledge question-answering method and system that can preserve the inherent structural semantics of images and adaptively select retrieval granularity and modal weights according to question type, thereby effectively improving the overall performance of meteorological knowledge question-answering systems in terms of accuracy, professionalism and usability.

[0007] In this embodiment of the invention, a meteorological knowledge question-and-answer method is provided, which includes the following steps: S1. Data preprocessing: Preprocess the multimodal knowledge resources in the meteorological field, identify the text content in the multimodal knowledge resources and perform paragraph segmentation, and save the non-text content as image form with semantic annotation. S2. Index Construction: Generate summary index and semantic factor index for each preprocessed text content or image data. The summary index represents the core topic summary of the content, and the semantic factor index consists of multiple independent phrase units that carry clear semantic meaning. S3. Initial retrieval: The query questions entered by the user are classified, and different retrieval strategies are selected based on different query question types to retrieve the summary index and / or semantic factor index, thereby obtaining a preliminary candidate index set; S4. Reordering: Input the candidate content corresponding to each candidate index in the preliminary candidate index set into the pre-trained multimodal reordering model to obtain the relevance score of each candidate content relative to the query question, and normalize and sort the scores. S5. Dynamic Fusion Weighting: Based on the normalized reordering score, calculate the text modality weight and image modality weight of the modality layer, fuse the calculated modality layer weight, the preset index layer weight, and the normalized relevance score, calculate the final comprehensive score of each candidate content, and select the top candidate content based on the comprehensive score. S6. Answer Generation: The selected candidate content is used as context and input into the visual language model along with the user's query question to generate the final answer.

[0008] In this embodiment of the invention, the method for generating the summary index and semantic factor index in the steps is as follows: For text content, input preset text summary templates and text factor templates into the large language model to extract the summary and semantic factor array in a formatted manner; For image content, a preset image summary template and image factor template are input into the visual language model to extract a summary of the image description and an array of semantic factors in a specific format.

[0009] In this embodiment of the invention, the steps further include: vectorizing the generated summary index and semantic factor index respectively, and storing them in a vector database.

[0010] In this embodiment of the invention, the step of selecting different retrieval strategies based on different query question types to retrieve the summary index and / or semantic factor index includes: For conceptual query questions, retrieve the summary index; For mechanism-based query problems, retrieve the semantic factor index; For comprehensive query problems, both summary index and semantic factor index are retrieved simultaneously, and the results are merged through a dynamic weighting strategy.

[0011] In this embodiment of the invention, in step S4, the scores of the candidate content are normalized using a minimum-maximum normalization method.

[0012] In this embodiment of the invention, the steps In this context, the method for calculating the dynamic weights of the modal layers is as follows: Let the text candidate set be... The sum of the normalized reordering scores is Image candidate set The sum of the normalized reordering scores is Then the text modal weights Image modal weights.

[0013] In this embodiment of the invention, the steps In the index layer, the summary index weights With semantic factor index weight Configure it as follows: In conceptual queries, summary index weight Greater than semantic factor index weight ; In mechanism-based queries, summary index weight Greater than semantic factor index weight ; In comprehensive queries, summary index weight Equal to semantic factor index weight .

[0014] In this embodiment of the invention, the steps In the middle, candidate content Final overall score The calculation method is as follows: First, calculate its index score: ,in and These are the similarity scores of its summary index and semantic factor index in the initial retrieval, respectively. Then, its score is calculated by combining modal weights: if the candidate content If it is text, then If candidates If it is an image, then .

[0015] In this embodiment of the invention, a meteorological knowledge question-and-answer system is also provided, which includes: The data preprocessing module is used to preprocess multimodal knowledge resources in the meteorological field, identify the text content in the multimodal knowledge resources and perform paragraph segmentation, and save the non-text content as image form with semantic annotation. The index building module is used to generate summary indexes and semantic factor indexes for each preprocessed text content or image data. The summary index represents the core topic summary of the content, and the semantic factor index consists of multiple independent phrase units that carry clear semantic meanings. The initial retrieval module is used to determine the type of the query question entered by the user, and select different retrieval strategies based on different query question types to retrieve the summary index and / or semantic factor index, thereby obtaining a preliminary candidate index set; The reordering module is used to input the candidate content corresponding to each candidate index in the preliminary candidate index set into a pre-trained multimodal reordering model, obtain the relevance score of each candidate content relative to the query question, and normalize and sort the scores. The dynamic fusion weighting module is used to calculate the text modality weights and image modality weights of the modality layer based on the normalized reordering scores. It fuses the calculated modality layer weights, the preset index layer weights, and the normalized relevance scores to calculate the final comprehensive score of each candidate content and selects the top candidate content based on the comprehensive score. The answer generation module is used to input multiple selected candidate contents as context, along with the user's query question, into the visual language model to generate the final answer.

[0016] In this embodiment of the invention, a computer-readable storage medium is also provided, on which a computer program is stored. When the program is executed by a processor, it implements the above-described meteorological knowledge question-answering method based on multimodal hierarchical indexing and dynamic fusion weighting.

[0017] Compared with existing technologies, the invented meteorological knowledge question-and-answer method and system have at least one of the following advantages: First, at the methodological level, this application proposes an integrated approach of "image → language → two-layer index": meteorological images (including text, tables, manuscripts, formulas, etc.) are directly input into a visual language model, and a summary index and semantic factor index isomorphic to the text are constructed based on this. This approach overcomes the limitations of simply... The resulting structural loss and semantic misalignment problems can be addressed by preserving spatial relationships, annotations, and numerical elements in images, thereby achieving seamless unified representation and retrieval of graphic and textual knowledge; Secondly, regarding the retrieval and ranking mechanism, this application adopts a "two-stage retrieval + unified reordering + two-layer dynamic weighting" architecture: the first stage uses cosine similarity for fast vector retrieval to ensure efficiency; the second stage uses a unified multimodal reorderer to fine-tune all candidates on the same scale, eliminating differences in scores between modalities; subsequently, a two-layer dynamic fusion strategy of modal statistical weights and index granular weights is introduced, which can adaptively determine the importance of text / images based on the candidate set, and allocate retrieval emphasis between summary and semantic factors according to the nature of the question. This fusion strategy is both adaptive and interpretable, which is beneficial for handling the differentiated needs of conceptual, mechanism-based, and comprehensive queries; Furthermore, in terms of retrieval granularity and semantic coverage, a dual-indexing system combining summary indexes and semantic factors is adopted. This enables the system to quickly locate terms and definitions, as well as accurately match causal and parameter details in complex mechanistic questions. Semantic factors are stored in the form of phrases as independent units of meaning, facilitating the combination and reconstruction of paragraph semantics and serving as re-ranking features, significantly improving the evidence coverage of complex question answering. Attached Figure Description

[0018] Figure 1 This is a flowchart of the meteorological knowledge question-and-answer method according to an embodiment of the present invention.

[0019] Figure 2 This is a schematic diagram of the meteorological knowledge question-and-answer system according to an embodiment of the present invention. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0021] The implementation of the present invention will be described in detail below with reference to specific embodiments.

[0022] like Figure 1 As shown in the figure, this embodiment of the invention provides a meteorological knowledge question-answering method based on multimodal hierarchical indexing and dynamic fusion weighting, which includes the following steps. These steps are described below.

[0023] Data preprocessing: Preprocess multimodal knowledge resources in the meteorological field, identify the text content in the multimodal knowledge resources and perform paragraph segmentation, and save the non-text content as image form with semantic annotation.

[0024] Specifically, for the meteorological text portion, its text content is directly read and segmented into paragraphs. For the meteorological document portion, optical character recognition (OCR) technology is used to extract the text content from the document, and then segmentation is performed. For text that cannot be extracted using OCR, [further details are omitted]. For parts that are poorly extracted or have poor extraction results, such as combined text and images, tables, manuscripts, formulas, etc., use [the following methods / methods]: The model (visual language model) identifies whether the content exists on the page. If it does, the page is saved as an image, and the image is semantically annotated. This invention embodiment... The model identifies text regions in a document and effectively reduces text fragmentation through intelligent reorganization techniques using vertical alignment and horizontally adjacent text blocks. This method preserves the document's layout structure while reducing text fragmentation. The impact of parsing errors on downstream tasks.

[0025] Index building: Generate summary index and semantic factor index for each preprocessed text content or image data. The summary index represents the core topic summary of the content, and the semantic factor index consists of multiple independent phrase units that carry clear semantic meaning.

[0026] In this embodiment of the invention, a two-level index is established for each text or image description: one is a summary index (denoted as...). The first is a summary or topic information of the content; the second is a semantic factor index (denoted as...). This represents the semantic details of the content. Specifically, prompt words can be used to guide the large model to extract the semantic content of text, which is then converted into its core summary and several main semantic factors. Alternatively, a summary can be generated from the first sentence of a paragraph, while extracting keywords or concepts from the article as semantic factors. Similarly, prompt words can be used to guide the visual language large model to extract the semantic content of images, and the semantic content involved in the image can be converted into its core summary and several main semantic factors. Then, each summary and semantic factor is vectorized to represent each knowledge unit. There are two types of vectors: and Here, all indices are converted into a 1024-dimensional dense vector space and stored in a vector database. This embodiment uses the " "Vector database. In this way, the semantic content of both text and image data in the document is indexed in textual form, facilitating subsequent unified retrieval. Specific prompt word templates are shown in the following four examples:" Text summary index generation template: { You are a meteorology expert. Task: Analyze the following Chinese text (...) Extract a more detailed summary, with the following requirements: 1) Cover the main phenomenon or object, and include significant spatiotemporal or numerical information (if any). 2) The length should be controlled between 20 and 40 Chinese characters; 3) The tone is academic and formal, and no suggestions are offered.

[0027] Please return strictly. Object: (" ":"<20-40 Chinese summary>" Input text: {TEXT} }

[0028] Text factor index generation template: { You are a meteorology expert. Task: Analyze the following Chinese paragraphs ( Extract semantic factors. The returned format is... An array where each item contains only one string (in phrase form), for example: ["Factor phrase 1","Factor phrase 2","Factor phrase 3",...] Requirements: Factor phrases should be able to independently express a semantic point (6–18 characters recommended), with no upper limit on the number, and should be able to return all relevant factors completely based on paragraph information.

[0029] Input text:

[0030] }

[0031] Image summary index generation template: { You are a meteorology expert. Task: Extract a 20-40 word summary in Chinese directly from the provided image file, covering the main phenomena and key elements of the image (as shown in the image, tables, curves, and formulas exist; please reflect the main points in the summary as much as possible). Strictly adhere to the return criteria. : (" ":"<20-40 Chinese summary>" Input image:

[0032] }

[0033] Image factor index generation template: { You are a meteorology expert. Task: Extract semantic factors directly from the provided image file and return... An array whose elements are only string phrases, for example: ["Factor phrase 1","Factor phrase 2","Factor phrase 3",...] Requirements: Each factor phrase should be as independent and understandable as possible (6–18 Chinese characters recommended). There is no upper limit to the number of semantic factors. If the image contains tables or formulas, please also output the core column headings of the table or the key parts of the formula expression as factors (in phrase form).

[0034] Input image:

[0035] }

[0036] Initial retrieval: The query questions entered by the user are classified, and different retrieval strategies are selected based on different query question types to retrieve the summary index and / or semantic factor index, thereby obtaining a preliminary candidate index set.

[0037] In this embodiment of the invention, a pre-trained query discrimination model (trained using a custom-labeled dataset) is first used to perform feature analysis on the meteorological questions input by the user. The extracted features include keyword coverage, syntactic complexity, and whether multi-level meteorological concepts are involved. Based on the analysis results, the queries are divided into the following three categories: 1. Conceptual queries: These typically focus on a specific meteorological phenomenon or technical term, such as "What is a downburst?". These queries have a clear theme and concise expression, making them more suitable for quick location using a summary index to quickly obtain standard definitions or core explanations. 2. Mechanism queries: These queries often involve causal relationships or internal mechanisms, such as "The main dynamic conditions for wind shear formation." Their semantic level is deeper, containing multiple semantic factors, making them more suitable for retrieval using a semantic factor index to capture detailed explanations. 3. Comprehensive queries: These questions contain both conceptual elements and involve mechanisms, impacts, or application scenarios, such as "The classification standards for tropical cyclones and their impact on air transport." For these questions, the system simultaneously retrieves the summary index and the semantic factor index, and fuses the results using a dynamic weighting strategy to balance accuracy and comprehensiveness. Once the model determines the query type, it will process the user query. Vectorization And calculate the cosine similarity with all corresponding type indices. Cosine similarity is defined as the cosine of the angle between two vectors: , Based on this, calculate queries and indexes. Summary similarity and semantic similarity The initial search can be performed using one of two methods, or simultaneously searching both the summary index and the semantic factor index. Then, the candidates are merged, here. For the selection of candidate indices, the cosine similarity should satisfy a similarity value of at least 0.5.

[0038] The training data for the aforementioned query discrimination model comes from a manually organized and categorized corpus of meteorological queries, with each sample corresponding to a clearly labeled query type. Based on this labeled data, a training set is constructed, and the model undergoes supervised learning. During training, strategies such as cross-validation, parameter regularization, and multi-round iterative optimization are employed to ensure the model's stability in classification accuracy and its good generalization ability, thus maintaining reliable discrimination performance when faced with real-world meteorological queries of varying structures and complexities.

[0039] Reordering: Input the candidate content corresponding to each candidate index in the preliminary candidate index set into the pre-trained multimodal reordering model to obtain the relevance score of each candidate content relative to the query question, and normalize and sort the scores.

[0040] The re-ranking model can fully analyze the relationship between the query and the content, and provide a relevance score, thereby enhancing retrieval recall. This invention employs... The provided " This model is specifically designed for multilingual, multimodal ranking of text and image content. The model takes a query and a set of candidate content as input and outputs a raw relevance score for each candidate content d. Then, the scores of this batch of candidates are normalized, which can be done using a minimum-maximum normalization method: Let , The normalized score is After normalization, the candidate scores are between 0 and 1, and the ranking relationship is maintained.

[0041] Dynamic fusion and weighting: Based on the normalized reordering score, the text modality weight and image modality weight of the modality layer are calculated. The calculated modality layer weight, the preset index layer weight, and the normalized relevance score are fused together to calculate the final comprehensive score of each candidate content. Based on this score, multiple candidate contents with the highest comprehensive scores are selected.

[0042] In this embodiment of the invention, a two-layer dynamic weighting mechanism is designed, targeting the modal level and the index level respectively.

[0043] • First-level weights (modal weights): These weights are applied to the text candidate set. The sum of the normalized scores is denoted as Image candidate set The score is recorded as Then the text modal weights Image modal weights Defined as , thus This design makes the overall relevance of the query text higher. Too large, or vice versa Too large.

[0044] • Second-level weighting (index layer weighting): The weights of the summary index and semantic factor index are preset based on the complexity of the query. and Conceptual questions often have clear answers and can be set. Systemic problems require comprehensive consideration of details and should be addressed by setting up [specific mechanisms]. For comprehensive problems, the two are set to be equal. For example, this paper considers a conceptual problem. Systemic problems Comprehensive problem . Based on the above weights, the comprehensive retrieval score (index score) of candidate content d is defined as:

[0045] Then, the final score is given by combining the modal weights: if the candidate content If it comes from the text, then its final score If from an image, then .according to The candidate paragraphs and images are sorted, and the paragraphs and images with the highest scores are selected. The selected candidate content must be in the top 50% of the score distribution of all content.

[0046] Answer generation: Multiple selected candidate contents are used as context and input into the visual language model along with the user's query question to generate the final answer.

[0047] In step S6, the visual language model selected is Specifically, it can be constructed By systematically combining relevant text and image information from user questions and candidate content, the visual language model can better understand the context and output more accurate answers.

[0048] like Figure 2 As shown, corresponding to the above-mentioned meteorological knowledge question answering method based on multimodal hierarchical indexing and dynamic fusion weighting, this embodiment of the invention also provides a meteorological knowledge question answering system based on multimodal hierarchical indexing and dynamic fusion weighting, which includes a data preprocessing module 1, an index building module 2, an initial retrieval module 3, a reordering module 4, a dynamic fusion weighting module 5, and an answer generation module 6.

[0049] The data preprocessing module 1 is used to preprocess multimodal knowledge resources in the meteorological field, identify the text content in the multimodal knowledge resources and perform paragraph segmentation, and save the non-text content as image form with semantic annotation.

[0050] The index building module 2 is used to generate a summary index and a semantic factor index for each preprocessed text content or image data. The summary index represents the core topic summary of the content, and the semantic factor index is composed of multiple independent phrase units that carry clear semantic meanings. The initial retrieval module 3 is used to determine the type of the query question input by the user, and select different retrieval strategies based on different query question types to retrieve the summary index and / or semantic factor index, thereby obtaining a preliminary candidate index set.

[0051] The reordering module 4 is used to input the candidate content corresponding to each candidate index in the preliminary candidate index set into the pre-trained multimodal reordering model, obtain the relevance score of each candidate content relative to the query question, and perform normalization and sorting on the score.

[0052] The dynamic fusion weighting module 5 is used to calculate the text modality weight and image modality weight of the modality layer based on the normalized reordering score, fuse the calculated modality layer weight, the preset index layer weight and the normalized relevance score, calculate the final comprehensive score of each candidate content, and select multiple candidate contents with the highest comprehensive scores based on the score.

[0053] The answer generation module 6 is used to input multiple selected candidate contents as context, along with the user's query question, into the visual language model to generate the final answer.

[0054] In this embodiment of the invention, a computer-readable storage medium is also provided, on which a computer program is stored. When the program is executed by a processor, it implements the above-described meteorological knowledge question-answering method based on multimodal hierarchical indexing and dynamic fusion weighting.

[0055] In summary, the invented meteorological knowledge question-answering method and system based on multimodal hierarchical indexing and dynamic fusion weighting has at least one of the following advantages: First, at the methodological level, this application proposes an integrated approach of "image → language → two-layer index": meteorological images (including text, tables, manuscripts, formulas, etc.) are directly input into a visual language model, and a summary index and semantic factor index isomorphic to the text are constructed based on this. This approach overcomes the limitations of simply... The resulting structural loss and semantic misalignment problems can be addressed by preserving spatial relationships, annotations, and numerical elements in images, thereby achieving seamless unified representation and retrieval of graphic and textual knowledge; Secondly, regarding the retrieval and ranking mechanism, this application adopts a "two-stage retrieval + unified reordering + two-layer dynamic weighting" architecture: the first stage uses cosine similarity for fast vector retrieval to ensure efficiency; the second stage uses a unified multimodal reorderer to fine-tune all candidates on the same scale, eliminating differences in scores between modalities; subsequently, a two-layer dynamic fusion strategy of modal statistical weights and index granular weights is introduced, which can adaptively determine the importance of text / images based on the candidate set, and allocate retrieval emphasis between summary and semantic factors according to the nature of the question. This fusion strategy is both adaptive and interpretable, which is beneficial for handling the differentiated needs of conceptual, mechanism-based, and comprehensive queries; Furthermore, in terms of retrieval granularity and semantic coverage, a dual-indexing system combining summary indexes and semantic factors is adopted. This enables the system to quickly locate terms and definitions, as well as accurately match causal and parameter details in complex mechanistic questions. Semantic factors are stored in the form of phrases as independent units of meaning, facilitating the combination and reconstruction of paragraph semantics and serving as re-ranking features, significantly improving the evidence coverage of complex question answering.

[0056] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A meteorological knowledge question-and-answer method, characterized in that, Includes the following steps: S1. Data preprocessing: Preprocess the multimodal knowledge resources in the meteorological field, identify the text content in the multimodal knowledge resources and perform paragraph segmentation, and save the non-text content as image form with semantic annotation. S2. Index Construction: Generate summary index and semantic factor index for each preprocessed text content or image data. The summary index represents the core topic summary of the content, and the semantic factor index consists of multiple independent phrase units that carry clear semantic meaning. S3. Initial retrieval: The query questions entered by the user are classified, and different retrieval strategies are selected based on different query question types to retrieve the summary index and / or semantic factor index, thereby obtaining a preliminary candidate index set; S4. Reordering: Input the candidate content corresponding to each candidate index in the preliminary candidate index set into the pre-trained multimodal reordering model to obtain the relevance score of each candidate content relative to the query question, and normalize and sort the scores. S5. Dynamic Fusion Weighting: Based on the normalized reordering score, calculate the text modality weight and image modality weight of the modality layer, fuse the calculated modality layer weight, the preset index layer weight, and the normalized relevance score, calculate the final comprehensive score of each candidate content, and select the top candidate content based on the comprehensive score. S6. Answer Generation: The selected candidate content is used as context and input into the visual language model along with the user's query question to generate the final answer.

2. The method according to claim 1, characterized in that, In step S2, the method for generating the summary index and semantic factor index is as follows: For text content, input a pre-set text summary template and text factor template into the large language model, and extract the summary and semantic factor array in JSON format; For image content, input preset image summary templates and image factor templates into the visual language model, and extract the image description summary and semantic factor array in JSON format.

3. The method according to claim 2, characterized in that, Step S2 further includes: vectorizing the generated summary index and semantic factor index respectively, and storing them in a vector database.

4. The method according to claim 1, characterized in that, In step S3, different retrieval strategies are selected based on different query question types to retrieve the summary index and / or semantic factor index, including: For conceptual query questions, retrieve the summary index; For mechanism-based query problems, retrieve the semantic factor index; For comprehensive query problems, both summary index and semantic factor index are retrieved simultaneously, and the results are merged through a dynamic weighting strategy.

5. The method according to claim 1, characterized in that, In step S4, the scores of the candidate content are normalized using a minimum-maximum normalization method.

6. The method according to claim 1, characterized in that, In step S5, the method for calculating the dynamic weights of the modal layer is as follows: Let M_text be the sum of the normalized re-ranking scores of the text candidate set D_text, and M_img be the sum of the normalized re-ranking scores of the image candidate set D_img; then the text modal weight α = M_text / (M_text+M_img), and the image modal weight β = M_img / (M_text+M_img).

7. The method according to claim 6, characterized in that, In step S5, the summary index weight w_sum and the semantic factor index weight w_fac in the index layer are set in the following way: In conceptual queries, the summary index weight w_sum is greater than the semantic factor index weight w_fac; In mechanism-based queries, the summary index weight w_fac is greater than the semantic factor index weight w_sum; In comprehensive queries, the summary index weight w_sum is equal to the semantic factor index weight w_fac.

8. The method according to claim 7, characterized in that, In step S5, the final comprehensive score Score(d) of candidate content d is calculated as follows: First, calculate the index score of candidate content d: Score_index(d)=w_sum×Score(I_sum)+w_fac×Score(I_fac), where Score(I_sum) and Score(I_fac) are the similarity scores of its summary index and semantic factor index in the initial retrieval, respectively. Then, the final comprehensive score Score(d) of the candidate content d is calculated by combining the modal weights: if the candidate content d is text, then Score(d) = α × Score_index(d); if the candidate d is an image, then Score(d) = β × Score_index(d).

9. A meteorological knowledge question-and-answer system, characterized in that, include: The data preprocessing module is used to preprocess multimodal knowledge resources in the meteorological field, identify the text content in the multimodal knowledge resources and perform paragraph segmentation, and save the non-text content as image form with semantic annotation. The index building module is used to generate summary indexes and semantic factor indexes for each preprocessed text content or image data. The summary index represents the core topic summary of the content, and the semantic factor index consists of multiple independent phrase units that carry clear semantic meanings. The initial retrieval module is used to determine the type of the query question entered by the user, and select different retrieval strategies based on different query question types to retrieve the summary index and / or semantic factor index, thereby obtaining a preliminary candidate index set; The reordering module is used to input the candidate content corresponding to each candidate index in the preliminary candidate index set into a pre-trained multimodal reordering model, obtain the relevance score of each candidate content relative to the query question, and normalize and sort the scores. The dynamic fusion weighting module is used to calculate the text modality weights and image modality weights of the modality layer based on the normalized reordering scores. It fuses the calculated modality layer weights, the preset index layer weights, and the normalized relevance scores to calculate the final comprehensive score of each candidate content and selects the top candidate content based on the comprehensive score. The answer generation module is used to input multiple selected candidate contents as context, along with the user's query question, into the visual language model to generate the final answer.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the meteorological knowledge question-and-answer method as described in any one of claims 1-8.

Images