Agricultural large model training and RAG enhanced question and answer method

Abstract

The present application relates to a kind of based on agricultural big model training and RAG enhanced question and answer method, belong to the field of agricultural big data, method includes: collection agricultural data, and constructs agricultural big model training data set;The input embedding of constructing Transform and Transform encoder are used to train the big model based on Transform architecture;After user input question text, big model executes dynamic word embedding disambiguation, regional technical term replacement, entity perception position coding in turn, generates the enhanced representation of input sequence, then, Transform encoder extracts deep semantic features, classification task directly outputs prediction label, and generation task is then word by word generated answer by decoder;RAG enhanced question and answer is carried out in combination with retrieval enhancement generation mechanism.The present application effectively solves the polysemy problem of agricultural technical terms in different contexts and regions by dynamic word embedding mechanism, significantly improves the accuracy of agricultural semantic understanding.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence and agricultural big data, and in particular to a method for training an agricultural large model and enhancing a question and answer based on RAG. BACKGROUND

[0002] With the diversification and large-scale of agricultural production data, a large amount of structured, semi-structured and unstructured data has been generated in the field of agriculture, such as temperature and humidity data collected by sensors, agricultural yearbook reports, scientific papers, etc. Traditional data processing and analysis methods can often only handle single type of data, lacking effective fusion of multi-source heterogeneous data. In addition, the polysemy of agricultural terminology and the regional differences of dialects have brought challenges to agricultural intelligent systems. In recent years, with the continuous development of deep learning technology, especially the maturity of Transformer architecture, many natural language processing models have been introduced into the field of agriculture. However, the existing models still have problems such as poor adaptability to term disambiguation and regional differences, and insufficient knowledge coverage when dealing with complex agricultural data and tasks.

[0003] In modern agricultural production, a large amount of agricultural data is being generated in various forms, such as temperature and humidity, soil pH value, etc. Sensor data, agricultural meteorological and agricultural machinery data, pest and disease monitoring reports, and unstructured text data such as agricultural manuals and scientific papers. These data can provide important support for agricultural management, decision-making and precision agriculture. However, traditional data processing and analysis methods mainly rely on single type of data, resulting in limitations in data utilization and inability to fully support the intelligent needs of the agricultural field. The existing agricultural large models have the following problems in the training and application process: 1. Traditional agricultural intelligent technology mostly relies on single type of data source, such as structured data or unstructured data, lacking effective fusion of multi-source heterogeneous data, resulting in insufficient utilization of information; 2. Existing natural language processing methods are difficult to effectively handle the polysemy of agricultural terminology, especially unable to flexibly adjust the semantic representation of terms according to different contexts or regional differences; 3. In existing technologies, many models cannot adapt to regional differences in the field of agriculture, especially the processing of dialects and local terms often cannot correspond to standard terms, resulting in inaccurate semantic expression; 4. Many agricultural large models cannot balance the effects of fine-grained tasks (such as term recognition) and global tasks (such as document classification) when performing multi-task learning, resulting in unstable performance of the model between different tasks. SUMMARY

[0004] The present application aims to overcome the shortcomings of the prior art and provides a method for training an agricultural large model and enhancing a question and answer based on RAG, solving the deficiencies existing in the prior art.

[0005] The objective of this invention is achieved through the following technical solution: a method based on agricultural large-scale model training and RAG-enhanced question answering, the method comprising:

[0006] Step 1: Collect agricultural data and construct a training dataset for a large agricultural model;

[0007] Step 2: The input embedding of the Transformer is formed sequentially through disambiguation of agricultural terminology and dynamic word embedding initialization, regional language pattern enhancement encoding, and agricultural entity perception location encoding. Then, a Transformer encoder is constructed to train a large model based on the Transformer architecture.

[0008] Step 3: After the user inputs the question text, the large model sequentially performs dynamic word embedding disambiguation, regional terminology replacement, and entity-aware location encoding to generate an enhanced representation of the input sequence. Then, the Transformer encoder extracts deep semantic features. The classification task directly outputs the predicted label, while the generation task generates the answer word by word through the decoder.

[0009] Step 4: Combine search enhancement generation mechanism to perform RAG enhanced question answering.

[0010] The aforementioned agricultural terminology polysemy disambiguation and dynamic word embedding initialization specifically include the following:

[0011] A1. Constructing an agricultural ontology graph: Constructing a set of agricultural entities as follows Let the set of semantic relationships between entities be defined as Then the agricultural ontology map is ,in, Represents the total number of agricultural entities. Indicates the first One agricultural entity, Indicates the first agricultural entity. Indicates the second agricultural entity. Indicates the first One agricultural entity;

[0012] A2. Context-aware disambiguation: The local context window word sequence of the target word is encoded by bidirectional LSTM to generate a context vector. The cosine similarity between the target word and the pre-trained vectors of each entity in the agricultural ontology graph is calculated. After the similarity discrimination control coefficient is adjusted, the semantic similarity score between the target word and each entity is obtained by normalization through the natural exponential function.

[0013] A3. Dynamic word embedding generation: Based on the semantic similarity scores of the target word's base word vector and each entity vector, the entity vector fusion result is obtained by weighted summation. The fusion weight of the base word vector is calculated by the Sigmoid function. The fusion results of the base word vector and the entity vector are added according to the weights to generate a context-aware dynamic word embedding vector.

[0014] The regional language pattern enhancement coding specifically includes the following:

[0015] B1. Construction of regional dictionary: Extract the mapping relationship between dialect terms and standard terms from agricultural yearbook data, and construct a dialect-standard term mapping table;

[0016] B2. Bidirectional substitution mask generation: For each word in the input sentence, if a dialect term mapping exists, a standard term substitution mask value of 1 is generated; otherwise, it is 0.

[0017] B3. Fusion Encoding Representation: The mask weighted result of the dynamic word embedding vector and the standard term vector is concatenated, and the fusion projection matrix and bias vector are linearly transformed after training to generate regionally enhanced word vectors.

[0018] The agricultural entity sensing location coding specifically includes the following:

[0019] C1. Entity Location Marking: Identify the location of all agricultural entities in the input sequence and construct a set of location indexes for each entity;

[0020] C2. Relative distance matrix calculation: Calculate the shortest absolute distance from each word position to the set of position indices of each entity;

[0021] C3. Entity-aware position coding generation: Based on the shortest distance matrix, the standard sinusoidal position codes of each entity are weighted by the distance decay factor, normalized, and then summed to generate an enhanced position coding vector.

[0022] The construction of the Transformer encoder specifically includes:

[0023] D1. Constructing the basic layer structure: Adopting a block-aggregation strategy, capturing features within blocks through local attention, realizing inter-block interaction through a gating mechanism, and fusing local and global representations in a layered manner;

[0024] D2. Stacking: Repeatedly stacking the basic layer structure Second-rate, The stacked layers are the number of layers, and the output of the stack is the context representation of the encoder of the last layer, which is achieved by repeatedly stacking the base layer structure. Next, final The output of the last layer is used as the result of the entire encoder, and the output feature of the last layer encoder is set as follows: ;

[0025] D3. Output: The output of the last encoder layer is used as the context representation of the document for the classification task, and is also input to the decoder to generate the answer.

[0026] D1 specifically includes the following:

[0027] D101, Semantic Blocking and Local Attention Calculation: The input document is divided into semantic blocks, and the projections of the query matrix, key matrix and value matrix are calculated within each block. Local attention output is generated by scaling dot product attention and normalized exponential function.

[0028] D102, Global Attention Gated Aggregation: The local attention output of each block is averaged and compressed into a block representation, which is then weighted by hyperbolic tangent activation and Sigmoid gating weights and aggregated into a global document representation;

[0029] D103. Layer Normalization and Residual Connectivity Based on Hierarchical Weighted Fusion: The local attention output and the global document representation are weighted and summed according to the hierarchical fusion coefficients, then residually connected with the output features of the previous layer. After layer normalization, the current layer features are output. The output features based on hierarchical weighted fusion and residual connectivity are defined as follows: ;

[0030] D104, Feedforward Neural Network: The feedforward neural network adopts the standard feedforward neural network of Transformer. The input is the output feature of layer normalization based on layer weighted fusion. The activation function of the feedforward neural network adopts the ReLU activation function.

[0031] D105, Standard Layer Normalization and Residual Connections: The standard layer normalization and residual connections of the Transformer are used. The input is the output features of the feedforward neural network, which, after layer normalization, are then... The residuals are summed to output the output features of the current layer. The output features based on layer normalization and residual connection, which are weighted by layer fusion, are defined as follows: .

[0032] The D3 specifically includes the following:

[0033] D301, Classification Task: [The task is to...] The first word vector is input into a fully connected layer, and the classification result is generated by Softmax.

[0034] D302, Generate Task: Input a Transformer decoder, which uses the encoder output as key-value pairs and the previous word as the query target to iteratively generate a sequence of answers;

[0035] During the training phase of the decoder, the input to the decoder consists of three parts: the encoder output, the answer sequence offset input, and the self-attention mask; specifically, it includes the following:

[0036] Encoder output: Contextual representation from the last encoder layer, i.e., the deep semantic features of the document, including dynamically disambiguated agricultural terms, regionally standardized expressions, and entity spatial relationship information;

[0037] Answer sequence offset input: A sequence of answers to the task generated from the constructed training dataset. And shift it one position to the right, that is, add a start marker at the starting position, as the initial input of the decoder;

[0038] Self-attention mask: Ensures that during decoding, each position only focuses on the generated left-hand word, avoiding information leakage;

[0039] During the training phase, the decoder performs the following steps layer by layer:

[0040] Masked Self-Attention Layer: Calculates internal dependencies for the currently generated word sequence;

[0041] Encoder-decoder attention layer: Aligns the mask self-attention output with the encoder's context representation, focusing on key information;

[0042] Feedforward network and residual connection: fuse features and pass them to the next layer.

[0043] The training of the large model based on the Transformer architecture specifically includes the following:

[0044] E1. Multi-granularity loss function optimization: A dual-granularity joint loss function is adopted, which strengthens semantic consistency learning through term-level contrastive loss and dynamically weights and fuses it with document-level task loss to achieve multi-level supervised optimization.

[0045] E2. Model Iterative Training and Parameter Update: A multimodal adaptive gradient modulator is adopted to achieve fine-grained parameter updates through gradient sensitivity analysis, parameter clustering and grouping, and modal learning rate scheduling.

[0046] E3. Stopping Iteration Condition Judgment: Continuously monitor the performance of the validation set during training, and calculate the total loss of the validation set after each iteration. The classification accuracy metric is set. If the verification loss does not decrease for 5 consecutive rounds, or the task metric reaches the predetermined threshold, the early stop mechanism is triggered. At the same time, the maximum number of iterations is set as a mandatory termination condition.

[0047] Step four specifically includes the following:

[0048] Retrieval Phase: User queries are converted into vectors by the input embedding of the Transformer, and Top-K relevant paragraphs are retrieved from the constructed agricultural knowledge base;

[0049] Augmented input: The retrieved paragraphs are concatenated with the original question and input into the trained agricultural big data model;

[0050] Answer generation: The agricultural big data model integrates the search results with its own knowledge to generate structured answers.

[0051] Step one specifically includes the following:

[0052] Collect multi-source heterogeneous agricultural data, including structured data, semi-structured data, and unstructured data;

[0053] Outlier removal, timestamp alignment, and feature standardization are performed on structured time-series data to form a continuous sensor data stream;

[0054] For semi-structured data, key fields are extracted using table parsing tools and converted into structured records in key-value pair format. At the same time, fields are extracted as category labels.

[0055] For unstructured text data, sentence segmentation, noise reduction, and semantic paragraph division are performed to ensure logical integrity and to construct question-answer pairs;

[0056] A unified encoding method is used for multi-task datasets, encoding all data into a text sequence format, with numerical data embedded through bucketing discretization.

[0057] The present invention has the following advantages:

[0058] 1. Through deep fusion and cleaning of multi-source heterogeneous data, the modeling capability of complex datasets in the agricultural field has been improved, especially when dealing with data with temporal and structured characteristics, the model performance has been significantly improved.

[0059] 2. The dynamic word embedding mechanism effectively solves the problem of polysemy in agricultural terms across different contexts and regions, and significantly improves the accuracy of agricultural semantic understanding.

[0060] 3. By enhancing the coding module with regional language patterns, the model can recognize and convert dialect terms, thus improving the processing effect of agricultural data from different regions.

[0061] 4. Through joint optimization of multi-granularity loss, a more balanced performance improvement can be achieved between classification and generation tasks, thus optimizing the learning effect of complex tasks in the agricultural field. Attached Figure Description

[0062] Figure 1 This is a schematic diagram of the process of the present invention;

[0063] Figure 2 A diagram showing the comparison of disambiguation performance of agricultural terms;

[0064] Figure 3 This is a schematic diagram comparing the performance of classification tasks in ablation experiments using multi-granularity loss functions.

[0065] Figure 4 This is a schematic diagram comparing the performance of the generation task in the multi-granularity loss function ablation experiment.

[0066] Figure 5 A schematic diagram illustrating model performance under different document lengths;

[0067] Figure 6 A comparative diagram illustrating the enhanced question-and-answer performance of RAG. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of this application provided below with reference to the accompanying drawings is not intended to limit the scope of protection of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application. The present invention will be further described below with reference to the accompanying drawings.

[0069] This invention specifically relates to a method for training agricultural large-scale models and RAG-enhanced question answering. It combines structured, semi-structured, and unstructured data for agricultural data collection and fusion, providing a rich data source for intelligent applications in agriculture and solving the problem of relying on only a single data source in existing technologies. It employs a three-level terminology disambiguation mechanism, combined with an agricultural ontology graph and a context-aware model, to generate dynamic word embeddings, addressing the shortcomings of traditional static word embedding methods in addressing the polysemy and regional differences in agricultural terminology. Through an adaptive word segmentation enhancement module and a regional dictionary, dialect terms are mapped to standard terms, achieving dynamic conversion of local languages ​​and terms in agriculture and improving the model's adaptability to data from different regions. Finally, it adopts an improved Transformer architecture, combined with multi-granularity loss function optimization, to strengthen the learning of terminology-level semantic consistency and document-level overall semantics, enabling multi-task learning in agriculture to balance detail recognition and overall understanding.

[0070] like Figure 1 As shown, it specifically includes the following:

[0071] S1, Agricultural Data Acquisition;

[0072] Multi-source heterogeneous agricultural data collection was conducted, including structured data, semi-structured data, and unstructured data, as follows:

[0073] (1) Structured data: real-time collection of time-series data such as temperature, humidity, pH value, and crop growth indicators from agricultural meteorological stations, soil sensors, and agricultural machinery IoT devices, with a sampling frequency ≥1Hz;

[0074] (2) Semi-structured data: crawl agricultural yearbooks, pest and disease monitoring reports, and agricultural product market transaction reports in PDF / Excel format, and use OCR and table parsing tools to extract key fields;

[0075] (3) Unstructured data: collect agricultural research papers, planting technology manuals, and farmer consultation dialogue texts, covering staple foods such as rice and wheat as well as regional economic crops.

[0076] S2, Construction of the training dataset for the large-scale agricultural model;

[0077] After completing the collection of multi-source heterogeneous agricultural data, a normalized dataset suitable for training large Transformer models was constructed. The three types of data collected in S1 were deeply cleaned and aligned, and enhanced into a multi-task format to support question-answer pair generation and classification tasks, as detailed below:

[0078] (1) For structured time-series data: outlier removal, timestamp alignment and feature standardization are performed to form a continuous sensor data stream; further, question-answer pairs are generated based on time-series features, such as generating the question "Is the current temperature suitable for rice growth?" based on temperature anomalies, and the answer "No, irrigation is needed to cool down";

[0079] (2) For semi-structured data: extract key fields such as the incidence rate of pests and diseases and the price fluctuation of agricultural products through table parsing tools, convert them into structured records in the form of key-value pairs, and extract fields as classification labels, such as mapping pest and disease types to category IDs: 0=rust, 1=mildew;

[0080] (3) For unstructured text data: After sentence segmentation, noise reduction and semantic paragraph division, logical integrity is ensured; further question-answer pairs are constructed, such as directly extracting question-answer pairs from farmers' consultation dialogue text (e.g., question: "What are the symptoms of corn leaf spot disease?", answer: "Brown spots appear on the leaves").

[0081] For research papers and technical manuals, template matching is used to generate synthetic question-answer pairs (e.g., based on the entity "rice", the question is generated: "High-yield fertilization scheme for rice?", and the answer is: "Extract relevant paragraphs from the manual").

[0082] Furthermore, a unified encoding is performed on the multi-task dataset, that is, all data is uniformly encoded into a text sequence format, and numerical data is embedded through bucket discretization.

[0083] Based on this, each sample contains:

[0084] 1. Input sequence, i.e., raw text or sensor data encoding.

[0085] 2. Task Tags:

[0086] 2.1 Classification Task: Tags , The total number of categories, such as crop disease types;

[0087] 2.2 Generating the task: Target answer sequence That is, the answer part in a question-and-answer pair.

[0088] The dataset is divided into training, validation, and test sets in a 7:2:1 ratio to ensure coverage of different crop types, regional characteristics, and agricultural scenarios. Furthermore, during the division, question-answer pairs and classification samples are sampled independently to avoid data leakage.

[0089] S3. Construct the input embedding of the Transformer;

[0090] S301, Disambiguation of polysemy in agricultural terminology and initialization of dynamic word embedding;

[0091] Agricultural terminology exhibits significant regional ambiguity; for example, "apple" can refer to either a fruit or an electronics brand. Conventional static word embedding methods cannot distinguish between contextual semantic differences, resulting in a lack of dynamic adaptability in terminology representation. This invention employs a three-level terminology disambiguation mechanism, generating dynamic word embedding representations through the construction of an agricultural ontology graph, context-aware disambiguation, and dynamic word embedding generation. The specific steps are as follows:

[0092] (1) Constructing an agricultural ontology graph: Define the agricultural entity set and the relationships between entities, and construct a structured ontology graph as the basis for disambiguation. Specifically, construct the agricultural entity set as follows: Define the set of semantic relationships between entities as Then the agricultural ontology map is , Represents the total number of agricultural entities. Indicates the first One agricultural entity, Indicates the first agricultural entity. Indicates the second agricultural entity. Indicates the first Agricultural entities.

[0093] (2) Context-aware disambiguation: A local context window word sequence of the target word is encoded using a bidirectional LSTM (a special recurrent neural network structure) to generate a context vector. Then, the cosine similarity between the target word and the pre-trained vectors of each entity in the agricultural ontology graph is calculated. After adjustment by the similarity discrimination control coefficient, the semantic similarity score between the target word and each entity is obtained by normalization using the natural exponential function, as shown below:

[0094] ,

[0095] in, Indicator With entity semantic similarity score, It is a natural exponential function. Let i be the target word, i.e., the term that needs to be disambiguated. The local context encoding vector is calculated as follows: , Indicates surrounding the target word A local context window containing the location arrive The word sequence, BiLSTM(·) is a bidirectional long short-term memory network function, Representing entities The pre-trained vectors, Representing entities The pre-trained vectors, The first in the agricultural ontology map One entity, This represents the similarity discrimination control coefficient, such as... , This represents the function for calculating cosine similarity.

[0096] (3) Dynamic word embedding generation: Based on the semantic similarity scores of the base word vectors and entity vectors of the target word, the entity vector fusion result is obtained by weighted summation. Then, the fusion weights of the base word vectors are calculated by the Sigmoid function. The fusion results of the base word vectors and entity vectors are added according to the weights to generate context-aware dynamic word embedding vectors, as follows:

[0097] ,

[0098] in, Indicates the first A generated dynamic word embedding vector, Indicator The basic word vectors, Indicator With entity semantic similarity score, represents the fusion weight of the basic word vectors, and the calculation method is expressed as , where Sig(·) represents the Sigmoid activation function, represents the dimension of the word vector, represents the vector concatenation operation.

[0099] For example Figure 2 As shown, to verify the effectiveness of the dynamic word embedding technology proposed in this invention in solving the problem of polysemy of agricultural terms, by comparing the disambiguation accuracy rates of the static word embedding method, the general language model baseline, and the method of this invention on three types of typical agricultural terms, namely common terms, polysemous terms, and regional terms, the height of the column in the figure represents the accuracy level, and the scatter points on the top of the column simulate the data fluctuations of multiple experiments, enhancing the authenticity. The experimental results show that the method of this invention has achieved significant improvements on various types of terms, especially in the aspects of polysemous terms and regional terms, which verifies that the three-level term disambiguation mechanism can effectively capture the regional characteristics and context relevance of agricultural terms, and solves the core problem that traditional methods cannot distinguish the semantic differences of some terms in different agricultural scenarios.

[0100] S302. Enhancement coding of regional language patterns;

[0101] There are a large number of regional dialect terms in agricultural texts. For example, "baomi" refers to corn, and conventional word segmenters cannot recognize such non-standard terms, resulting in semantic representation deviation and performance degradation of downstream tasks. This invention adopts an adaptive word segmentation enhancement module to achieve the dynamic conversion of dialect terms to standard terms through constructing a regional dictionary, generating a standard term replacement mask, and fusing the coding representation. The specific steps are as follows:

[0102] (1) Construction of the regional dictionary: Extract the mapping relationship between dialect terms and standard terms from materials such as agricultural yearbooks, and construct a dialect-standard term mapping table, which is expressed as:

[0103] ,

[0104] where represents the dialect-standard term mapping table, represents the dialect term, represents the standard term, represents the th mapping pair of the dialect term and its corresponding standard term, represents the total number of entries in the mapping table.

[0105] (2) Generation of bidirectional replacement masks: For each word in the input sentence, if there is a dialect term mapping, generate a standard term replacement mask value of 1, otherwise 0, which is expressed as:

[0106] ,

[0107] in, Indicates the first The replacement mask value for each word. A quantifier indicating existence, used to determine whether a dialect term mapping exists.

[0108] (3) Fusion encoding representation: The masked weighted result of the dynamic word embedding vector and the standard term vector is concatenated, and then linearly transformed by a trainable fusion projection matrix and bias vector to generate the regional enhanced word vector, represented as:

[0109] ,

[0110] in, Indicates the generated first Regionally enhanced word vectors The fusion projection matrix represents trainable parameters. This indicates the standard terminology replacement mask value. Standard terminology The vector representation of , This represents the fusion bias vector, which is a trainable parameter.

[0111] S303, Location coding for agricultural entities;

[0112] Traditional positional encoding methods, such as sine coding in Transformer, only consider the absolute position of words, neglecting the importance of the relative distance between entities in agricultural texts for semantic understanding. For example, the distance between words describing pests and diseases and crops directly affects their relevance, and conventional methods cannot effectively capture such features. This invention designs a relative distance encoding mechanism for entity perception. By marking entity positions, calculating the shortest distance matrix, and fusing distance-weighted positional encoding, it enhances the model's ability to perceive spatial relationships between entities. The specific steps are as follows:

[0113] (1) Entity location labeling: Identify the location of all agricultural entities in the input sequence and construct a set of location indices for each entity, represented as:

[0114] ,

[0115] in, Representing entities The set of position indices in the sequence. For the first in the set of agricultural entities One entity, Representing entities No. The position index of the second occurrence. Representing entities The position index of the first occurrence. Representing entities The position index of the second occurrence.

[0116] (2) Calculation of relative distance matrix: Calculate the shortest absolute distance from each word position to the set of position indices of each entity, expressed as:

[0117] ,

[0118] in, Indicator To the entity The shortest distance, Representing entities Position index in the sequence, Representing entities Location index set, Indicates the word position index, This represents the absolute value operation. This indicates the minimum value operation.

[0119] (3) Entity-aware location coding generation: Based on the shortest distance matrix, the standard sinusoidal location codes of each entity are weighted by a distance attenuation factor, normalized, and then summed to generate an enhanced location coding vector, represented as:

[0120] ,

[0121] in, Indicates the first Enhanced positional encoding vectors of each word, Represents the distance attenuation factor, such as, , This represents the standard sinusoidal position coding function.

[0122] S304. Form the input embedding of the Transformer;

[0123] The combined geographic-enhanced word vectors and enhanced location-encoded vectors are used as the input embeddings of the Transformer. That is, the input embeddings of the Transformer include geographic-enhanced word vectors and enhanced location-encoded vectors.

[0124] S4. Construct the Transformer encoder;

[0125] S401. Construct the basic layer structure;

[0126] (1) Multi-head self-attention sub-layer (using a multi-head self-attention sub-layer based on a hierarchical attention enhancement mechanism to replace or replace the standard multi-head self-attention sub-layer for self-attention calculation);

[0127] Agricultural documents, such as pest and disease research reports, often contain long-range semantic dependencies, with descriptions spanning multiple paragraphs. The self-attention mechanism of conventional Transformer models is difficult to effectively capture this hierarchical structural feature, leading to the loss of key semantic information.

[0128] This invention employs a block-aggregation strategy, capturing intra-block features through local attention, achieving inter-block interaction through a gating mechanism, and hierarchically fusing local and global representations. The specific steps are as follows:

[0129] 1. Semantic Blocking and Local Attention Calculation: The input document is divided into semantic blocks. Within each block, the projections of the query matrix, key matrix, and value matrix are calculated. Local attention output is generated using scaled dot product attention and a normalized exponential function, expressed as:

[0130] ,

[0131] in, Indicates the first Local attention output of semantic blocks, Indicates the first The query matrix of the block. Indicates the first The key matrix of the block, Indicates the first The value matrix of the block, Indicates querying the projection matrix. Represents the key projection matrix. Indicates transpose. This represents the normalized exponential function.

[0132] 2. Global Attention Gated Aggregation: The output of each local attention block is averaged and compressed into a block representation, then weighted by hyperbolic tangent activation and sigmoid gating, and aggregated into a global document representation, as follows:

[0133] ,

[0134] in, The global document representation representing the aggregation. Indicates the first The compressed representation of a semantic block is calculated as follows: , This indicates the average pooling operation. Indicates the first The gating weights of each semantic block are calculated as follows: , This represents the hyperbolic tangent activation function. Represents the gating weight vector. for transpose, Represents the gated projection matrix. This represents the total number of semantic blocks, i.e., the number of blocks into which the input document is divided.

[0135] 2. Layer Normalization and Residual Connection Based on Hierarchical Weighted Fusion: The local attention output and the global document representation are weighted and summed according to the hierarchical fusion coefficients, then connected to the residual of the output features of the previous layer. After layer normalization, the current layer features are output, represented as:

[0136] ,

[0137] ,

[0138] in, Indicates the first The mixed output characteristics of the layer Indicates the first The output features of the layer Indicates the first The output features of the layer Presentation layer normalization operation, This represents the output of local attention. Indicates the first The layer fusion coefficient, for example, is set to 0.1 times the layer index.

[0139] It should be noted that the layer normalization and residual connection module based on hierarchical weighted fusion adopts a hierarchical fusion strategy to enhance semantic transmission. First, local attention output... As an intra-block semantic feature, the aggregated global document representation As a cross-block aggregation feature, the two are fused through hierarchical fusion coefficients. Weighted summation is performed to generate a mixture, which is then residually concatenated with the output features of the previous layer. This mixture is then input into a normalization layer, and the output features of the current layer are output. The output features based on layer-level weighted fusion, layer normalization, and residual concatenation are defined as follows: .

[0140] 3. Feedforward Neural Network: The feedforward neural network adopts the standard feedforward neural network of Transformer. The input is the output feature of layer normalization based on layer weighted fusion. The activation function of the feedforward neural network is the ReLU activation function.

[0141] 4. Standard Layer Normalization and Residual Connections: Standard layer normalization and residual connections are implemented using Transformer. The input is the output features of the feedforward neural network, which, after layer normalization, are then... The residuals are summed to output the output features of the current layer. The output features based on layer normalization and residual connection, which are weighted by layer fusion, are defined as follows: .

[0142] S402, stacking;

[0143] Repeatedly stacked base layer structure Second-rate, For the number of stacking layers, such as This indicates a stack of 12 layers. Specifically, the output of the stack is the context representation of the last layer encoder, achieved by repeatedly stacking the base layer structure. Next, final The output of the last layer is used as the result of the entire encoder, and the output feature of the last layer encoder is defined as follows: .

[0144] S403, Large Model Output;

[0145] The output of the final encoder layer serves as the context representation of the document for classification tasks, and is also input to the decoder for generating the answer. Specifically, the output features of the final encoder layer... As a contextual representation of the document, it is used for classifying and generating tasks, as follows:

[0146] 1. Classify tasks: The first word vector ([CLS] label) is input into the fully connected layer, and the classification result, such as crop disease type, is generated by Softmax.

[0147] 2. Generate task: Input the Transformer decoder and generate answers through autoregression. For example, for technical guidance text, the decoder uses the encoder output as key-value pairs, the previous word as the query target, and iteratively generates a sequence of answers.

[0148] Specifically, for the generation task, the decoder works based on the autoregressive architecture of Transformer, the core of which is to use the context representation output by the encoder as the knowledge base to gradually generate a sequence of answers that conforms to agricultural semantics.

[0149] During the training phase of the decoder, the input to the decoder consists of three parts: the encoder output, the answer sequence offset input, and the self-attention mask, as follows:

[0150] a) Encoder output: Contextual representation from the last encoder layer of S402, i.e., the deep semantic features of the document, including dynamically disambiguated agricultural terms, regionally standardized expressions, and entity spatial relationship information;

[0151] b) Answer Sequence Offset Input: The answer sequence of the task generated from the training dataset constructed using S2. However, it is shifted one position to the right, that is, a [START] marker is added to the starting position, which serves as the initial input to the decoder;

[0152] c) Self-attention mask: Ensures that each position only focuses on the generated left-hand word during decoding, avoiding information leakage.

[0153] During the training phase, the decoder performs the following steps layer by layer:

[0154] a) Masked Self-Attention Layer: Calculates internal dependencies for the currently generated word sequence, such as causal logic in agricultural technology descriptions: "fertilization" → "irrigation";

[0155] b) Encoder-decoder attention layer: Aligns the mask self-attention output with the context representation of the encoder to focus on key information, such as extracting the "disease cycle" feature from pest and disease reports;

[0156] c) Feedforward network and residual connection: fuse features and pass them to the next layer.

[0157] Finally, the hidden states of the top-level output are transformed by linear transformation and Softmax to generate a vocabulary probability distribution. During training, the decoder is forced to predict the current word based on the word before the real answer through a teacher-forced strategy. For example, when the input is "The symptoms of corn leaf spot disease are", the predicted output is "brown spots", so that the model learns the semantic logic of agricultural question answering.

[0158] In the inference phase of the decoder, an autoregressive generation method is adopted, and the specific steps are as follows:

[0159] a) Starting with the representation of the user question (e.g., “Prevention and control of rice sheath blight?”) obtained by the encoder and the [START] marker.

[0160] b) Decoder generates answers word by word: Each step predicts the next word based on the historical generated words and the encoder context, such as generating "1." in the first step, generating "before sowing" in the second step, until the marker is output or the length limit is reached.

[0161] c) The generation process dynamically invokes the agricultural knowledge patterns learned during training:

[0162] For questions and answers generated from structured data (such as responses to abnormal temperatures), output concise instructions ("Irrigation and cooling required").

[0163] For unstructured data (such as agricultural technology manuals), generate paragraph-style technical guidance ("Spray with 50g / mu of Jinggangmycin diluted in water").

[0164] S5. Training large models based on the Transformer architecture;

[0165] S501, Multi-granularity loss function optimization;

[0166] Tasks in the agricultural domain require capturing both fine-grained term-level semantics and document-level overall semantics. However, conventional single loss functions are difficult to take into account the supervision signals at both levels, resulting in an imbalance in model performance between detailed tasks such as entity recognition and overall tasks such as document classification.

[0167] This invention employs a dual-granularity joint loss function, which enhances semantic consistency learning through term-level contrastive loss and dynamically weights and fuses it with document-level task loss to achieve multi-level supervised optimization. The specific steps are as follows:

[0168] 1. Term-level contrast loss calculation: Cosine similarity is calculated for instance representation vectors of identical term identifiers in a batch. Normalization using the natural exponential function and temperature coefficient is applied to generate a contrast loss, enhancing semantic consistency. This is represented as:

[0169] ,

[0170] in, Indicates term-level contrast loss. For the first The representation vector of each term instance, used as an anchor point. To and Another term in the same class (the first) The representation vector of each instance is used as a positive sample. For the first A representation vector for each term instance, used for negative sample computation. Represents the cosine similarity function. Indicates the temperature coefficient, such as, , Indicates batch size, Indicates an indicator function, when The value is 1 when the time is right, and 0 otherwise. This represents a logarithmic function, with the default base being the natural constant. For the first The term identifier for a term instance. For the first A term identifier for a term instance.

[0171] 2. Document-level task loss calculation: Based on the true labels and predicted probabilities of the classification task, calculate the cross-entropy loss, expressed as:

[0172] ,

[0173] In the formula, Indicates document-level task loss; Indicates the number of documents in the batch; This indicates the number of categories, specifically the categories for downstream document-level tasks, such as crop type, pest and disease type, or regional category in document classification tasks, which are classified according to the content of agricultural documents; Indicates the first The true tags of the document in the first... One-hot encoding of the category; Indicates the first The predicted probability of a document in the th order of ... The softmax probability of the category.

[0174] 3. Generation Loss Calculation: The negative log-likelihood loss function is used to calculate the cross-entropy loss between the model's predicted sequence and the target answer sequence in the generation task, expressed as:

[0175] ,

[0176] In the formula, Indicates the generation loss; Indicates the length of the answer sequence; Represents the sequence of target answers The Middle The word element at each position; The representation model generates words based on history. and the output features of the last encoder layer Predict the probability of the current word; It is a word generated from history.

[0177] 4. Joint Optimization of Multi-Granularity Losses: The term-level comparison loss and document-level task loss are weighted and summed according to the term loss weight coefficients to obtain the total loss, expressed as:

[0178] ,

[0179] In the formula, The total loss represents the final optimization objective function; This refers to the loss weighting coefficient of the term, such as, .

[0180] like Figure 3 The image shows a performance comparison chart for classification tasks, as follows: Figure 4The figure shows a performance comparison chart for the generation task. To verify the optimization effect of the multi-granularity joint loss function on multi-task learning in agriculture, the performance of the single-task loss, the terminology-free loss, and the complete method on the dual tasks of classification and generation are compared. The parallel bars in the figure clearly show the difference in the effect of each ablation setting on the two tasks, and the dark bars indicate the specific performance values. The experimental results show that the complete method achieves the best results in both classification accuracy and generation quality. This indicates that the terminology-level contrast loss strengthens the semantic consistency of the same terms, and the document-level task loss ensures the overall supervision effect. The two work together to solve the balance problem of fine-grained terminology recognition and overall document understanding in agricultural tasks, enabling the model to accurately identify crop disease types and generate technical guidance text that conforms to agricultural semantics.

[0181] S502, Model Iterative Training and Parameter Update;

[0182] Large-scale agricultural models require simultaneous optimization of dynamic word embedding parameters, hierarchical attention weights, and task head parameters. However, conventional optimizers such as Adam use a uniform learning rate for all parameters, which cannot adapt to the significant differences in parameter sensitivity between modules, resulting in unstable convergence.

[0183] This invention employs a multimodal adaptive gradient modulator, and achieves refined parameter updates through gradient sensitivity analysis, parameter clustering, and modal-specific learning rate scheduling. The specific steps are as follows:

[0184] 1. Parameter gradient sensitivity analysis: Calculate the gradient sensitivity coefficient of each parameter tensor to quantify the influence of the parameters on the total loss, expressed as:

[0185] ,

[0186] In the formula, Represents a parameter tensor The gradient sensitivity coefficient quantifies the effect of updating this parameter on the total loss. The intensity of the impact; Indicates the first The parameter tensor is the trainable parameters of the model; This represents the expectation operator, used to calculate the average of the gradient norm; Indicates total loss Regarding parameters The gradient; This represents the L2 norm, i.e., the Euclidean norm, and is used to calculate the length of the gradient vector. This represents the total loss function, such as multi-granularity joint loss. This represents the variance operator, used to calculate the degree of dispersion of the gradient; This represents a small constant used to prevent the denominator from being zero and to ensure numerical stability, such as... .

[0187] 2. Parameter Clustering and Grouping: Based on the gradient sensitivity coefficient threshold, the parameters are divided into four groups: high sensitivity group, medium sensitivity group, low sensitivity group, and very low sensitivity group, as shown below:

[0188] Highly sensitive group, specifically referring to dynamic embedding layer parameters: ;

[0189] The medium-sensitive group specifically refers to attention gating parameters: ;

[0190] Low-sensitivity group, specifically referring to standard feedforward neural network parameters: ;

[0191] The extremely low sensitivity group specifically refers to the position encoding parameters: ;

[0192] In the formula, The parameter set representing the highly sensitive group; This represents the set of parameters for the sensitive group. The parameter set representing the low-sensitivity group; The set of parameters representing the extremely low sensitivity group; Indicates the first A parameterized tensor; Indicates parameters The gradient sensitivity coefficient, as defined in 503.1; This indicates "makes" or "satisfies the condition," and is used in set definitions to specify the conditions for grouping parameters. Indicates the high sensitivity threshold; Indicates the medium sensitivity threshold; This represents a low sensitivity threshold. In one embodiment, the threshold is set as follows: , , .

[0193] 3. Modal Learning Rate Scheduling: Each group of parameters is assigned an independent base learning rate. The learning rate is then dynamically adjusted based on the number of training epochs, changes in loss, and the decay coefficient, as shown below:

[0194] ,

[0195] In the formula, Indicates the first During the nth iteration, the 1st The learning rate of the group parameters; Indicates the first The base learning rate for the group parameters; This represents a function that takes the minimum value. Indicates the current training iteration round number; Indicates the first The change in loss over the next iteration; The scaling factor represents the scaling factor for changes in loss, controlling the sensitivity of learning rate adjustments to loss fluctuations, such as... ; It is a natural constant; Indicates the first The attenuation coefficient of the group parameters; Indicates the total number of iteration rounds, e.g., .

[0196] The settings need to be designed differently according to the characteristics of the parameter group, referring to the threshold grouping rules. The specific settings are shown in Table 1 below:

[0197] Table 1. Reference Threshold Grouping Rules Table

[0198]

[0199] 4. Gradient Modulation Update: An improved Adam optimizer is used to calculate the first and second momentum of the gradient, inject a gradient correction term determined by the sign function and the infinity norm threshold, and update the parameters according to the group learning rate, as shown below:

[0200] ,

[0201] ,

[0202] ,

[0203] ,

[0204] In the formula, Indicates the first The first-order momentum of the next iteration; Indicates the first The first-order momentum of the next iteration; The decay rate of first-order momentum, such as, ; Indicates the first The gradient vector of the next iteration; Indicates the first The second-order momentum of the next iteration; Indicates the first The second-order momentum of the next iteration; The decay rate of the second momentum, such as, ; Indicates element-wise multiplication; Represents the modulation gradient vector; The weights of the gradient correction term, such as, ; Represents a symbolic function; Indicates an indicator function, when The value is 1 when the time is right, and 0 otherwise. Gradient The infinite norm; This represents the gradient magnitude threshold, used to determine whether to apply a correction term; Indicates the first The set of parameter tensors in the nth iteration represents the... In the next iteration, the high-dimensional tensor composed of all trainable parameters has the same dimension as the model structure. Indicates the first The set of parameter tensors in the nth iteration represents the... In the next iteration, the high-dimensional tensor composed of all trainable parameters has the same dimension as the model structure.

[0205] like Figure 5 As shown, the performance of the entity-aware location encoding mechanism of this invention on the processing capability of long documents is analyzed. By measuring the performance trends of conventional models, relative location encoding methods and the method of this invention on agricultural document classification tasks as the document length changes from short text to long document, the curve trends and discrete point distribution in the figure simulate the performance fluctuations in real experiments. The experimental results show that as the document length increases, the performance of conventional methods drops sharply, while the method of this invention maintains stable and excellent performance. This proves that the entity-aware relative distance encoding mechanism can effectively capture the spatial relationship features between the description of pests and diseases and crop entities in agricultural texts, overcoming the defect of traditional location encoding that only considers word order position and ignores the semantic association of entities.

[0206] S503, Determining the stopping iteration condition;

[0207] The performance of the validation set is continuously monitored during training, and the total loss of the validation set is calculated after each iteration. The classification accuracy metric is set. If the verification loss does not decrease for 5 consecutive rounds, or the task metric reaches a predetermined threshold, such as accuracy > 95%, an early stop mechanism is triggered. At the same time, a maximum number of iterations is set as a mandatory termination condition, such as 10,000 rounds, to avoid overfitting.

[0208] S6, Large Model Inference Applications;

[0209] The trained model is deployed as an agricultural knowledge service engine. Specifically, after the user inputs a question text (such as "methods for controlling corn leaf spot disease"), the model sequentially performs dynamic word embedding disambiguation, regional terminology replacement, and entity-aware location encoding to generate an enhanced representation of the input sequence. Then, the Transformer encoder extracts deep semantic features, the classification task directly outputs the predicted label, and the generation task generates the answer word by word through the decoder.

[0210] S7, RAG Enhanced Question Answering; RAG is a model architecture that combines retrieval and generation. By retrieving relevant information from a large-scale external knowledge base and incorporating it into the generation process, it enhances the knowledge and accuracy of the model, thereby generating richer and more accurate text content.

[0211] To enhance the ability to answer complex questions, RAG-enhanced question answering is implemented by combining retrieval enhancement generation mechanisms, as detailed below:

[0212] 1) Retrieval stage: User queries are converted into vectors by the embedding model, and Top-K relevant paragraphs are retrieved from the agricultural knowledge base constructed by S1, such as papers and manuals in the agricultural knowledge base;

[0213] 2) Augmented input: The retrieved paragraphs are concatenated with the original question and input into the trained agricultural big data model;

[0214] 3) Generating answers: The model integrates the retrieved content with its own knowledge to generate structured answers, such as step-by-step prevention and control plans, thereby improving its coverage of long-tail knowledge such as time-sensitive policies and regional agricultural techniques.

[0215] like Figure 6 As shown, this study evaluates the ability of the retrieval enhancement generation mechanism to improve the solution of complex agricultural problems. By comparing the accuracy of the basic model and the RAG-enhanced model in answering four typical agricultural problems—common problems, professional problems, regional problems, and timely policies—the grouped bars in the figure visually show the performance differences in each scenario, and the scatter points inside the bars represent the distribution of experimental data. The experimental results show that the RAG enhancement mechanism has the most significant improvement on professional knowledge and timely content. First, the user's question is converted into a vector to retrieve professional literature and the latest policies from the relevant knowledge base. Then, the retrieved content is concatenated with the original question and input into the model to generate a structured answer, which effectively makes up for the problem of insufficient coverage of long-tail knowledge such as regional agricultural technology and timely policies by the large model.

[0216] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and improvements, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

Claims

1. A method for enhancing question answering based on large agricultural model training and RAG (Rapid Application Group) is characterized by: The method includes: Step 1: Collect agricultural data and construct a training dataset for a large agricultural model; Step 2: The input embedding of the Transformer is formed sequentially through disambiguation of agricultural terminology and dynamic word embedding initialization, regional language pattern enhancement encoding, and agricultural entity perception location encoding. Then, a Transformer encoder is constructed to train a large model based on the Transformer architecture. Step 3: After the user inputs the question text, the large model sequentially performs dynamic word embedding disambiguation, regional terminology replacement, and entity-aware location encoding to generate an enhanced representation of the input sequence. Then, the Transformer encoder extracts deep semantic features. The classification task directly outputs the predicted label, while the generation task generates the answer word by word through the decoder. Step 4: Perform RAG-enhanced question answering by combining search enhancement generation mechanisms; The aforementioned agricultural terminology polysemy disambiguation and dynamic word embedding initialization specifically include the following: A1. Constructing an agricultural ontology graph: Constructing a set of agricultural entities as follows Let the set of semantic relationships between entities be defined as Then the agricultural ontology map is ,in, Represents the total number of agricultural entities. Indicates the first One agricultural entity, Indicates the first agricultural entity. Indicates the second agricultural entity. Indicates the first One agricultural entity; A2. Context-aware disambiguation: The local context window word sequence of the target word is encoded by bidirectional LSTM to generate a context vector. The cosine similarity between the target word and the pre-trained vectors of each entity in the agricultural ontology graph is calculated. After the similarity discrimination control coefficient is adjusted, the semantic similarity score between the target word and each entity is obtained by normalization through the natural exponential function. A3. Dynamic word embedding generation: Based on the semantic similarity scores of the target word's base word vector and each entity vector, the entity vector fusion result is obtained by weighted summation. The fusion weight of the base word vector is calculated by the Sigmoid function. The fusion results of the base word vector and the entity vector are added according to the weights to generate a context-aware dynamic word embedding vector.

2. The method for agricultural large-scale model training and RAG-enhanced question answering according to claim 1, characterized in that: The regional language pattern enhancement coding specifically includes the following: B1. Construction of regional dictionary: Extract the mapping relationship between dialect terms and standard terms from agricultural yearbook data, and construct a dialect-standard term mapping table; B2. Bidirectional substitution mask generation: For each word in the input sentence, if a dialect term mapping exists, a standard term substitution mask value of 1 is generated; otherwise, it is 0. B3. Fusion Encoding Representation: The mask weighted result of the dynamic word embedding vector and the standard term vector is concatenated, and the fusion projection matrix and bias vector are linearly transformed after training to generate regionally enhanced word vectors.

3. The method for agricultural large-scale model training and RAG-enhanced question answering according to claim 1, characterized in that: The agricultural entity sensing location coding specifically includes the following: C1. Entity Location Marking: Identify the location of all agricultural entities in the input sequence and construct a set of location indexes for each entity; C2. Relative distance matrix calculation: Calculate the shortest absolute distance from each word position to the set of position indices of each entity; C3. Entity-aware position coding generation: Based on the shortest distance matrix, the standard sinusoidal position codes of each entity are weighted by the distance decay factor, normalized, and then summed to generate an enhanced position coding vector.

4. The method for agricultural large-scale model training and RAG-enhanced question answering according to claim 1, characterized in that: The construction of the Transformer encoder specifically includes: D1. Constructing the basic layer structure: Adopting a block-aggregation strategy, capturing features within blocks through local attention, realizing inter-block interaction through a gating mechanism, and fusing local and global representations in a layered manner; D2. Stacking: Repeatedly stacking the basic layer structure Second-rate, The stacked layers are the number of layers, and the output of the stack is the context representation of the encoder of the last layer, which is achieved by repeatedly stacking the base layer structure. Next, final The output of the last layer is used as the result of the entire encoder, and the output feature of the last layer encoder is set as follows: ; D3. Output: The output of the last encoder layer is used as the context representation of the document for the classification task, and is also input to the decoder to generate the answer.

5. The method for agricultural large-scale model training and RAG-enhanced question answering according to claim 4, characterized in that: D1 specifically includes the following: D101, Semantic Blocking and Local Attention Calculation: The input document is divided into semantic blocks, and the projections of the query matrix, key matrix and value matrix are calculated within each block. Local attention output is generated by scaling dot product attention and normalized exponential function. D102, Global Attention Gated Aggregation: The local attention output of each block is averaged and compressed into a block representation, which is then weighted by hyperbolic tangent activation and Sigmoid gating weights and aggregated into a global document representation; D103. Layer Normalization and Residual Connectivity Based on Hierarchical Weighted Fusion: The local attention output and the global document representation are weighted and summed according to the hierarchical fusion coefficients, then residually connected with the output features of the previous layer. After layer normalization, the current layer features are output. The output features based on hierarchical weighted fusion and residual connectivity are defined as follows: ; D104, Feedforward Neural Network: The feedforward neural network adopts the standard feedforward neural network of Transformer. The input is the output feature of layer normalization based on layer weighted fusion. The activation function of the feedforward neural network adopts the ReLU activation function. D105, Standard Layer Normalization and Residual Connections: The standard layer normalization and residual connections of the Transformer are used. The input is the output features of the feedforward neural network, which, after layer normalization, are then... The residuals are summed to output the output features of the current layer. The output features based on layer normalization and residual connection, which are weighted by layer fusion, are defined as follows: .

6. The method for agricultural large-scale model training and RAG-enhanced question answering according to claim 4, characterized in that: The D3 specifically includes the following: D301, Classification Task: [The task is to...] The first word vector is input into a fully connected layer, and the classification result is generated by Softmax. D302, Generate Task: Input a Transformer decoder, which uses the encoder output as key-value pairs and the previous word as the query target to iteratively generate a sequence of answers; During the training phase of the decoder, the input to the decoder consists of three parts: the encoder output, the answer sequence offset input, and the self-attention mask; specifically, it includes the following: Encoder output: Contextual representation from the last encoder layer, i.e., the deep semantic features of the document, including dynamically disambiguated agricultural terms, regionally standardized expressions, and entity spatial relationship information; Answer sequence offset input: A sequence of answers to the task generated from the constructed training dataset. And shift it one position to the right, that is, add a start marker at the starting position, as the initial input of the decoder; Self-attention mask: Ensures that during decoding, each position only focuses on the generated left-hand word, avoiding information leakage; During the training phase, the decoder performs the following steps layer by layer: Masked self-attention layer: calculates internal dependencies for the currently generated word sequence; Encoder-decoder attention layer: Aligns the mask self-attention output with the encoder's context representation, focusing on key information; Feedforward network and residual connection: The features are merged and passed to the next layer.

7. The method for agricultural large-scale model training and RAG-enhanced question answering according to claim 1, characterized in that: The training of the large model based on the Transformer architecture specifically includes the following: E1. Multi-granularity loss function optimization: A dual-granularity joint loss function is adopted, which strengthens semantic consistency learning through term-level contrastive loss and dynamically weights and fuses it with document-level task loss to achieve multi-level supervised optimization. E2. Model Iterative Training and Parameter Update: A multimodal adaptive gradient modulator is adopted to achieve fine-grained parameter updates through gradient sensitivity analysis, parameter clustering and grouping, and modal learning rate scheduling. E3. Stopping Iteration Condition Judgment: Continuously monitor the performance of the validation set during training, and calculate the total loss of the validation set after each iteration. The classification accuracy metric is set. If the verification loss does not decrease for 5 consecutive rounds, or the task metric reaches the predetermined threshold, the early stop mechanism is triggered. At the same time, the maximum number of iterations is set as a mandatory termination condition.

8. The method for agricultural large-scale model training and RAG-enhanced question answering according to claim 1, characterized in that: Step four specifically includes the following: Retrieval Phase: User queries are converted into vectors by the input embedding of the Transformer, and Top-K relevant paragraphs are retrieved from the constructed agricultural knowledge base; Augmented input: The retrieved paragraphs are concatenated with the original question and input into the trained agricultural big data model; Answer generation: The agricultural big data model integrates the search results with its own knowledge to generate structured answers.

9. A method for agricultural large-scale model training and RAG-enhanced question answering according to any one of claims 1-8, characterized in that: Step one specifically includes the following: Collect multi-source heterogeneous agricultural data, including structured data, semi-structured data, and unstructured data; Outlier removal, timestamp alignment, and feature standardization are performed on structured time-series data to form a continuous sensor data stream; For semi-structured data, key fields are extracted using table parsing tools and converted into structured records in key-value pair format. At the same time, fields are extracted as category labels. For unstructured text data, sentence segmentation, noise reduction, and semantic paragraph division are performed to ensure logical integrity and to construct question-answer pairs; A unified encoding method is used for multi-task datasets, encoding all data into a text sequence format, with numerical data embedded through bucketing discretization.

Images