A method, system, computer equipment, and media for constructing a large-scale tea technology model.

Abstract

This invention provides a method, system, computer equipment, and medium for constructing a large-scale tea technology model, belonging to the field of agricultural informatization. The method includes acquiring tea images, tea tree gene sequences, tea technology knowledge text data, and molecular structure diagrams of tea compounds; training a Transformer model based on the tea knowledge information to obtain an initial model, extracting image features, text features, gene features, and molecular structures respectively; fusing features based on a cross-modal attention mechanism to obtain fused features; outputting human-readable text based on the fused features; extracting question-answer pairs from the tea knowledge information using prompt word templates through the initial model, constructing a tea knowledge question-answer pair dataset based on the question-answer pairs; training the initial model based on the tea knowledge question-answer pair dataset to obtain a large-scale tea technology model after fine-tuning. This method ensures the adaptability of the constructed model for application in the field of tea technology.

Description

Technical Field

[0001] This invention belongs to the field of agricultural informatization, specifically involving a method, system, computer equipment, and medium for constructing a large-scale tea technology model. Background Technology

[0002] Tea, a beloved beverage in daily life, has seen its research extend beyond traditional cultivation and processing to the molecular level. This involves not only knowledge of the tea plant's genome and transcription factors, but also the identification of 2D and 3D molecular structures and gene sequences. This research using modern science and technology to explore the molecular aspects of tea is collectively referred to as tea science. However, tea science currently faces numerous challenges in its intelligent development, such as fragmented knowledge distribution and a lack of unified semantic organization, which limits the effective utilization of tea knowledge by intelligent systems.

[0003] To address the aforementioned shortcomings, with the rapid development of artificial intelligence, especially the widespread application of large language models—for example, models like Deepseek, Doubao, and Kimi have demonstrated excellent capabilities in various text question-answering tasks—the agricultural technology field has begun to incorporate them into research and production. Tea technology is gradually entering a new intelligent stage driven by data and empowered by models. While these general-purpose models possess a certain ability to understand and generate knowledge, and can answer some commonly used questions in tea technology, they struggle with the identification and analysis of molecular-level information such as molecular structures and gene sequences, resulting in insufficient adaptability for practical applications in the tea technology field. Summary of the Invention

[0004] To address the issue of insufficient adaptability of existing general models in the field of tea technology, this invention provides a method, system, computer equipment, and storage medium for constructing a large-scale tea technology model.

[0005] To achieve the above objectives, the present invention provides the following technical solution:

[0006] A method for constructing a large-scale tea technology model includes:

[0007] Acquire tea knowledge information, which includes tea images, tea tree gene sequences, textual data on tea science and technology, and molecular structure diagrams of tea compounds;

[0008] A Transformer model is trained based on tea knowledge information to obtain an initial model. This initial model includes encoders for images, gene sequences, text, and molecular structures, a cross-modal fusion module, and an output module. The encoders extract image features, text features, gene features, and molecular structures from tea images, tea tree gene sequences, tea science and technology knowledge text data, and tea compound molecular structure diagrams, respectively. The cross-modal fusion module fuses image features, text features, gene features, and molecular structure features based on a cross-modal attention mechanism to obtain fused features. The output module outputs human-readable text based on the fused features.

[0009] Based on prompt word templates, question-answer pairs are extracted from tea knowledge information through an initial model, and a tea knowledge question-answer pair dataset is constructed based on the question-answer pairs.

[0010] The question-and-answer dataset is filtered, and an initial model is trained on the dataset based on the filtered tea knowledge questions and answers, resulting in a large-scale tea technology model that has been fine-tuned according to instructions.

[0011] Optionally, the method for constructing a large-scale tea technology model provided by the present invention further includes:

[0012] The tea image is divided into multiple image blocks by an image encoder. Position encoding is added to the multiple image blocks and the encoding process is performed to obtain image features.

[0013] The text data on tea technology knowledge was divided into multiple semantic units by using a tokenizer. Positional encoding was added to these semantic units and the encoding process was performed to obtain text features.

[0014] The tea tree gene sequence was segmented into multiple consecutive k-mer fragments using a gene sequence encoder based on the sliding window method. The k-mer fragments were then encoded using the DNABERT model to obtain gene features.

[0015] The molecular structure diagram of tea compound is converted by a molecular structure encoder to obtain the SMILES molecular expression, and a molecular diagram is generated based on the SMILES molecular expression. The molecular diagram is then encoded to obtain molecular features.

[0016] The cross-modal attention between image features, text features, gene features and molecular features is calculated through the cross-modal fusion module. Based on the cross-modal attention, the image features, text features, gene features and molecular features are weighted and fused to obtain the tea fusion feature.

[0017] The token sequence is determined by the tea fusion characteristics through the output module, and the token sequence is converted into human-readable text.

[0018] Optionally, in the method for constructing a large-scale tea technology model provided by the present invention, a LoRA low-rank matrix is ​​set in the linear layer of the encoder corresponding to each of the image, gene sequence, text and molecular structure.

[0019] Optionally, the initial model also includes a quality scoring module. The method for constructing a large-scale tea technology model provided by this invention also includes:

[0020] The quality scoring module is used to score the quality of the tea knowledge Q&A dataset, and the quality score results for each dataset are obtained.

[0021] The quality score is corrected by using a score transition matrix to obtain a corrected quality score. The score transition matrix is ​​determined based on the consistency distribution of the quality score results.

[0022] The similarity between each sample in the tea knowledge question-answering dataset and its semantically neighboring samples is calculated using the mean cosine similarity, thus obtaining the semantic diversity score for each dataset.

[0023] The tea knowledge question-answer pair dataset was filtered based on quality correction scores and semantic diversity to obtain a subset of question-answer pair data.

[0024] The initial model is trained using a subset of question-and-answer pairs to obtain a large-scale tea technology model after fine-tuning.

[0025] Optionally, the samples include single samples, sample pairs, and sample triples. The method for constructing a large-scale tea technology model provided by this invention also includes:

[0026] The score distribution is calculated based on the quality score results of a single sample to obtain the first-order consensus vector.

[0027] The second-order consensus vector is obtained by calculating the score offset probability based on the quality score results of the sample pairs.

[0028] The score offset probability is calculated based on the quality score results of the sample triples, resulting in a third-order consensus vector;

[0029] The score transition matrix and the probability distribution of the true scores are determined based on the first-order consensus vector, the second-order consensus vector, and the third-order consensus vector.

[0030] The quality score is corrected by adjusting the score transition matrix and the probability distribution of the true score to obtain the quality-corrected score.

[0031] Optionally, the method for constructing a large-scale tea technology model provided by the present invention further includes:

[0032] By inputting a subset of question-and-answer pair data into the tea technology big model after fine-tuning the training instructions, suboptimal response data is obtained.

[0033] By merging the question-answer pair data subset and the suboptimal response data, a human preference judgment dataset is obtained.

[0034] Based on the human preference judgment dataset, the tea technology big model, which was fine-tuned by instructions, was trained by direct preference optimization to obtain the tea technology big model optimized by direct preference.

[0035] Optionally, the method for constructing a large-scale tea technology model provided by the present invention further includes:

[0036] A tea knowledge base was constructed based on tea science and technology knowledge text data and knowledge graphs. The knowledge graphs were constructed from tea tree genome knowledge texts using entity recognition and relation extraction algorithms.

[0037] The data in the tea knowledge base is encoded into text block vectors, and a vector database is constructed based on the text block vectors;

[0038] Obtain tea knowledge questions and calculate the cosine similarity between the questions and multiple text block vectors to obtain the cosine similarity between the multiple text block vectors.

[0039] The tea knowledge question information and the text block vector with the highest cosine similarity are input into the tea technology big model optimized by direct preference to obtain the tea knowledge answer result.

[0040] This invention also provides a tea technology large-scale model construction system, comprising:

[0041] The data import module is used to acquire tea knowledge information, which includes tea images, tea tree gene sequences, tea science and technology knowledge text data, and tea compound molecular structure diagrams.

[0042] The pre-training module is used to train the Transformer model based on tea knowledge information to obtain an initial model. This initial model includes encoders for images, gene sequences, text, and molecular structures, a cross-modal fusion module, and an output module. The encoders extract image features, text features, gene features, and molecular structures from tea images, tea tree gene sequences, tea science and technology knowledge text data, and tea compound molecular structure diagrams, respectively. The cross-modal fusion module fuses image features, text features, gene features, and molecular structure features based on a cross-modal attention mechanism to obtain fused features. The output module outputs human-readable text based on the fused features.

[0043] The question-answer pair dataset construction module is used to extract question-answer pairs from tea knowledge information based on prompt word templates and an initial model, and to construct a tea knowledge question-answer pair dataset based on the question-answer pairs.

[0044] The instruction fine-tuning module is used to train an initial model on a dataset based on tea knowledge questions and answers, and then obtain a large-scale tea technology model after instruction fine-tuning training.

[0045] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement any of the steps in a method for constructing a large model of tea technology.

[0046] The present invention also provides a computer-readable storage medium storing a computer program that, when loaded by a processor, can execute any step of a method for constructing a large-scale tea technology model.

[0047] The method for constructing a large-scale tea technology model provided by this invention has the following beneficial effects:

[0048] Because the method for constructing a large-scale tea technology model provided by this invention introduces images, gene sequences, text, and molecular structures... Figure 4 By using multimodal data and designing corresponding dedicated encoders, the model accurately analyzes and represents molecular-level information of tea leaves. Through a cross-modal attention fusion module, the semantic information of different modalities is aligned, and the constructed tea technology big data model can identify and respond to molecular-level information such as molecular structure and gene sequence. This meets the diverse tea knowledge questioning needs of users, ensures the adaptability of the tea technology big data model to tea-related knowledge, and provides a scalable technical path for the intelligentization of tea technology. Attached Figure Description

[0049] To more clearly illustrate the embodiments and design schemes of the present invention, the accompanying drawings required for this embodiment will be briefly described below. The drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a schematic diagram of a method for constructing a large-scale tea technology model according to an embodiment of the present invention;

[0051] Figure 2 This is a schematic diagram of the overall framework of the large-scale tea technology model provided in the embodiments of the present invention;

[0052] Figure 3 This is a flowchart illustrating the steps of a method for constructing a large-scale tea technology model according to an embodiment of the present invention.

[0053] Figure 4 This is a schematic diagram of a large-scale tea technology model construction system provided in an embodiment of the present invention. Detailed Implementation

[0054] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. The following embodiments are only used to more clearly illustrate the technical solutions of the present invention and should not be construed as limiting the scope of protection of the present invention.

[0055] While existing general-purpose models possess some knowledge understanding and generation capabilities, and can answer some commonly used questions in tea science, their answers are not precise enough, and almost none can identify molecular-level information such as molecular structures and gene sequences. Therefore, the tea industry lacks a large-scale model framework with cross-modal understanding capabilities, support for professional data modeling, and scalability and systematization. This paper proposes a solution by combining high-quality data screening mechanisms, knowledge enhancement methods, and preference alignment for optimized training, along with a scientific and quantifiable performance evaluation system, to truly serve the intelligent upgrading and professional research of tea science.

[0056] Based on the aforementioned needs, this invention constructs a multimodal large-scale model supporting 2D and 3D molecular image recognition, gene sequence recognition, transcription factor and gene knowledge question answering. It combines this with a data construction method based on semantic filtering and confidence assessment for instruction fine-tuning, and uses a training approach that combines instruction fine-tuning and alignment training to optimize the large-scale tea technology model. Addressing the potential for "large-scale model illusion" caused by instruction fine-tuning, this invention utilizes a retrieval-enhanced generation mechanism to mitigate this phenomenon, thereby resolving issues such as insufficient knowledge coverage, lack of molecular structure and gene sequence recognition functions in existing large-scale models when handling tea technology-related tasks.

[0057] Example 1

[0058] This invention provides a method for constructing a large-scale model of tea technology, specifically as follows: Figure 1 As shown, it includes the following steps:

[0059] Step 11: Obtain tea knowledge information, which includes tea images, tea tree gene sequences, tea science and technology knowledge text data, and molecular structure diagrams of tea compounds.

[0060] Specifically, the method for constructing a large-scale tea technology model provided by this invention first collects relevant data on tea technology, encompassing almost all scientific research papers and textbooks in the field; images of the molecular structures of compounds in tea; and knowledge information such as tea tree gene sequences, genomes, and transcription factors. Among these, the textual data on tea technology knowledge includes textual data related to tea technology, such as genes and transcription factors, including a large amount of professional data, academic papers, industry reports, case studies, and professional images in the tea field. Tea text and images are obtained from these sources. The compounds in tea are then used to create 3D structural images from their 2D molecular formulas using software, resulting in molecular structure diagrams of tea compounds. Data on tea tree genomes, transcription factors, and regulatory pathways are also acquired for subsequent model training.

[0061] For example, this invention collects the 2D molecular formulas and amino acid sequence information of almost all compounds in the tea field, and uses ChemDraw and Chem3D to create 3D structural formulas of the compound molecules based on the collected 2D molecular formulas. Furthermore, to make molecular recognition more accurate, features are supplemented and labeled for different molecules to enhance the model's ability to express complex structures. In addition, to make the model's knowledge coverage more comprehensive, this invention also collects various genome-related data such as gene exons, introns, mRNA, transcription factors, and regulatory pathways, thereby achieving multimodal data acquisition of tea gene sequences and tea compound molecular structure diagrams.

[0062] Step 12: Train the Transformer model based on tea knowledge information to obtain an initial model. The initial model includes encoders for images, gene sequences, text, and molecular structures, a cross-modal fusion module, and an output module. The encoders are used to extract image features, text features, gene features, and molecular structures from tea images, tea tree gene sequences, tea science and technology knowledge text data, and tea compound molecular structure diagrams, respectively. The cross-modal fusion module is used to fuse image features, text features, gene features, and molecular structure features based on a cross-modal attention mechanism to obtain fused features. The output module is used to output human-readable text based on the fused features.

[0063] Step 12 includes:

[0064] Step 121: Divide the tea image into multiple image blocks using an image encoder, add positional encoding to the multiple image blocks and perform encoding processing to obtain image features;

[0065] Step 122: Divide the tea technology knowledge text data into multiple semantic units using the Tokenizer segmenter, add positional encoding to the multiple semantic units and encode them to obtain text features;

[0066] Step 123: Using a gene sequence encoder, the tea tree gene sequence is segmented into multiple continuous k-mer fragments based on the sliding window method. The k-mer fragments are then encoded using the DNABERT model to obtain gene features.

[0067] Step 124: Convert the molecular structure diagram of tea compound into SMILES molecular expression through molecular structure encoder, generate molecular diagram based on SMILES molecular expression, and encode the molecular diagram to obtain molecular features.

[0068] Step 125: Calculate the cross-modal attention between image features, text features, gene features, and molecular features through the cross-modal fusion module, and then weight and fuse the image features, text features, gene features, and molecular features based on the cross-modal attention to obtain the tea fusion features;

[0069] Step 126: Determine the token sequence from the tea fusion features through the output module, and convert the token sequence into human-readable text.

[0070] During model pre-training, the data is processed into a unified JSONL format, and images are tagged with paths. After cleaning and deduplication, a pre-training dataset is obtained. This pre-training dataset is then used for large-scale language model pre-training. The AdamW optimizer is employed during model pre-training, with appropriate hyperparameters such as learning rate, weight decay coefficient, and number of training epochs set. A learning rate scheduling strategy combining linear warm-up and cosine annealing is used, along with mixed precision training and gradient accumulation techniques to improve training efficiency and adapt to memory resource constraints. This enables the TeaTech large-scale model to understand and generate answers for relevant tasks.

[0071] Specifically, this invention can build a multimodal large-scale model based on the Transformer architecture of multimodal fusion, and train the model with collected data to obtain the final tea technology large-scale model. This tea technology large-scale model can be a large language model or other types of models; this application does not impose any restrictions. The tea technology large-scale model includes an encoder, a cross-modal fusion module, and a unified language model output module. Considering that the input image may be a general tea technology task image, a 2D molecular formula image, or a 3D molecular formula image, and different tasks require different encoders, this invention incorporates an image pre-judgment mechanism in the task scheduling module to prevent model invocation errors. If the image is determined to be a molecular image, the molecular structure encoder is invoked; if it is a general image, the image encoder is invoked. The encoder module includes an image encoder, a text encoder, a gene sequence encoder, and a molecular structure encoder.

[0072] Furthermore, before inputting data into the model, the task scheduling and control module determines the type of encoder to be invoked for the input task. Also, considering that the input molecular structure image is also a pixel image, an image pre-judgment mechanism is added to the task scheduling module: if it is determined to be a molecular image, the molecular structure encoder is invoked; otherwise, the image encoder is invoked.

[0073] The image encoder is built based on the visual Transformer to extract and encode features from the image data. For the image in the input model, it is divided into N image blocks, and each image block is flattened to obtain a vector, as shown in formula (1):

[0074] (1)

[0075] in, It is the i-th image patch. Let d represent the embedding vector of the i-th image patch, where d represents the dimension. Represents a mapping function. Indicates the image block Mapping to vector space Then, a positional encoding vector is added to each image block vector, as shown in formula (2):

[0076] (2)

[0077] in, Let x represent the initial input vector matrix after position encoding, where x represents the image vector and p represents the position encoded vector.

[0078] The vector with added positional encoding is then encoded using a Transformer to obtain image features, as shown in formula (3):

[0079] (3)

[0080] Among them, image features The text encoder, built upon the language Transformer, extracts and encodes text data. For example, it uses a Tokenizer to divide the text into tokens, adds positional encoding, and then encodes them using a Transformer to obtain text features. .

[0081] Considering that the input is a 2D or 3D molecular structure image, the molecular structure encoder integrates a specially trained image classifier to determine whether the information in the input model is a 2D molecular formula or a 3D structural formula. It then uses molecular structure image recognition models such as DECIMER to convert the molecular formula or structural formula image into a standard SMILES molecular expression. When the input model information is a 2D molecular formula image, chemical tools such as RDKit convert SMILES into the graph structure form required by the graph neural network, where atoms are nodes and chemical bonds are edges. When the input model information is a 3D structural formula image, RDKit's conformation generator or 3D structure prediction model generates a 3D perceptual graph structure containing atomic topological features, 3D coordinates, and bond geometric features based on the SMILES. Finally, a graph neural network is used to encode the molecular graph, extracting molecular-level structural features, and training it in conjunction with task labels. Through end-to-end learning, automatic recognition and property prediction of molecular images are achieved.

[0082] The gene sequence encoder is built upon DNABERT and employs a Transformer architecture to model DNA sequences. Specifically, the gene sequence input to the model is first segmented using a sliding window method with fixed-length k-mer segments to generate continuous, overlapping k-mer fragments. These fragments are used to construct token sequences with biologically relevant local contextual meaning. Subsequently, the k-mer sequences are used as input and encoded by the DNABERT model to obtain the contextual representation of each position and the semantic embedding vector of the entire sequence. This encoder enables the model to recognize and classify genes.

[0083] After multiple encoders set up in parallel extract image features, text features, gene features, and molecular features, the cross-modal fusion module uses a cross-attention mechanism to dynamically align and fuse features between images and text, text and text, molecular images and their text labels, or gene sequences and their text labels, thereby improving the model's semantic alignment and reasoning capabilities under multimodal conditions.

[0084] For example, when fusing image and text features, the computation process of cross-modal attention mechanisms is typically a variant of self-attention operations between feature sequences from two modalities. For instance, for text feature sequences... and image feature sequences When performing cross-modal attention computation, the image feature sequence is processed through... The projection yields the Query, and the text feature sequences are respectively processed through... and The projection yields the Key and Value, according to the attention mechanism shown in formula (4):

[0085] (4)

[0086] in, , , Q is the query vector, K is the key vector, and V is the value vector. , and Let be the projection matrix. Cross-modal attention is shown in equation (5):

[0087] (5)

[0088] in, For the dimension of attention head, Each element in can be determined based on... The correlation of elements in the middle is obtained The weighted representation of corresponding elements in the dataset enables cross-modal information fusion. Similarly, it is also possible to calculate... right Cross-modal attention is used to obtain fused features based on the cross-modal attention fusion between various features. Finally, the unified language model output module converts the fused semantic representation into user-understandable natural language or structured form and outputs the representation.

[0089] Next, the output module calculates hidden vectors from the fused features and maps them to the vocabulary space to obtain the probability distribution of each token. Then, it uses sampling or Top-k decoding to progressively select token sequences from the probability distribution. Finally, a token segmenter converts the token sequences into the final human-readable text.

[0090] Step 13: Based on the prompt word template, extract question-answer pairs from tea knowledge information through the initial model, and construct a tea knowledge question-answer pair dataset based on the question-answer pairs.

[0091] Step 14: Filter the question-and-answer dataset, train the initial model on the dataset based on tea knowledge questions and answers, and obtain the large tea technology model after fine-tuning the training.

[0092] Specifically, the tea technology large-scale model construction method provided by this invention can guide the model to extract key knowledge from collected tea technology data knowledge to construct a question-and-answer pair set through the constructed prompt word template. This effectively solves the problem of difficulty in creating question-and-answer pair datasets during instruction fine-tuning and significantly reduces manual costs. For non-text data, the model can analyze and describe images, and generate question-and-answer pairs based on the analysis and description. This ensures that the generated question-and-answer pairs have a knowledge base and professional accuracy, while retaining the professional terminology, standard definitions, and domain specifications in the original dataset, thus improving the quality and efficiency of automated data construction. For example, the format of the tea knowledge question-and-answer pair dataset can include both questions and answers, such as "Q: What are the characteristics of the transcription factor WRKY? A: The characteristics of the transcription factor WRKY are…". Furthermore, for question-and-answer datasets containing images, an image path information is added, so that the corresponding image can be used during training.

[0093] The initial model also includes a quality scoring module, and step 14 includes:

[0094] Step 141: Use the quality scoring module to score the quality of the tea knowledge Q&A dataset and obtain the quality score results for each dataset.

[0095] Step 142: Correct the quality score results using the score transition matrix to obtain the corrected quality score. The score transition matrix is ​​determined based on the consistency distribution of the quality score results.

[0096] The samples include single samples, sample pairs, and sample triples. Step 142 includes:

[0097] Step 1421: Calculate the score distribution based on the quality score results of a single sample to obtain the first-order consensus vector.

[0098] Step 1422: Calculate the score offset probability based on the quality score results of the sample pairs to obtain the second-order consensus vector.

[0099] Step 1423: Calculate the score offset probability based on the quality score results of the sample triplet to obtain the third-order consensus vector.

[0100] Step 1424: Determine the score transition matrix and the probability distribution of the true scores based on the first-order consensus vector, the second-order consensus vector, and the third-order consensus vector.

[0101] Step 1425: Correct the quality score result based on the score transition matrix and the probability distribution of the true score to obtain the quality corrected score.

[0102] Step 143: Calculate the similarity between each sample in the tea knowledge question-answering dataset and its semantically neighboring samples using mean cosine similarity, and obtain the semantic diversity score for each dataset.

[0103] Step 144: Filter the tea knowledge question-answer pair dataset based on the quality correction score and semantic diversity score to obtain a subset of question-answer pair data.

[0104] Step 145: Train the initial model based on the question-and-answer pair data subset to obtain the tea technology model after fine-tuning the training.

[0105] Specifically, after pre-training the tea technology model based on the collected data, the question-answer pairs in the tea knowledge question-answer pair dataset can be further filtered to determine the instruction fine-tuning dataset, and the tea technology model can be trained again based on the instruction fine-tuning dataset.

[0106] First, the generated question-and-answer pairs data are initially scored based on rarity, large language, complexity, and information content using the tea technology model, resulting in a quality score. Considering that different models may have scoring biases in specific domains, a score transition matrix is ​​introduced to correct the error of the model output score results in order to improve the credibility and robustness of the score, as shown in formula (6):

[0107] (6)

[0108] in, for The transition matrix, To embed feature vectors, The ratings generated for the tea technology model For the actual scoring, K represents the scoring range of 0 points. 5 points To get the real rating Transition to tea technology model to generate predictive scores The probability. When the tea technology model predicts a score that matches the actual score, that is... n, If the transition matrix is ​​equal to the identity matrix, then no error will occur.

[0109] Considering the difficulty in directly obtaining the true scores of question-answer pairs, this invention uses K-NN score clustering, meaning that similar embedded features should belong to the same score category, which is a kind of "semantic similarity". The assumption of "similar quality" is used to estimate scores based on the tea technology model. By statistically analyzing the consistent distribution of scores locally, a "consensus vector" is constructed. We can then deduce the rating transition matrix T and the probability distribution p of the true score, as shown in formulas (7), (8), and (9):

[0110] (7)

[0111] (8)

[0112] (9)

[0113] in, The first-order consensus vector is determined by calculating the single-sample rating distribution. It is a second-order consensus vector, determined by the rating offset probability between sample pairs. The consensus vector is a third-order vector, determined by the score shift probability of the sample triplet. Let represent the Hadamard product, p be the distribution of the true scores, and T be the rating transition matrix to be estimated. K is the number of categories. and These are the cyclic shift parameters used to construct higher-order consensus vectors. and It is a circular right shift of the columns of the rating transition matrix T. , The new matrix after the bits, This indicates that the score j is cyclically shifted modulo K. Finally, a system of equations is constructed based on formulas (7), (8), and (9) to calculate the score transition matrix K and the probability distribution p of the true score.

[0114] Then, the posterior probability of all possible true scores from 0 to 5 is calculated using Bayes' theorem, as shown in formula (10):

[0115] (10)

[0116] Where k represents all possible scores, and the score with the highest posterior probability is selected from them. The final quality correction score is shown in formula (11):

[0117] (11)

[0118] Furthermore, to prevent the model from misjudging high-quality samples during the calibration process, a confidence probability mechanism with a confidence level of 0.5 is introduced to maintain sample selection stability within the scoring uncertainty region. Then, a correction suggestion is generated, and the correction score is... .

[0119] While ensuring quality, the selection of tea knowledge Q&A datasets should also consider the semantic diversity of samples. Specifically, the mean cosine similarity is used to calculate the similarity between each sample and its semantic nearest neighbor in order to select long-tail samples with unique content value, thereby enhancing the knowledge coverage and representation ability of the dataset, as shown in formula (12):

[0120] (12)

[0121] Where n is the number of nearest neighbors that are semantically most similar to the sample, and v is the index of the neighbor sample, representing each neighbor in the nearest neighbor set. and The semantic embedding representation obtained by encoding, for The set of n nearest neighbors, The average cosine similarity is calculated between each sample and its semantic nearest neighbors.

[0122] Finally, a high-quality subset of data for model fine-tuning and knowledge base construction was selected based on a weighted strategy of quality and diversity scores. This method significantly reduces the cost of manual annotation and enhances the coverage and accuracy of the selection results. By avoiding performance degradation caused by low-rated or redundant data, smaller, higher-quality datasets can achieve better results, thereby improving the performance of model fine-tuning. A similar method was used to select high-quality datasets for text-to-image question-answering pairs, which were then used as the training dataset for fine-tuning the TeaTech model.

[0123] Based on this, considering that training methods such as instruction fine-tuning during model training may cause the model to forget its original knowledge, LoRA technology is introduced in the instruction fine-tuning stage. By introducing low-rank matrices into the attention mechanism inside the encoder and the linear layers of the feedforward network, model fine-tuning is achieved without making significant modifications to the entire model structure, while enhancing the model's response accuracy and professional performance in tea technology tasks.

[0124] Specifically, the LoRA principle states that the weights of a certain linear layer are... LoRA introduces two low-rank matrices and ,in , is the rank, and the adjusted weight is calculated as shown in formula (13):

[0125] (13)

[0126] in, is the scaling factor used to control the magnitude of LoRA updates. BA is the low-rank update matrix. During training, only A and B are optimized, while the original W remains frozen.

[0127] For example, the selected tea knowledge question-and-answer dataset is first converted into Alpaca and ShareGPT formats. A series of hyperparameters required for fine-tuning training are configured, such as learning rate, batch size, and number of training rounds. The AdamW optimizer is used, and the rank (r), scaling factor (alpha), and dropout probability of the LoRA matrix are set before fine-tuning the tea technology model. During training, in order to improve training efficiency and reduce memory usage, bf16 mixed precision is used for the accuracy type, and a cosine annealing strategy is used to gradually reduce the learning rate to avoid overfitting. For the loss function, we use the average cross-entropy loss to measure the difference between the predicted distribution and the true label, thereby achieving a more stable training process. The formula for calculating the average cross-entropy loss is shown in formula (14):

[0128] (14)

[0129] Where C represents the input instruction. To output the real token at position t in the sequence, This represents all tokens generated by the model before time t, where T is the total number of tokens in the output sequence. The above steps complete the fine-tuning of the Tea Technology large-scale model, generate a weight file, and merge it with the original Tea Technology large-scale model.

[0130] Step 15: Input the question-answer pair data subset into the tea technology big model after fine-tuning the training instructions to obtain suboptimal response data.

[0131] Step 16: Merge the question-answer pair data subset and the suboptimal response data to obtain the human preference judgment dataset.

[0132] Step 17: Based on the human preference judgment dataset, perform direct preference optimization training on the tea technology big model after fine-tuning the instructions to obtain the tea technology big model after direct preference optimization.

[0133] Specifically, after the question-answer pair screening is completed, the selected question-answer pair dataset is used as the preferred response data, and the preferred response data is input into the tea technology big model to generate the second-best response data. Then, the preferred response data and the second-best response data are merged to obtain the human preference judgment dataset for DPO training of the tea technology big model.

[0134] Direct Preference Optimization (DPO) is an alignment training method based on human preference datasets, aiming to guide the model to generate responses that better align with human values ​​and expression habits. This invention utilizes a large-scale tea technology model to generate a human preference judgment dataset and trains the model using a contrastive learning mechanism. Compared to traditional reinforcement learning, this avoids extensive manual annotation, significantly reducing annotation costs and improving the feasibility and efficiency of achieving accurate alignment in professional scenarios. The optimal response represents more professional, accurate, and valuable question-and-answer data within the tea technology field, while the suboptimal response is a less optimal answer generated based on the same question, such as increasing the generated temperature, limiting the length, or adding misleading information. These answers may lack professionalism or accuracy. After creating the human preference judgment dataset, the target model is trained using the human preference judgment dataset via DPO.

[0135] Based on this, the present invention combines contrastive learning and uses the DPO algorithm to train the model to distinguish between high-quality and low-quality answers, thereby internalizing human preference standards. This enables the model to not only answer questions correctly, but also to express these answers in the way that humans expect. The DPO loss function is shown in formula (15):

[0136] (15)

[0137] Furthermore, by minimizing this loss, the model learns to give higher probabilities of generating high-quality answers, reflecting an internalized alignment with human preferences, as shown in Equation (16):

[0138] (16)

[0139] Where W is the input, To select the best response, This is a suboptimal response. It is the conditional probability of the model generating the answer R; For the loss function of direct preference optimization, It is the trainable parameter vector of the model. The loss function is the average value over the entire preference dataset, ensuring that the model optimization covers all samples. The parameter that minimizes the expected loss. It's the sigmoid function, used to convert probability differences into preference probabilities. Then, z represents the mathematical expectation, and z is the input to the activation function. .

[0140] Through this dual-track training method of fine-tuning with instructions and optimizing with direct preferences, the final trained large-scale tea technology model not only understands professional knowledge but can also express this knowledge according to the standards of human experts, thus demonstrating higher value in practical applications.

[0141] Step 18: Construct a tea knowledge base based on tea science and technology knowledge text data and knowledge graph. The knowledge graph is constructed from tea tree genome knowledge text in the tea science and technology knowledge text data through entity recognition and relation extraction algorithms.

[0142] Specifically, in the method for constructing a large-scale tea technology model provided by this invention, based on entity recognition and relation extraction algorithms and genomic data such as promoters, introns, and regulatory information, a hierarchical and structured transcription factor knowledge graph covering the relationship between "transcription factors-regulatory genes-functional annotation-regulatory pathways-physiological phenotypes" is constructed. For example, a knowledge graph in the form of a triplet including "WRKY75, regulation, tea polyphenol synthesis pathway" is provided.

[0143] Furthermore, the tea technology big data model constructed in this invention can incorporate a knowledge graph as part of a knowledge base via HTTP calls. For example, when a user asks "What is the effect of WRKY75 on tea trees?", the model initiates an HTTP request to the deployed transcription factor knowledge graph to obtain the relevant triple "WRKY75, regulation, tea polyphenol synthesis". The result is further parsed into an embedding vector, which is then input into the model along with the user's question to achieve semantic generation after knowledge enhancement, such as outputting the text information "WRKY75 has the effect of regulating tea polyphenol synthesis".

[0144] Step 19: Encode the data in the tea knowledge base into text block vectors, and construct a vector database based on the text block vectors.

[0145] Step 20: Obtain tea knowledge question information, and calculate the cosine similarity between the tea knowledge question information and multiple text block vectors to obtain the cosine similarity corresponding to the multiple text block vectors.

[0146] Step 21: Input the tea knowledge question information and the text block vector with the highest cosine similarity into the tea technology big model optimized by direct preference to obtain the tea knowledge answer result.

[0147] Specifically, this invention provides a method for constructing a large-scale tea technology model. It can also build a knowledge base based on collected data, filtered question-and-answer pairs, and constructed knowledge graphs, achieving retrieval-enhanced generation and thus alleviating the problem of the "big model illusion." The "big model illusion" refers to the phenomenon where the generated results of a large model contain unfounded or erroneous content; the model fabricates information that does not exist in the context or knowledge base, affecting the accuracy and reliability of the output results. RAG technology can effectively alleviate this problem through retrieval and generation. The core idea is to introduce external knowledge during the generation process, enhancing the model's generation capabilities by retrieving relevant information. Specifically, the RAG mechanism first receives an input, such as a question, sentence, or paragraph, then retrieves relevant documents from a pre-built knowledge base based on the input, and inputs these documents along with the question into the model for answer generation. Furthermore, the knowledge base supports dynamic expansion and real-time updates, allowing it to be updated in real-time based on the latest tea knowledge, ensuring the timeliness of information and avoiding the huge overhead of retraining the model due to data changes.

[0148] For example, a knowledge base is constructed using collected data, filtered question-and-answer pairs, and a constructed knowledge graph. Long texts in the knowledge base are segmented into paragraphs or sentences, and the data within these text blocks is uniformly encoded into vectors using the bge-m3 embedding model to obtain text block vectors. Then, a vector database is constructed based on these text block vectors to store and retrieve vector data. Subsequently, user-input tea knowledge questions are obtained and converted into vectors using the bge-m3 embedding model. Then, a search is performed in the vector database based on cosine similarity to determine the text block vector closest to the vector converted from the tea knowledge question information, as shown in formula (17).

[0149] (17)

[0150] in, For the first in the vector database Vector encoding of text blocks, The user input question is vector-encoded. The k highest-scoring text blocks are selected using formula (18):

[0151] (18)

[0152] Finally, the retrieved text blocks are input into the tea technology big data model along with the question, and the tea technology big data model combines the context and the question to generate the final answer, thus achieving enhanced tea technology knowledge response.

[0153] Furthermore, to comprehensively verify the effectiveness of the proposed tea technology large-scale model construction method, this invention uses a combination of quantitative indicators and subjective evaluation. On one hand, to verify the impact of data quality on model performance, the model was fine-tuned using both the original question-and-answer dataset before filtering and the high-quality dataset after knowledge filtering. The performance of the two models was compared on the same test set. The model performance was mainly judged using metrics such as BLEU score, ROUGE-1 and ROUGE-L F1 scores, recall, and precision.

[0154] ROUGE-1 is used to measure the degree of overlap between the model-generated answer and the reference answer at the word level. It mainly reflects whether the model answer covers keywords. Its definition is shown in formula (19):

[0155] (19)

[0156] in, Indicates a single word, This indicates that the word belongs to the reference answer text. "To iterate through the reference answer text S, the total number of 1-grams (single words) that match the generated text," “S represents the total number of 1-grams of the reference answer text. ROUGE-L is a matching method based on the longest common subsequence, used to measure the sequence structure similarity between the generated answer and the reference answer, and its definition is shown in formula (20):

[0157] (20)

[0158] Wherein, n-gram is a semantic segment formed by a combination of n consecutive words in the text, and LCS is the longest common subsequence. The BLEU score is used to evaluate the degree of overlap between the model's answer and the reference answer in n-grams, which is usually 1-4, and focuses on reflecting the naturalness of grammatical structure and word order, as shown in formulas (21) and (22):

[0159] (twenty one)

[0160] (twenty two)

[0161] Where c is the length of the generated answer and r is the length of the reference answer. The exact matching rate of n-grams; is the weighting coefficient for each n-gram, and BP is the length penalty factor. On the other hand, in the subjective evaluation stage, a human preference review method is adopted, with reviewers scoring the model's responses from four aspects: professionalism, logic, expression preference, and factual consistency. The response quality of the model output before and after DPO training is compared, and the results show that the model performs better in all the above dimensions after DPO training.

[0162] In summary, the tea technology large-scale model construction method provided by this invention combines a multimodal fusion large language model construction method, enabling the model to possess rich knowledge understanding and reasoning capabilities. By constructing the model and feeding it with large-scale data in the tea technology field, this invention significantly enhances the model's knowledge mastery in this domain, particularly excelling in the recognition of genome-related knowledge and molecular 2D / 3D structures. Simultaneously, the introduction of instruction fine-tuning and direct preference optimization effectively improves the quality of the model's responses to user questions, enhances its ability to discriminate between accurate and semantically consistent responses, and improves the controllability and reliability of the generated results. To further effectively alleviate the model's "knowledge blind spots" and "illusion" problems and enhance its real-time domain knowledge update capabilities, a retrieval-enhanced generation mechanism is introduced, dynamically calling relevant knowledge to assist the large model in generating responses. Furthermore, to improve data quality, reduce low-quality and unnecessary data, and conserve training resources, a large model scoring system combined with a score transition matrix is ​​used to filter high-quality question-answer pairs, while also avoiding the negative impact of low-quality data on model training.

[0163] Example 2

[0164] Based on Example 1, this invention also provides a complete example of a tea technology large-scale model construction process:

[0165] like Figure 2 As shown, a large-scale model for answering tea-related questions is first built on the basis of the Transformer model. The model is then trained through pre-training, instruction fine-tuning, direct preference optimization, and retrieval enhancement to obtain a large-scale tea technology model. Finally, the performance of the large-scale tea technology model is evaluated.

[0166] The specific process is as follows: Figure 3 As shown, the first step was to comprehensively collect and integrate all data in the field of tea science and technology, ensuring that the data covered multiple key modalities and data types in tea research. This included collecting a large amount of tea gene-related references, genome and gene sequence data, 2D molecular formulas of compounds in tea, and creating 3D structural images using software. This data integration and processing provided a solid foundation for subsequent model training and optimization.

[0167] Subsequently, based on the Transformer model and utilizing a cross-modal attention mechanism, a large-scale tea technology model was constructed, consisting of encoders for images, gene sequences, text, and molecular structures, a cross-modal fusion module, a task scheduling and control module, and a unified language model output module. The cross-modal fusion module employs a cross-modal attention mechanism to achieve deep fusion of image, gene sequence, and text information. The task scheduling and control module dynamically allocates computational resources and inference paths based on the input data type and downstream tasks. The unified language model output module generates user-understandable natural language from the multimodal fused features. Residual connections and layer normalization mechanisms are introduced between the encoders and fusion modules to ensure effective information transfer and stable training. The multimodal model, trained using collected tea technology data, possesses the basic knowledge understanding and general representation capabilities required for tea technology tasks.

[0168] Subsequently, a pre-designed prompt word template guides the large-scale model to construct question-and-answer pairs from collected tea technology data. Semantic evaluation and quality checks are performed on the initially constructed question-and-answer dataset. Low-quality data may degrade model performance; therefore, a combination of manual review and automatic quality assessment is used to select high-quality question-and-answer pairs for fine-tuning training. The constructed tea knowledge question-and-answer pair dataset is used as the optimal response dataset, and the large-scale model is guided to generate a suboptimal response dataset based on this dataset. The optimal and suboptimal responses are then merged to form a human preference judgment training dataset. Fine-tuning of the pre-trained tea technology large-scale model is performed based on the tea knowledge question-and-answer pair dataset, and Direct Preference Optimization (DPO) training is performed on the fine-tuned model based on the human preference dataset. Considering that pre-training primarily provides the model with a large amount of basic knowledge in the tea technology field but is difficult to accurately adapt to specific tasks, fine-tuning focuses on task alignment and behavioral guidance. Training with high-quality, diverse question-and-answer data enables the model to better follow task instructions and generate expected outputs. To mitigate the possibility that fine-tuning training might cause the model to forget previously learned knowledge, LoRA fine-tuning is used.

[0169] Then, based on entity recognition and relation extraction algorithms, a knowledge graph is constructed using collected genomic and transcription factor data. For example, using collected genomic and transcription factor data, a hierarchical and structured transcription factor knowledge graph covering the relationships of "transcription factor-regulatory gene-functional annotation-regulatory pathway-physiological phenotype" is constructed, such as the triple form of "WRKY75, regulation, tea polyphenol synthesis pathway". The knowledge graph is incorporated into the knowledge base via HTTP calls. A knowledge retrieval-enhanced generation mechanism is used to assist the large model in mitigating the "illusion" problem. Subsequently, a knowledge base is constructed based on a large amount of collected tea science and technology text data and the constructed knowledge graph, and the above data is encoded into a unified vector form based on the bge-m3 embedding model to construct a vector database to achieve retrieval-enhanced generation, improve model performance, and alleviate the illusion problem. The comprehensive performance of the large tea science and technology model in tasks such as text-image question answering and gene sequence recognition, as well as in terms of human preferences, is systematically evaluated to verify the effectiveness of optimization and training methods, such as comparing ROUGE's F1 score, recall, precision, and BLEU score. For example, the tea technology model was fine-tuned using datasets before and after screening, and various metrics were compared to demonstrate the effectiveness of the method. The model trained by DPO was scored by human evaluation to determine whether it conformed to human preferences and whether it was superior to the model fine-tuned by instructions.

[0170] Example 3

[0171] This invention also provides a large-scale tea technology model construction system, such as... Figure 4 As shown, it includes:

[0172] The data import module 31 is used to acquire tea knowledge information, which includes tea images, tea tree gene sequences, tea science and technology knowledge text data, and molecular structure diagrams of tea compounds.

[0173] The pre-training module 32 is used to train the Transformer model based on tea knowledge information to obtain an initial model. The initial model includes encoders corresponding to images, gene sequences, text, and molecular structures, a cross-modal fusion module, and an output module. The encoders are used to extract image features, text features, gene features, and molecular structures from tea images, tea tree gene sequences, tea science and technology knowledge text data, and tea compound molecular structure diagrams, respectively. The cross-modal fusion module is used to fuse image features, text features, gene features, and molecular structure features based on a cross-modal attention mechanism to obtain fused features. The output module is used to output human-readable text based on the fused features.

[0174] The question-answer pair dataset construction module 33 is used to extract question-answer pairs from tea knowledge information based on prompt word templates and an initial model, and to construct a tea knowledge question-answer pair dataset based on the question-answer pairs.

[0175] The instruction fine-tuning module 34 is used to train an initial model on the dataset based on tea knowledge questions and answers, and obtain a large tea technology model after instruction fine-tuning training.

[0176] Specifically, a large-scale tea technology model construction system includes a pre-training module, an instruction fine-tuning training module, a direct preference optimization training module, a retrieval enhancement generation module, and a model performance evaluation module.

[0177] The pre-training module trains the model using a tea technology knowledge corpus, endowing the large model with fundamental knowledge and multimodal understanding capabilities in the tea technology field. The instruction fine-tuning training module optimizes the model's responses to user tasks using high-quality question-answering data. The direct preference optimization training module trains the fine-tuned model using a constructed preference judgment dataset, adjusting the model's behavior to conform to human preferences and expectations. The retrieval enhancement generation module constructs a knowledge base and vector database, using RAG technology to ensure the model can import new knowledge in real time, mitigating the illusion problem and improving model accuracy. The model performance evaluation module systematically evaluates the comprehensive performance of the large-scale tea technology model across multiple tasks, including text-based question answering and gene sequence recognition, verifying the effectiveness of optimization and training strategies. Evaluation methods include quantitative metrics and subjective human evaluation.

[0178] The present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory. The processor executes the computer program to implement the steps in an embodiment of a tea technology large-scale model construction method. Specific implementation methods can be found in the method embodiments, and will not be repeated here.

[0179] Furthermore, the present invention also provides a computer-readable storage medium containing instructions on which a computer program is stored. For example, a memory containing instructions that can be executed by a processor of a computer device to perform the above-described method. For example, the computer-readable storage medium may be a hard disk, ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc. When the computer program is executed by the processor, it can implement the steps in an embodiment of a tea technology large-scale model construction method. Specific implementation methods can be found in the method embodiments, which will not be repeated here.

[0180] Those skilled in the art will understand that embodiments of the present invention can provide methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0181] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0182] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0183] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0184] It should be noted that the specific embodiments described above enable those skilled in the art to more fully understand the present invention, but do not limit the present invention in any way. Therefore, although the present invention has been described in detail in this specification and embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the present invention; and all technical solutions and improvements that do not depart from the spirit and scope of the present invention are covered within the protection scope of the present invention patent. No reference numerals in the claims should be construed as limiting the scope of the claims. Any simple variations or equivalent substitutions of technical solutions that can be readily obtained by those skilled in the art within the scope of the technology disclosed in the present invention are within the protection scope of the present invention.

Claims

1. A method for constructing a large-scale tea technology model, characterized in that, include: Acquire tea knowledge information, wherein the tea knowledge information includes tea images, tea tree gene sequences, tea science and technology knowledge text data, and tea compound molecular structure diagrams; A Transformer model is trained based on the tea knowledge information to obtain an initial model. This initial model includes encoders for each of the image, gene sequence, text, and molecular structure data, a cross-modal fusion module, an output module, and a quality scoring module. The encoders extract image features, text features, gene features, and molecular structures from the tea image, tea tree gene sequence, tea science and technology knowledge text data, and tea compound molecular structure diagram, respectively. The cross-modal fusion module fuses the image features, text features, gene features, and molecular structure features based on a cross-modal attention mechanism to obtain fused features. The output module outputs human-readable text based on the fused features. Based on prompt word templates, question-answer pairs are extracted from tea knowledge information through the initial model, and a tea knowledge question-answer pair dataset is constructed based on the question-answer pairs. The quality scoring module performs quality scoring on the tea knowledge question-and-answer dataset to obtain quality score results for each dataset. These results include quality score results for single samples, sample pairs, and sample triplets. A score distribution is calculated based on the quality score results of the single samples to obtain a first-order consensus vector. A score shift probability is calculated based on the quality score results of the sample pairs to obtain a second-order consensus vector. A score shift probability is calculated based on the quality score results of the sample triplets to obtain a third-order consensus vector. The score transition matrix and the probability of the true score are determined based on the first-order, second-order, and third-order consensus vectors. The probability distribution of the score transition matrix and the true score is used to correct the quality score result, resulting in the corrected quality score. The score transition matrix is ​​determined based on the consistency distribution of the quality score result. The similarity between each sample in the tea knowledge question-and-answer pair dataset and its semantically neighboring samples is calculated using the mean cosine similarity, resulting in a semantic diversity score for each dataset. The tea knowledge question-and-answer pair dataset is then filtered based on the corrected quality score and the semantic diversity score to obtain a subset of question-and-answer pair data. The tea technology big model is trained using the subset of question-and-answer pair data to obtain the fine-tuned tea technology big model.

2. The method for constructing a large-scale tea technology model according to claim 1, characterized in that, Based on the tea knowledge information, the model is trained to obtain the following initial model: The tea image is divided into multiple image blocks by an image encoder. Position encoding is added to the multiple image blocks and the encoding process is performed to obtain image features. The tea technology knowledge text data is divided into multiple semantic units by a tokenizer, and positional encoding is added to the multiple semantic units and then encoded to obtain text features. The tea tree gene sequence is segmented into multiple consecutive k-mer fragments using a gene sequence encoder based on a sliding window method. The k-mer fragments are then encoded using the DNABERT model to obtain gene features. The molecular structure diagram of the tea compound is converted into a SMILES molecular expression by a molecular structure encoder, and a molecular diagram is generated based on the SMILES molecular expression. The molecular diagram is then encoded to obtain molecular features. The cross-modal fusion module calculates the cross-modal attention between the image features, text features, gene features, and molecular features, and then weights and fuses the image features, text features, gene features, and molecular features based on the cross-modal attention to obtain the tea fusion feature; The output module determines the token sequence based on the tea fusion characteristics and converts the token sequence into human-readable text.

3. The method for constructing a large-scale tea technology model according to claim 2, characterized in that, The linear layers of the encoders corresponding to the images, gene sequences, texts, and molecular structures are equipped with LoRA low-rank matrices.

4. The method for constructing a large-scale tea technology model according to claim 1, characterized in that, After training the tea technology big data model using the question-and-answer pair data subset to obtain the instruction-fine-tuned tea technology big data model, the process further includes: The suboptimal response data is obtained by inputting the subset of question-and-answer pair data into the tea technology big model after fine-tuning the training instructions. The question-and-answer pair data subset and the suboptimal response data are merged to obtain the human preference judgment dataset; Based on the human preference judgment dataset, the tea technology big model after fine-tuning the instructions is trained by direct preference optimization to obtain the tea technology big model after direct preference optimization.

5. The method for constructing a large-scale tea technology model according to claim 4, characterized in that, The tea science and technology knowledge text data includes tea tree genome knowledge text. Based on the human preference judgment dataset, the tea science and technology large-scale model, after being fine-tuned and trained according to the instructions, undergoes direct preference optimization training. After obtaining the directly preference-optimized tea science and technology large-scale model, it also includes: A tea knowledge base is constructed based on the tea science and technology knowledge text data and knowledge graph, wherein the knowledge graph is constructed from the tea tree genome knowledge text through entity recognition and relation extraction algorithms; The data in the tea knowledge base is encoded into text block vectors, and a vector database is constructed based on the text block vectors; Obtain tea knowledge question information, and calculate the cosine similarity between the tea knowledge question information and multiple text block vectors to obtain the cosine similarity corresponding to the multiple text block vectors; The tea knowledge question information and the text block vector with the highest cosine similarity are input into the tea technology big model optimized by direct preference to obtain the tea knowledge answer result.

6. A tea technology large-scale model construction system, characterized in that, include: The data import module is used to acquire tea knowledge information, which includes tea images, tea tree gene sequences, tea science and technology knowledge text data, and tea compound molecular structure diagrams. A pre-training module is used to train a Transformer model based on the tea knowledge information to obtain an initial model. This initial model includes encoders corresponding to images, gene sequences, text, and molecular structures, a cross-modal fusion module, an output module, and a quality scoring module. The encoders extract image features, text features, gene features, and molecular structures from the tea images, tea tree gene sequences, tea science and technology knowledge text data, and tea compound molecular structure diagrams, respectively. The cross-modal fusion module fuses the image features, text features, gene features, and molecular structure features based on a cross-modal attention mechanism to obtain fused features. The output module outputs human-readable text based on the fused features. The question-answer pair dataset construction module is used to extract question-answer pairs from tea knowledge information based on prompt word templates and an initial model, and to construct a tea knowledge question-answer pair dataset based on the question-answer pairs. The instruction fine-tuning module is used to perform quality scoring on the tea knowledge Q&A dataset through the quality scoring module, obtaining quality scoring results for each dataset. These quality scoring results include quality scoring results for single samples, sample pairs, and sample triplets. A score distribution is calculated based on the quality scoring results of the single samples to obtain a first-order consensus vector. A score shift probability is calculated based on the quality scoring results of the sample pairs to obtain a second-order consensus vector. A score shift probability is calculated based on the quality scoring results of the sample triplets to obtain a third-order consensus vector. The score transition matrix and the true score are determined based on the first-order, second-order, and third-order consensus vectors. The probability distribution of scores; the quality score is corrected based on the score transition matrix and the probability distribution of the true scores to obtain the quality-corrected score, wherein the score transition matrix is ​​determined based on the consistency distribution of the quality score; the similarity between each sample in the tea knowledge question-and-answer pair dataset and its semantically neighboring samples is calculated using the mean cosine similarity to obtain the semantic diversity score corresponding to each dataset; the tea knowledge question-and-answer pair dataset is filtered based on the quality-corrected score and the semantic diversity score to obtain a subset of question-and-answer pair data; the tea technology big model is trained using the subset of question-and-answer pair data to obtain the tea technology big model after fine-tuning the instructions.

7. A computer device, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the tea technology large model construction method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is loaded by the processor, it is able to execute the steps of the tea technology large model construction method according to any one of claims 1 to 5.

Images