Agricultural condition consultation service method and system based on size model cooperation

Abstract

The application provides a kind of agricultural situation consultation service method and system based on size model cooperation, wherein the method comprises: obtaining multi-modal agricultural situation consultation question;Carry out key word extraction and expansion on agricultural situation consultation question, obtain expanded agricultural situation consultation question;The expanded agricultural situation consultation question is input into the pre-constructed intention recognition model, and the user intention corresponding to the agricultural situation consultation question is obtained;Based on user intention, judge whether the agricultural situation consultation question needs to be answered by a professional small model, if yes, input the agricultural situation consultation question into the pre-constructed target professional small model, obtain the original consultation result, input the agricultural situation consultation question and the original consultation result into the large model, obtain the first consultation result of the agricultural situation consultation question, otherwise, directly input the agricultural situation consultation question into the large model, obtain the second consultation result of the agricultural situation consultation question.The application improves the accuracy, professionalism and efficiency of consultation answer, has strong professional scene adaptability, and improves user experience.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of agricultural information technology, and in particular to a crop condition consultation service method and system based on large and small model cooperation. BACKGROUND

[0002] With the development of agricultural modernization, the demand for crop condition consultation is increasing. In recent years, with the rapid development of artificial intelligence, various large models are increasingly widely used in intelligent question answering, content creation and other aspects, and agricultural intelligent consultation systems have also emerged as the times require. However, the existing agricultural intelligent consultation system simply relies on pre-trained large models, and the solution to the fine problems of specific scenarios in the agricultural field is not accurate and deep enough. The use of large model fine-tuning has high cost and poor timeliness, which affects the user experience. SUMMARY

[0003] The present application provides a crop condition consultation service method and system based on large and small model cooperation, to solve the defects of the existing agricultural intelligent consultation system that simply relies on large models, poor accuracy, poor professional scene adaptability, and slow timeliness.

[0004] In a first aspect, the present application provides a crop condition consultation service method based on large and small model cooperation, comprising:

[0005] Obtaining a multi-modal crop condition consultation question input by a user;

[0006] Performing key word extraction and expansion on the crop condition consultation question to obtain an expanded crop condition consultation question;

[0007] Inputting the expanded crop condition consultation question into a pre-constructed intent recognition model to obtain a user intent corresponding to the crop condition consultation question output by the intent recognition model;

[0008] Based on the user intent, determining whether the crop condition consultation question needs to be answered by a professional small model, if yes, inputting the crop condition consultation question into a pre-constructed target professional small model to obtain an original consultation result of the crop condition consultation question output by the target professional small model, inputting the crop condition consultation question and the original consultation result into a large model to obtain a first consultation result of the crop condition consultation question output by the large model, otherwise, directly inputting the crop condition consultation question into the large model to obtain a second consultation result of the crop condition consultation question output by the large model;

[0009] The intent recognition model is trained based on the expanded sample crop condition consultation question and the user intent label corresponding to the sample crop condition consultation question.

[0010] The target professional small model is trained based on a multi-modal sample agricultural condition consultation question and an original consultation result label of the sample agricultural condition consultation question.

[0011] In some embodiments, the focusing word extraction and expansion of the agricultural condition consultation question obtains an expanded agricultural condition consultation question, including:

[0012] The agricultural condition consultation question is segmented to obtain a word sequence of the agricultural condition consultation question, and the focusing word extraction is performed on the word sequence of the agricultural condition consultation question to obtain a focusing word set of the agricultural condition consultation question.

[0013] The vectorization processing is performed on the word sequence of the agricultural condition consultation question to obtain a word vector sequence of the agricultural condition consultation question, the word vector sequence of the agricultural condition consultation question is input into an improved FastText model to obtain an association score of the agricultural condition consultation question and each focusing word in a pre-constructed focusing word library output by the improved FastText model.

[0014] Based on the association score of the agricultural condition consultation question and each focusing word, a preset association score threshold, and the focusing word set of the agricultural condition consultation question, a plurality of expanded focusing words of the agricultural condition consultation question are determined.

[0015] Based on the plurality of expanded focusing words, the agricultural condition consultation question is expanded to obtain an expanded agricultural condition consultation question.

[0016] The improved FastText model includes an input layer, a hidden layer, and an output layer. The input layer introduces a position-sensitive attention mechanism, and the hidden layer includes a capsule network. The improved FastText model is trained based on a sample word vector sequence of a sample agricultural condition consultation question and an association score label of a sample agricultural condition consultation question and each focusing word in the focusing word library.

[0017] In some embodiments, the focusing word extraction of the word sequence of the agricultural condition consultation question obtains the focusing word set of the agricultural condition consultation question, including:

[0018] The co-occurrence frequency of adjacent words in the word sequence of the agricultural condition consultation question is counted, and based on a preset co-occurrence frequency threshold, a plurality of word blocks are determined from the word sequence of the agricultural condition consultation question.

[0019] The number of times that each word in the word sequence of the agricultural condition consultation question appears in the agricultural condition consultation question and the number of times that the word appears in the corresponding word block are calculated to obtain the term frequency (TF) of the word.

[0020] Calculate an inverse word frequency (IWF) of each word in combination with a semantic similarity between different words in the word sequence of the agricultural condition consultation question;

[0021] Multiply the TF of each word and the IWF of each word to obtain a TF-IWF value of each word;

[0022] Sort the word TF-IWF value of each word to obtain a sorting result, and determine a key word set of the agricultural condition consultation question according to the sorting result.

[0023] In some embodiments, the intent recognition model includes an improved bert-base-chinese model and an improved deep pyramid convolutional neural network; the improved bert-base-chinese model includes an input layer, a gated multi-head self-attention layer, a feedforward network layer, and a normalization layer; and the improved deep pyramid convolutional neural network includes a feature mapping layer, a region embedding layer, a convolution block layer, a channel attention layer, a pooling fusion layer, and a full connection layer.

[0024] In some embodiments, the inputting the agricultural condition consultation question and the original consultation result into a large model to obtain the first consultation result of the agricultural condition consultation question output by the large model includes:

[0025] Inputting the agricultural condition consultation question and the original consultation result into a large model, searching for related information in a pre-constructed external knowledge base based on the agricultural condition consultation question by the large model, supplementing the original consultation result according to the found related information to obtain the first consultation result of the agricultural condition consultation question;

[0026] The inputting the agricultural condition consultation question into the large model to obtain the second consultation result of the agricultural condition consultation question output by the large model includes:

[0027] Inputting the agricultural condition consultation question into a large model, searching for related information in the external knowledge base based on the agricultural condition consultation question by the large model, and generating the second consultation result of the agricultural condition consultation question based on the found related information.

[0028] In some embodiments, the construction process of the external knowledge base includes:

[0029] Obtaining a plurality of documents related to the agricultural field;

[0030] Segmenting each document to obtain a plurality of text blocks and establishing a mapping relationship between the text blocks and the documents;

[0031] identifying an entity of each text block, associating a picture and / or a table in the corresponding document with the entity of each text block, and establishing a mapping relationship between the picture and / or the table and the entity;

[0032] determining a syntactic dependency relationship between different entities, constructing an initial knowledge graph with the entities as nodes and the syntactic dependency relationship between different entities as edges, extracting features of each node and features of each edge in the initial knowledge graph based on a graph neural network, and determining a relationship type between different entities based on the features of each node and the features of each edge;

[0033] extracting embedding information of each sentence of each text block, fusing the embedding information of each sentence of each text block with the initial knowledge graph, and obtaining a knowledge graph;

[0034] performing community division and hierarchy construction on the knowledge graph, constructing a multi-dimensional index based on entities, relationships, communities and hierarchies, and performing semantic enhancement on the multi-dimensional index.

[0035] In some embodiments, the method further comprises:

[0036] obtaining feedback information of the user on the first consultation result or the second consultation result;

[0037] determining whether the user is satisfied with the first consultation result or the second consultation result according to the feedback information;

[0038] if the user is not satisfied with the first consultation result or the second consultation result, determining a target online expert according to the key word set of the agricultural condition consultation question, sending the agricultural condition consultation question to the corresponding target expert end, and receiving a third consultation result of the agricultural condition consultation question sent by the target expert end.

[0039] In some embodiments, the target professional small model is one of a target detection small model, a classification detection small model and a yield estimation small model; the target detection small model comprises an improved YOLOv10 network, the improved YOLOv10 network comprises a backbone network, a neck network, a one-to-many prediction head and a one-to-one prediction head, and the backbone network comprises a pyramid compression attention (PSA) module; the classification detection small model comprises an improved ConvNeXt V2 network, the improved ConvNeXt V2 network comprises a full convolution mask autoencoder framework, a global response normalization layer and a convolution block attention module (CBAM).

[0040] In some embodiments, the training process of the intent recognition model comprises:

[0041] obtaining a multi-modal sample agricultural condition consultation question;

[0042] perform key word extraction and expansion on the sample agricultural condition consultation question to obtain an expanded sample agricultural condition consultation question;

[0043] determine a user intent label corresponding to the sample agricultural condition consultation question;

[0044] use the expanded sample agricultural condition consultation question as training data and use the user intent label corresponding to the sample agricultural condition consultation question as a training label to train an initial intent recognition model, and after the training is completed, obtain the intent recognition model.

[0045] In a second aspect, the present application further provides an agricultural condition consultation service system based on large and small model cooperation, comprising:

[0046] a first acquisition unit configured to acquire a multi-modal agricultural condition consultation question input by a user;

[0047] an expansion unit configured to perform key word extraction and expansion on the agricultural condition consultation question to obtain an expanded agricultural condition consultation question;

[0048] an intent recognition unit configured to input the expanded agricultural condition consultation question into a pre-constructed intent recognition model to obtain a user intent corresponding to the agricultural condition consultation question output by the intent recognition model;

[0049] a question answering unit configured to determine, based on the user intent, whether the agricultural condition consultation question needs to be answered by a professional small model, and if so, input the agricultural condition consultation question into a pre-constructed target professional small model to obtain an original consultation result of the agricultural condition consultation question output by the target professional small model, input the agricultural condition consultation question and the original consultation result into a large model to obtain a first consultation result of the agricultural condition consultation question output by the large model, and if not, directly input the agricultural condition consultation question into the large model to obtain a second consultation result of the agricultural condition consultation question output by the large model;

[0050] The intent recognition model is obtained by training based on an expanded sample agricultural condition consultation question and a user intent label corresponding to the sample agricultural condition consultation question.

[0051] The target professional small model is obtained by training based on a multi-modal sample agricultural condition consultation question and an original consultation result label of the sample agricultural condition consultation question.

[0052] The application provides a farmland condition consultation service method and system based on size model cooperation, which obtains a multi-modal farmland condition consultation question input by a user, determines a corresponding farmland condition consultation question, extracts key words from the farmland condition consultation question, and obtains an expanded farmland condition consultation question, identifies a user intention based on a pre-constructed intention recognition model according to the expanded farmland condition consultation question, and answers the farmland condition consultation question based on the user intention by combining a target professional small model and a large model or directly using the large model to answer the farmland condition consultation question, thereby improving the accuracy, professionalism and efficiency of question answering, improving the adaptability of professional scenes, meeting diversified consultation demands of the user, and improving the user experience. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the application or prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0054] Figure 1 is a flowchart of the farmland condition consultation service method based on size model cooperation provided by the embodiment of the application.

[0055] Figure 2 is a flowchart of the key word extraction and expansion of the farmland condition consultation question provided by the embodiment of the application.

[0056] Figure 3 is a structural diagram of the improved FastText model provided by the embodiment of the application.

[0057] Figure 4 is a structural diagram of the intention recognition model provided by the embodiment of the application.

[0058] Figure 5 is a flowchart of the training process of the intention recognition model provided by the embodiment of the application.

[0059] Figure 6 is a structural diagram of the farmland condition consultation service system based on size model cooperation provided by the embodiment of the application. DETAILED DESCRIPTION

[0060] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in conjunction with the drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the protection scope of the present application.

[0061] Figure 1 A flowchart of a method for agricultural condition consulting service based on size model cooperation provided by an embodiment of the present application is shown in FIG. 1. As shown in FIG. 1, in some embodiments, a method for agricultural condition consulting service based on size model cooperation is provided, which includes the following steps: step 110, step 120, step 130 and step 140. The method flow steps are only used as one possible implementation of the present application. Figure 1

[0062] Step 110, obtaining a multi-modal agricultural condition consulting question input by a user;

[0063] Step 120, extracting key words from the agricultural condition consulting question and expanding the agricultural condition consulting question to obtain an expanded agricultural condition consulting question;

[0064] Step 130, inputting the expanded agricultural condition consulting question into a pre-constructed intent recognition model to obtain a user intent corresponding to the agricultural condition consulting question output by the intent recognition model;

[0065] Step 140, judging whether the agricultural condition consulting question needs to be answered by a professional small model based on the user intent. If yes, inputting the agricultural condition consulting question into a pre-constructed target professional small model to obtain an original consulting result of the agricultural condition consulting question output by the target professional small model, inputting the agricultural condition consulting question and the original consulting result into a large model to obtain a first consulting result of the agricultural condition consulting question output by the large model, otherwise, directly inputting the agricultural condition consulting question into the large model to obtain a second consulting result of the agricultural condition consulting question output by the large model;

[0066] The intent recognition model is obtained by training based on an expanded sample agricultural condition consulting question and a user intent label corresponding to the sample agricultural condition consulting question; and the target professional small model is obtained by training based on a multi-modal sample agricultural condition consulting question and an original consulting result label of the sample agricultural condition consulting question.

[0067] The multi-modal agricultural condition consulting question includes text, picture, voice and the like.

[0068] Optionally, the multi-modal agricultural condition consulting question is preprocessed, such as data cleaning and data conversion, for example, converting voice into text.

[0069] ​Optionally, based on the intention recognition model, feature extraction is performed on the expanded agricultural condition consultation question to obtain a feature vector of the expanded agricultural condition consultation question, the user intention is classified according to the feature vector of the expanded agricultural condition consultation question to obtain a category of the user intention.

[0070] Optionally, the user intention is classified into a specific professional question and answer and others, if the user intention is the specific professional question and answer, the professional small model is required to answer, and if the user intention is others, the professional small model is not required to answer.

[0071] The specific professional question and answer includes but is not limited to animal quantity statistics, crop instance segmentation, planting state recognition, disease and pest diagnosis, growth stage discrimination, crop yield estimation, market price inquiry, breeding knowledge question and answer, and planting knowledge question and answer.

[0072] Optionally, before the agricultural condition consultation question is input into the target professional small model constructed in advance, the user intention is matched with a plurality of professional small models constructed in advance, and the target professional small model is determined from the plurality of professional small models according to a matching result.

[0073] In the embodiment of the present application, the obtained agricultural condition consultation question is subjected to key word extraction and expansion to obtain an expanded agricultural condition consultation question, the user intention is recognized based on a pre-constructed intention recognition model according to the expanded agricultural condition consultation question, and the agricultural condition consultation question is answered by combining a target professional small model and a large model or directly using the large model to answer the agricultural condition consultation question, thereby improving the accuracy, professionalism and efficiency of question answering, improving the adaptability of professional scenarios, meeting the diversified consultation needs of users, and improving the user experience.

[0074] Figure 2 A flowchart for key word extraction and expansion of an agricultural condition consultation question is provided in the embodiment of the present application. Figure 2 As shown in FIG. 1, in some embodiments, step 120 performs key word extraction and expansion on the agricultural condition consultation question to obtain an expanded agricultural condition consultation question, including:

[0075] Step 121, performing word segmentation on the agricultural condition consultation question to obtain a word sequence of the agricultural condition consultation question, performing key word extraction on the word sequence of the agricultural condition consultation question to obtain a key word set of the agricultural condition consultation question;

[0076] Step 122, performing vectorization processing on the word sequence of the agricultural condition consultation question to obtain a word vector sequence of the agricultural condition consultation question, inputting the word vector sequence of the agricultural condition consultation question into an improved FastText model to obtain an association score of the agricultural condition consultation question and each key word in a pre-constructed key word library output by the improved FastText model;

[0077] Step 123, based on the association score of the crop condition consultation question and each key word, the preset association score threshold and the key word set of the crop condition consultation question, determining a plurality of expanded key words of the crop condition consultation question;

[0078] Step 124, based on the plurality of expanded key words, expanding the crop condition consultation question to obtain an expanded crop condition consultation question;

[0079] The improved FastText model is trained based on the sample word vector sequence of the sample crop condition consultation question and the association score label of each key word in the key word library.

[0080] Figure 3 The structure diagram of the improved FastText model provided by the embodiment of the application is shown in the figure. Figure 3 As shown in the figure, in some embodiments, the improved FastText model includes an input layer, a hidden layer and an output layer; the input layer introduces a position-sensitive attention mechanism, the hidden layer includes a capsule network, and the output layer introduces a key word library; wherein the word vector sequence of the crop condition consultation question .

[0081] Optionally, the position encoding of each word vector of the crop condition consultation question is calculated through the input layer, each word vector is added with its position encoding, the weighted word vector sequence is calculated by using a multi-head attention mechanism, and a feature vector of the crop condition consultation question is obtained; the input layer introduces a position-sensitive attention mechanism, which can better capture the position information of each word in the text, and is helpful for understanding the semantic structure of the text.

[0082] It should be noted that, in the hidden layer, a structure based on a capsule network is adopted instead of a traditional fully connected layer. The capsule network is composed of a plurality of capsules, and each capsule is a group of neurons for representing different attributes of an entity; for the feature vector of the input crop condition consultation question, a text feature vector output by the capsule is obtained after a series of transformations. Compared with the basic fully connected layer, the hidden layer can better capture the hierarchical structure and semantic relationship between words in the text.

[0083] Optionally, based on the output layer, a weight vector of each key word is determined, and the association score of the sample crop condition consultation question and each key word is calculated according to the weight vector of each key word and the text feature vector.

[0084] Optionally, a key word with an association score greater than the preset association score threshold and not in the key word set of the crop condition consultation question is selected as an expanded key word; the expanded key word is added to the key word set to complete the expansion of the key word.

[0085] Optionally, the calculation formula of the association score of the agricultural condition consultation question and each key word in the pre-constructed key word library is:

[0086] ;

[0087] wherein, is the text feature vector of the capsule network output of the hidden layer, is the cosine similarity between the feature vector and the key word weight vector; is an indicator function, which is 1 if the key word k is associated with the category i , and 0 otherwise; is the weight vector of the key word.

[0088] Optionally, the training process of the improved FastText model comprises:

[0089] obtaining a sample agricultural condition consultation question, determining the association score label of the sample agricultural condition consultation question and each key word in the key word library, performing word segmentation on the sample agricultural condition consultation question to obtain a sample word sequence of the sample agricultural condition consultation question, performing vectorization processing on the sample word sequence of the sample agricultural condition consultation question to obtain a sample word vector sequence of the sample agricultural condition consultation question, training an initial improved FastText model using the sample word vector sequence of the sample agricultural condition consultation question as training data and the association score label of the sample agricultural condition consultation question and each key word in the key word library as training label, and obtaining the improved FastText model after the training is completed.

[0090] In some embodiments, the key word extraction on the word sequence of the agricultural condition consultation question in step 121 to obtain the key word set of the agricultural condition consultation question comprises:

[0091] Step 1211, counting the co-occurrence frequency of adjacent words in the word sequence of the agricultural condition consultation question, and determining a plurality of word blocks from the word sequence of the agricultural condition consultation question based on a preset co-occurrence frequency threshold;

[0092] Step 1212, calculating the number of times each word in the word sequence of the agricultural condition consultation question appears in the agricultural condition consultation question and the number of times each word appears in the corresponding word block to obtain the TF of each word;

[0093] Step 1213, calculating the IWF of each word in combination with the semantic similarity between different words in the word sequence of the agricultural condition consultation question;

[0094] Step 1214, multiplying the TF of each word and the IWF of each word to obtain the TF-IWF value of each word;

[0095] Step 1215, sorting the TF-IWF value of each word to obtain a sorting result, and determining the key word set of the agricultural condition consultation question according to the sorting result.

[0096] Optionally, set the word sequence in the agricultural condition consultation question , calculate the number of times of co-occurrence of adjacent words and in the agricultural condition consultation question , set a threshold , if , then and are regarded as a word block .

[0097] Therefore, when calculating the TF, for each word t, not only the number of times of occurrence of the word t in the text of the agricultural condition consultation question is calculated, but also the number of times of occurrence of the word block containing the word t is calculated. The improved calculation formula of the word frequency is as follows:

[0098] ;

[0099] In the formula, tf(t) represents the word frequency of the word t, T is the set of all words in the text, B is the set of all word blocks in the text, is the number of times of occurrence of the word block t , is the total number of all word blocks in B, is a weight parameter, and the value is between 0 and 1. It should be noted that the basic IWF mainly calculates the frequency of occurrence of a word in a document set, and ignores semantic information. The embodiment of the present application introduces semantic information by using the pre-trained Chinese word vector model, that is, the Word Embedding of the Enhanced Representation through Knowledge Integration (ERNIE) to enhance semantic representation model. For a word t, the semantic similarity of the word t with all other words in the document set is calculated. Assuming that there are N words in the document set, the semantic similarity of the word t with the word is

[0100] , and the calculation formula of the semantic similarity of the word t with the word is as follows:

[0101] ;

[0102] In the formula, is the word vector of the word t and the word vector is the word vector of the word .

[0103] Optionally, the improved calculation formula of the inverse word frequency is as follows:​​​

[0104] ;

[0105] In the formula, Indicator t Inverse word frequency, It is a word t Document word frequency in a document collection, i.e., the frequency of words contained in the document. t The number of documents, It is a hyperparameter used to balance the impact of semantic similarity and document frequency on IWF calculation.

[0106] Multiply the improved word frequency and the reverse word frequency to obtain the improved result. = ;According to all words in the text The values ​​are sorted in descending order, and the top k words are selected as the key words of the text.

[0107] Optionally, for each word, the semantic similarity between the word and all other words in the document set is calculated, and the document frequency of each word in the document set, i.e. the number of documents containing the word, is calculated. The total number of words contained in the document set is counted, and based on these data, the inverse frequency of each word is calculated.

[0108] Figure 4 This is a schematic diagram of the structure of an intent recognition model provided in an embodiment of the present invention. Figure 4 As shown, in some embodiments, the intent recognition model includes an improved BERT-base-chinese model and an improved deep pyramid convolutional neural network; the improved BERT-base-chinese model includes an input layer, a gated multi-head self-attention layer, a feedforward network layer, and a normalization layer; the improved deep pyramid convolutional neural network includes a feature mapping layer, a region embedding layer, a convolutional block layer, a channel attention layer, a pooling fusion layer, and a fully connected layer. In the figure, Conv represents a convolutional layer, Pooling represents a pooling layer, Regionembedding represents a region embedding layer, and Softmax is a mathematical function for the classification problem.

[0109] It should be noted that BERT is a pre-trained model based on the Transformer architecture, which uses bidirectional encoding to capture contextual information of the text. The improved BERT-base-chinese model increases the network depth by adding extra feedforward layers and introduces a variant of the multi-head self-attention mechanism—the gated multi-head self-attention layer—to enhance attention to key information.

[0110] Optionally, the input to the input layer is an expanded agricultural information consultation question. ,in nfor a length of the text sequence, each token corresponding to a word embedding , a position embedding and a segment embedding ; wherein the word embedding converts discrete words into continuous vector representations, the position embedding allows the model to perceive the order of words, and the segment embedding is used to distinguish different sentences; the generated word embedding, position embedding and paragraph embedding vectors are superimposed and output, and the sum of the three provides rich information for subsequent processing.

[0111] Optionally, the output of the input layer is , , .

[0112] Optionally, based on the gated multi-head self-attention layer, first calculate the query Q, key K and value V, then calculate the attention score, and select important information through the gating unit. The calculation formula of the query Q, key K and value V is as follows:

[0113] ;

[0114] ;

[0115] wherein, is the dimension of the input vector, is the dimension of the query and key, is the output of the previous layer, is used to find relevant information, represents the feature identifier of each token, represents the feature information of each token, is the weight matrix of Q, is the weight matrix of K, is the weight matrix of V.

[0116] Optionally, the calculation formula of the attention score is as follows:

[0117] ;

[0118] wherein, represents the attention score of .

[0119] Optionally, the multi-head attention has h heads, and the output of the i-th head is , and the multi-head attention score is represented as follows:

[0120] .

[0121] Optionally, a gating mechanism is introduced, and the vector that controls the flow of information is:

[0122] ;

[0123] wherein, represents the output tensor of the previous layer, with dimension , G is a vector for controlling information flow; is a weight matrix of G , with dimension ; represents a bias vector, with dimension .

[0124] Optionally, the output of the gated multi-head self-attention layer is: , represents element-wise multiplication, and important information can be selected more flexibly through the gating unit.

[0125] It should be noted that the feedforward network layer adds two additional layers on the basis of the original feedforward network layer, and the input of the two additional layers is the output of the gated multi-head attention layer , and the output is obtained after two-layer network and activation function processing.

[0126] The first layer: , , is the dimension of the feedforward network hidden layer, is a nonlinear activation function, used to enhance the nonlinear ability thereof; is a weight matrix, used to linearly map to the dimension of the feedforward network hidden layer , to learn the linear combination of features; is a bias vector, used to add an offset to the linear transformation result, to enhance the fitting ability of the model.

[0127] The second layer: , , to further enhance the nonlinear transformation of the features;

[0128] The output of the feedforward network layer is:

[0129] .

[0130] Optionally, the input of the normalization layer is the output of the feedforward network layer , and the output result of the normalization layer is , i.e.

[0131] ;

[0132] In the formula, is the mean of , is the variance of , is a small constant, is a learnable scaling parameter, is a learnable translation parameter.

[0133] It should be noted that the improved deep pyramid convolutional neural network adjusts the size of the convolution kernel on the basis of the original deep pyramid convolutional neural network, adopts different sizes of convolution kernel in parallel to capture features of different scales, and introduces a channel attention mechanism to let the model automatically learn the importance of each channel feature. At the same time, the depth of the network is increased, and more convolution blocks are added.

[0134] Optionally, the improved deep pyramid convolutional neural network comprises:

[0135] (1) Feature mapping layer: used for feature mapping the output of the last layer of the BERT model to obtain , wherein , , , is the output of the last layer of the BERT model, representing the semantic features of the text, and the dimension is , is a feature suitable for DPCNN processing; represents a weight matrix, used for linearly transforming from a dimension of to a dimension of ; represents a bias vector, with a dimension of , used to increase the flexibility and fitting ability of the model, avoid the linear transformation of feature mapping being too monotonous, and help the model learn more complex feature representations.

[0136] (2) Region embedding layer: used for obtaining spliced features by different size convolution kernels from the feature mapping output . The convolution kernel size is designed as , three, and the number is . After convolution operation, different scale local features are spliced to obtain .

[0137] (3) Convolution block layer: used for processing the output of the above region embedding layer to obtain , each convolution block contains two convolution layers, and the convolution size is , and the number of convolution kernels is .

[0138] The output of the first convolution layer is: , , and the activation function , denotes a bias vector with dimension , denotes a convolution kernel weight matrix, 3 in the above equation denotes the spatial size of the convolution kernel (e.g., 3x3), is the number of input channels, i.e., three kinds of convolution kernels are used in the region embedding layer, and the number is , and the total number of channels is ; is the number of output channels (i.e., the number of convolution kernels).

[0139] The output of the second convolution layer is: , , the activation function , denotes a bias vector with dimension , denotes a convolution kernel weight matrix.

[0140] Optionally, the connection is made by residual connection to obtain , and further through a pooling operation to obtain , so as to reduce the feature dimension, and the above process is repeated 4 times to obtain .

[0141] (4) Channel attention layer: used to obtain feature map after the output of the convolution block layer is weighted by channel attention.

[0142] Firstly, global average pooling is used to obtain , H and W are the height and width of the feature map, and the channel attention weighting weight is obtained through a fully connected layer, wherein , , r is the reduction ratio, , , and finally the feature map is obtained by weighting. Through this step, important channel feature values are highlighted.

[0143] (5) Pooling fusion layer: used to obtain the output of the channel attention layer, the result of global average pooling and the result of global maximum pooling are spliced to obtain the spliced feature .

[0144] Specifically, the output of the channel attention layer is subjected to global average pooling to obtain , and global maximum pooling is performed to obtain , and finally the two are spliced to obtain .

[0145] (6) Full connection layer:

[0146] The output of the pooling fusion is used to obtain the classification result After linear transformation and activation function processing, the classification result is obtained .

[0147] Specifically, after the first full connection layer, we get: , , ; Then, after the activation function: ;

[0148] After the second full connection layer: , where , , C is the number of classifications, and the output is finally obtained: .

[0149] Optionally, the bert-base-chinese model is fused with the DPCNN network, the word vector library expanded by the FastText model is fine-tuned by the bert model, and multiple d-dimensional vectors are generated for each word in the text , where i represents the ith word in the text, and these vectors are concatenated into an (n, d) dimensional matrix , then the dot product calculation is performed by the convolution kernel X, the activation function and the pooling operation are used to generate a new feature matrix , and finally the full connection layer and softmax are used for classification, where and represent the bias. The classification result is represented as: .

[0150] Optionally, the accuracy, precision and recall are used to evaluate the classification results of the intent recognition model.

[0151] Accuracy (Accuracy): ;

[0152] Precision (Precision): ;

[0153] Recall (Recall): ;

[0154] Where TP (True Positive) is the true positive, i.e., the number of samples correctly predicted as positive; TN (True Negative) is the true negative, i.e., the number of samples correctly predicted as negative; FP (False Positive) is the false positive, i.e., the number of samples incorrectly predicted as positive; and FN (False Negative) is the false negative, i.e., the number of samples incorrectly predicted as negative.

[0155] In some embodiments, the agricultural condition consultation question and the original consultation result are input into the large model to obtain a first consultation result of the agricultural condition consultation question output by the large model, which includes:

[0156] The agricultural condition consultation question and the original consultation result are input into the large model, and the large model finds relevant information in a pre-constructed external knowledge base based on the agricultural condition consultation question, supplements the original consultation result according to the found relevant information, and obtains a first consultation result of the agricultural condition consultation question.

[0157] The agricultural condition consultation question is input into the large model to obtain a second consultation result of the agricultural condition consultation question output by the large model, which includes:

[0158] The agricultural condition consultation question is input into the large model, and the large model finds relevant information in an external knowledge base based on the agricultural condition consultation question, and generates a second consultation result of the agricultural condition consultation question based on the found relevant information.

[0159] The external knowledge base includes prior knowledge in multiple aspects of the agricultural field, such as planting technology, breeding management, agricultural product processing, and marketing.

[0160] It can be understood that, by means of the external knowledge base, the large model can not only supplement the original consultation result to make the answer more accurate and comprehensive, but also directly generate a consultation result based on the question to improve user experience.

[0161] In some embodiments, the construction process of the external knowledge base includes:

[0162] Obtaining a plurality of documents related to the agricultural field;

[0163] Segmenting each document to obtain a plurality of text blocks and establishing a mapping relationship between the text blocks and the documents;

[0164] Identifying entities of each text block, associating pictures and / or tables in the corresponding document with the entities of each text block, and establishing a mapping relationship between the pictures and / or tables and the entities;

[0165] Determine the syntactic dependency relationship between different entities, construct an initial knowledge graph with entities as nodes and syntactic dependency relationships between different entities as edges, extract the features of each node and the features of each edge in the initial knowledge graph based on the graph neural network, and determine the relationship type between different entities based on the features of each node and the features of each edge.

[0166] Extract the embedding information of each sentence of each text block, fuse the embedding information of each sentence of each text block with the initial knowledge graph, and obtain the knowledge graph.

[0167] Perform community division and hierarchy construction on the knowledge graph, and construct a multi-dimensional index based on entities, relationships, communities and hierarchies, and perform semantic enhancement on the multi-dimensional index.

[0168] Optionally, the plurality of documents related to the agricultural field at least include variety information, agricultural product prices, planting techniques, pest control knowledge, agricultural news policies, supply and demand information, and related data such as agricultural production.

[0169] Optionally, the document is segmented into text blocks TextUnits, and each text block is represented by embedding. The semantic information of the text block is expressed by an embedding vector; the picture and the table are expressed by using words, and a mapping relationship between the text block and the total document is established.

[0170] Optionally, the vectorization representation of each sentence is obtained by encoding the document, and the similarity between two sentence vectors is calculated, that is, An adaptive threshold is set The threshold is dynamically adjusted according to the overall semantic complexity of the document, that is, by calculating the variance of all sentence vectors in the document , combined with a proportion coefficient k to obtain When , two sentences can be divided into the same segment to ensure the coherence of the semantics.

[0171] Optionally, an entity dictionary is constructed, and each entity of each text block is recognized by a named entity recognition model based on an attention mechanism and in combination with the entity dictionary.

[0172] It should be noted that the <entity, relationship, statement> triple of each text block TextUnit is extracted, and these elements are represented in a structured manner, that is, an entity and a relationship list are used to further express subgraph information. After generating the entity and relationship graph, further summarize each entity and relationship by a large language model LLM to generate corresponding abstract information. The graph is optimized by domain-enhanced NER and multi-modal fusion.

[0173] Domain-enhanced NER: Entities from the knowledge graph are used as prior knowledge to construct an entity dictionary. For input document fragments, an attention-based NER model is used to process the input text sequence. The vector is obtained after word embedding layer. Then, a context-aware representation is obtained through multiple Transformer layers. At the same time, the entities in the entity dictionary are represented as vectors. Calculate the vector for each word With all entity vectors similarity Similarity information is incorporated into the attention calculation of the NER model to enhance entity recognition capabilities in specific agricultural fields. The formula for calculating the attention score is as follows:

[0174] ;

[0175] In the formula, , These are the learnable weight matrices. It is a hyperparameter that controls the degree of influence of similarity.

[0176] Multimodal fusion: In addition to textual information, table extraction, image recognition technology, and the professional models involved in the embodiments of this invention are fully utilized to convert the images and data in the tables in the document into text, associate them with entities in the text, and establish mapping relationships to achieve multimodal information fusion.

[0177] Optionally, for complex semantic relationships in the agricultural field, a graph neural network (GNN)-based method is used for relationship mining. This involves using the entities extracted in the above steps as nodes, constructing an initial graph structure based on the entity's position and syntactic relationships in the text, determining the syntactic dependencies between entities through syntactic analysis, and using these as edges in the graph. Then, GraphSAGE is used to learn features from the graph. The feature calculation formula for each node is as follows:

[0178] ;

[0179] in, Is node v at the th Layer feature representation, It is the set of neighboring nodes of node v. and It is a weight matrix. It is an activation function.

[0180] Optionally, the relationship features in the knowledge graph are integrated into the learning process of the GNN, and the knowledge graph relationship vector related to the current entity pair is taken as an additional input feature; a classifier is used to predict the relationship type between entities based on the learned node features.

[0181] Optionally, two methods of knowledge graph fusion embedding and hierarchical semantic embedding are used to extract the embedding information of each sentence of each text block; the knowledge graph fusion embedding method can make the text embedding vector integrate the structural information of the knowledge graph, and enhance the semantic expression ability; the hierarchical semantic embedding method can better capture the semantic information of different levels according to the hierarchical characteristics of agricultural knowledge, and improve the accuracy of text representation.

[0182] The knowledge graph fusion embedding refers to text embedding combined with the structural information of the knowledge graph. First, the sentences in the document segment are encoded to obtain a sentence vector s; then the entity and relationship information in the knowledge graph are integrated into the sentence vector, for each sentence, find the entities involved therein, construct a subgraph according to the adjacency relationship of the entities in the knowledge graph; use the graph attention network GAT to learn the features of the subgraph, and obtain the representation of the entity under the structure of the knowledge graph .

[0183] Optionally, the calculation formula of the sentence vector is:

[0184] ;

[0185] wherein, is a hyperparameter for controlling the influence degree of the knowledge graph structural information.

[0186] The hierarchical semantic embedding refers to the hierarchical embedding method for the hierarchical structure of agricultural knowledge, which divides the text into different levels (paragraph, sentence, phrase), and respectively embeds each level.

[0187] For example, for paragraph embedding p, the sentence embedding in the paragraph is pooled to obtain: ; then, the embeddings of different levels are fused through a multi-layer neural network to obtain the final text embedding vector, so as to better capture the hierarchical semantics of agricultural knowledge.

[0188] Optionally, the above knowledge graph (including entity, relationship and text embedding information) is divided into communities and constructed into levels. The former uses a semantic-relation joint community division algorithm, which can fully consider the semantic and relationship characteristics of agricultural knowledge, so that the division result is more consistent with the internal logic of the knowledge; the latter can more clearly show the hierarchical relationship of the knowledge, and is convenient for knowledge organization and query.

[0189] Community division: First, calculate the semantic similarity between entities according to the text embedding vector At the same time, calculate the relationship closeness according to the weight of the edge between entities in the knowledge graph Integrate the two similarities to get a comprehensive similarity:

[0190] , where is a hyperparameter that balances the semantic and relationship similarities, and then use the density-based community discovery algorithm DBSCAN to divide the community based on the comprehensive similarity.

[0191] Hierarchical construction: Construct a hierarchical structure based on the classification system of agricultural knowledge (such as crop classification, agricultural technology classification, breeding variety classification, subject classification, etc.); Determine the root node in the knowledge graph, and then gradually build the hierarchy according to the category information and relationships of entities.

[0192] Optionally, use multi-dimensional index construction and semantic index enhancement for retrieval. By using multi-dimensional index, information in the knowledge graph can be quickly located from different angles, improving the flexibility of retrieval. By using semantic index enhancement, the accuracy and efficiency of retrieval can be improved by utilizing semantic information, better meeting users' query needs for agricultural knowledge.

[0193] Multi-dimensional index construction: Construct multi-dimensional indexes based on entities, relationships, communities, and hierarchies. For entities, use entity names and their unique identifiers as index keys, and associate detailed information of the entity; for relationships, use relationship types and related entity pairs as index keys, and store detailed descriptions of the relationship and related knowledge graph fragments; for communities, use community identifiers as index keys, and associate entities, relationships within the community, and community feature descriptions (such as topics, hierarchy ranges, etc.); for hierarchies, use hierarchy identifiers as index keys, and store entities, communities within the hierarchy, and upper and lower hierarchy relationships.

[0194] Semantic index enhancement: Use text embedding vectors and semantic information of the knowledge graph to enhance the index. That is, for each index item, in addition to storing basic information, also store the summary information of the text embedding vector related to it. In retrieval, by calculating the similarity between the query vector and the text embedding summary information of the index item, potential relevant index items are quickly filtered out, improving the accuracy and efficiency of retrieval.

[0195] It should be noted that the embodiment of the present application mainly realizes the utilization of the external knowledge base in the form of a knowledge vector library and a knowledge graph, that is, the advantages of the knowledge graph and the retrieval enhancement generation RAG are exerted by using GraphRAG, a knowledge graph database is constructed, the global knowledge base is segmented and summarized, and the complex relationship between each document fragment is captured. By decomposing the input content into different text blocks, and then through vectorization processing, combining retrieval enhancement and knowledge graph technology, accurate problem retrieval is realized.

[0196] In some embodiments, the above method further comprises:

[0197] Obtaining feedback information of the user on the first consultation result or the second consultation result;

[0198] According to the feedback information, determining whether the user is satisfied with the first consultation result or the second consultation result;

[0199] In the case that the user is not satisfied with the first consultation result or the second consultation result, according to the key word set of the agricultural condition consultation question, determining a target online expert, sending the agricultural condition consultation question to the corresponding target expert end, and receiving a third consultation result of the agricultural condition consultation question sent by the target expert end.

[0200] In some embodiments, the target professional small model is one of a target detection small model, a classification detection small model and a yield estimation small model; the target detection small model comprises an improved YOLOv10 network, the improved YOLOv10 network comprises a backbone network, a neck network, a one-to-many prediction head and a one-to-one prediction head, and the backbone network comprises a pyramid squeeze attention (PSA) module; the classification detection small model comprises an improved ConvNeXt V2 network, and the improved ConvNeXt V2 network comprises a full convolution mask autoencoder framework, a global response normalization layer and a convolutional block attention module (CBAM).

[0201] It should be noted that the classification detection small model is used for classification problems in agricultural production, and uses an improved ConvNeXt V2 model to solve problems such as variety type, quality grading, etc.

[0202] Optionally, the yield estimation small model mainly extracts visible light vegetation index, texture color, geometric shape and other features in the RGB image to evaluate the growth and health status of the vegetation.

[0203] Figure 5 The flowchart of the training process of the intent recognition model provided by the embodiment of the present application is shown in FIG. 1. Figure 5As shown, in some embodiments, the training process of the intent recognition model includes:

[0204] Step 510, acquiring a multi-modal sample agricultural condition consultation question;

[0205] Step 520, extracting and expanding the key words of the sample agricultural condition consultation question to obtain an expanded sample agricultural condition consultation question;

[0206] Step 530, determining the user intent label corresponding to the sample agricultural condition consultation question;

[0207] Step 540, training an initial intent recognition model using the expanded sample agricultural condition consultation question as training data and the user intent label corresponding to the sample agricultural condition consultation question as a training label, and obtaining the intent recognition model after the training is completed.

[0208] Optionally, the extracting and expanding the key words of the sample agricultural condition consultation question to obtain an expanded sample agricultural condition consultation question includes:

[0209] segmenting the sample agricultural condition consultation question to obtain a sample word sequence of the sample agricultural condition consultation question, extracting the key words of the sample agricultural condition consultation question from the sample word sequence of the sample agricultural condition consultation question to obtain a sample key word set of the sample agricultural condition consultation question;

[0210] vectorizing the sample word sequence of the sample agricultural condition consultation question to obtain a sample word vector sequence of the sample agricultural condition consultation question, inputting the sample word vector sequence of the sample agricultural condition consultation question into the improved FastText model to obtain an association score of the sample agricultural condition consultation question and each key word in the pre-constructed key word library output by the improved FastText model;

[0211] determining a plurality of sample expansion key words of the sample agricultural condition consultation question based on the association score of the sample agricultural condition consultation question and each key word, a preset association score threshold, and the sample key word set of the sample agricultural condition consultation question;

[0212] expanding the sample agricultural condition consultation question based on the plurality of sample expansion key words to obtain an expanded sample agricultural condition consultation question.

[0213] The agricultural condition consultation service system based on the size model cooperation provided by the embodiments of the present application will be described below. The agricultural condition consultation service system based on the size model cooperation described below can be correspondingly referred to the agricultural condition consultation service method based on the size model cooperation described above.

[0214] Figure 6 The structure diagram of the agricultural condition consultation service system based on the size model cooperation provided by the embodiments of the present application is shown in Figure 6 The agricultural condition consultation service system 600 includes:

[0215] The first acquisition unit 610 is configured to acquire a multi-modal agricultural condition consultation question input by a user, and determine a corresponding agricultural condition consultation question.

[0216] The expansion unit 620 is configured to extract key words from the agricultural condition consultation question and expand the agricultural condition consultation question to obtain an expanded agricultural condition consultation question.

[0217] The intent recognition unit 630 is configured to input the expanded agricultural condition consultation question into a pre-constructed intent recognition model to obtain a user intent corresponding to the agricultural condition consultation question output by the intent recognition model. The intent recognition model is obtained by training based on an expanded sample agricultural condition consultation question and a user intent label corresponding to the sample agricultural condition consultation question.

[0218] The question answering unit 640 is configured to determine, based on the user intent, whether the agricultural condition consultation question needs to be answered by a professional small model. If yes, the agricultural condition consultation question is input into a pre-constructed target professional small model to obtain an original consultation result of the agricultural condition consultation question output by the target professional small model. The agricultural condition consultation question and the original consultation result are input into a large model to obtain a first consultation result of the agricultural condition consultation question output by the large model. If no, the agricultural condition consultation question is directly input into the large model to obtain a second consultation result of the agricultural condition consultation question output by the large model. The target professional small model is obtained by training based on a multi-modal sample agricultural condition consultation question and an original consultation result label of the sample agricultural condition consultation question.

[0219] Optionally, the key word extraction and expansion of the agricultural condition consultation question to obtain the expanded agricultural condition consultation question include:

[0220] The agricultural condition consultation question is segmented to obtain a word sequence of the agricultural condition consultation question. The key word extraction is performed on the word sequence of the agricultural condition consultation question to obtain a key word set of the agricultural condition consultation question. The vectorization processing is performed on the word sequence of the agricultural condition consultation question to obtain a word vector sequence of the agricultural condition consultation question. The word vector sequence of the agricultural condition consultation question is input into an improved FastText model to obtain an association score of the agricultural condition consultation question and each key word in a pre-constructed key word library output by the improved FastText model. Based on the association score of the agricultural condition consultation question and each key word, a preset association score threshold, and the key word set of the agricultural condition consultation question, a plurality of expansion key words of the agricultural condition consultation question are determined. Based on the plurality of expansion key words, the expansion is performed on the agricultural condition consultation question to obtain the expanded agricultural condition consultation question.

[0221] The improved FastText model comprises an input layer, a hidden layer and an output layer; the input layer introduces a position-sensitive attention mechanism, and the hidden layer comprises a capsule network; the improved FastText model is trained based on a sample word vector sequence of a sample agricultural situation consultation question and an association score label of each key word in a key word library and the sample agricultural situation consultation question.

[0222] Optionally, key words are extracted from the word sequence of the agricultural situation consultation question to obtain a key word set of the agricultural situation consultation question, comprising:

[0223] The co-occurrence frequency of adjacent words in the word sequence of the agricultural situation consultation question is counted, and a plurality of word blocks are determined from the word sequence of the agricultural situation consultation question based on a preset co-occurrence frequency threshold; the number of times each word in the word sequence of the agricultural situation consultation question appears in the agricultural situation consultation question and the number of times each word appears in the corresponding word block are calculated to obtain the word frequency of each word; the inverse word frequency of each word is calculated in combination with the semantic similarity between different words in the word sequence of the agricultural situation consultation question; the word frequency of each word is multiplied by the inverse word frequency of each word to obtain the word frequency-inverse word frequency value of each word; the word frequency-inverse word frequency value of each word is sorted to obtain a sorting result, and the key word set of the agricultural situation consultation question is determined according to the sorting result.

[0224] Optionally, the intent recognition model comprises an improved bert-base-chinese model and an improved deep pyramid convolutional neural network; the improved bert-base-chinese model comprises an input layer, a gated multi-head self-attention layer, a feedforward network layer and a normalization layer; and the improved deep pyramid convolutional neural network comprises a feature mapping layer, a region embedding layer, a convolution block layer, a channel attention layer, a pooling fusion layer and a full connection layer.

[0225] Optionally, the agricultural situation consultation question and the original consultation result are input into the large model to obtain a first consultation result of the agricultural situation consultation question output by the large model, comprising: inputting the agricultural situation consultation question and the original consultation result into the large model, searching for related information in a pre-constructed external knowledge base based on the agricultural situation consultation question by the large model, supplementing the original consultation result based on the found related information to obtain the first consultation result of the agricultural situation consultation question.

[0226] Optionally, the agricultural situation consultation question is input into the large model to obtain a second consultation result of the agricultural situation consultation question output by the large model, comprising: inputting the agricultural situation consultation question into the large model, searching for related information in the external knowledge base based on the agricultural situation consultation question by the large model, and generating the second consultation result of the agricultural situation consultation question based on the found related information.

[0227] Optionally, the construction process of the external knowledge base comprises: acquiring a plurality of documents related to the agricultural field; segmenting each document to obtain a plurality of text blocks, and establishing a mapping relationship between the text blocks and the documents; identifying entities of each text block, associating pictures and / or tables in the corresponding document with the entities of each text block, and establishing a mapping relationship between the pictures and / or tables and the entities; determining the syntactic dependency relationship between different entities, constructing an initial knowledge graph taking the entities as nodes and the syntactic dependency relationship between different entities as edges, extracting the features of each node and the features of each edge in the initial knowledge graph based on a graph neural network, and determining the relationship types between different entities based on the features of each node and the features of each edge; extracting embedding information of each sentence of each text block, fusing the embedding information of each sentence of each text block with the initial knowledge graph, and obtaining a knowledge graph; performing community division and hierarchical construction on the knowledge graph, and constructing a multi-dimensional index based on entities, relationships, communities and hierarchies, and performing semantic enhancement on the multi-dimensional index.

[0228] Optionally, the agricultural condition consultation service system 600 based on the size model collaboration further comprises:

[0229] The second acquisition unit is configured to acquire feedback information of the user on the first consultation result or the second consultation result; the determination unit is configured to determine whether the user is satisfied with the first consultation result or the second consultation result according to the feedback information; and the expert collaboration unit is configured to, in a case where the user is not satisfied with the first consultation result or the second consultation result, determine a target online expert according to the key word set of the agricultural condition consultation question, send the agricultural condition consultation question to the corresponding target expert terminal, and receive a third consultation result of the agricultural condition consultation question sent by the target expert terminal.

[0230] Optionally, the target professional small model is one of a target detection small model, a classification detection small model and a yield measurement small model; the target detection small model comprises an improved YOLOv10 network, the improved YOLOv10 network comprises a backbone network, a neck network, a one-to-many prediction head and a one-to-one prediction head, and the backbone network comprises a pyramid compression attention (PSA) module; the classification detection small model comprises an improved ConvNeXt V2 network, the improved ConvNeXt V2 network comprises a full convolution mask autoencoder framework, a global response normalization layer and a convolution block attention module (CBAM).

[0231] Optionally, the training process of the intent recognition model comprises: acquiring a plurality of sample agricultural condition consultation questions in multiple modalities; performing key word extraction and expansion on the sample agricultural condition consultation questions to obtain expanded sample agricultural condition consultation questions; determining user intent labels corresponding to the sample agricultural condition consultation questions; training an initial intent recognition model taking the expanded sample agricultural condition consultation questions as training data and taking the user intent labels corresponding to the sample agricultural condition consultation questions as training labels, and obtaining the intent recognition model after the training is completed.

Claims

1. A crop condition consultation service method based on size model cooperation, characterized by, The method comprises the following steps: acquiring a multi-modal agricultural condition consultation question input by a user; extracting key words from the agricultural condition consultation question and expanding the agricultural condition consultation question to obtain an expanded agricultural condition consultation question; inputting the expanded agricultural condition consultation question into a pre-constructed intention recognition model to obtain a user intention corresponding to the agricultural condition consultation question output by the intention recognition model; based on the user intention, determining whether the agricultural condition consultation question needs to be answered by a professional small model, if yes, inputting the agricultural condition consultation question into a pre-constructed target professional small model to obtain an original consultation result of the agricultural condition consultation question output by the target professional small model, inputting the agricultural condition consultation question and the original consultation result into a large model to obtain a first consultation result of the agricultural condition consultation question output by the large model, otherwise, directly inputting the agricultural condition consultation question into the large model to obtain a second consultation result of the agricultural condition consultation question output by the large model; wherein the intention recognition model is obtained by training based on an expanded sample agricultural condition consultation question and a user intention label corresponding to the sample agricultural condition consultation question; the intention recognition model comprises an improved bert-base-chinese model and an improved deep pyramid convolutional neural network; the improved bert-base-chinese model comprises an input layer, a gated multi-head self-attention layer, a feedforward network layer and a normalization layer; the improved deep pyramid convolutional neural network comprises a feature mapping layer, a region embedding layer, a convolution block layer, a channel attention layer, a pooling fusion layer and a full connection layer; wherein the target professional small model is obtained by training based on a multi-modal sample agricultural condition consultation question and an original consultation result label of the sample agricultural condition consultation question.

2. The size model-based collaborative crop condition consultation service method according to claim 1, characterized by, The method of extracting key words from the agricultural condition consultation question and expanding the agricultural condition consultation question to obtain an expanded agricultural condition consultation question comprises the following steps: performing word segmentation on the agricultural condition consultation question to obtain a word sequence of the agricultural condition consultation question, extracting key words from the word sequence of the agricultural condition consultation question to obtain a key word set of the agricultural condition consultation question; performing vectorization processing on the word sequence of the agricultural condition consultation question to obtain a word vector sequence of the agricultural condition consultation question, inputting the word vector sequence of the agricultural condition consultation question into an improved FastText model to obtain an association score of each key word in a pre-constructed key word library with the agricultural condition consultation question output by the improved FastText model; based on the association score of each key word with the agricultural condition consultation question, a preset association score threshold and the key word set of the agricultural condition consultation question, determining a plurality of expanded key words of the agricultural condition consultation question; based on the plurality of expanded key words, expanding the agricultural condition consultation question to obtain an expanded agricultural condition consultation question; The improved FastText model comprises an input layer, a hidden layer and an output layer; the input layer introduces a position-sensitive attention mechanism, and the hidden layer comprises a capsule network; the improved FastText model is trained based on a sample word vector sequence of a sample agricultural situation consultation question and an association score label of the sample agricultural situation consultation question and each key word in the key word library.

3. The crop condition consultation service method based on size model cooperation according to claim 2, characterized in that, The key word extraction on the word sequence of the agricultural situation consultation question obtains a key word set of the agricultural situation consultation question, and comprises the following steps: statistically acquiring a co-occurrence frequency of adjacent words in the word sequence of the agricultural situation consultation question, and determining a plurality of word blocks from the word sequence of the agricultural situation consultation question based on a preset co-occurrence frequency threshold; calculating a number of times each word in the word sequence of the agricultural situation consultation question appears in the agricultural situation consultation question and a number of times the word appears in a corresponding word block to obtain a word frequency of the word; calculating an inverse word frequency of the word in combination with a semantic similarity between different words in the word sequence of the agricultural situation consultation question; multiplying the word frequency of the word and the inverse word frequency of the word to obtain a word frequency-inverse word frequency value of the word; sorting the word frequency-inverse word frequency value of the word to obtain a sorting result, and determining a key word set of the agricultural situation consultation question according to the sorting result.

4. The size model collaboration-based crop condition consultation service method according to claim 1, characterized in that, The inputting of the agricultural situation consultation question and the original consultation result into the large model to obtain the first consultation result of the agricultural situation consultation question output by the large model comprises: inputting the agricultural situation consultation question and the original consultation result into the large model, searching for relevant information in a pre-constructed external knowledge base based on the agricultural situation consultation question by the large model, supplementing the original consultation result according to the found relevant information to obtain the first consultation result of the agricultural situation consultation question; The inputting of the agricultural situation consultation question into the large model to obtain the second consultation result of the agricultural situation consultation question output by the large model comprises: inputting the agricultural situation consultation question into the large model, searching for relevant information in the external knowledge base based on the agricultural situation consultation question by the large model, and generating the second consultation result of the agricultural situation consultation question based on the found relevant information.

5. The size model collaboration-based crop condition consultation service method according to claim 4, characterized in that, The construction process of the external knowledge base comprises: acquiring a plurality of documents related to the agricultural field; segmenting each document to obtain a plurality of text blocks and establishing a mapping relationship between the text blocks and the documents; identifying an entity of each text block, associating pictures and / or tables in the corresponding document with the entity of each text block, and establishing a mapping relationship between the pictures and / or tables and the entity; determining a syntactic dependency relationship between different entities, constructing an initial knowledge graph by taking the entities as nodes and the syntactic dependency relationships between the different entities as edges, extracting a feature of each node and a feature of each edge in the initial knowledge graph based on a graph neural network, and determining a relationship type between different entities based on the feature of each node and the feature of each edge. Extract embedding information of each sentence of each text block, fuse the embedding information of each sentence of each text block with the initial knowledge graph, and obtain a knowledge graph; Community division and hierarchy construction are performed on the knowledge graph, and a multi-dimensional index based on entities, relationships, communities and hierarchies is constructed, and semantic enhancement is performed on the multi-dimensional index.

6. The size model collaboration-based crop condition consultation service method according to claim 1, characterized in that, The method further comprises: obtaining feedback information of the user on the first consultation result or the second consultation result; According to the feedback information, it is determined whether the user is satisfied with the first consultation result or the second consultation result; In the case that the user is not satisfied with the first consultation result or the second consultation result, according to the key word set of the agricultural situation consultation question, a target online expert is determined, the agricultural situation consultation question is sent to the corresponding target expert terminal, and a third consultation result of the agricultural situation consultation question sent by the target expert terminal is received.

7. The size model collaboration-based crop condition consultation service method according to claim 1, characterized in that, The target professional small model is one of a target detection small model, a classification detection small model and a yield estimation small model; the target detection small model comprises an improved YOLOv10 network, the improved YOLOv10 network comprises a backbone network, a neck network, a one-to-many prediction head and a one-to-one prediction head, and the backbone network comprises a pyramid compression attention (PSA) module; the classification detection small model comprises an improved ConvNeXt V2 network, the improved ConvNeXt V2 network comprises a full convolution mask autoencoder framework, a global response normalization layer and a convolution block attention module (CBAM). 8.The crop condition consultation service method based on size model cooperation according to claim 1, wherein, The training process of the intent recognition model comprises: obtaining a multi-modal sample agricultural situation consultation question; extracting and expanding the key words of the sample agricultural situation consultation question to obtain an expanded sample agricultural situation consultation question; determining the user intent label corresponding to the sample agricultural situation consultation question; using the expanded sample agricultural situation consultation question as training data and the user intent label corresponding to the sample agricultural situation consultation question as training label, training an initial intent recognition model, and obtaining the intent recognition model after training.

9. A crop condition consultation service system based on size model cooperation, characterized by, It comprises: a first acquisition unit configured to acquire a multi-modal agricultural situation consultation question input by a user; an expansion unit configured to extract and expand key words of the agricultural situation consultation question to obtain an expanded agricultural situation consultation question; an intent recognition unit configured to input the expanded agricultural situation consultation question into a pre-constructed intent recognition model to obtain a user intent corresponding to the agricultural situation consultation question output by the intent recognition model; The problem solving unit is configured to determine whether the agricultural condition consultation question needs to be answered by a professional small model based on the user intention, and if yes, input the agricultural condition consultation question into a target professional small model constructed in advance to obtain an original consultation result of the agricultural condition consultation question output by the target professional small model, input the agricultural condition consultation question and the original consultation result into a large model to obtain a first consultation result of the agricultural condition consultation question output by the large model, and if not, directly input the agricultural condition consultation question into the large model to obtain a second consultation result of the agricultural condition consultation question output by the large model. The intention recognition model is obtained by training based on the expanded sample agricultural condition consultation question and a user intention label corresponding to the sample agricultural condition consultation question. The intention recognition model comprises an improved bert-base-chinese model and an improved deep pyramid convolutional neural network. The improved bert-base-chinese model comprises an input layer, a gated multi-head self-attention layer, a feedforward network layer and a normalization layer. The improved deep pyramid convolutional neural network comprises a feature mapping layer, a region embedding layer, a convolution block layer, a channel attention layer, a pooling fusion layer and a full connection layer. The target professional small model is obtained by training based on a multi-modal sample agricultural condition consultation question and an original consultation result label of the sample agricultural condition consultation question.

Images