Adaptation verification methods, systems, and media for large language models

Abstract

This invention provides a method, system, and medium for adapting and validating large language models. The method includes: standardizing corpus instruction information to generate standard instruction information; performing semantic segmentation on the standard instruction information and extracting key elements, generating corresponding fragment element label sequences based on the key elements; classifying the fragment element label sequences according to intent using a pre-trained BERT model to output latent intent models and corresponding probability distribution sequences; obtaining model scores for latent intent models, calculating the fusion score of each latent intent model based on the model scores and probability distribution sequences, and selecting the latent intent model with the highest fusion score as a reference model; calculating the target similarity between each standard model and the reference model in the model library, and selecting the standard model with the highest similarity as the target model. This invention can adapt and select a suitable large language model based on user input, improving the user experience.

Description

Technical Field

[0001] This invention belongs to the field of large model technology, and relates to a method for adapting large language models, and particularly to an adaptation verification method, system and medium for large language models. Background Technology

[0002] Large Language Models (LLMs) are deep learning models trained on massive amounts of text data, enabling them to generate natural language text or understand the meaning of language text. These models can provide in-depth knowledge and language production on a wide range of topics through training on large datasets. Their core idea is to learn patterns and structures of natural language through large-scale unsupervised training, mimicking human language cognition and generation processes to some extent.

[0003] LLMs perform exceptionally well across a variety of applications, capable of handling not only simple language tasks such as spell checking and grammar correction, but also complex tasks like text summarization, machine translation, sentiment analysis, dialogue generation, and content recommendation. Through pre-training on large-scale datasets, large language models acquire powerful general modeling and generalization capabilities.

[0004] However, there are many large language models on the market, and different large language models have different focuses. Therefore, users often cannot choose the most suitable large model when using it, which affects the user experience. Summary of the Invention

[0005] The purpose of this invention is to provide a method, system, and medium for adapting and verifying large language models, in order to solve the problem in the prior art that the inability to select the corresponding large model according to user needs affects the user experience.

[0006] In a first aspect, the present invention provides an adaptation verification method for large language models, the method comprising:

[0007] The system acquires user-input corpus instruction information and generates standard instruction information after standardizing the corpus instruction information.

[0008] The standard instruction information is semantically segmented and key elements are extracted. A corresponding fragment element label sequence is generated based on the key elements.

[0009] The fragment element label sequence is classified according to the pre-trained BERT model to output the latent intent model and the corresponding probability distribution sequence.

[0010] Obtain the model score of the latent intent model, calculate the fusion score of each latent intent model based on the model score and the probability distribution sequence, and select the latent intent model with the highest fusion score as the reference model;

[0011] The target similarity between each standard model in the model library and the reference model is calculated, and the standard model with the highest similarity is selected as the target model to complete the model adaptation verification. The model library includes multiple standard models of different types.

[0012] In one implementation of the first aspect, the step of acquiring user-input corpus instruction information and generating standard instruction information after standardizing the corpus instruction information includes:

[0013] The character encoding detector is activated to decompose the corpus instruction information to obtain multiple split characters;

[0014] A standardization engine is used to process the split characters in a standardized manner to obtain a unified character representation, and a regular expression engine is used to extract features from the character representation to obtain basic character features;

[0015] The basic character features are processed to calculate the corresponding importance score. Based on the importance score, interference information is removed to generate standard character features. The standard character features are then combined to generate the standard instruction information.

[0016] The process of processing the basic character features to calculate the corresponding importance score includes:

[0017] Calculate the fundamental frequency and inverse frequency of each of the basic character features;

[0018] The frequency score of the basic character feature is calculated based on the basic frequency and the inverse frequency.

[0019] Obtain each potential corpus category corresponding to the basic character feature in the corpus, and calculate the occurrence probability of the basic character feature in each potential corpus category through an intent classifier, and calculate the information entropy of each basic character feature based on the occurrence probability;

[0020] The number of basic character features in each potential corpus category is counted to obtain a statistical count, and the corresponding verification score is calculated based on the statistical count.

[0021] The importance score of each basic character feature is calculated based on the frequency score, the information entropy, and the verification score.

[0022] In one implementation of the first aspect, the importance score is calculated by weighted summation of the frequency score, the information entropy, and the check score.

[0023] In one implementation of the first aspect, the step of semantically segmenting the standard instruction information and extracting key elements, and generating a corresponding fragment element label sequence based on the key elements, includes:

[0024] After performing semantic boundary detection on the standard instruction information based on the attention mechanism, the semantic boundary is obtained. The standard instruction information is then split into multiple instruction fragments based on the semantic boundary, and the weights of the instruction fragments are calculated.

[0025] The instruction fragment is subjected to element extraction according to the preset element type to obtain multiple fragment elements, and the comprehensive score of each fragment element is calculated.

[0026] Based on the comprehensive score and the segment weight, the corresponding segment element labels are generated, and the segment element labels are arranged together in sequence to form the segment element label sequence.

[0027] In one implementation of the first aspect, the step of performing intent classification on the fragment element label sequence based on a pre-trained BERT model to output a latent intent model and a corresponding probability distribution sequence includes:

[0028] After converting the fragment element label sequence into the BERT standard input format, dynamic word vectors are generated by the Transformer encoder, and the dynamic word vectors are input into the BERT model to output the corresponding latent intent.

[0029] The potential intent model corresponding to each potential intent is obtained according to the preset mapping relationship;

[0030] After mapping the fully connected layer to the intent category recognition space, the recognition result is obtained, and the softmax function is used to convert the recognition result into the corresponding probability distribution to obtain the probability distribution sequence.

[0031] In one implementation of the first aspect, obtaining the model score of the latent intent model and calculating the fusion score of each latent intent model based on the model score and the probability distribution sequence includes:

[0032] Based on preset score dimensions and preset weights, the model score of each latent intent model is calculated in the model library. The score dimensions include accuracy dimension, confidence dimension, and consistency dimension.

[0033] The model score and the probability distribution sequence are weighted to obtain the fusion score for each latent intent model.

[0034] In one implementation of the first aspect, calculating the target similarity between each of the standard models in the model library and the reference model includes:

[0035] Obtain the internal representation information and external behavior information of the reference model;

[0036] The internal similarity between each standard model and the reference model is calculated based on the internal representation information, and the external similarity between each standard model and the reference model is calculated based on the external behavior information.

[0037] Knowledge layer similarity is calculated by evaluating the consistency of the performance of the reference model and the standard model on the same knowledge exploration dataset.

[0038] The internal similarity, external similarity, and knowledge layer similarity are weighted and summed to obtain the target similarity between each standard model and the reference model.

[0039] Secondly, the present invention also discloses an adaptation verification system for large language models, the system comprising:

[0040] The information processing module is used to acquire the corpus instruction information input by the user, and to generate standard instruction information after standardizing the corpus instruction information;

[0041] The extraction and generation module is used to perform semantic segmentation on the standard instruction information and extract key elements, and generate a corresponding fragment element label sequence based on the key elements.

[0042] The classification module is used to classify the segment element label sequence according to the pre-trained BERT model to output the latent intent model and the corresponding probability distribution sequence.

[0043] The selection module is used to obtain the model score of the latent intent model, calculate the fusion score of each latent intent model based on the model score and the probability distribution sequence, and select the latent intent model with the highest fusion score as the reference model.

[0044] The adaptation module is used to calculate the target similarity between each standard model in the model library and the reference model, select the standard model with the highest similarity as the target model, and complete the model adaptation verification. The model library includes multiple standard models of different types.

[0045] Thirdly, the present invention also discloses a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the above-described adaptation verification method for large language models.

[0046] As described above, the adaptation verification method, system, and medium for large language models of the present invention have the following beneficial effects:

[0047] This invention generates standard instruction information by standardizing user-input corpus instructions. This standard instruction information is then processed to obtain multiple potentially suitable latent intent models and corresponding probability distribution sequences. Based on the probability distribution sequences and corresponding model scores, a suitable reference model is selected. This allows for matching the model with the highest target similarity from the model library, thus completing the large-scale model adaptation process. The invention outputs the optimal adapted model for the user-input corpus instructions, improving the user experience of using the large-scale model. It facilitates flexible adaptation of the large-scale model for different users and different input content, meeting the usage needs of different scenarios. Attached Figure Description

[0048] Figure 1 The flowchart shown is a method for adapting and verifying large language models according to an embodiment of the present invention.

[0049] Figure 2 The diagram shown is a structural block diagram of the adaptation verification system for large language models according to an embodiment of the present invention. Detailed Implementation

[0050] The following specific examples illustrate the implementation of the present invention. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that, unless otherwise specified, the following embodiments and features described therein can be combined with each other.

[0051] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of the present invention. The illustrations only show the components related to the present invention and are not drawn according to the actual number, shape and size of the components in the actual implementation. In the actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0052] See Figure 1 and Figure 2The following embodiments of the present invention provide a method, system, and medium for adapting and verifying large language models. After standardizing the user-input corpus command information, standard command information is generated. This standard command information is then processed to obtain multiple potentially adaptable latent intent models and corresponding probability distribution sequences. Based on the probability distribution sequences and corresponding model scores, a suitable reference model is selected. This allows for matching the model with the highest target similarity in the model library as the target model, thereby completing the adaptation of the large model. The optimal adapted model is output for the user-input corpus command information, improving the user experience of using the large model. This facilitates flexible adaptation of the large model for different users and different input content, meeting the usage needs of different scenarios.

[0053] The technical solutions of the present invention will now be described in detail with reference to the accompanying drawings.

[0054] like Figure 1 As shown, in one embodiment, the present invention provides an adaptation verification method for large language models, the method comprising the following steps:

[0055] S100: Obtain the corpus instruction information input by the user, and generate standard instruction information after standardizing the corpus instruction information.

[0056] In this embodiment, after the user inputs corpus instruction information, the corpus instruction information is standardized to remove interference and invalid information, so as to facilitate the subsequent adaptation of an optimal large language model based on the standard instruction information.

[0057] In some embodiments, the step of obtaining user-input corpus instruction information and generating standard instruction information after standardizing the corpus instruction information includes:

[0058] The character encoding detector is activated to decompose the corpus instruction information to obtain multiple split characters;

[0059] A standardization engine is used to process the split characters in a standardized manner to obtain a unified character representation, and a regular expression engine is used to extract features from the character representation to obtain basic character features;

[0060] The basic character features are processed to calculate the corresponding importance score. Based on the importance score, interference information is removed to generate standard character features. The standard character features are then combined to generate the standard instruction information.

[0061] Specifically, after the user inputs the corpus instruction information, the character encoding detector is first activated to decompose the corpus instruction information to obtain multiple split characters. The split characters are then standardized by a standardization engine to obtain a unified character representation. The character representation is then feature-extracted by a regular expression engine to obtain the corresponding basic character features. After that, the importance score of the basic character features is calculated to remove interference information and generate standard character features. Finally, the standard character features are combined to obtain the standard instruction information.

[0062] The character encoding detector, standardization engine, and regular expression engine all utilize existing technologies, and this solution does not impose any special restrictions on them, so they will not be elaborated on here.

[0063] In some other embodiments, the processing of the basic character features to calculate the corresponding importance score includes:

[0064] Calculate the fundamental frequency and inverse frequency of each of the basic character features;

[0065] The frequency score of the basic character feature is calculated based on the basic frequency and the inverse frequency.

[0066] Obtain each potential corpus category corresponding to the basic character feature in the corpus, and calculate the occurrence probability of the basic character feature in each potential corpus category through an intent classifier, and calculate the information entropy of each basic character feature based on the occurrence probability;

[0067] The number of basic character features in each potential corpus category is counted to obtain a statistical count, and the corresponding verification score is calculated based on the statistical count.

[0068] The importance score of each basic character feature is calculated based on the frequency score, the information entropy, and the verification score.

[0069] Specifically, the basic frequency and inverse frequency of each basic character feature are first calculated. The basic frequency A is the frequency of the basic character feature d among all basic character features corresponding to the entire corpus instruction information, while the inverse frequency B measures the prevalence of the current basic character feature d in the entire corpus instruction information. The final frequency score S is the product of the basic frequency and the inverse frequency. The specific calculation process satisfies the following formula:

[0070]

[0071] Where T represents the total number of basic character features, T dThis indicates the number of times the basic character feature of the current type appears among all the basic character features.

[0072] Since each basic character feature corresponds to at least one potential corpus category, each potential corpus category corresponding to the basic character feature is obtained from the corpus, and the occurrence probability of the basic character feature in each potential corpus category is calculated using an intent classifier. This allows for the calculation of the information entropy of each basic character feature based on the occurrence probability. A higher information entropy indicates a more dispersed and even distribution of the feature, while a lower entropy indicates a more concentrated distribution. The calculation process of the information entropy satisfies the following formula:

[0073]

[0074] Among them, C d The information entropy of the basic character feature d is represented by n, and the number of latent corpus categories is represented by P. i This represents the probability of the basic character feature appearing in the i-th potential corpus category.

[0075] The information entropy of the basic character feature d can be calculated through the above process. Then, the number of basic character features in each potential corpus category is counted to obtain the statistical quantity. The corresponding verification score is calculated based on the statistical quantity. The verification score is determined by multiplying the statistical quantity and the attenuation coefficient.

[0076] Furthermore, the importance score is calculated by weighted summation of the frequency score, the information entropy, and the verification score. The weights of the frequency score, the information entropy, and the verification score can be preset values, empirical values, or calculated using the analytic hierarchy process (AHP). Since assigning weights using the AHP is a prior art technique, this solution does not impose any special limitations on it and will not elaborate further here.

[0077] After obtaining the frequency score, information entropy, and verification score, they are weighted and summed according to preset weights to obtain the importance score of each basic character feature, so as to facilitate the subsequent screening of basic character features.

[0078] S200: Perform semantic segmentation on the standard instruction information and extract key elements, and generate a corresponding fragment element label sequence based on the key elements.

[0079] In some embodiments, the step of semantically segmenting the standard instruction information and extracting key elements, and generating a corresponding fragment element label sequence based on the key elements, includes:

[0080] After performing semantic boundary detection on the standard instruction information based on the attention mechanism, the semantic boundary is obtained. The standard instruction information is then split into multiple instruction fragments based on the semantic boundary, and the weights of the instruction fragments are calculated.

[0081] The instruction fragment is subjected to element extraction according to the preset element type to obtain multiple fragment elements, and the comprehensive score of each fragment element is calculated.

[0082] Based on the comprehensive score and the segment weight, the corresponding segment element labels are generated, and the segment element labels are arranged together in sequence to form the segment element label sequence.

[0083] In this embodiment, in order to perform semantic segmentation on standard instruction information to extract key elements and generate a fragment element label sequence, the semantic boundary is first obtained by performing semantic boundary detection on the standard instruction information using an attention mechanism in the prior art. The calculation process of the semantic boundary probability specifically satisfies the following formula:

[0084]

[0085] P b Represents semantic boundary probability. This represents the activation function. Let w represent the bias term, and h represent the weight matrix. i h represents the context representation vector of the i-th word in the standard instruction information. i+1 This represents the context representation vector of the (i+1)th word in the standard instruction information. The concatenation of the context representation vectors of the i-th word and the (i+1)-th word.

[0086] After determining the semantic boundary probability, the boundary positions where the semantic boundary probability is greater than the probability threshold are selected as semantic boundaries. The standard instruction information is then split into multiple instruction fragments based on these semantic boundaries. This splitting occurs either between adjacent semantic boundaries or between the outermost semantic boundary and its adjacent end. The weight of each instruction fragment is then calculated to obtain the fragment weight, specifically satisfying the following formula:

[0087]

[0088] Let W represent the weight of the j-th segment, softmax represent the normalization function, v represent the attention weight vector, and W represent the weight of the j-th segment. s S represents the semantic transformation matrix, which is obtained by backpropagation on the training data. j Let represent the representation vector of the j-th instruction segment. represents the bias value, which is a fixed parameter, and tanh represents the hyperbolic tangent activation function.

[0089] After obtaining the segment weights, the instruction segments are extracted according to preset element types to obtain multiple segment elements, and a comprehensive score is calculated for each segment element. The comprehensive score is a weighted sum of the information density score and the semantic saliency score. The information density score is the occurrence ratio of the segment element, while the semantic saliency score is the similarity between the element semantic vector and the global semantic vector of the segment element. The element semantic vector is the result of a weighted average of the vector representations of segments containing segment element a, while the global semantic vector is calculated through a fixed window. Since the calculation process of the global semantic vector is existing technology, it will not be described in detail here.

[0090] After obtaining the comprehensive score, the segment element label corresponding to each segment element is generated based on the comprehensive score and the segment weight. The segment element label includes the comprehensive score, segment weight and segment element. Then, the segment element labels are arranged together in sequence to form the segment element label sequence, which facilitates subsequent intent classification based on the element label sequence.

[0091] S300. Based on the pre-trained BERT model, perform intent classification on the segment element label sequence to output the latent intent model and the corresponding probability distribution sequence.

[0092] In some embodiments, the step of classifying the segment element label sequence according to a pre-trained BERT model to output a latent intent model and a corresponding probability distribution sequence includes:

[0093] After converting the fragment element label sequence into the BERT standard input format, dynamic word vectors are generated by the Transformer encoder, and the dynamic word vectors are input into the BERT model to output the corresponding latent intent.

[0094] The potential intent model corresponding to each potential intent is obtained according to the preset mapping relationship;

[0095] After mapping the fully connected layer to the intent category recognition space, the recognition result is obtained, and the softmax function is used to convert the recognition result into the corresponding probability distribution to obtain the probability distribution sequence.

[0096] Specifically, the obtained text is first converted into the BERT standard input format: [CLS] text[SEP], where [CLS] represents a special classification label, and its final output serves as a classification feature; [SEP] represents a sentence separator. Then, feature extraction is performed, and dynamic word vectors are generated through a 12 / 24-layer Transformer encoder (BERT-base / large). The self-attention mechanism captures long-distance dependencies between words, and bidirectional encoding considers contextual information. The dynamic word vectors are then input into the BERT model, which outputs the corresponding latent intent. According to a preset mapping relationship, the latent intent model corresponding to each latent intent is obtained. After mapping to the intent category recognition space through a fully connected layer, the recognition result is obtained. The softmax function is then used to convert the recognition result into the corresponding probability distribution to obtain the probability distribution sequence corresponding to the latent intent model.

[0097] Since the dynamic word vectors are obtained by using the BERT model, and the recognition results are obtained by mapping them to the intent category recognition space through a fully connected layer, and the recognition results are converted into the corresponding probability distribution by the softmax function, all of these are implemented using the existing BERT model. This solution does not involve any improvement to the BERT model itself, and will not be elaborated here.

[0098] S400. Obtain the model score of the latent intent model, calculate the fusion score of each latent intent model based on the model score and the probability distribution sequence, and select the latent intent model corresponding to the highest fusion score as the reference model.

[0099] In some embodiments, obtaining the model score of the latent intent model and calculating the fusion score of each latent intent model based on the model score and the probability distribution sequence includes:

[0100] Based on preset score dimensions and preset weights, the model score of each latent intent model is calculated in the model library. The score dimensions include accuracy dimension, confidence dimension, and consistency dimension.

[0101] The model score and the probability distribution sequence are weighted to obtain the fusion score for each latent intent model.

[0102] Specifically, for each latent intent model matched in the model library, its model score corresponds to four dimensions: accuracy, confidence, and consistency. Accuracy reflects the historical prediction accuracy of the current model; confidence reflects the certainty of the current model's historical output; and consistency reflects the degree of collaboration between the current model and other models. These scores are recorded within the model and updated synchronously as the model is used, and will not be elaborated further here. Then, a weighted summation is performed on each latent intent model based on its model score and probability distribution sequence to obtain a corresponding fusion score. This facilitates the subsequent selection of the most suitable model as a reference model based on the fusion score, and allows for the selection of the model closest to the reference model for adaptation.

[0103] S500. Calculate the target similarity between each standard model in the model library and the reference model, select the standard model with the highest similarity as the target model, and complete the model adaptation verification. The model library includes multiple standard models of different types.

[0104] After selecting a reference model, the target similarity between each standard model in the model library and the reference model is calculated, and the standard model with the highest similarity is selected as the target model, thereby completing the model adaptation verification.

[0105] In other embodiments, calculating the target similarity between each of the standard models in the model library and the reference model includes:

[0106] Obtain the internal representation information and external behavior information of the reference model;

[0107] The internal similarity between each standard model and the reference model is calculated based on the internal representation information, and the external similarity between each standard model and the reference model is calculated based on the external behavior information.

[0108] Knowledge layer similarity is calculated by evaluating the consistency of the performance of the reference model and the standard model on the same knowledge exploration dataset.

[0109] The internal similarity, external similarity, and knowledge layer similarity are weighted and summed to obtain the target similarity between each standard model and the reference model.

[0110] Specifically, by inputting the same batch of text into both the standard and reference models, the hidden state vectors extracted at the last layer can serve as internal representation information. Typically, average pooling is applied to the sequence dimension, and the internal similarity is calculated using a predefined cosine similarity formula. External behavioral information, on the other hand, is the output result obtained after inputting the same task; the external similarity is obtained by calculating the similarity between the two outputs. Simultaneously, similarity is calculated by evaluating the consistency of the models' performance on the same knowledge probe set. For example, the sets of correct (or incorrect) answers for all questions in the standard and reference models can be represented as vectors (e.g., using TF-IDF or BERT embedding), and the similarity between the two sets can be calculated to obtain the knowledge layer similarity, such as the Jaccard similarity coefficient. The calculation of this coefficient is a matter of existing technology and will not be elaborated upon here.

[0111] Then, the internal similarity, the external similarity, and the knowledge layer similarity are weighted and summed according to preset weights to obtain the target similarity between each standard model and the reference model. This allows the standard model with the highest target similarity to be selected as the target model, thereby completing the target model adaptation work, selecting a suitable model for the user, and improving the user experience.

[0112] The scope of protection of the adaptation verification method for large language models described in this embodiment is not limited to the execution order of the steps listed in this embodiment. Any solution implemented by adding, subtracting, or replacing steps in the prior art based on the principles of this invention is included within the scope of protection of this invention.

[0113] This invention also discloses an adaptation verification system for large language models, with reference to... Figure 2 The system includes:

[0114] The information processing module 201 is used to acquire the corpus instruction information input by the user, and to generate standard instruction information after standardizing the corpus instruction information;

[0115] The extraction and generation module 202 is used to perform semantic segmentation on the standard instruction information and extract key elements, and generate a corresponding fragment element label sequence based on the key elements.

[0116] The classification module 203 is used to classify the segment element label sequence according to the pre-trained BERT model to output the latent intent model and the corresponding probability distribution sequence.

[0117] Selection module 204 is used to obtain the model score of the latent intent model, calculate the fusion score of each latent intent model based on the model score and the probability distribution sequence, and select the latent intent model corresponding to the highest fusion score as the reference model;

[0118] The adaptation module 205 is used to calculate the target similarity between each standard model in the model library and the reference model, select the standard model with the highest similarity as the target model, and complete the model adaptation verification. The model library includes multiple standard models of different types.

[0119] The adaptation verification system for large language models described herein can implement the adaptation verification method for large language models described in this invention. However, the implementation device for the adaptation verification method for large language models described in this invention includes, but is not limited to, the structure of the adaptation verification system for large language models listed in this embodiment. All structural modifications and substitutions of the prior art made in accordance with the principles of this invention are included within the protection scope of this invention.

[0120] Since the structure and principle of each module of the above system correspond one-to-one with the steps of the aforementioned method, they will not be repeated here.

[0121] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed, implements the above-described adaptation verification method for large language models.

[0122] This invention also provides a computer-readable storage medium storing a computer program that, when executed by an electronic device, implements the above-described adaptation verification method for large language models.

[0123] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing a processor, and the program can be stored in a computer-readable storage medium that is non-transitory. Transitory media, such as random access memory, read-only memory, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof. The aforementioned storage media can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. This available medium can be magnetic media (e.g., floppy disk, hard disk, magnetic tape), optical media (e.g., digital video disc (DVD)), or semiconductor media (e.g., solid-state disk (SSD)).

[0124] In the embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, or methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules / units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or units may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection of apparatuses or modules or units may be electrical, mechanical, or other forms.

[0125] The modules / units described as separate components may or may not be physically separate. The components shown as modules / units may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules / units can be selected to achieve the objectives of the embodiments of the present invention, depending on actual needs. For example, the functional modules / units in the various embodiments of the present invention may be integrated into one processing module, or each module / unit may exist physically separately, or two or more modules / units may be integrated into one module / unit.

[0126] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0127] The descriptions of the processes or structures corresponding to the above figures each have their own emphasis. For parts of a process or structure that are not described in detail, please refer to the relevant descriptions of other processes or structures.

[0128] The above embodiments are merely illustrative of the principles and effects of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in the present invention should still be covered by the claims of the present invention.

Claims

1. A method for fitting verification of large language models, characterized in that, The method includes: The system acquires user-input corpus instruction information and generates standard instruction information after standardizing the corpus instruction information. The standard instruction information is semantically segmented and key elements are extracted. A corresponding fragment element label sequence is generated based on the key elements. The fragment element label sequence is classified according to the pre-trained BERT model to output the latent intent model and the corresponding probability distribution sequence. Obtain the model score of the latent intent model, calculate the fusion score of each latent intent model based on the model score and the probability distribution sequence, and select the latent intent model with the highest fusion score as the reference model; Calculate the target similarity between each standard model in the model library and the reference model, select the standard model with the highest similarity as the target model, and complete the model adaptation verification. The model library includes multiple standard models of different types. The step of semantically segmenting the standard instruction information and extracting key elements, and generating corresponding fragment element label sequences based on the key elements, includes: After performing semantic boundary detection on the standard instruction information based on the attention mechanism, the semantic boundary is obtained. The standard instruction information is then split into multiple instruction fragments based on the semantic boundary, and the weights of the instruction fragments are calculated. The instruction fragment is subjected to element extraction according to the preset element type to obtain multiple fragment elements, and the comprehensive score of each fragment element is calculated. Based on the comprehensive score and the segment weight, corresponding segment element labels are generated, and the segment element labels are arranged together in order to form the segment element label sequence; The step of classifying the segment element label sequence according to the pre-trained BERT model to output a latent intent model and a corresponding probability distribution sequence includes: After converting the fragment element label sequence into the BERT standard input format, dynamic word vectors are generated by the Transformer encoder, and the dynamic word vectors are input into the BERT model to output the corresponding latent intent. The potential intent model corresponding to each potential intent is obtained according to the preset mapping relationship; After mapping the fully connected layer to the intent category recognition space, the recognition result is obtained, and the softmax function is used to convert the recognition result into the corresponding probability distribution to obtain the probability distribution sequence.

2. The adaptation verification method for large language models according to claim 1, characterized in that, The step of acquiring user-input corpus instruction information and generating standard instruction information after standardizing the corpus instruction information includes: The character encoding detector is activated to decompose the corpus instruction information to obtain multiple split characters; A standardization engine is used to process the split characters in a standardized manner to obtain a unified character representation, and a regular expression engine is used to extract features from the character representation to obtain basic character features; The basic character features are processed to calculate the corresponding importance score. Based on the importance score, interference information is removed to generate standard character features. The standard character features are then combined to generate the standard instruction information.

3. The adaptation verification method for large language models according to claim 2, characterized in that, The process of processing the basic character features to calculate the corresponding importance score includes: Calculate the fundamental frequency and inverse frequency of each of the basic character features; The frequency score of the basic character feature is calculated based on the basic frequency and the inverse frequency. Obtain each potential corpus category corresponding to the basic character feature in the corpus, and calculate the occurrence probability of the basic character feature in each potential corpus category through an intent classifier, and calculate the information entropy of each basic character feature based on the occurrence probability; The number of basic character features in each potential corpus category is counted to obtain a statistical count, and the corresponding verification score is calculated based on the statistical count. The importance score of each basic character feature is calculated based on the frequency score, the information entropy, and the verification score.

4. The adaptation verification method for large language models according to claim 3, characterized in that, The importance score is calculated by weighted summation of the frequency score, the information entropy, and the check score.

5. The adaptation verification method for large language models according to claim 1, characterized in that, The step of obtaining the model score of the latent intent model and calculating the fusion score of each latent intent model based on the model score and the probability distribution sequence includes: Based on preset score dimensions and preset weights, the model score of each latent intent model is calculated in the model library. The score dimensions include accuracy dimension, confidence dimension, and consistency dimension. The model score and the probability distribution sequence are weighted to obtain the fusion score for each latent intent model.

6. The adaptation verification method for large language models according to any one of claims 1 to 5, characterized in that, Calculating the target similarity between each of the standard models in the model library and the reference model includes: Obtain the internal representation information and external behavior information of the reference model; The internal similarity between each standard model and the reference model is calculated based on the internal representation information, and the external similarity between each standard model and the reference model is calculated based on the external behavior information. Knowledge layer similarity is calculated by evaluating the consistency of the performance of the reference model and the standard model on the same knowledge exploration dataset. The internal similarity, external similarity, and knowledge layer similarity are weighted and summed to obtain the target similarity between each standard model and the reference model.

7. An adaptation verification system for large language models, characterized in that, The system includes: The information processing module is used to acquire the corpus instruction information input by the user, and to generate standard instruction information after standardizing the corpus instruction information; The extraction and generation module is used to perform semantic segmentation on the standard instruction information and extract key elements, and generate a corresponding fragment element label sequence based on the key elements. The classification module is used to classify the segment element label sequence according to the pre-trained BERT model to output the latent intent model and the corresponding probability distribution sequence. The selection module is used to obtain the model score of the latent intent model, calculate the fusion score of each latent intent model based on the model score and the probability distribution sequence, and select the latent intent model corresponding to the highest fusion score as the reference model. The adaptation module is used to calculate the target similarity between each standard model in the model library and the reference model, select the standard model with the highest similarity as the target model, and complete the model adaptation verification. The model library includes multiple standard models of different types. The step of semantically segmenting the standard instruction information and extracting key elements, and generating corresponding fragment element label sequences based on the key elements, includes: After performing semantic boundary detection on the standard instruction information based on the attention mechanism, the semantic boundary is obtained. The standard instruction information is then split into multiple instruction fragments based on the semantic boundary, and the weights of the instruction fragments are calculated. The instruction fragment is subjected to element extraction according to the preset element type to obtain multiple fragment elements, and the comprehensive score of each fragment element is calculated. Based on the comprehensive score and the segment weight, corresponding segment element labels are generated, and the segment element labels are arranged together in order to form the segment element label sequence; The step of classifying the segment element label sequence according to the pre-trained BERT model to output a latent intent model and a corresponding probability distribution sequence includes: After converting the fragment element label sequence into the BERT standard input format, dynamic word vectors are generated by the Transformer encoder, and the dynamic word vectors are input into the BERT model to output the corresponding latent intent. The potential intent model corresponding to each potential intent is obtained according to the preset mapping relationship; After mapping the fully connected layer to the intent category recognition space, the recognition result is obtained, and the softmax function is used to convert the recognition result into the corresponding probability distribution to obtain the probability distribution sequence.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed, it implements the adaptation verification method for large language models as described in any one of claims 1 to 6.

Images