A multi-level value system construction method and system

Abstract

The application relates to the technical field of natural language generation, and discloses a multi-level value system construction method and system. The method comprises the following steps: constructing a first layer of a value system based on artificially set value anchor words; mining value units from a real-world corpus; generating candidate phrases based on a part-of-speech tagged word sequence, and training a phrase quality evaluator by using a distant supervision integrated training technology to screen the candidate phrases, so that the screened phrases are used as the value units; clustering the value units with similar semantics under the guidance of the value anchor words to form a third layer of the value system; and taking a generalization phrase induced based on the clustering result as a second layer of the value system, so that a multi-level value system is constructed. The method disclosed by the application has high automation, emphasizes multi-technology path fusion and real-world corpus driving, and the constructed value system is widely covered and rich in connotation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language generation, and in particular to a multi-level value system construction method and system. BACKGROUND

[0002] A value system with rich levels and profound connotations can play a key role in value dissemination, education and training, and large model alignment and other application scenarios.

[0003] Early research on value system construction methods mainly relied on expert manual induction. Although this method can ensure the accuracy of values, it is inefficient and requires regular manual maintenance, making it difficult to respond to social changes in real time. In recent years, some research has focused on using topic models and word embedding techniques to construct value systems. Latent Dirichlet Allocation (LDA) model is widely used in document topic mining, but its parallel topic structure is difficult to capture hierarchical semantic relationships (such as the hierarchy of abstract and concrete concepts). To overcome this limitation, Blei et al. proposed a hierarchical topic model (hLDA) that builds a tree-like topic structure in the nested process, enabling hierarchical topic mining. However, hLDA still relies on data-driven methods and lacks domain knowledge guidance, resulting in semantic bias in knowledge systems constructed in specific domains such as values. In terms of lexical semantic representation, static word embedding models such as Word2Vec and GloVe learn word vectors through context co-occurrence relationships, but they fail to fully model word order and direction information. Subsequent research such as Skip-gram variants attempts to introduce position vectors, but the description of complex semantic relationships is still insufficient. In addition, current clustering methods based on word embeddings (such as K-means) can generate class divisions, but the semantic consistency within classes and the distinction between classes are limited, making it difficult to support fine-grained knowledge system construction. SUMMARY

[0004] To solve the above technical problems, the application provides a multi-level value system construction method and system based on fine-grained value unit mining, which emphasizes multi-technology path fusion and real world corpus driving. Specifically, the application starts from a small number of value anchor words set by artificial, designs a fine-grained value unit mining module and a value unit clustering learning module, which are respectively used to construct the content of the third layer of connotation knowledge and the second layer of field subdivision of the value system. In the fine-grained value unit mining module, the application adopts a distant supervision integrated training technology, constructs multiple heterogeneous sub-models by introducing multiple weak supervision signals, and fuses the prediction results to alleviate the interference of single label noise, so as to obtain a robust phrase quality evaluator. The fine-grained value unit can be mined from a large amount of related value corpus as a candidate unit of the third layer of the value system. In the value unit clustering learning module, the application proposes an adaptive semantic clustering algorithm, which uses the semantic features of the fine-grained value unit for clustering learning, and summarizes the clustering results in a man-machine cooperative manner to form the second layer of the value system and the corresponding third layer content, so as to construct a multi-level value system with rich connotation and clear hierarchy.

[0005] To solve the above technical problems, the application adopts the following technical solutions:

[0006] In a first aspect, the application provides a multi-level value system construction method, comprising:

[0007] Constructing a first layer of the value system based on value anchor words set by artificial;

[0008] Mining value units from a real world corpus, wherein the mining of the value units comprises: generating candidate phrases based on part-of-speech annotated word sequences, and training a phrase quality evaluator using a distant supervision integrated training technology to screen the candidate phrases from the candidate phrases, and taking the screened phrases as value units;

[0009] Clustering value units with similar semantics under the guidance of the value anchor words to form a third layer of the value system: classifying according to the distance of the value unit representation to the cluster center, then calculating each centroid by the weighted sum of all value unit representations, and then dividing the value units into different classes through clustering loss; taking the summary phrases induced based on the clustering results as the second layer of the value system, thereby completing the construction of the multi-level value system.

[0010] In one of the embodiments, the generating of the candidate phrases based on the part-of-speech annotated word sequences specifically comprises:

[0011] Processing the real world corpus into part-of-speech annotated word sequences, dividing the phrase boundaries by calculating the part-of-speech quality score, and generating the candidate phrases.

[0012] In one of the embodiments, the processing of the real-world corpus into a part-of-speech tagged word sequence, the division of phrase boundaries by calculating part-of-speech quality scores, and the generation of candidate phrases specifically include:

[0013] Let the corpus be processed into a part-of-speech tagged word sequence of length ; denote the pair of the i-th word and the part-of-speech tag corresponding to the i-th word ;

[0014] denote the sequence of part-of-speech tags of all the part-of-speech tags , denote any sub-sequence of consecutive part-of-speech tags in , denoted as ; the part-of-speech quality score of the sub-sequence of tags is defined as the conditional probability that the corresponding word sequence is a complete semantic unit, in the form of

[0015] ;

[0016] When is greater than a set threshold, the corresponding word sequence is taken as a candidate phrase; is the conditional probability used to mark the boundaries of the word sequence.

[0017] In one of the embodiments, the part-of-speech quality score is specifically calculated in the following manner:

[0018] ;

[0019] wherein denotes the part-of-speech tag of the word at the beginning of the r-th phrase, denotes the part-of-speech tag of the word preceding the word at the beginning of the r-th phrase, denotes the probability that is adjacent to in the phrase of the corpus and precedes denotes the probability that is adjacent to in the phrase of the corpus and precedes denote the part-of-speech tags corresponding to the j-1-th word and the j-th word, respectively.

[0020] In one of the embodiments, is defined as: ​

[0021] ;

[0022] ;

[0023] denotes any pair of adjacent words in the corpus; function for generating the part-of-speech tag of the input word.

[0024] In one of the embodiments, the phrase quality evaluator is trained by using the distant supervision ensemble training technique to screen the candidate phrases, and the screened phrases are taken as the value units, which specifically comprises:

[0025] The keywords and phrases with a frequency exceeding a preset value in the public knowledge base are taken as weak supervision signals to construct a positive phrase sample pool; the corpus is processed by using an n-gram method to generate a phrase set, and the phrases in the phrase set not appearing in the public knowledge base are classified into a negative phrase sample pool;

[0026] The phrase samples are randomly extracted from the positive phrase sample pool and the negative phrase sample pool to construct a perturbation training set, and an unpruned decision tree is used as an initial heterogeneous sub-model; a plurality of independently trained heterogeneous sub-models are integrated to obtain the phrase quality evaluator ;

[0027] A candidate phrase is input into the phrase quality evaluator , the number of the heterogeneous sub-models predicting the candidate phrase as a positive phrase sample in the is taken as the quality score of the candidate phrase :

[0028] ;

[0029] For the same candidate phrase, the set of the heterogeneous sub-models predicting the candidate phrase as a positive phrase sample in the , For the same candidate phrase, the set of the heterogeneous sub-models predicting the candidate phrase as a negative phrase sample in the ;

[0030] All the candidate phrases satisfying are taken as the value units, wherein is a preset phrase quality threshold.

[0031] In one of the embodiments, the phrase quality evaluator contains Y heterogeneous sub-models, and the prediction error of the phrase quality evaluator is expressed as:

[0032] ;

[0033] in, This represents the empirical error of the heteroproton model. Represents the number of combinations. As an intermediate variable;

[0034] The optimal number of heteroproton models in the phrase quality evaluator is calculated using the prediction error.

[0035] In one embodiment, the clustering of value units with similar semantics to form the third layer of the value system involves: classifying value units based on their distance from the cluster center, calculating each centroid using a weighted sum of all value unit representations, and then dividing the value units into different classes using a clustering loss. Specifically, this includes:

[0036] For a batch Each value unit Encoding as a value unit representation The set representing value units The set represented by the cluster center As input to the clustering module, the distance matrix between the value unit representation and the center representation is obtained. ;

[0037] According to the distance matrix To classify the representation of value units:

[0038] ;

[0039] in, The representation of the k-th cluster center is initialized as a random vector; Represents the value unit representation The degree to which it does not belong to every category This refers to the argmin function. This represents the value unit representation. The index of the class to which it was assigned;

[0040] Clustering loss for: ; For the first Representation of each cluster center;

[0041] Represents Euclidean distance; These are the model parameters that the clustering module can learn;

[0042] The weights of all value unit representations are obtained from the distance matrix, and the cluster center representations are updated based on the weighted sum of the value unit representations.

[0043] ;

[0044] is the number of field sub-divisions under each value anchor word;

[0045] When the training of the clustering module is completed, the distance matrix between the value unit representations and the center representations is calculated , and the clustering results of each value unit are obtained.

[0046] In one embodiment, the distance matrix between the value unit representations and the center representations is The calculation method is as follows:

[0047] ;

[0048] ;

[0049] The element in the distance matrix between the value unit representations and the center representations is the normalized distance between the i-th cluster center representation and the j-th value unit representation .

[0050] In a second aspect, the present application provides a computer system comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the method of any one of the embodiments of the first aspect when executing the computer program.

[0051] Compared with the prior art, the present application has the beneficial technical effects that:

[0052] In the fine-grained value unit mining module, the present application uses a distant supervision integrated training technology, constructs multiple heterogeneous sub-models by introducing multiple weak supervision signals, and fuses the prediction results to alleviate the interference of single label noise, thereby obtaining a robust phrase quality evaluator. In the value unit clustering learning module, the present application proposes an adaptive semantic clustering algorithm, which uses the semantic features of fine-grained value units for clustering learning, avoids the dependence of traditional co-occurrence clustering methods on corpus quality, and summarizes the clustering results in a human-computer collaborative manner to form the second layer of the value system and the corresponding third layer of fine-grained value unit content. Unlike the value system construction technology that highly depends on artificial expert writing in the past, the method proposed by the present application has high automation, emphasizes the fusion of multiple technical paths and real-world corpus driving, and constructs a value system that is widely covered and rich in connotation, which is suitable for various application scenarios such as value dissemination, education and training, and large model alignment. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 ​​​A method flowchart in an embodiment of the present application;

[0054] Figure 2 An overall architecture schematic diagram in an embodiment of the present application. DETAILED DESCRIPTION

[0055] A preferred embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0056] As shown in the drawings, Figure 1 A multi-level value system construction method of the present application includes the following steps:

[0057] S1, constructing a first layer of the value system based on artificially set value anchor words;

[0058] S2, mining value units from a real-world corpus, wherein the mining of value units includes: generating candidate phrases based on a part-of-speech tagged word sequence, and training a phrase quality evaluator using a distant supervision integrated training technique to screen the candidate phrases from the candidate phrases, and taking the screened phrases as value units;

[0059] S3, clustering value units of similar semantics under the guidance of value anchor words to form a third layer of the value system: classifying according to the distance of value unit representation to the cluster center, then calculating each centroid through the weighted sum of all value unit representations, and further dividing the value units into different classes through clustering loss; taking the summary phrases induced based on the clustering results as the second layer of the value system, thereby constructing a multi-level value system.

[0060] The present application takes artificially set highly concise value anchor words as the first layer of the value system, further mines corresponding value connotations from real-world corpus as value units of the third layer of the value system, and clusters and learns these value connotations into more concise summary phrases as the second layer of the value system, thereby constructing a multi-level value system with rich connotations and distinct levels.

[0061] The overall architecture of the present application is shown in the drawings, Figure 2 Mainly includes a fine-grained value unit mining module and a value unit clustering learning module. The fine-grained value unit mining module uses the distant supervision integrated training technique to efficiently and robustly extract high-quality phrases in the text as value units. The value unit clustering learning module further clusters semantically related value units as the third layer of the value system under the guidance of value anchor words, and induces more concise summary phrases for each category of the third layer as the second layer of the value system.

[0062] 1. Fine-grained value unit mining module.

[0063] The goal of this module is to automatically extract fine-grained value units from massive real-world corpus, which are defined as phrases that may embody the connotation of values hidden in the corpus and are important components of the third layer of value system. For the above task, this module first generates candidate phrases based on part-of-speech tagging, then trains a phrase quality evaluator using distant supervision ensemble learning technology, which can effectively reduce the training cost and training error of the phrase quality evaluator. Finally, the trained phrase quality evaluator is used to efficiently and robustly filter high-quality phrases from candidate phrases as fine-grained value units. Specifically, the fine-grained value unit mining module mainly includes the following contents:

[0064] (1) Part-of-speech guided candidate phrase generation:

[0065] Part-of-speech tags provide shallow, language-specific knowledge that can help improve the accuracy of candidate phrase generation, especially at the syntactic component boundaries of the language. Based on the part-of-speech-based candidate phrase division, the final value unit can be more complete.

[0066] Let the corpus be processed into a part-of-speech tagged word sequence of length , denoted as , where denotes a pair consisting of a word and its part-of-speech tag . Part-of-speech guided phrase segmentation is to divide this sequence into segments by a phrase boundary index sequence , satisfying . The th segment refers to .

[0067] Given the part-of-speech tag sequence of the entire corpus of length , denoted as , the label subsequence (for clarity, denoted as ) is defined as the conditional probability that its corresponding word sequence is a complete semantic unit. Formally:

[0068] .

[0069] The part-of-speech quality score aims to reward phrases that correctly identify the part-of-speech sequence. The specific form of is:

[0070] ;

[0071] where is the part-of-speech tag The phrase in the corpus in which the word is located The probability of being adjacent and preceding. In this formula, the first term represents the probability of there being a phrase boundary between the words at indices and while the latter product represents the probability of all the part-of-speech tags in being in the same phrase. This part-of-speech quality score naturally penalizes longer segments, as for all , regardless of how the corpus is segmented, and exactly one of

[0072] Mathematically, is defined as:

[0073] ;

[0074] As it depends on how the document is segmented into phrases, is initialized uniformly and will be learned during the phrase segmentation process.

[0075] (2) Phrase quality evaluator based on distant supervision ensemble training:

[0076] In the construction of the training set, the present application adopts a distant supervision training method to construct a large amount of training data at a very low cost. Public knowledge bases (such as Wikipedia, policy and regulation knowledge bases, etc.) are widely recognized as a source of high-quality information due to their authoritative and accurate content. Therefore, the present application regards the frequently occurring keywords and phrases in the knowledge base as high-quality phrases, and uses them as weak supervision signals to construct a positive phrase sample pool. For the negative phrase sample pool, the present application generates a phrase set from the corpus by the n-gram method, wherein the phrases not appearing in the knowledge base are classified into the negative phrase sample pool. Although this method avoids manual annotation, it brings the problem of noise, i.e. some high-quality phrases may be incorrectly classified into the negative phrase sample pool due to not being included.

[0077] To reduce the impact of noise, the present application adopts an ensemble learning method to train the phrase quality evaluator. First, the method randomly extracts phrase samples from the positive phrase sample pool and the negative phrase sample pool to construct a perturbed training set, and uses an unpruned decision tree as an initial heterogeneous model to ensure complete fitting of the phrase samples. Then, the results of multiple independently trained heterogeneous models are integrated to obtain a robust phrase quality evaluator . The present application takes the proportion of results predicted as positive in all heterogeneous models as the quality score of the candidate phrase :

[0078] ;

[0079] Phrase quality evaluator The prediction error of the Y heterogeneous models (decision trees) can be expressed as:

[0080]

[0081] wherein, is the empirical error of the heterogeneous model, represents the combination number. The optimal number of heterogeneous model construction can be calculated by the above formula.

[0082] Through the above method, the module first uses the part-of-speech quality score of the label sub-sequence to mine a large number of candidate phrases with strong integrity from the corpus, then uses a robust phrase quality evaluator to screen out high-quality phrases and use them as the third layer of the value system.

[0083] 2. Value unit clustering learning module.

[0084] The goal of this module is to cluster similar value units in the semantic space and summarize more concise summary phrases as the second layer of the value system. The present application uses a two-layer fully connected neural network as the clustering module and designs a clustering algorithm to cluster value units with similar semantics. The clustering algorithm classifies value units according to the distance from the cluster center (centroid), then calculates each centroid by the weighted sum of all value unit representations, and finally designs a clustering loss to optimize the clustering network to divide value units into different classes.

[0085] In detail, for a batch of value units, the present application uses a pre-trained language model (such as the BERT model) to encode each value unit to obtain a d-dimensional value unit representation , and the clustering module takes the set of value unit representations and the set of cluster center representations as input, where is the number of domain subdivisions under each value anchor word, is a randomly initialized vector. Then the present application can obtain the distance matrix between the value unit representation and the cluster center representation:

[0086]

[0087]

[0088] Herein​​​​ It is the first Each cluster center represents With the Each value unit represents The normalized distance between them represents the value unit representation. Not belonging to the first The degree of class; These are learnable clustering module parameters. It is Euclidean distance.

[0089] Next, the present invention can be based on the distance matrix To classify the representation of value units:

[0090] .

[0091] in Represents the value unit representation The degree to which it does not belong to every category The function represents the value unit based on the minimum distance. It is then assigned to the appropriate class. Therefore, This indicates that the value unit represents the class index to which it is divided.

[0092] To train clustering modules to obtain the optimal clustering paradigm, this invention proposes a clustering loss to minimize the distance between value unit representations and their corresponding cluster centers. The goal of the clustering loss is to find an optimal set of clustering module parameters. This allows for the calculation of optimal cluster centers, such that after a value unit representation passes through this clustering module, its distance to its corresponding cluster center is smaller, while its distance to other cluster centers is larger. Clustering loss. As shown below:

[0093] .

[0094] Subsequently, this invention obtains the weights of all value unit representations based on the distance matrix, and updates the cluster center representations based on the weighted sum of the value unit representations. The update formula is as follows:

[0095] .

[0096] After the clustering module is fully trained, it is calculated... This yields the clustering results for each value unit, which constitutes the third layer of the value system. Furthermore, this invention utilizes an advanced large language model to summarize the general phrases for each category in the third layer, serving as the content for the second layer, the domain classification layer, thereby constructing a rich and multi-layered value system.

[0097] The application can be applied to a large language model training platform, automatically constructs a deep value system that matches the value of contemporary society, and lays a foundation for value alignment of the large language model. In the implementation level, it can be embedded in an intelligent system as software, efficiently integrated into various application scenarios, and comprehensively improve the value orientation and decision quality of the intelligent system.

[0098] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. As used herein, the terms "comprises", "comprising", "includes", "including" and the like are specifically intended to be open-ended and to mean that other features, steps, operations, and / or components can be added.

[0099] It should be understood that although the steps in the flowcharts of the specification drawings are shown in sequence according to the direction of the arrows, these steps are not necessarily executed in sequence according to the direction of the arrows. Unless otherwise specified herein, the execution of these steps is not strictly limited in sequence, and these steps can be executed in other sequences. Moreover, at least part of the steps in the flowcharts of the specification drawings can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution sequence of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0100] In one of the embodiments, the present application further provides a computer system, which can be a server. The computer system comprises a processor, a memory and a network interface connected through a system bus. The processor of the computer system is used to provide computing and control capabilities. The memory of the computer system comprises a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer system is used to store data used in the above method. The network interface of the computer system is used to communicate with external terminals through network connection. The computer program is executed by the processor to implement the above method.

[0101] The technical features of the above embodiments can be combined in any manner. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present application.

[0102] It will be obvious to a person skilled in the art that the application is not limited to the details of the foregoing exemplary embodiments and can be implemented in other concrete forms without departing from the spirit or essential characteristics of the application. The embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the application being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein and no

[0103] Furthermore, it should be understood that although the description is made on the basis of embodiments, not every embodiment contains only one independent technical solution, and the description is made in this way only for the sake of clarity, and a person skilled in the art should consider the description as a whole, and the technical solutions in each embodiment can also be combined appropriately to form other embodiments that can be understood by a person skilled in the art.

Claims

1. A multi-level value system construction method characterized by, The method comprises the following steps: a first layer of the value system is constructed based on a value anchor word set artificially; value units are mined from a real-world corpus, wherein the mining of the value units comprises: generating candidate phrases based on a part-of-speech tagged word sequence, and training a phrase quality evaluator by using a distant supervision integrated training technique to screen the candidate phrases, and taking the screened phrases as the value units; similar semantic value units are clustered under the guidance of the value anchor words to form a third layer of the value system: the value units are classified according to the distances of the value unit representations to the cluster centers, then each centroid is calculated through the weighted sum of all value unit representations, and the value units are divided into different classes through a clustering loss, and a generalization phrase based on the clustering result is taken as the second layer of the value system, thereby completing the construction of the multi-level value system.

2. The method according to claim 1, wherein, The generating of the candidate phrases based on the part-of-speech tagged word sequence comprises: processing the real-world corpus into the part-of-speech tagged word sequence, dividing the phrase boundaries through the calculation of a part-of-speech quality score, and generating the candidate phrases.

3. The method of claim 2, wherein, The processing of the real-world corpus into the part-of-speech tagged word sequence, the dividing of the phrase boundaries through the calculation of the part-of-speech quality score, and the generating of the candidate phrases comprise: A corpus is processed into a sequence of morphologically tagged words of length ; ; denotes the i-th word and the pair consisting of the i-th word and its corresponding morphological tag ; a sequence of part-of-speech tags , any of the tag subsequences consisting of consecutive part-of-speech tags , denoted by ; a part-of-speech quality score of the tag subsequence, defined as the conditional probability that the corresponding word sequence is a complete semantic unit, of the form: ； When greater than a set threshold, the corresponding sequence of words as a candidate phrase; conditional probability, for marking boundaries of sequences of words.

4. The method of claim 3, wherein, The part-of-speech quality score is calculated in the following manner: ； wherein, denotes the part-of-speech tag of the word at the beginning of the rth phrase, denotes the part-of-speech tag of the word preceding the word at the beginning of the rth phrase, denotes the probability that the word preceding the word at the beginning of the rth phrase is adjacent to and precedes the word at the beginning of the rth phrase in phrases of the corpus, denotes the probability that the word preceding the word at the beginning of the rth phrase is adjacent to and precedes the word at the beginning of the rth phrase in phrases of the corpus, denote the part-of-speech tags corresponding to the (j-1)th word and the jth word, respectively.

5. The method of claim 4, wherein, is defined as: ； ； denotes any pair of adjacent words in the corpus; function for generating a part-of-speech tag for the input word.

6. The method of claim 1, wherein, The training of the phrase quality evaluator by using the distant supervision integrated training technique to screen the candidate phrases, and taking the screened phrases as the value units comprises: taking the keywords and phrases with a frequency exceeding a set value in a public knowledge base as weak supervision signals to construct a positive phrase sample pool, processing the corpus by using an n-gram method to generate a phrase set, and taking the phrases in the phrase set that do not appear in the public knowledge base into a negative phrase sample pool; The phrase samples are randomly stratified from the positive phrase sample pool and the negative phrase sample pool to construct a disturbance training set, and an unpruned decision tree is used as an initial heterogeneous sub-model; multiple independently trained heterogeneous sub-models are integrated to obtain a phrase quality evaluator ; inputting a candidate phrase to a phrase quality evaluator , the proportion of the number of heterogeneous sub-models that predict the candidate phrase as a positive phrase sample among all heterogeneous sub-models as a quality score of the candidate phrase : ； a set of heterogeneous sub-models that predict the same candidate phrase as a positive phrase sample, a set of heterogeneous sub-models that predict the same candidate phrase as a positive phrase sample, a set of heterogeneous sub-models that predict the same candidate phrase as a positive phrase sample, a set of heterogeneous sub-models that predict the same candidate phrase as a positive phrase sample, All candidate phrases that satisfy are taken as value units, wherein, is a preset phrase quality threshold.

7. The method of claim 6, wherein, Phrase quality evaluator Including Y heterogeneous models, the prediction error of the phrase quality evaluator Is represented as: ； wherein is the empirical error for the heteronuclear model, denotes the number of combinations, is an intermediate variable; the optimal number of heterogeneous sub-models in the phrase quality evaluator is calculated through a prediction error.

8. The method of claim 1, wherein, The clustering of the similar semantic value units to form the third layer of the value system: the value units are classified according to the distances of the value unit representations to the cluster centers, then each centroid is calculated through the weighted sum of all value unit representations, and the value units are divided into different classes through a clustering loss, comprises: For a batch of value units, each value unit is encoded as a value unit representation A set of value unit representations is obtained As input to a clustering module, a distance matrix between value unit representations and center representations is obtained ; According to the distance matrix the value unit representations are classified: ； wherein, is a representation of the k-th cluster center, initialized as a random vector; represents a value unit representation does not belong to each class, represents an argmin function, represents a value unit representation an index of the class into which it is partitioned; cluster loss is: ; is a representation of the th cluster center; denotes the Euclidean distance; are model parameters learnable by the clustering module; the weights of all value unit representations are obtained according to a distance matrix, and the cluster center representation is updated according to the weighted sum of the value unit representations: ； is the number of field segments under each value anchor word; When the training of the clustering module is completed, the clustering result of each value unit is obtained by calculating that is, the clustering result of each value unit is obtained.

9. The method of claim 8, wherein, a distance matrix between the value unit representations and the center representation is calculated as ； ； elements in is the th cluster center representation normalized distance between the th value unit representation th value unit representation 10. A computer system comprising a memory and a processor, said memory storing a computer program, characterized in that, The processor executes the computer program to realize the steps of the method in any one of claims 1 to 9.

Images