Explainable text classification method based on large model concept generation

Abstract

The application provides an interpretable text classification method based on large model concept generation, and relates to the field of natural language processing. In the training stage, first, a stable and interpretable concept feature and its corresponding dimension are extracted and labeled for each sample by a large model to construct a task-aware concept system; then each labeled sample is encoded for prediction modeling to obtain a high-interpretable text classification model. In the reasoning stage, first, an experienced sample is selected for a new sample based on a screening strategy combining double semantic consistency and sample diversity enhancement; then the concept and its corresponding dimension of the new sample are labeled by a large model using the task-aware concept system and the experienced sample, and the labeling result is fed back to the text classification model to complete the final prediction. The application improves the processing efficiency and prediction accuracy of the text classification task, enhances the transparency, stability and business controllability of the classification model output, and meets the demand of application scenarios with high requirements for interpretability.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of natural language processing, in particular to an interpretable text classification method based on large model concept generation. BACKGROUND

[0002] Text classification is one of the most fundamental and important tasks in natural language processing, which aims to automatically classify a given text document into one or more predefined categories.

[0003] Currently, the text classification method based on semantic embedding vectors mainly converts the original text into semantic vector representation and uses it as the input feature of the classifier to complete the classification task. Although this method improves the semantic modeling ability of the text to some extent, its core defect is poor interpretability, with typical "black box" properties. As an abstract mathematical representation, the semantic vector is difficult to intuitively reflect the causal relationship between the key semantics in the text and the classification result, and users cannot understand "why the text is judged as this category". Especially in scenarios that require clear decision basis, the lack of perceptible and traceable semantic chain limits the application range of this method. In addition, semantic embedding is sensitive to context and expression, and the difference in vectors under different contexts is difficult to control, further weakening its explanation consistency and stability.

[0004] Another method attempts to assist text classification through topic modeling (such as Latent Dirichlet Allocation LDA) to enhance the interpretability of the classification process. This method extracts the topic distribution of the text and uses the topic as an intermediate variable to guide the semantic modeling and category determination of the text, which improves the interpretability of the model to some extent. However, topic modeling methods have the problems of coarse topic granularity and unstable semantics, and the generated topics often cannot finely describe complex concept semantics. In addition, this method relies on pre-set topic numbers and manually set bag-of-words models, lacks flexibility and scalability, and is difficult to dynamically adapt to the semantic evolution needs of new fields. Therefore, its adaptability and classification effect in complex business scenarios are limited.

[0005] In summary, existing technologies either lack interpretability or have limited classification ability, making it difficult to balance high performance and interpretability. SUMMARY

[0006] (I) Technical problems solved

[0007] To solve the technical problems of the prior art, the present application provides an interpretable text classification method based on large model concept generation, which solves the technical problem of how to balance high performance and interpretability.

[0008] (II) Technical solutions

[0009] To achieve the above object, the present application is implemented by the following technical solutions:

[0010] An explainable text classification method based on large model concept generation, comprising:

[0011] Based on the structured reasoning of the large model, the concept and its corresponding dimension of each sample in the training set are labeled to construct a task-aware concept system; wherein the training set contains a plurality of documents and their classification labels, and each document corresponds to a sample;

[0012] The labeled results of each sample are encoded, combined with the classification labels, and the text classification model is trained until convergence;

[0013] Based on the screening strategy combined with double semantic consistency and sample diversity enhancement, a plurality of labeled samples are selected from the training set as experience samples of new samples;

[0014] The task-aware concept system, the new sample and the experience sample are jointly used as the input of the large model, the concept and its corresponding dimension of the new sample are labeled, and after encoding, they are used as the input of the converged text classification model to obtain the text classification result of the new sample.

[0015] Preferably, the structured reasoning based on the large model labels the concept and its corresponding dimension of each sample in the training set to construct a task-aware concept system, comprising:

[0016] Divide each sample by sentence and obtain a summary through the large model to abstract concepts with dimension information from the summary;

[0017] After traversing all samples, remove redundant concepts through the large model;

[0018] Hierarchize the concepts after removing redundancy through the large model to obtain a hierarchical concept card composed of parent concepts and child concepts; wherein the parent concept contains a plurality of child concepts, and the child concept corresponds one-to-one to the concept after removing redundancy;

[0019] Select samples with different types of classification labels from the training set in equal amounts, and use the selected samples and the hierarchical concept card as the input of the large model to label the corresponding dimension of the concept of each sample;

[0020] According to the frequency requirement, unify the concept and its corresponding dimension of all samples to simplify and construct the task-aware concept system.

[0021] Preferably, the removing redundant concepts through the large model comprises:

[0022] The encoder calculates semantic similarity of names and descriptions of different concepts in pairs to filter out concepts with similarity greater than a threshold and put them into a merging candidate set;

[0023] The large model merges the same or similar concepts in the merging candidate set to remove the redundant concepts.

[0024] Preferably, the text classification model is a binary classification model or a multi-classification model.

[0025] Preferably, the process of obtaining the dual semantic consistency is as follows:

[0026] Obtain the embedding vectors of each parent concept containing several child concepts in the task-aware concept system, each labeled sample in the training set, and the new sample;

[0027] Calculate the first similarity between the embedding vectors of each labeled sample and the new sample;

[0028] Calculate the second similarity between each labeled sample and each child concept contained in the parent concept;

[0029] Fuse the first similarity and the second similarity to obtain the relevance score of each labeled sample to represent the dual semantic consistency.

[0030] Preferably, the screening strategy combined with dual semantic consistency and sample diversity enhancement includes:

[0031] Select a preset number of labeled samples from the training set at a time, and obtain multiple training subsets after repeating the operation;

[0032] Based on the relevance score of any two labeled samples in the current training subset and the similarity kernel between them, obtain a relevance weighted kernel;

[0033] Based on the relevance weighted kernel, construct a conditional determinant point process kernel matrix of the current training subset;

[0034] Based on the conditional determinant point process kernel matrix, use the matrix determinant identity to derive the first log probability of the current training subset;

[0035] Based on the adjustable hyperparameter, perform exponential transformation on the relevance score in the first log probability to obtain a second log probability, which is used to represent the dual semantic consistency of each labeled sample in the training subset and the diversity between them.

[0036] Preferably, the second log probability is expressed as:

[0037]

[0038] wherein log is a logarithm function; is a determinant of a matrix; e is a mathematical constant; is a determinant point process kernel matrix indexed by the training subset S, is a conditional determinant point process kernel matrix after applying an exponential transformation; is a relevance score of the i th labeled sample relative to the new sample; is an adjustable hyperparameter.

[0039] An interpretable text classification system based on large model concept generation, comprising:

[0040] a concept generation module configured to label the concept and its corresponding dimension of each sample in a training set based on structured reasoning of a large model, to construct a task-aware concept system; wherein the training set comprises a plurality of documents and their classification labels, and each document corresponds to a sample;

[0041] a training module configured to encode the labeled results of each sample, combine the classification labels, and train a text classification model until convergence;

[0042] a concept labeling module configured to select a plurality of labeled samples from the training set as experience samples of a new sample based on a screening strategy combining dual semantic consistency and sample diversity enhancement;

[0043] an interpretable prediction module configured to jointly input the task-aware concept system, the new sample and the experience samples into the large model as inputs, label the concept and its corresponding dimension of the new sample, and encode the new sample as an input of the converged text classification model to obtain a text classification result of the new sample.

[0044] A storage medium storing a computer program for interpretable text classification based on large model concept generation, wherein the computer program causes a computer to execute the interpretable text classification method as described above.

[0045] An electronic device, comprising:

[0046] one or more processors; memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs comprise a program for executing the interpretable text classification method as described above.

[0047] (Three) beneficial effects

[0048] The present application provides an interpretable text classification method based on large model concept generation. Compared with the prior art, the following beneficial effects are achieved:

[0049] In the present application: in the training stage, first, the stable and interpretable concept features and their corresponding dimensions are extracted and labeled for each sample by a large model to construct a task-aware concept system; then each labeled sample is encoded for prediction modeling to obtain a high-interpretable text classification model. In the reasoning stage, first, based on the screening strategy combining double semantic consistency and sample diversity enhancement, a number of labeled samples are screened and used as experience samples of new samples; then the constructed task-aware concept system and the screened experience samples are used to label the concepts and their corresponding dimensions of the new samples by the large model, and then the labeling results are fed back to the text classification model to complete the final prediction, thereby forming a closed loop. The present application not only improves the processing efficiency and prediction accuracy of the text classification task, but also enhances the transparency, stability and business controllability of the classification model output, meeting the needs of application scenarios with high requirements for interpretability. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor.

[0051] Figure 1 A block diagram of an interpretable text classification method based on large model concept generation provided by the embodiment of the present application;

[0052] Figure 2 A technical flowchart of an experience playback labeling method with consistency and diversity double constraints provided by the embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical scheme and advantages of the embodiments of the present application more clear, the technical scheme in the embodiments of the present application is described clearly and completely. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0054] The embodiments of the present application provide an interpretable text classification method based on large model concept generation, which solves the technical problem of how to balance high performance and interpretability, and is suitable for various text semantic discrimination scenes such as news, finance, medical treatment and law, and has strong interpretability.

[0055] The technical scheme in the embodiments of the present application is to solve the above technical problems, and the general idea is as follows:

[0056] (1) Overall closed-loop coupling processing flow:

[0057] One of the core technical improvements of the embodiments of the present application is to design the text intelligent classification task as an integrated closed-loop system tightly coupled with three modules of "concept card construction, interpretable prediction, and enhanced concept labeling". Compared with the traditional stage-by-stage processing method, the closed-loop architecture fully utilizes the synergistic effect of concept knowledge driving and model prediction capability through dynamic interaction between the front and rear modules: in the training stage, first, a large model is used to extract and label stable and interpretable concept features and their corresponding dimensions for each sample to construct a task-aware concept system; then, each labeled sample is encoded for prediction modeling to obtain a highly interpretable text classification model. In the reasoning stage, first, based on the screening strategy combining double semantic consistency and sample diversity enhancement, a number of labeled samples are screened and used as experience samples for new samples; then, using the constructed task-aware concept system and the screened experience samples, the large model is used to label the concepts and their corresponding dimensions of the new samples, and then the labeling results are fed back to the text classification model to complete the final prediction, thereby forming a closed loop.

[0058] (2) Innovative concept merging strategy of "encoder-driven + large model assisted + manually adjustable":

[0059] The embodiments of the present application innovatively propose a fusion strategy of encoder-driven, large model-assisted, and manually adjustable threshold in the concept merging stage. The strategy first calculates the semantic similarity between candidate concepts using a semantic encoder, then sets a manually adjustable threshold to screen similar concept groups according to specific business scenarios, and finally inputs the concept groups into a large model for concept merging. This strategy effectively combines the semantic accuracy of the encoder, the language abstraction ability of the large model, and the flexibility of manual rules, overcoming the uncontrollability of traditional large models and the locality problem of encoders, significantly improving the quality and global consistency of concept merging, and enhancing the stability and reuse value of the final concept card.

[0060] (3) Labeled sample screening strategy of "double semantic consistency + diversity enhancement":

[0061] The embodiment of the application proposes a labeled text screening strategy combining "double semantic consistency" and "sample diversity enhancement" in the reasoning stage, which is used to construct a high-quality enhanced prompt input large model. Specifically, the strategy measures: (1) the overall semantic similarity between new samples and candidate labeled texts, and (2) the semantic similarity between candidate labeled texts and sub-concepts in the task-aware concept system, to ensure that the selected labeled samples have high semantic consistency with the current task; On this basis, the sample diversity index is introduced, and the mutual exclusivity between the selected samples is controlled by a weighted method to avoid selecting redundant semantic samples. The strategy can effectively improve the generalization ability and stability of the large model in the concept annotation task, and enhance the precision and consistency of the reasoning result.

[0062] (4) Support for knowledge reuse and incremental evolution capability:

[0063] The embodiment of the application relies on a (hierarchical) concept card as the core carrier to build the system, so that the entire process has natural knowledge reuse and incremental evolution capability. When new data or concepts are added, the concept card and the annotation module can be updated locally, without the need for full retraining to realize system iteration and capability expansion. This capability greatly reduces the system maintenance cost, improves the flexibility of the model to adapt to different tasks and time periods, and supports long-term stable application.

[0064] In order to better understand the above technical solutions, the above technical solutions will be described in detail below in conjunction with the drawings in the specification and specific embodiments.

[0065] Embodiment 1:

[0066] As shown in Figure 1 , the embodiment of the application provides an interpretable text classification method based on large model concept generation, which includes:

[0067] S1, structured reasoning based on a large model, annotating the concepts and their corresponding dimensions of each sample in the training set to construct a task-aware concept system; wherein the training set contains a plurality of documents and their classification labels, and each document corresponds to a sample;

[0068] S2, encode the annotation results of each sample, combine the classification labels, and train a text classification model until convergence;

[0069] S3, based on the screening strategy combining double semantic consistency and sample diversity enhancement, select a plurality of labeled samples from the training set as experience samples for new samples;

[0070] S4, taking the task-aware concept system, the new sample and the experience sample as inputs of the large model, labeling the concept of the new sample and the corresponding dimension, and taking the encoded new sample as an input of a converged text classification model to obtain a text classification result of the new sample.

[0071] The embodiment of the application not only improves the processing efficiency and prediction accuracy of the text classification task, but also enhances the transparency, stability and business controllability of the classification model output, meeting the demand of application scenarios with high requirement for explainability.

[0072] Next, the above scheme will be described in detail:

[0073] In step S1, based on the structured reasoning of the large model, the concept of each sample in the training set and the corresponding dimension are labeled to construct a task-aware concept system; wherein the training set contains a plurality of documents and their classification labels, and each document corresponds to a sample.

[0074] In an optional embodiment, the step includes:

[0075] S11, dividing each sample according to a sentence, and obtaining a summary through the large model to abstract a concept with dimension information from the summary.

[0076] The obtained summary can be input into the large model, each summary corresponds to a number, and the large model is guided to cluster summaries with similar semantics, and a concise concept is abstracted for the summaries in each group. Then for each concept, the following four components are used to define its profile: ① concept name: a phrase composed of two to four words. ② Concept description: concise definition of the concept, avoid referencing superordinate terms, and try to reduce dependence on other concepts. ③ Concept dimension: each concept is limited to two dimensions to ensure clarity and binary classification. ④ Summary ID: list the IDs of all summaries belonging to the concept.

[0077] S12, after traversing all samples, removing redundant concepts through the large model.

[0078] After obtaining the concept set in the above step, redundant and repetitive concepts are removed. In this process, the embodiment of the application proposes an innovative strategy of encoder, large model and manual adjustment. The traditional concept merging method mainly has two limitations: (1) directly calling the large model to merge concepts, the result is uncontrollable, and lacks adjustability and consistency guarantee; (2) only relying on the calculation of semantic similarity based on the encoder to directly merge, which is easy to ignore the overall semantic relationship of the concept, and the merging quality is limited.

[0079] In an optional embodiment, the removing redundant concepts through the large model includes:

[0080] S121, calculating semantic similarity of names and descriptions of different concepts in pairs by using an encoder to filter out concepts with similarity greater than a threshold and put them into a merging candidate set.

[0081] It can be understood that the similarity threshold can be dynamically adjusted according to actual application requirements.

[0082] S122, merging the same or similar concepts in the merging candidate set by using the large model to remove the redundant concepts.

[0083] Specifically, the related steps are as follows:

[0084] ① Analyzing the name of each concept to determine overlapping or similar semantics. ② Defining unified concepts while avoiding overgeneralization and avoiding using the word “and” in concept names. ③ Defining concise definitions of concepts, avoiding reference to superordinate terms, and minimizing dependence on other concepts. ④ Determining the dimensionality of the concept, limiting each concept to only two dimensions to ensure clear binary classification. After the above four steps are completed, the merged concept is output, and the field contains the merged name, description, and dimension attribute.

[0085] S13, hierarchizing the de-redundant concepts by using the large model to obtain a hierarchical concept card composed of parent concepts and child concepts; wherein the parent concept contains a plurality of child concepts, and the child concept corresponds to the de-redundant concept one by one.

[0086] By introducing a sentence pair semantic similarity calculation mechanism, the present application can merge and clean concepts with diverse expression forms but similar semantics, and construct a hierarchical concept system with clear structure and consistent semantics. In principle, K layers can be generated, but the present embodiment only considers two layers, i.e., a hierarchical concept card composed of parent concepts and child concepts.

[0087] Specifically, the process of obtaining a hierarchical concept card composed of parent concepts and child concepts includes two core stages: generating an initial hierarchical structure and iterative optimization, as follows:

[0088] (I) Generating an initial hierarchical structure:

[0089] Input the de-redundant concept set into the large model, and guide the large model to perform the following steps: ① Check the concept name and description to determine groups with similar patterns and contexts. ② Select a group and create a parent concept based on the child concepts of the group. ③ Define the parent concept ID and name, which should be concise (2-4 words) and specific enough to avoid confusion with other existing parent concepts. ④ Build the parent concept description.

[0090] (II) Iterative optimization:

[0091] For sub-concepts not included in any parent concept in the construction of the initial structure, they are considered as "missing concepts". Each missing concept is analyzed separately, and if the concept can be integrated into an existing parent concept, it is merged; if a concept cannot be integrated into any existing parent concept, a new parent concept is created for it.

[0092] S14, from the training set, samples with different types of classification labels are selected in equal amounts, and the selected samples are input into the hierarchical concept card as input of the large model to label the dimension corresponding to each sample concept; Specifically:

[0093] First, the Prompt context is constructed by selecting documents with different labels from the training set in equal amounts, that is, inputting an equal amount of multi-class samples. Then these samples and the aforementioned hierarchical concept card are input into the large model, and it is specified in the Prompt that it is necessary to "identify whether the concept appears and its dimension"; the input / output format is strictly defined, so that the model has structured perception ability; without informing the true label of any sample text. Further guide the large model to complete the following operations: ① Familiarize yourself with the concepts in this set of concept cards. ② Evaluate each concept according to its name, description, and dimension to determine its relevance to any part of the input text content. If relevant, assign it a relevance score of 1; otherwise, assign it a relevance score of 0. ③ For concepts with a relevance score of 1, select the appropriate dimension value from the available range of concept dimensions consistent with the content. For concepts with a relevance score of 0, the dimension value is "none". ④ Jointly combine the relevance score and the dimension value into a semicolon-separated string. After the above steps are completed, output the dimension value corresponding to each sample concept.

[0094] S15, according to the frequency requirement, uniformly simplify the concepts and their corresponding dimensions of all samples to construct the task-aware concept system; Specifically:

[0095] According to the frequency of the concept, low-frequency concepts are filtered out to improve the model's generalization ability and computational efficiency, and finally a refined and efficient hierarchical concept card is formed as the task-aware concept system.

[0096] In step S2, the annotation results of each sample are encoded, and the text classification model is trained in combination with the classification labels until convergence.

[0097] After the above concept generation based on large model structured reasoning is completed, a task-aware concept system has been constructed, which contains high-value concepts and their corresponding dimensions after filtering out low-frequency concepts. This step aims to further convert these concepts into input features that can be used by the text classification model to train the model to achieve interpretable prediction of text.

[0098] Specifically, the step includes:

[0099] S21, encoding concept and its dimension:

[0100] Each concept is composed of two dimensions, which are used to represent the characteristics of the concept in different semantic scenarios. For example, in the news topic classification task, the concept "financial elements" can include "financial related" and "non-financial related" two dimensions, the concept "industry terminology" can include "specialized" and "non-specialized" two dimensions, and the concept "market background" can include "international market" and "domestic market" two dimensions. Assuming that after processing the given news text by the concept generation module, the following dimension labeling results are obtained:

[0101] Financial elements → (1, 0) (representing financial related);

[0102] Industry terminology → (0, 1) (representing non-specialized terminology);

[0103] Market background → (1, 0) (representing involving international market).

[0104] Concatenate the dimension vectors of all concepts in a fixed order to form a complete concept feature vector. If there are M concepts and each concept contains d dimensions, the final vector length is Mxd. For the above example, the vector x=[1, 0, 0, 1, 1, 0] is obtained after concatenating the three sets of concept dimension vectors, and the concatenated concept dimension vector is used as the input of the classification model.

[0105] S22, build a classification model:

[0106] Based on the concept dimension vector corresponding to each sample and its classification label, train the text classification model until it converges.

[0107] It should be noted that for binary classification tasks, a logistic regression model can be used for modeling, which has good interpretability and can analyze the influence direction and strength of each semantic concept through regression coefficients; if it is a multi-classification task, gradient boosting tree models such as Light Gradient Boosting Machine (LightGBM), eXtreme Gradient Boosting (XGBoost) can be selected to achieve higher prediction performance and stability while maintaining certain interpretability.

[0108] It is necessary to point out that after the model training is completed, the prediction result not only gives the classification label of the text (such as 'financial news' or 'non-financial news'), but also can determine which concepts and their dimensions mainly affect the prediction result by backtracking the model weight. For example, if the dimension weight of 'financial elements (financial related)' is high and the value is 1, it means that this dimension is the key factor leading to the prediction of 'financial news'.

[0109] In step S3, a number of labeled samples are selected from the training set as experience samples of new samples based on a screening strategy combining double semantic consistency and sample diversity enhancement.

[0110] Due to the uncertainty of the large model probability generation mechanism, the behavior of the large model in annotating concepts and dimensions in this stage may not be consistent, and even may deviate from the concept generation behavior in the training stage. In order to keep the preference between the generation of the large model in the training and the recognition in the reasoning consistent, the embodiment of the application introduces a screening strategy combining double semantic consistency and sample diversity enhancement to overcome the problem that the traditional large model reasoning process usually relies on a single input text, lacks structured retrieval and utilization of historical annotation knowledge, and is easy to produce understanding deviation, affecting the accuracy and stability of concept recognition.

[0111] The embodiment of the application actively introduces the labeled samples in the training stage as 'experience memory' in the large model reasoning stage, and integrates the In-Context Learning capability to assemble these samples together with the new samples and the task-aware concept system into a structured enhanced Prompt, which is input to the large model to guide it to complete the concept recognition task. This design not only improves the calling ability of the large model to historical knowledge, but also fully plays its context reasoning capability.

[0112] The selection of experience samples is based on two key constraints: the selected labeled samples not only meet the relevance to the new text, but also meet the diversity among the selected samples. Therefore, the embodiment of the application proposes an experience replay annotation method with double constraints of consistency and diversity, and the technical process is as shown in Figure 2 .

[0113] In an optional embodiment, the acquisition process of the double semantic consistency is as follows:

[0114] S31, obtaining the embedding vectors of a plurality of child concepts contained in each parent concept in the task-aware concept system, each labeled sample in the training set, and the new sample;

[0115] S32, calculating the first similarity between the embedding vectors of each labeled sample and the new sample;

[0116] S33, calculating a second similarity between each of the labeled samples and each of the several child concepts contained in each of the parent concepts;

[0117] S34, fusing the first similarity and the second similarity to obtain a relevance score of each of the labeled samples to represent the dual semantic consistency.

[0118] On this basis, the screening strategy combined with dual semantic consistency and sample diversity enhancement includes:

[0119] S35, selecting a preset number of labeled samples from the training set at a time, and obtaining a plurality of training subsets after repeating the operation;

[0120] S36, obtaining a relevance weighted kernel based on the relevance scores of any two labeled samples in the current training subset and the similarity kernel between them;

[0121] S37, constructing a conditional determinant point process kernel matrix of the current training subset based on the relevance weighted kernel;

[0122] S38, deriving a first log probability of the current training subset based on the conditional determinant point process kernel matrix and using the matrix determinant identity;

[0123] S39, performing exponential transformation on the relevance score in the first log probability based on an adjustable hyperparameter to obtain a second log probability, and the second log probability is used to represent the dual semantic consistency of each labeled sample in the training subset and the diversity between them.

[0124] Specifically:

[0125] (I) Relevance (dual semantic consistency):

[0126] For relevance, a common method is to retrieve demonstration examples based on the overall semantic similarity between labeled samples and new samples. However, this method is not optimal for this task because high overall similarity does not necessarily mean that the labeled sample provides relevant information for the specific concept in the new input sample.

[0127] In order to consider fine-grained similarity calculation and imitate the labeling mode in the training phase, the embodiments of the present application decompose the recognition task into parent sub-tasks. For each parent concept and its child concepts, a single contextual learning cue is constructed, which only focuses on the child concepts under the parent concept. In order to construct each cue, the similarity between the new sample and each labeled sample is calculated, as well as the similarity between the labeled sample and the target concept subset.

[0128] Then, the two similarity scores are averaged to obtain a joint relevance score. A text embedding model (e.g., a miniaturized text embedding model, text-embedding-3-small) is used to obtain the embedding vectors of the labeled samples, the sub-concepts contained in each parent concept in the task-aware concept system, and the new samples, respectively represented as follows: , , ,in Let be the set of real numbers, d be the embedding dimension, N be the number of labeled samples, and Q be the number of child concepts contained in the current parent concept.

[0129] Therefore, the relevance score of the labeled samples, which characterizes the dual semantic consistency, can be calculated as the average of the following two similarity scores:

[0130]

[0131] in, For the first i The relevance score of each labeled sample relative to a new sample, where sim is the cosine similarity function; For the first i Embedding vectors of labeled samples; This is the embedding vector of the new sample; This indicates that it belongs to the u-th parent concept. The set of sub-concepts; It is the v-th child concept of the u-th parent concept; This represents the embedding vector of the aforementioned v-th sub-concept; Let be the size of the set of child concepts belonging to the u-th parent concept.

[0132] (ii) Diversity constraints:

[0133] Besides similarity, diversity among labeled samples is also crucial. If all retrieved labeled samples are too homogeneous, large models may be instructed to narrow their pattern range and fail to generalize. The Determinantal Point Process (DPP) is a commonly used method to select a diverse subset from a pool of candidate samples by suppressing redundancy. DPP selects samples based on a similarity kernel matrix, where the probability of selecting a subset increases with the determinant of the corresponding submatrix. This is because the determinant can be interpreted as the square of the volume of the parallelepiped formed by the eigenvectors of the selected items; the more orthogonal the vectors (i.e., the lower the similarity), the larger this volume.

[0134] To simultaneously optimize relevance and diversity, this embodiment of the invention extends the original DPP by prioritizing a subset containing labeled samples highly relevant to the input, while preserving internal diversity. Given a corresponding embedding vector... Annotated training sample set , the original DPP defines a positive semi-definite kernel matrix containing elements where is a similarity kernel (e.g. cosine similarity), are embedding vectors of the i th and j th annotated sample, respectively. The probability of selecting a subset is:

[0135]

[0136] where, denotes the DPP kernel matrix indexed by S, which measures diversity; I is the identity matrix, denotes the determinant of a matrix. To incorporate relevance into the selection of new samples , a relevance-weighted kernel is defined as:

[0137]

[0138] Note that DPP essentially models the joint probability of selecting a subset through a positive semi-definite kernel matrix L, rather than the importance of individual samples. If only is considered, it measures the "relevance between the th annotated sample and the new sample , but DPP decides whether a group of samples in subset S is selected or not, and the quality of subset S depends on the joint performance of all samples in it. Multiplying , means that when the th annotated sample and the

[0139] th annotated sample are both selected, their joint contribution depends on their respective relevance to the new sample. If one of the samples has weak relevance to the new sample (e.g. ≈0), even if the other sample is strong, we do not want this pair to contribute too much to the overall selection, so the product is suppressed. Using the above relevance-weighted kernel, a conditional DPP kernel matrix is defined, where r is the relevance score vector. For a subset S, the corresponding conditional DPP kernel matrix is

[0140] . By applying the matrix determinant identity where A, B represent different matrices, the log probability of selecting subset S (first log probability) is derived as:

[0141] where log is the logarithm function, denotes the acquisition matrix wherein, π is the identity matrix, π is the diagonal element, π is the multiplication symbol.

[0142] Further, in order to be able to explicitly control the trade-off between relevance and diversity, embodiments of the present application do not directly use relevance scores but introduce a tunable hyper-parameter λ>0 and apply an exponential transformation to adjust their influence, thus deriving a modified target (second log-probability):

[0143]

[0144] wherein, e is a mathematical constant; is the conditional determinant point process kernel matrix after applying the exponential transformation .

[0145] Obviously, by calculating the second log-probability as described above, a subset of experience samples that take into account both relevance and diversity can be screened out for the new sample after comparison.

[0146] In step S4, the task-aware concept system, the new sample and the experience sample are jointly used as input of the large model, the concept of the new sample and its corresponding dimension are labeled, and after encoding, the text classification result of the new sample is obtained as input of the converged text classification model.

[0147] In the inference phase, embodiments of the present application will task-aware concept system, new sample and experience sample that takes into account both relevance and diversity together to form an enhanced Prompt, input to the large model, and label the concept of the new sample and its corresponding dimension.

[0148] For example, in order to ensure the accuracy and consistency of the concept dimension labeling of the new sample, embodiments of the present application design a structured Prompt template to guide the large model to reason and label. The template includes the following key elements:

[0149] 1. Persona

[0150] The labeling task is performed by an expert role with experience in news analysis and topic classification, which is responsible for identifying concepts related to financial elements, industry terms and market background from news text, and determining whether they appear in the text.

[0151] 2. Instruction

[0152] Given a news article and a set of concept profiles, the annotation task requires determining whether each concept is relevant to the news article and further annotating the relevant concepts with their dimension values, such as "appeared" or "not appeared." Relevance and dimension values ​​will be used together in the subsequent prediction model. Annotation should reference historical experience data to ensure consistency and interpretability.

[0153] 3. Input Format

[0154] The input is a dictionary containing three parts:

[0155] ①News: The news ID is the key and the value is the news text;

[0156] ②Concept Profiles: Using concept ID as the key and the value as the corresponding concept profile, each concept contains "Concept ID", "Concept Name", "Concept Description" and optional "Sub-concept Dimensions";

[0157] ③Experiences: Historical annotation examples, used for annotation reference.

[0158] 4. Output Format

[0159] The output is in JSON format, with the news ID as the key and a dictionary as the value. Each concept ID serves as the key, and the key-value pair is in the form of "relevance score; subconcept value".

[0160] ① The relevance score ∈ {0,1} indicates whether the concept is relevant to the news;

[0161] ② The sub-concept value comes from "Concept Dimensions". If there is no dimension information, fill in "None".

[0162] 5. Annotation Example

[0163] {

[0164] "Input": {

[0165] News: A company released its annual financial report, projecting future revenue growth.

[0166] "Concept Profiles": {

[0167] "C0": {

[0168] "Concept ID": "C0",

[0169] "Concept Name": "Financial Elements",

[0170] "Concept Description": "Information related to finance",

[0171] "Concept Dimensions": ["Finance-related", "Non-finance-related"]

[0172] },

[0173] "C1": {

[0174] "Concept ID": "C1",

[0175] "Concept Name": "Industry Terminology",

[0176] "Concept Description": "Type of specialized terminology used in news",

[0177] "Concept Dimensions": ["Specialized", "Non-specialized"]

[0178] },

[0179] "C2": {

[0180] "Concept ID": "C2",

[0181] "Concept Name": "Market Context",

[0182] "Concept Description": "Range of markets that news content relates to",

[0183] "Concept Dimensions": ["International Market", "Domestic Market"]

[0184] }

[0185] },

[0186] "Experiences": {

[0187] "Experiences_1": "Historical labeling example 1",

[0188] "Experiences_2": "Historical labeling example 2"

[0189] }

[0190] },

[0191] "Output": {

[0192] "News": {

[0193] "C0": "1; Finance-related",

[0194] "C1": "1; Non-professional",

[0195] "C2": "0;None"

[0196] }

[0197] }

[0198] }

[0199] By completing the above steps to define the concept of the new sample and its corresponding dimension annotation, the sample is further input into the text classification model that has been built during the training phase to complete the final prediction of the new sample.

[0200] Thus, this embodiment of the invention completes an integrated closed-loop reasoning path from concept card construction, annotation enhancement, and interpretable prediction, ensuring the controllability, consistency, and interpretability of the overall system.

[0201] Example 2:

[0202] This invention provides an interpretable text classification system based on large model concept generation, comprising:

[0203] The concept generation module is used for structured reasoning based on a large model, labeling the concept and its corresponding dimension for each sample in the training set to construct a task-aware concept system; wherein the training set contains several documents and their classification labels, and each document corresponds to a sample;

[0204] The training module is used to encode the annotation results of each sample and, in conjunction with the classification labels, train the text classification model until it converges.

[0205] The concept annotation module is used to select several annotated samples from the training set as empirical samples for new samples based on a screening strategy that combines dual semantic consistency and sample diversity enhancement.

[0206] An interpretable prediction module is used to take the task-aware concept system, the new sample, and the experience sample as input to the large model, label the concept of the new sample and its corresponding dimension, encode it, and use it as input to the converged text classification model to obtain the text classification result of the new sample.

[0207] Example 3:

[0208] The embodiment of the application provides a storage medium which stores a computer program for explainable text classification based on large model concept generation, wherein the computer program causes a computer to execute the explainable text classification method as described in embodiment 1.

[0209] Embodiment 4:

[0210] The embodiment of the application provides an electronic device, comprising:

[0211] one or more processors; a memory; and one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the program comprises a program for executing the explainable text classification method as described in embodiment 1.

[0212] It can be understood that the explainable text classification system based on large model concept generation, the storage medium and the electronic device provided by the embodiment of the application correspond to the explainable text classification method based on large model concept generation provided by the embodiment of the application, and the relevant content of the explanation, examples and advantages of the method can be referred to the corresponding part, which will not be repeated here.

[0213] To sum up, compared with the prior art, the following advantages are achieved:

[0214] 1. A closed-loop reasoning and prediction system is constructed to support end-to-end automated application

[0215] The embodiment of the application forms a complete technical closed loop from semantic extraction, concept organization to large model reasoning and final prediction, and realizes full-process system coupling. The closed loop structure can support automatic labeling, explainable prediction and continuous learning, and effectively improves the overall task processing efficiency and the systematicness of model operation and maintenance.

[0216] 2. Enhance the generalization ability and stability of the model under complex semantic tasks

[0217] The embodiment of the application constructs a more robust Prompt input large model by means of a sample screening mechanism enhanced by double semantic consistency and diversity, significantly reduces the result fluctuation and semantic deviation while improving the reasoning accuracy, and ensures the reliability and consistency of the labeling output under complex business context.

[0218] 3. Consider the automation capability and manual control, and enhance the flexibility of the system

[0219] The embodiment of the application introduces a "manual adjustable threshold" mechanism in the key processing link (such as concept merging), realizes effective intervention on the large model semantic generation process, guarantees the consistency of processing, retains the controllability of human decision-making, adapts to diversified business needs, and improves the flexibility and landing ability of system application.

[0220] 4. With knowledge reuse and long-term evolution capability, reduce system maintenance cost

[0221] Based on the explicit concept-driven structure, the embodiment of the present application has good knowledge reuse capability and evolution adaptation capability. With the introduction of new concepts, the system can be seamlessly extended without overall retraining, thereby significantly reducing the long-term model update cost and improving the maintainability and sustainability of the system.

[0222] It should be noted that, in this article, the relationship terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between the entities or operations. Moreover, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or equipment including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or equipment. Without more limitations, the element defined by the statement "including a" does not exclude the presence of other identical elements in the process, method, article or equipment including the element.

[0223] The above embodiments are only used to illustrate the technical solutions of the present application, but not to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An interpretable text classification method based on large model concept generation, characterized in that, The method comprises the following steps: Based on large model structured reasoning, the concept and its corresponding dimension of each sample in the training set are labeled to construct a task-aware concept system; wherein the training set contains a plurality of documents and their classification labels, and each document corresponds to a sample; Encode the labeling results of each sample, combine the classification labels, and train the text classification model until convergence; Based on the screening strategy combined with double semantic consistency and sample diversity enhancement, a plurality of labeled samples are selected from the training set as experience samples of new samples; The task-aware concept system, the new sample and the experience sample are jointly used as the input of the large model to label the concept and its corresponding dimension of the new sample, and the encoded result is used as the input of the converged text classification model to obtain the text classification result of the new sample; The large model structured reasoning based on the concept and its corresponding dimension of each sample in the training set are labeled to construct a task-aware concept system, comprising: Divide each sample by sentence and obtain a summary through the large model to abstract concepts with dimension information from the summary; After traversing all samples, remove redundant concepts through the large model; Hierarchize the redundant-removed concepts through the large model to obtain a hierarchical concept card composed of parent concepts and child concepts; wherein the parent concept contains a plurality of child concepts, and the child concept corresponds to the redundant-removed concept one by one; Select samples with different types of classification labels from the training set, and use the selected samples and the hierarchical concept card as the input of the large model to label the corresponding dimension of each sample's concept; According to the frequency requirement, unify the concept and its corresponding dimension of all samples to simplify and construct the task-aware concept system; The process of obtaining double semantic consistency is as follows: Obtain the embedding vectors of each parent concept containing a plurality of child concepts in the task-aware concept system, each labeled sample in the training set, and the new sample; Calculate the first similarity between the embedding vectors of each labeled sample and the new sample; Calculate the second similarity between each labeled sample and each parent concept containing a plurality of child concepts; Fuse the first similarity and the second similarity to obtain the relevance score of each labeled sample to represent the double semantic consistency; The screening strategy combined with double semantic consistency and sample diversity enhancement comprises: Select a preset number of labeled samples from the training set at a time, and obtain a plurality of training subsets after repeating the operation; Based on the relevance score of any two labeled samples in the current training subset and the similarity kernel between them, obtain a relevance weighted kernel; Based on the relevance weighted kernel, construct the conditional determinant point process kernel matrix of the current training subset; Based on the conditional determinant point process kernel matrix, derive the first logarithmic probability of the current training subset using the matrix determinant identity. The second log probability is obtained by exponentially transforming the correlation score in the first log probability based on an adjustable hyperparameter, and is used to represent the dual semantic consistency of each of the labeled samples in the training subset and the diversity among them.

2. The interpretable text classification method of claim 1, wherein, The removing redundant concepts by the large model comprises: The semantic similarity of the names and descriptions of different concepts is calculated by the encoder to filter out all concepts with a similarity greater than a threshold and put them into a merging candidate set; The same or similar concepts in the merging candidate set are merged by the large model to remove the redundant concepts.

3. The interpretable text classification method of claim 1, wherein, The text classification model is a binary classification model or a multi-classification model.

4. The interpretable text classification method of claim 1, wherein, The second log probability is represented as: where log is the logarithm function; is the determinant of matrix; e is the mathematical constant; is the determinant point process kernel matrix indexed by the training subset S, is the conditional determinant point process kernel matrix after applying the exponential transform; is the relevance score of the i th labeled sample relative to the new sample; is the tunable hyper-parameter.

5. An interpretable text classification system based on large model concept generation, characterized by, The computer program product for performing the interpretable text classification method of claim 1 comprises: A concept generation module is configured to annotate the concepts and corresponding dimensions of each sample in the training set based on the structured reasoning of the large model to construct a task-aware concept system, wherein the training set contains a plurality of documents and their classification labels, and each document corresponds to a sample; A training module is configured to encode the annotation results of each sample and combine the classification labels to train a text classification model until convergence; A concept annotation module is configured to filter a plurality of labeled samples from the training set as experience samples of a new sample based on a screening strategy combining dual semantic consistency and sample diversity enhancement; An interpretable prediction module is configured to jointly input the task-aware concept system, the new sample and the experience samples into the large model to annotate the concepts and corresponding dimensions of the new sample, and encode the new sample as an input of the converged text classification model to obtain a text classification result of the new sample.

6. A storage medium, characterized by The computer program product for performing the interpretable text classification method of claim 1~4 comprises:

7. An electronic device, comprising: One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs comprise a program for performing the interpretable text classification method of claim 1~4. One or more processors; Memory; And one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the programs comprise a program for performing the interpretable text classification method of claim 1~4.

Images