Text processing method and device and electronic equipment

Abstract

The invention provides a text processing method and device and electronic equipment, and the method comprises the steps: coding a to-be-classified text, and obtaining the feature representation of the to-be-classified text; the feature representation is divided into N feature subsets, N is an integer larger than 1, the value of N is determined through the training process of the target model, and the feature subsets correspond to the processing layers one to one; for each processing layer, determining the layer probability of the to-be-classified text belonging to each category under the layer based on the feature subset of the layer; synthesizing the layer probability of each processing layer to obtain a comprehensive probability that the to-be-classified text belongs to each category; and determining the semantic category of the to-be-classified text based on the comprehensive probability.

Description

Technical Field

[0001] This disclosure relates to the field of natural language processing technology, specifically to a text processing method, apparatus, and electronic device. Background Technology

[0002] Existing technologies, when handling semantic recognition tasks of text, suffer from prediction bias due to the influence of confounding variables, making it difficult to capture the complex semantics of text. Summary of the Invention

[0003] In view of the above problems, this disclosure provides a text processing method, apparatus and electronic device.

[0004] This disclosure provides a text processing method, comprising: encoding the text to be classified to obtain a feature representation of the text to be classified; dividing the feature representation into N feature subsets, where N is an integer greater than 1, and the value of N is determined through the training process of the target model, and each feature subset corresponds one-to-one with a processing layer; for each processing layer, determining the layer probability of the text to be classified belonging to each category under that layer based on the feature subset of that layer; combining the layer probabilities of each processing layer to obtain the comprehensive probability of the text to be classified belonging to each category; and determining the semantic category of the text to be classified based on the comprehensive probability.

[0005] According to embodiments of this disclosure, dividing the feature representation into N feature subsets includes: dividing the feature dimensions of the feature representation into N groups, each group containing continuous feature dimensions and / or non-continuous feature dimensions; and using the feature values ​​corresponding to each group of feature dimensions as feature subsets.

[0006] According to embodiments of this disclosure, the partitioning is performed in any of the following ways: the feature dimensions are uniformly divided into N groups; if the total number of feature dimensions is not divisible by N, the remaining feature dimensions are allocated to the target processing layer or discarded; or if the total number of feature dimensions is not divisible by N, the value of N is adjusted to a divisible value that satisfies a preset condition.

[0007] According to embodiments of this disclosure, the method further includes: dividing N feature subsets into at least one combined feature subset, each combined feature subset containing at least two feature subsets; each combined feature subset corresponding one-to-one with a combined processing layer; for each combined processing layer, determining the combined probability of the text to be classified belonging to each category under that combined processing layer; and determining the comprehensive probability based on the combined probability of each combined processing layer and the layer probability of each processing layer.

[0008] According to embodiments of this disclosure, for each combined processing layer, the combined probability of the text to be classified belonging to each category under that combined processing layer is determined, including: fusing feature subsets of each processing layer within each combined processing layer and determining the combined probability based on the fused feature subsets; or fusing the layer probabilities of each processing layer within each combined processing layer to obtain the combined probability.

[0009] According to embodiments of this disclosure, for each processing layer, the layer probability of the text to be classified belonging to each category under that layer is determined based on the feature subset of that layer, including: obtaining the category prototype of each category under each processing layer; calculating the similarity between the feature subset of each processing layer and the category prototype; and calculating the layer probability of the text to be classified belonging to each category under each processing layer based on the similarity.

[0010] According to embodiments of this disclosure, obtaining category prototypes for each category at each processing layer includes: calculating category prototypes based on feature subsets of the support set at each processing layer, wherein the support set contains at least one labeled sample of the actual category to which the text to be classified belongs.

[0011] According to embodiments of this disclosure, determining the semantic category of a text to be classified based on comprehensive probability includes: determining the category with the highest probability as the semantic category of the text to be classified based on comprehensive probability; or determining at least one category with a comprehensive probability exceeding a preset threshold as the semantic category.

[0012] This disclosure also provides a text processing apparatus, comprising: an encoding module for encoding text to be classified to obtain a feature representation of the text to be classified; a partitioning module for partitioning the feature representation into N feature subsets, wherein N is an integer greater than 1, and the value of N is determined through the training process of a target model, and each feature subset corresponds one-to-one with a processing layer; a determination module for determining, based on the feature subsets of each processing layer, the layer probability of the text to be classified belonging to each category in that layer; a synthesis module for synthesizing the layer probabilities of each processing layer to obtain a comprehensive probability of the text to be classified belonging to each category; and a judgment module for determining the semantic category of the text to be classified based on the comprehensive probability.

[0013] This disclosure further provides an electronic device, including: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the following methods: encoding text to be classified to obtain a feature representation of the text to be classified; dividing the feature representation into N feature subsets, where N is an integer greater than 1, and the value of N is determined through the training process of a target model, with each feature subset corresponding to a processing layer; for each processing layer, determining the layer probability of the text to be classified belonging to each category under that layer based on the feature subset of that layer; combining the layer probabilities of each processing layer to obtain a comprehensive probability of the text to be classified belonging to each category; and determining the semantic category of the text to be classified based on the comprehensive probability.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0015] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0016] Figure 1 The illustrations depict application scenarios of the text processing method, apparatus, and electronic device according to embodiments of the present disclosure.

[0017] Figure 2 A flowchart illustrating the determination of the number of layers N according to an embodiment of the present disclosure is shown schematically;

[0018] Figure 3 A schematic diagram illustrating the overall structure of a target model according to an embodiment of the present disclosure is shown.

[0019] Figure 4 A schematic diagram illustrating a causal intervention layering according to an embodiment of the present disclosure is shown.

[0020] Figure 5 A flowchart illustrating a text processing method according to an embodiment of the present disclosure is shown schematically.

[0021] Figure 6 A schematic block diagram of a text processing apparatus according to embodiments of the present disclosure is shown; and

[0022] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a text processing method according to an embodiment of the present disclosure. Detailed Implementation

[0023] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0024] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0025] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0026] It should be noted that the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information in this disclosed technical solution comply with relevant laws and regulations, necessary confidentiality measures have been taken, and it does not violate public order and good morals. In this disclosed technical solution, user authorization or consent has been obtained before acquiring or collecting user personal information.

[0027] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0028] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0029] With the rapid development of artificial intelligence technology, intelligent customer service has become an important means of enterprise customer service. However, existing intelligent customer service systems still face many challenges when processing semantic recognition tasks of short texts. On the one hand, due to the complex characteristics of text, such as polysemy and long-distance dependencies, few-shot learning methods from the field of computer vision are difficult to directly apply to natural language processing tasks. At the same time, current models mostly use convolutional neural networks (CNNs) as feature extractors, which are limited by local receptive fields and cannot fully capture long-distance contextual semantics, resulting in insufficient semantic representation and inaccurate intent recognition. On the other hand, most existing technologies are based on relevance modeling and have failed to effectively overcome the prediction bias caused by confounding variables (such as synonyms and related words) in pre-trained knowledge. Although some causal intervention methods attempt to introduce a hierarchical mechanism through backdoor adjustment, the number of hierarchies depends on manual setting and lacks adaptive optimization, which not only results in high experimental costs but also limits the model's generalization ability. In addition, the scarcity of labeled data further restricts the model's performance on a few intent categories.

[0030] To address the aforementioned issues, related technologies have proposed that the introduction of attention mechanisms can optimize prototype generation, but this still does not fundamentally solve the problem of confounding bias. While existing causal intervention methods can adjust pre-trained knowledge in layers, they do not address the gradient vanishing problem caused by parallel multi-classifiers, and the layering parameters still need to be set empirically.

[0031] In view of the above, this disclosure provides a text processing method, apparatus, and electronic device, which will be described below with reference to the accompanying drawings.

[0032] Figure 1 Figure 100 schematically illustrates an application scenario of a text processing method, apparatus, and electronic device according to embodiments of the present disclosure.

[0033] It is important to note that Figure 1 The examples shown are merely examples of scenarios in which the embodiments of this disclosure can be applied, to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0034] like Figure 1 As shown, application scenario 100 according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0035] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to submit intent recognition requests containing user query text and obtain text processing results. Relevant users or system operators (such as customer service administrators, model maintainers, and business configuration personnel) can perform operations such as uploading support set samples, configuring classification tasks, and viewing recognition results through the terminal devices.

[0036] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with data processing and communication capabilities and support access to the system network, including but not limited to smartphones, tablets, portable computers, and desktop computers.

[0037] Server 105 can be a backend server providing intelligent text processing and classification services, such as an intelligent customer service system deployed with the text processing method based on the embodiments of this disclosure, to determine the semantic category of user query text and then output the corresponding semantic recognition result to drive automatic response or business flow (only an example). The server can schedule and monitor the status of text processing tasks, record text processing logs, and return the semantic recognition result to the corresponding user terminal for further business flow, automatic reply, or manual processing.

[0038] It should be noted that the text processing method provided in this embodiment can generally be executed by server 105. Correspondingly, the text processing device and electronic device provided in this embodiment can generally be located in server 105. The text processing method provided in this embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the text processing device provided in this embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0039] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0040] The following will be based on Figure 1 The described scene, through Figures 2-5 The text processing method of the present disclosure embodiments will be described in detail.

[0041] Before applying the text processing method provided in the embodiments of this disclosure, the target model can be trained. For example... Figure 2 As shown, the training process may include: sampling from the training data to obtain a simulated support set and a simulated query set; initializing the number of layers N to an initial value (e.g., N=1), processing the simulated support set and the simulated query set based on the current N value to obtain the prediction result; calculating the loss between the prediction result and the true label of the simulated query set, and adjusting the trainable parameters in the target model through backpropagation and gradient descent, including the number of layers N; when the training meets the termination condition (such as reaching a preset training round or loss convergence), fixing the optimized N value as a parameter for partitioning feature subsets in subsequent practical applications.

[0042] Throughout the training process, N was used as a learnable parameter to achieve adaptive optimization, reducing the cost of manual parameter tuning and enhancing the model's generalization performance under different data distributions and task scenarios.

[0043] Figure 3 An overall structural diagram of the target model according to an embodiment of the present disclosure is shown schematically.

[0044] like Figure 3 As shown, the target model can be built based on a language model pre-trained on a large-scale corpus. In one specific implementation, pre-trained weights can be loaded, some low-level parameters can be frozen, and other relevant parameters can be set, such as setting the initial learning rate to 2e-5, the batch size to 4, and iterative training for 20,000 rounds. The input to the target model includes support set samples and query set samples. The input text can first be preprocessed, including word segmentation, fixing the text length (e.g., 50 words), and entity location annotation. Then, it can be input into the embedding layer of the target model for encoding to generate a high-dimensional semantic feature representation. This feature representation can be normalized by a batch normalization layer (BN layer) to alleviate the gradient vanishing problem that occurs in parallel computation of multi-classifiers and improve training stability. Afterward, the features can be fed into the prototype layer, where a class prototype for each category is calculated based on the features of the support set samples (e.g., the average of the sample feature vectors of each category in the support set is taken to generate the class prototype of that category to represent the semantic center of the corresponding intent). Finally, in the Metric Layer, the similarity between the query set sample features and various prototypes is calculated, and the prediction results are output.

[0045] Figure 4 A schematic diagram illustrating a causal intervention layering according to an embodiment of the present disclosure is shown.

[0046] In the prototype layer, the complete class prototype of each category (taking "Category 1" as an example) can be decomposed into N processing layers, forming a sub-prototype within each processing layer (e.g., sub-prototype 1 to sub-prototype N). Each sub-prototype represents the typical features of that category in the specific semantic dimension that the corresponding processing layer focuses on. Correspondingly, the feature vector of the query set samples can also be divided into N feature subsets, each of which is input into the corresponding processing layer. Within each processing layer (e.g., the k-th layer), independent similarity calculation is performed, that is, the query feature subset of that layer is compared with the sub-prototypes of all categories in that layer to calculate the classification probability (i.e., "layer probability") of that layer. Finally, by fusing the layer probabilities output from all N processing layers, a comprehensive classification prediction result can be obtained. This hierarchical backdoor adjustment strategy, which corresponds one-to-one with "feature subset - processing layer - sub-prototype", effectively blocks the interference of confounding variables between different semantic dimensions by decoupling the overall features to independent subspaces for processing, thereby improving the ability to recognize complex semantics and the model's robustness to interference.

[0047] Understandably, by combining the target model with an adaptive hierarchical causal intervention mechanism, the semantic representation and classification performance in small sample scenarios are significantly enhanced, and the stability and generalization ability of the model are improved.

[0048] like Figure 5 As shown, the text processing method of this embodiment may include S510~S550.

[0049] In S510, the text to be classified is encoded to obtain the feature representation of the text to be classified.

[0050] The text to be classified can be raw text data from user queries, comments, or conversations, such as user questions in a customer service system, sentiment feedback on social media, or complex statements in multi-turn conversations. The text to be classified can be input in natural language, and its semantics can include a single intent, multiple intents, or implicit sentiment.

[0051] The encoding process of text to be classified can transform unstructured natural language sequences into structured numerical vectors (i.e., feature representations) rich in semantic information. This process can be achieved through deep neural network models to capture deep semantic relationships and long-distance contextual dependencies between words, thereby generating highly discriminative and robust feature representations, providing a data foundation for subsequent hierarchical classification processing.

[0052] In some embodiments, encoding can be implemented using a pre-trained language model based on the Transformer architecture. This model can dynamically evaluate the relevance weights between any two words in a text sequence using a self-attention mechanism, thereby overcoming the limitations of local windows and achieving deep fusion modeling of global contextual information. Specifically, the text to be classified can be input into the model, and through layer-by-layer nonlinear transformation and information aggregation of multiple Transformer blocks, the vector of the entire text or the [CLS] flag can be extracted as a feature representation.

[0053] In other embodiments, to further enhance the discriminative power of the feature representation, auxiliary features can be introduced for fusion based on the encoding of the main model described above. For example, auxiliary feature vectors such as syntactic features, dictionary features, or statistical features can be extracted from the original text in parallel. Then, the main feature representation generated by the main model can be concatenated, weighted, or fused with one or more auxiliary feature vectors based on attention to form the final comprehensive feature representation.

[0054] In S520, the feature representation is divided into N feature subsets, where N is an integer greater than 1, and the value of N is determined through the training process of the target model. Each feature subset corresponds one-to-one with a processing layer.

[0055] The partitioning process decouples the feature space. In one embodiment, the overall feature representation can be mapped to N different subspaces using a learnable projection matrix or a linear transformation layer, with the output vector of each subspace being a subset of features.

[0056] In another embodiment, the partitioning can be achieved through an attention-guided masking mechanism. For example, N sets of attention weights (masks) can be generated by a learnable gating network, and each set can be element-wise multiplied or weighted with the original feature representation to extract N feature subsets.

[0057] It should be noted that the determination of the number of strata N breaks through the limitation of existing causal intervention methods that require manual pre-setting of the number of strata. In the embodiments of this disclosure, N is constructed as a learnable parameter of the model (or indirectly represented by a learnable surrogate parameter). During the meta-learning training process of the model, the classification weights and the number of strata N can be optimized simultaneously through gradient backpropagation, enabling the model to adaptively determine the appropriate semantic decomposition granularity based on data characteristics and task complexity. After training, N can be fixed and used in all inference processes during the application phase.

[0058] In S530, for each processing layer, the layer probability of the text to be classified belonging to each category under that layer is determined based on the feature subset of that layer.

[0059] Each processing layer can be configured as an independent classifier (e.g., a fully connected layer followed by a normalized exponential function), with its input being a subset of the features corresponding to that layer. In this way, each classifier is constrained to make decisions within its assigned feature subspace, thereby forcing the model to learn to separate semantic information of different dimensions (such as sentiment, intent, and topic) into different processing layers, achieving parallel and decoupled analysis of complex text semantics.

[0060] For example, in a hybrid sentiment and intent classification task for customer service scenarios, after training, the model can associate a subset of features from the first processing layer with the sentiment semantic attributes of the text (such as lexical sentiment tendency and intensity), enabling this layer to focus on outputting the layer probability of the text belonging to sentiment categories such as "positive," "negative," or "neutral." Simultaneously, a subset of features from the second processing layer can be associated with the intent semantic attributes of the text (such as action verbs and interrogative words), enabling this layer to focus on outputting the layer probability of the text belonging to intent categories such as "query," "operation," or "complaint."

[0061] Through layering and decoupling mechanisms, interference between different semantic dimensions can be effectively isolated, and independent and focused discrimination can be made for each semantic aspect, thus laying the foundation for subsequent holistic and robust classification decisions.

[0062] In S540, the layer probabilities of each processing layer are combined to obtain the combined probability of the text to be classified belonging to each category.

[0063] To integrate local judgments from different processing layers and form a globally consistent classification decision, the layer probabilities of all layers can be fused.

[0064] In one embodiment, the fusion strategy can be a simple weighted average, which involves summing the probabilities of each processing layer belonging to the same category to obtain the overall probability of that category. The weights can be preset fixed values ​​or dynamically generated through sub-networks (e.g., gating networks). These sub-networks can take the overall or hierarchical features of the text to be classified as input and output the weight allocation corresponding to each processing layer, thus adaptively adjusting the contribution of each layer based on the semantic characteristics of the current text. For example, when processing text with strong emotional content, the system can assign higher weights to the processing layer responsible for sentiment analysis.

[0065] In another embodiment, the fusion strategy can also employ attention-based hierarchical aggregation. Specifically, for each candidate category, an attention relevance score can be calculated between the category and the representations output by each processing layer, and the layer probabilities of each layer can be weighted and aggregated based on this score, so that when finally determining each category, the focus can be placed on the semantic level most relevant to it.

[0066] In S550, the semantic category of the text to be classified is determined based on the comprehensive probability.

[0067] Based on the obtained comprehensive probability vector, the category with the highest probability value can be selected as the final semantic category output. This output is the model's judgment on the overall semantic content of the text, which can be directly applied to downstream automated business processes.

[0068] To improve the reliability of classification results and control the risk of misclassification, a confidence level determination mechanism can be introduced. Specifically, a confidence level threshold can be set. When determining the semantic category, the category with the highest overall probability is first identified, and then it is determined whether its probability value exceeds the confidence level threshold. If it exceeds the threshold, that category is determined as the final semantic category; if it does not exceed the threshold, the text to be classified is marked as a "low-confidence sample," and a pre-defined subsequent processing flow is triggered, such as routing it to human customer service for review, or classifying it as an "unclear intent" category for special handling.

[0069] For example, in an intelligent customer service system, a user inputs the query text "Where is the nearest ATM?". First, the query text is encoded to obtain its 768-dimensional feature representation. Based on the training-determined N=4, the model divides the feature representation into four feature subsets, each corresponding to a processing layer. Within each processing layer, based on prototypes of categories such as "ATM query," "account query," and "branch query" in the support set, the layer probability of the query text at each layer is calculated. Then, the combined probabilities of all layers are combined to obtain the highest overall probability that the query text belongs to the "ATM query" category. Finally, the semantic category of the query text can be determined as "ATM query," triggering the corresponding automatic reply or map-guided process.

[0070] For example, in a user feedback processing scenario, a user inputs "The service attitude was very poor, but the problem was solved." After encoding this text, it is divided into feature subsets based on N=3 determined during training, corresponding to three processing layers: "emotional polarity," "problem description," and "transitional semantics." Within each layer, layer probabilities are calculated based on the prototypes of emotional categories such as "positive," "negative," and "neutral." After combining the probabilities of each layer, since "negative" is dominant in the emotional polarity layer and "positive" is significantly reflected in the transitional semantics layer, the system can ultimately determine that the text has a semantic category of "mixed emotion (mainly negative)," and trigger a tiered processing or manual follow-up process.

[0071] For example, in an e-commerce customer service scenario, a user inputs "want to learn about product information and hopes to purchase." After encoding, the features are divided into two feature subsets according to the training-determined N=2, corresponding to the "consultation intent" and "purchase intent" processing layers, respectively. Within each layer, layer probabilities are calculated based on prototypes of categories such as "information consultation," "purchase intent," and "after-sales consultation." After combining the probabilities of the two layers, the system identifies that the text has high confidence in both the "information consultation" and "purchase intent" categories, thus determining its semantic category as "composite intent, i.e., consultation and purchase," and triggering a combined business process of information push and purchase guidance.

[0072] Understandably, by adaptively dividing the feature representation into N processing layers and dynamically optimizing the number of layers N during training, decoupling and parallel analysis of multidimensional semantic information in text are achieved. This method not only effectively captures long-distance dependencies and ambiguities in text, improving the semantic understanding of complex queries, but also significantly reduces the interference of confounding variables on prediction results through a hierarchical causal intervention mechanism, enhancing the model's robustness and prediction stability. Furthermore, the adaptive hierarchical mechanism avoids the cost and limitations of manual parameter tuning, enabling the model to flexibly adapt to different datasets and task requirements. It is suitable for small-sample scenarios with scarce data, significantly improving the generalization performance and overall accuracy of text semantic relationship recognition.

[0073] Based on the above embodiments, in this embodiment, the feature representation is divided into N feature subsets, including: dividing the feature dimensions of the feature representation into N groups, each group containing continuous feature dimensions and / or non-continuous feature dimensions; and taking the feature values ​​corresponding to each group of feature dimensions as feature subsets.

[0074] Since deep semantic features can exhibit different distribution and association patterns across different dimensions, this embodiment provides a flexible feature grouping strategy to adapt to the semantic decoupling granularity requirements of different tasks.

[0075] For example, continuous dimension grouping can be performed, that is, grouping multiple consecutive dimensions with adjacent indices in the feature vector together. For instance, for a 768-dimensional feature vector, if N=4, it can be evenly divided into four groups of consecutive dimensions: dimensions 1-192, dimensions 193-384, dimensions 385-576, and dimensions 577-768, with each group corresponding to a feature subset. This approach assumes that semantic information has local continuity in the feature space in order to capture the relevant semantic patterns carried by adjacent dimensions.

[0076] For example, discontinuous dimension grouping can be performed, that is, grouping multiple dimensions in the feature vector where the ordinal numbers are not consecutive and are distributed at intervals. For instance, dimensions can be extracted at fixed intervals, with dimensions 1, 5, 9, 13, etc., grouped into the first group, dimensions 2, 6, 10, 14, etc., grouped into the second group, and so on. This approach can aggregate feature information in a distributed manner from a global perspective, which helps to decouple semantic factors that are relatively discretely distributed in the feature space.

[0077] In other embodiments, the two methods described above can be combined, with some groups consisting of continuous dimensions and others consisting of non-continuous dimensions, thereby taking into account both local relevance and global diversity.

[0078] For example, in a classification task that mixes sentiment and intent, if continuous grouping is used, the feature dimensions of the main sentiment response words can be grouped into one group, and the feature dimensions of the main intent response keywords can be grouped into another group, thus achieving a preliminary separation of sentiment and intent at the feature subset level. If non-continuous grouping is used, each feature subset can contain feature components from different semantic aspects, thereby maintaining a certain degree of semantic diversity within each subset, which is beneficial for more fine-grained cross-analysis.

[0079] Understandably, by flexibly grouping feature dimensions, feature subsets with different semantic concentrations and diversity can be constructed based on actual task requirements and data characteristics. This provides a structured data foundation for focused and complementary semantic analysis in subsequent processing layers. Furthermore, this partitioning mechanism does not depend on the specific content of the input text, but rather on the ordinal numbers of the feature dimensions or grouping rules learned through training, ensuring the universality and transferability of the partitioning.

[0080] Based on the above embodiments, in this embodiment, the division is performed in any one of the following ways: the feature dimensions are evenly divided into N groups; if the total number of feature dimensions is not divisible by N, the remaining feature dimensions are allocated to the target processing layer or discarded; or if the total number of feature dimensions is not divisible by N, the value of N is adjusted to a divisible value that meets the preset conditions.

[0081] Since in practical applications, the total number of dimensions of feature representation (e.g., 768 dimensions) and the number of layers N may not be an integer multiple, this embodiment provides a variety of configurable partitioning strategies to ensure the robustness and feasibility of feature partitioning operations under different parameter configurations, and to provide structurally regular inputs for the independent computation of each subsequent processing layer.

[0082] In the uniform partitioning mode, the system can directly divide the total number of dimensions D by N and round down to obtain the baseline number of dimensions for each group. It can also divide the continuous or regularly distributed dimensions into N groups with an equal number of dimensions to achieve simple partitioning. This is beneficial for each processing layer to obtain a balanced semantic information load and is suitable for scenarios where D is divisible by or close to divisible by N.

[0083] If the total number of dimensions D is not divisible by N, a remainder distribution or discarding mode can be used. Specifically, a uniform partition can be performed first, and then the dimensions that cannot be evenly divided and remainder R can be further processed. For example, all remaining dimensions R can be appended to a pre-specified target processing layer (e.g., the first or last layer), giving that layer more dimensions than other layers, thus completing the partition without loss of information; or, these R dimensions can be discarded directly, using only the D' dimensions (D'=DR) that can be evenly divided for partitioning, to maintain the consistency of input dimensions across all processing layers and simplify subsequent parallel computation.

[0084] If it is necessary to strictly ensure that the number of dimensions in each group is exactly the same, an adaptive adjustment mode for the number of strata can be adopted. In this mode, when D is not divisible by N, the system can adaptively adjust the value of N to make it a value M that can divide D. This adjustment can be made according to preset conditions, such as "taking the closest divisible value to N". Specific adjustment methods can include: rounding up to the smallest divisible value greater than or equal to N, rounding down to the largest divisible value less than or equal to N, or skipping the current value of N and selecting an adjacent divisible value (e.g., when N=7, if D is not divisible by 7, skip 7 and use 6 or 8 as the actual number of groups).

[0085] For example, suppose the total number of feature dimensions D=768, and the initial N=5 obtained from training. Since 768 is not divisible by 5, if the remainder distribution mode is used, the 768 dimensions can be evenly divided into 5 groups of 153 dimensions each, and the remaining 3 dimensions can be added to the first processing layer to give it 156 dimensions. If the discard mode is used, the 765 dimensions (768-3) can be evenly divided into 153 dimensions each. If the adjustment mode is used, the largest value M=4 less than 5 that is divisible by 768 can be found by rounding down, and the even division can be done with 768÷4=192, or N can be adjusted to the closest divisible value M=6 (768÷6=128).

[0086] Understandably, by providing the above-mentioned multiple optional partitioning execution strategies, it is possible to flexibly choose the balance between information preservation integrity, computational regularity and implementation complexity when facing different dimensional scales and hierarchical requirements, thereby enhancing the adaptability and deployment convenience of text processing under different hardware configurations and task constraints.

[0087] Based on the above embodiments, in this embodiment, for each processing layer, the layer probability of the text to be classified belonging to each category under that layer is determined based on the feature subset of that layer, including: obtaining the category prototype of each category under each processing layer; calculating the similarity between the feature subset of each processing layer and the category prototype; and calculating the layer probability of the text to be classified belonging to each category under each processing layer based on the similarity.

[0088] For the current processing layer, firstly, the corresponding category prototype for each candidate category at this layer can be obtained. This prototype represents the typical features of the corresponding category in the semantic dimensions of interest at this layer. Then, the feature subset of the input text at the current processing layer can be compared one by one with the category prototypes of all candidate categories at this layer, and the similarity metric between the two (e.g., Euclidean distance, cosine similarity, etc.) can be calculated to evaluate the proximity of the input text to each category from the semantic perspective of this layer. Based on the calculated similarity values, they can be mapped to a probability distribution using a preset transformation function (e.g., normalized exponential function). This distribution represents the layer probability of the text to be classified belonging to each category at the current processing layer, reflecting the likelihood of the text belonging to each category from the perspective of this layer.

[0089] To adapt to the different requirements of computational efficiency and discrimination precision in different scenarios, the similarity calculation and probability conversion in the above process can be implemented in different ways.

[0090] In Implementation Method 1 (Unified Discrimination Mode), all processing layers can use the same similarity metric function (such as Euclidean distance) and probability transformation function (such as Softmax) to achieve uniformity and computational efficiency. This is suitable for task scenarios where the semantic dimensions of each processing layer are clearly distinguishable and the discrimination logic is relatively consistent. For example, in sentiment classification tasks, regardless of whether the processing layer focuses on sentiment intensity, sentiment object, or sentiment cause, a unified distance metric can be used to measure the difference between the text and various sentiment prototypes.

[0091] In Implementation Method Two (Adaptive Discrimination Mode), different processing layers can be allowed to employ adaptive similarity calculation or probability transformation strategies based on the characteristics of their semantic concerns. For example, for processing layers primarily focused on discrete category discrimination (such as intent recognition), distance-based discrimination can be used; for processing layers focused on continuous semantic intensity (such as sentiment intensity analysis), vector projection-based or correlation-based discrimination can be used. This mode, by customizing discrimination mechanisms for each layer, can improve the ability to model complex and heterogeneous semantic attributes.

[0092] For example, in a customer service query classification scenario involving three layers of processing, the first layer focuses on "query intent," with category prototypes including "inquiry," "complaint," and "processing," etc.; the second layer focuses on "emotional polarity," with category prototypes including "positive," "neutral," and "negative," etc.; and the third layer focuses on "urgency," with category prototypes including "normal" and "urgent." For a user query like "Why hasn't my order arrived yet? I'm very anxious!", the similarity between feature subsets and corresponding prototypes can be calculated at each layer and converted into layer probabilities. Ultimately, the first layer can provide a high probability for "inquiry," the second layer a high probability for "negative," and the third layer a high probability for "urgent," collectively constituting a multi-dimensional semantic description of the text.

[0093] Understandably, by calculating the layer probabilities separately, each processing layer can independently complete prototype-based category discrimination within its assigned feature subspace. This not only achieves semantic decoupling analysis but also provides reliable probabilistic input for subsequent fusion of the discrimination results from each layer to form a comprehensive and robust final classification decision.

[0094] Based on the above embodiments, in this embodiment, the category prototypes of each category under each processing layer are obtained, including: calculating the category prototypes based on the feature subsets of the support set under each processing layer, wherein the support set contains at least one labeled sample of the actual category to which the text to be classified belongs.

[0095] Since the text processing method provided in this embodiment can be applied to small sample learning scenarios, it is impossible to pre-store fixed prototypes of all possible categories. Therefore, each time a classification task is performed, the prototypes of each category can be calculated in real time using a small number of labeled samples (i.e., support sets) provided by the current task, so as to ensure that the prototypes can accurately reflect the specific feature distribution of each category in the current task.

[0096] Given a support set containing C categories, each with K labeled samples, we can obtain the category prototypes for each category at each processing layer. In this process, firstly, all samples in the support set can be input into the encoding model to obtain the overall feature representation of each sample. Then, following the same method as the text to be classified, the feature representation of each sample can be divided into N feature subsets. For the i-th sample, its k-th feature subset is denoted as... Here, k = 1, 2, ..., N represents the processing layer index. Then, for each processing layer k and its corresponding feature subspace, intra-layer category prototype aggregation can be performed. Specifically, for each category c (there are C categories in total), all samples belonging to category c can be selected from the support set. Then, the feature subset of these samples belonging to category c at layer k can be... (where S) cAggregation operations are performed on the set of sample indices representing category c. The aggregation operation can be used to calculate the mean, i.e., the hierarchical prototype of that category. It can be obtained from the following formula:

[0097]

[0098] in, This represents the number of samples supporting category c. The mean operation can effectively extract the central tendency of this category of samples in the current semantic dimension (the k-th layer), forming a representative prototype.

[0099] By performing the above aggregation on each of the C categories sequentially, we can obtain the prototype set of all C categories under processing layer k. .

[0100] By traversing all N processing layers, a complete prototype system can eventually be constructed, meaning that each category c has N prototypes. Each corresponds to N different processing layers and semantic dimensions.

[0101] For example, in a customer service intent recognition task, the support set provides five categories (e.g., "check balance," "transfer," "complaint," "consultation fee," and "report lost"), with K labeled samples for each category (K≥1). The model first encodes all samples in the support set and divides the feature representation of each sample into four feature subsets according to a preset partitioning rule. For the "check balance" category, the system can aggregate the feature subsets of all K samples at layer 1 (e.g., calculate the mean) to obtain the prototype of this category at layer 1; similarly, it can aggregate the feature subsets of all K samples at layers 2, 3, and 4 to obtain the prototypes of this category at layers 2, 3, and 4, respectively. The other four categories can be processed in the same way, thus calculating a prototype for each category at each processing layer.

[0102] Understandably, by dynamically calculating the category prototypes of each layer based on the task-specific support set, the model can quickly adapt to unseen categories while ensuring that the prototypes are highly correlated with the data distribution of the current task.

[0103] In embodiments of this disclosure, the text processing method further includes: dividing N feature subsets into at least one combined feature subset, each combined feature subset containing at least two feature subsets; each combined feature subset corresponding one-to-one with a combined processing layer; for each combined processing layer, determining the combined probability of the text to be classified belonging to each category under that combined processing layer; and determining the comprehensive probability based on the combined probability of each combined processing layer and the layer probability of each processing layer.

[0104] In other embodiments, the N basic feature subsets obtained from the division can be further grouped to form at least one combination. Each combination consists of two or more basic feature subsets, i.e., combined feature subsets.

[0105] During the grouping process, basic feature subsets with adjacent indices can be merged (e.g., combining layers 1-2 and 3-4 respectively). Alternatively, semantically complementary or related basic feature subsets can be merged based on semantic associations learned during training or predefined (e.g., combining a feature subset encoding sentiment polarity with a feature subset encoding sentiment intensity). Alternatively, a lightweight attention network can be used to dynamically calculate the association weights between each basic feature subset, and perform soft or hard grouping based on these weights. After grouping, a corresponding combination processing layer can be built for each combined feature subset. This combination processing layer is similar to the basic processing layer, but its input is a combined feature subset that integrates multiple basic semantic information.

[0106] In each combined processing layer, based on its combined feature subset, the combined probability of the text to be classified belonging to each category under that layer is determined. The specific calculation method is analogous to the basic processing layer. That is, based on the corresponding combined feature subset of the support set samples, the combined category prototype of each category under the current combined processing layer is calculated (e.g., taking the average of the combined feature subsets of samples of the same category). Then, the similarity between the combined feature subset of the text to be classified and the combined prototypes of each category can be calculated, and converted into a probability distribution through a normalization function, thus obtaining the combined probability of that combined processing layer.

[0107] When determining the overall probability of a text belonging to each category, we can consider not only the layer probabilities output by all N basic processing layers, but also the combined probabilities output by all combined processing layers. The fusion strategy can be a weighted average of all layer probabilities and combined probabilities, or a hierarchical attention mechanism can be used to dynamically assign different importance weights to layers of different types and granularities based on the current input, and then perform weighted aggregation.

[0108] For example, in a system with N=4, the basic processing layers focus on "emotional polarity," "topic," "intention," and "object," respectively. Through a combination mechanism, the two basic feature subsets, "emotional polarity" and "object," can be merged into a combined feature subset, corresponding to a combined processing layer. This layer can be used to analyze "emotions directed towards a specific object" (e.g., "anger towards a product"). Simultaneously, "topic" and "intention" can be merged into another combined feature subset, corresponding to another combined processing layer, used to analyze "intentions under a specific topic" (e.g., "consultation intentions under the after-sales topic"). Finally, by combining the probabilities of these four basic layers and the two combined layers, a more comprehensive and context-aware classification result can be obtained.

[0109] Understandably, by introducing a combination processing mechanism, a multi-level semantic analysis system from fine-grained (basic layer) to coarse-grained (combination layer) is constructed. This not only captures discriminative information on a single semantic dimension, but also effectively models the interaction and synergistic effects between different semantic dimensions. As a result, when faced with complex texts containing multiple and intertwined semantics, it provides more accurate and robust classification capabilities, further enhancing the practicality of the model in real-world application scenarios.

[0110] Based on the above embodiments, in this embodiment, for each combined processing layer, the combined probability of the text to be classified belonging to each category under that combined processing layer is determined, including: fusing feature subsets of each processing layer within each combined processing layer and determining the combined probability based on the fused feature subsets; or fusing the layer probabilities of each processing layer within each combined processing layer to obtain the combined probability.

[0111] In determining the combination probability, feature subsets from each base processing layer within the current combination processing layer can be fused to form a fused combination feature representation. Fusion operations can include, but are not limited to, vector concatenation, element-wise weighted summation, or attention-based dynamic fusion. For example, if a combination processing layer consists of feature subsets from processing layer A and processing layer B, a combination feature vector with doubled dimensions can be obtained through concatenation. Then, based on the support set samples, the combination category prototypes for each category under this combination processing layer can be calculated (e.g., averaging the fused features of samples of the same category). Finally, the similarity between the fused features of the text to be classified and the combination prototypes of each category can be calculated and converted into a probability distribution, thus obtaining the combination probability of this combination processing layer. By performing early fusion at the feature level, richer original information can be preserved, which is beneficial for the combination processing layer to learn more complex joint semantic representations beyond a single base layer.

[0112] In other embodiments, the layer probabilities output by each basic processing layer within the current combined processing layer can be directly fused. That is, for a combined processing layer, its input is the layer probabilities already calculated by each basic processing layer under its jurisdiction. The fusion operation can be a simple average (such as an arithmetic average), a weighted average, or based on a certain decision rule (such as taking the maximum value). For example, if a combined processing layer is associated with processing layer C and processing layer D, the average of the layer probabilities output by these two processing layers that belong to the same category can be directly taken as the combined probability of the combined processing layer for that category. The probability-based fusion method is more lightweight and efficient, without the need to recalculate prototypes and similarities, and is suitable for scenarios that are sensitive to computing resources or require fast reasoning.

[0113] For example, in an embodiment that combines the "emotional polarity layer" and the "intent type layer" into a "emotion-intent joint layer," if a feature fusion approach is used, the feature subsets (e.g., 64-dimensional) of the text to be classified in the emotional polarity layer and the feature subsets (e.g., 64-dimensional) in the intent type layer are concatenated to obtain a 128-dimensional combined feature vector. Then, the joint prototype is calculated using the similar combined features of the support set samples, and a similarity comparison is performed to finally output the combined probability. If a probability fusion approach is used, the layer probability distribution output by the emotional polarity layer (e.g., [positive: 0.8, negative: 0.2]) and the layer probability distribution output by the intent type layer (e.g., [consultation: 0.6, complaint: 0.4]) can be directly obtained. Then, for each possible emotional-intent combination category (e.g., "positive-consultation," "positive-complaint," "negative-consultation," "negative-complaint"), the combined probability of that joint category is calculated according to a certain rule (e.g., multiplying the corresponding emotional probability by the intent probability).

[0114] It is understandable that by providing the above two optional combinations of probabilities to determine the path, it is possible to flexibly choose the appropriate implementation strategy according to specific task requirements, model complexity constraints, and performance goals.

[0115] Based on the above embodiments, in this embodiment, determining the semantic category of the text to be classified based on comprehensive probability includes: determining the category with the highest probability as the semantic category of the text to be classified based on comprehensive probability; or determining at least one category with a comprehensive probability exceeding a preset threshold as the semantic category.

[0116] Due to the complexity of actual text semantics, user queries may contain a single explicit intent, or they may implicitly contain multiple or complex semantic attributes. To adapt to different business scenarios and decision-making needs, this embodiment provides different output logics.

[0117] In the single-label output logic, the category with the highest value in the comprehensive probability vector can be used as the final and unique semantic determination result. This logic follows the "maximum confidence" principle and is suitable for scenarios where categories are mutually exclusive and downstream business processes need to clearly define a unique classification result. For example, in a scenario where customer service dialogues are routed to different skill groups, a user query can ultimately only be assigned to one processing queue. If, after calculation, the comprehensive probability of the query "I want to change my password" for the "password reset" category is 0.92, which is much higher than other categories, then "password reset" can be directly output as its semantic category, triggering the corresponding password change process.

[0118] In the multi-label output logic, a confidence threshold can be pre-set, and all categories with a combined probability exceeding this threshold are output as the semantic category of the text. This logic acknowledges that text can be associated with multiple non-mutually exclusive semantic dimensions simultaneously, making it suitable for scenarios requiring the identification of multiple attributes or aspects of text. For example, when analyzing product reviews, a review might simultaneously contain praise for the "camera function" and criticism of "battery life." If the combined probability of both "camera-positive" and "battery life-negative" exceeds a threshold (e.g., 0.6), then both category labels can be output simultaneously, thus providing a more comprehensive and refined characterization of user feedback.

[0119] In other embodiments, besides determining the final category, all categories can be sorted from highest to lowest probability based on comprehensive probabilities, outputting a sorted list of categories and their corresponding probability values. This output method does not perform hard truncation but instead transmits all probability information completely to the downstream system, allowing the downstream business logic to make the final decision based on actual needs (such as setting dynamic thresholds or combining other features). For example, in understanding search engine query intent, categories such as "flight booking," "hotel search," and "travel guides" are sorted by probability and output for use by subsequent hybrid sorting modules.

[0120] In other embodiments, a rejection threshold (typically lower than the classification threshold) can be introduced. When the highest overall probability is lower than this rejection threshold, no specific category can be output. Instead, the text can be marked as "low confidence" or "unclear intent" and routed to manual review or a more complex processing flow. This effectively controls the risk of misclassification in automatic classification. This approach is suitable for scenarios with extremely high accuracy requirements or a large number of queries with unknown categories.

[0121] Understandably, through diverse output logic, it can flexibly adapt to various business needs, from strict single classification to open multi-label recognition, thereby achieving a balance between the accuracy, coverage, and business adaptability of classification results.

[0122] Figure 6 A block diagram of a text processing apparatus according to an embodiment of the present disclosure is shown schematically.

[0123] like Figure 6 As shown, the text processing device 600 includes an encoding module 610, a segmentation module 620, a determination module 630, a synthesis module 640, and a judgment module 650.

[0124] According to some embodiments of this disclosure, the text processing device 600 can be used to implement the reference. Figures 2-5 The text processing method described according to embodiments of this disclosure.

[0125] The encoding module 610 can perform, for example, operation S510, to encode the text to be classified and obtain a feature representation of the text to be classified.

[0126] The partitioning module 620 can perform, for example, operation S520, to partition the feature representation into N feature subsets, where N is an integer greater than 1, and the value of N is determined through the training process of the target model, with each feature subset corresponding to a processing layer.

[0127] The determination module 630 can perform, for example, operation S530, for determining the layer probability of the text to be classified belonging to each category under each processing layer based on a subset of features of that layer.

[0128] The integration module 640 can perform, for example, operation S540, to integrate the layer probabilities of each processing layer to obtain the integrated probability of the text to be classified belonging to each category.

[0129] The determination module 650 can perform, for example, operation S550, to determine the semantic category of the text to be classified based on the comprehensive probability.

[0130] For example, any plurality of the encoding module 610, partitioning module 620, determination module 630, synthesis module 640, and decision module 650 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of the present disclosure, at least one of the encoding module 610, partitioning module 620, determination module 630, synthesis module 640, and decision module 650 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the encoding module 610, the partitioning module 620, the determining module 630, the synthesis module 640, and the judgment module 650 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0131] Figure 7 A block diagram schematically illustrates an electronic device suitable for implementing a text processing method according to an embodiment of the present disclosure.

[0132] like Figure 7As shown, an electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage portion 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0133] RAM 703 stores various programs and data required for the operation of electronic device 700. Processor 701, ROM 702, and RAM 703 are interconnected via bus 704. Processor 701 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 702 and / or RAM 703. It should be noted that programs may also be stored in one or more memories other than ROM 702 and RAM 703. Processor 701 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in one or more memories.

[0134] According to embodiments of this disclosure, the electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to a bus 704. The electronic device 700 may also include one or more of the following components connected to the I / O interface 705: an input section 706 including target hardware, etc.; an output section 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card such as a LAN card, a modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0135] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0136] According to embodiments of this disclosure, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 702 and / or RAM 703 and / or one or more memories other than ROM 702 and RAM 703 described above.

[0137] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the text processing methods provided in the embodiments of this disclosure.

[0138] When the computer program is executed by the processor 701, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0139] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 709, and / or installed from a removable medium 711. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0140] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0141] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on a user's computing device, partially on a user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0142] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0143] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0144] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A text processing method, comprising: The text to be classified is encoded to obtain the feature representation of the text to be classified. The feature representation is divided into N feature subsets, where N is an integer greater than 1, and the value of N is determined through the training process of the target model. Each feature subset corresponds one-to-one with a processing layer. For each processing layer, the layer probability of the text to be classified belonging to each category under that layer is determined based on the feature subset of that layer; By combining the probabilities of each processing layer, the combined probability of the text to be classified belonging to each category is obtained; The semantic category of the text to be classified is determined based on the comprehensive probability.

2. The method according to claim 1, wherein dividing the feature representation into N feature subsets includes: The feature dimensions of the feature representation are divided into N groups, each group containing continuous feature dimensions and / or non-continuous feature dimensions. The feature values ​​corresponding to each feature dimension are taken as the feature subset.

3. The method according to claim 2, wherein, The partitioning is performed using any one of the following: The feature dimension is uniformly divided into N groups; If the total number of feature dimensions is not divisible by N, the remaining feature dimensions are allocated to the target processing layer or discarded; or If the total number of the feature dimensions is not divisible by N, adjust the value of N to a divisible value that meets the preset conditions.

4. The method according to claim 1, further comprising: The N feature subsets are divided into at least one combined feature subset, and each combined feature subset contains at least two feature subsets; Each of the aforementioned combined feature subsets corresponds one-to-one with the combined processing layer; For each of the combined processing layers, determine the combined probability of the text to be classified belonging to each category under that combined processing layer; The overall probability is determined based on the combination probability of each of the combined processing layers and the layer probability of each of the processing layers.

5. The method according to claim 4, wherein determining the combined probability of the text to be classified belonging to each category under each of the combined processing layers includes: By fusing feature subsets from each of the combined processing layers, the combined probability is determined based on the fused feature subsets; or The combined probability is obtained by fusing the layer probabilities of each processing layer within each of the combined processing layers.

6. The method according to claim 1, wherein determining the layer probability of the text to be classified belonging to each category under each processing layer based on a feature subset of that layer includes: Obtain the category prototypes for each category under each of the aforementioned processing layers; Calculate the similarity between the feature subsets of each processing layer and the prototypes of each category; Based on the similarity, the layer probability of the text to be classified belonging to each category under each of the processing layers is calculated.

7. The method according to claim 6, wherein obtaining the category prototypes of each category under each of the processing layers comprises: The category prototype is calculated based on the feature subsets of the support set under each of the processing layers, wherein the support set contains at least one labeled sample of the actual category to which the text to be classified belongs.

8. The method according to claim 1, wherein determining the semantic category of the text to be classified based on the comprehensive probability comprises: Based on the comprehensive probability, the category with the highest probability is determined as the semantic category of the text to be classified; or At least one category whose overall probability exceeds a preset threshold is identified as the semantic category.

9. A text processing apparatus, comprising: The encoding module is used to encode the text to be classified to obtain the feature representation of the text to be classified; The partitioning module is used to divide the feature representation into N feature subsets, where N is an integer greater than 1, and the value of N is determined through the training process of the target model. Each feature subset corresponds one-to-one with a processing layer. The determination module is used to determine the probability that the text to be classified belongs to each category in each processing layer based on the feature subset of that layer. The synthesis module is used to synthesize the layer probabilities of each processing layer to obtain the comprehensive probability that the text to be classified belongs to each category; The determination module is used to determine the semantic category of the text to be classified based on the comprehensive probability.

10. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors perform the following method: The text to be classified is encoded to obtain its feature representation; the feature representation is divided into N feature subsets, where N is an integer greater than 1, and the value of N is determined through the training process of the target model, with each feature subset corresponding to a processing layer; for each processing layer, the layer probability of the text to be classified belonging to each category under that layer is determined based on the feature subset of that layer; the layer probabilities of each processing layer are combined to obtain the comprehensive probability of the text to be classified belonging to each category; the semantic category of the text to be classified is determined based on the comprehensive probability.

Images