A text retrieval method, model, training method, and medium for a vertical domain
By constructing domain trees and large language models to generate text content and optimizing text classification models, the problem of lack of professional knowledge in general models is solved, and high efficiency, accuracy and rapid adaptability of vertical domain text retrieval are achieved.
Patent Information
- Application Number
- CN202611389269.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-08
- Publication Date
- 2026-11-17
AI Technical Summary
In existing technologies, general pre-trained models lack expertise in vertical domains, resulting in poor classification performance, difficulty in rapid synchronous evolution, and impact on the accuracy and efficiency of business text retrieval.
By constructing a domain tree, generating text content using a large language model, and optimizing the domain tree and text classification model based on confidence, an iterative optimization mechanism is formed to improve the accuracy of classification labels.
It improves the accuracy and efficiency of text retrieval in vertical domains, reduces reliance on manual annotation, and adapts to the rapid changes in knowledge within vertical domains.
Smart Images

Figure CN123115409A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of vertical domain text classification, and in particular to a vertical domain text retrieval method, model, training method and medium. Background Technology
[0002] In the text processing business of vertical industries, a large amount of unstructured text data is often generated. Taking the ICT hardware field as an example, text data includes chip specifications, design documents, process specifications, fault tickets, and industry standards. To facilitate retrieval, this text data needs to be mapped to corresponding category tags.
[0003] In related technologies, general-domain pre-trained models can be used for classification. However, these models lack expertise in specific domains and require manual annotation to train and adapt them to particular domains. As technology evolves, new technical branches emerge within the domain, rendering the existing labeling system inadequate. Updating the labeling system necessitates manually re-annotating large amounts of training data and retraining the model, making it difficult for the model to evolve rapidly in sync with constantly updated domain knowledge. This ultimately leads to poor classification performance and impacts the accuracy of business text retrieval. Summary of the Invention
[0004] In view of this, the embodiments of this specification provide a text retrieval method, model, training method, and medium for a vertical domain. Specifically, this specification is implemented through the following technical solutions: The first aspect of this specification provides a text retrieval method for a vertical domain, including: Obtain a pre-constructed domain tree based on a knowledge system of a specified vertical domain. The domain tree includes multiple levels, each level includes at least one node, and each node has path information and basic description information. The path information is used to characterize the relationship between the node and other nodes in the knowledge system, and the basic description information is used to characterize the scope of the semantics of the node in the knowledge system. For each node, at least one text content describing the node is generated using a large language model based on the node's path information and basic description information. For each piece of text content, the text content is input into a text classification model containing multiple levels of text classifiers, so that the text classification model classifies the text content layer by layer through the text classifiers at each level, and obtains at least one classification label of the text content and the confidence level corresponding to each classification label. The hierarchical structure of the text classification model is constructed based on the hierarchical structure of the domain tree, and the classification label is used to characterize the node to which the text content belongs in the domain tree. If each confidence level is not lower than a preset threshold, the text content is used as the input feature, and each classification label of the text content is used as the ground truth label to construct a sample for training the text classification model. If any of the confidence levels is lower than a preset threshold, the target node in the domain tree is optimized using the text content. The target node is the node represented by the classification label corresponding to the confidence level, so as to regenerate the text content and construct samples based on the optimized domain tree. The text classification model is trained using the samples, and the trained text classification model is used to generate classification labels for the business text in the vertical domain, so as to retrieve the business text based on the classification labels.
[0005] The second aspect of the embodiments of this specification provides a method for training a text classification model in a vertical domain, including: Obtain a pre-constructed domain tree based on a knowledge system of a specified vertical domain. The domain tree includes multiple levels, each level includes at least one node, and each node has path information and basic description information. The path information is used to characterize the relationship between the node and other nodes in the knowledge system, and the basic description information is used to characterize the scope of the semantics of the node in the knowledge system. For each node, at least one text content describing the node is generated using a large language model based on the node's path information and basic description information. For each piece of text content, the text content is input into a text classification model containing multiple levels of text classifiers, so that the text classification model classifies the text content layer by layer through the text classifiers at each level, and obtains at least one classification label of the text content and the confidence level corresponding to each classification label. The hierarchical structure of the text classification model is constructed based on the hierarchical structure of the domain tree, and the classification label is used to characterize the node to which the text content belongs in the domain tree. If each confidence level is not lower than a preset threshold, the text content is used as the input feature, and each classification label of the text content is used as the ground truth label to construct a sample for training the text classification model. If any of the confidence levels is lower than a preset threshold, the target node in the domain tree is optimized using the text content. The target node is the node represented by the classification label corresponding to the confidence level, so as to regenerate the text content and construct samples based on the optimized domain tree. The text classification model is trained using the samples.
[0006] The third aspect of the embodiments of this specification provides a vertical domain text classification model for generating classification labels for received business text, wherein the business text is text from a specified vertical domain. The text classification model is trained by the method described in the second aspect.
[0007] The fourth aspect of this specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods provided in the first and second aspects.
[0008] This specification's embodiments introduce a pre-constructed domain tree into the training process of a text classification model in a structured manner, incorporating a vertical domain's knowledge system. The path information and basic description information of nodes in this domain tree define the semantic boundaries and technical connotations of various professional terms within the vertical domain. Based on this, when generating training text content, the large language model can generate training text content within the domain scope defined by the path information and the semantic boundaries defined by the basic description information. This avoids the situation where a general-purpose large language model, lacking vertical domain expertise, generates semantically ambiguous or deviating text content from the actual business scenario, thus mitigating the difficulties faced when relying on manual annotation to obtain training samples.
[0009] Furthermore, by judging and processing the confidence level of the text classification model output, the target node in the domain tree is optimized when the confidence level is lower than a preset threshold. The optimized domain tree then guides the subsequent training of the text classification model, thereby forming a mutually driving iterative optimization mechanism between the domain tree and the text classification model. This improves the accuracy of the text classification model in classifying business text outputs, thus enhancing the efficiency and accuracy of the vertical domain text retrieval process. Attached Figure Description
[0010] To more clearly illustrate the technical solutions in the embodiments of this specification, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0011] Figure 1 This is a flowchart illustrating an exemplary embodiment of a text retrieval method in a vertical field.
[0012] Figure 2 This is a schematic diagram illustrating the application architecture of a text retrieval method as exemplified in the embodiments of this specification.
[0013] Figure 3This is a schematic diagram illustrating the hierarchical structure of the domain tree as an example of an embodiment of this specification.
[0014] Figure 4 This is a schematic diagram illustrating the hierarchical structure of a text classification model as an example of an embodiment of this specification.
[0015] Figure 5 This is a data interaction diagram illustrating the feedback optimization process as exemplified in the embodiments of this specification.
[0016] Figure 6 This is a flowchart illustrating a training method for a text classification model in a vertical domain, as provided in the embodiments of this specification. Detailed Implementation
[0017] In vertical industries involving text processing, a large amount of unstructured text data is typically generated. Taking ICT hardware fields such as integrated circuits, computing infrastructure, and communication networks as examples, the text data can include chip datasheets, printed circuit board design documents, packaging process specifications, device selection reports, after-sales service tickets, and relevant industry standards and academic literature. To facilitate the organization, archiving, and retrieval of this text data, it is usually necessary to classify the text data and map it to predefined category tags to support subsequent management and querying based on these tags.
[0018] However, the ICT hardware vertical field exhibits certain unique characteristics in text classification. First, this field involves numerous technical terms, such as BGA packaging, wide-bandgap semiconductors, and specific technical concepts like box switches and chassis switches. The semantic boundaries between these technical concepts are relatively narrow, and similar terms can have significantly different meanings in different contexts. Second, the knowledge system in this field has a finely hierarchical structure. For example, semiconductor devices can be further divided into passive devices, active devices, integrated circuits, and other subcategories, each of which can be further subdivided into smaller, more granular technical branches.
[0019] In related technologies, a common approach to classifying the aforementioned text data is to directly utilize pre-trained models in a general domain. This method leverages the general language understanding capabilities accumulated during the pre-training phase of these models, using prompts or a small number of samples to guide the model in outputting corresponding classification labels for given text. However, pre-trained models in a general domain primarily encounter general domain texts during pre-training, resulting in relatively limited understanding of specialized terminology and subdivided knowledge structures within vertical domains. Therefore, when directly applied to text classification in vertical domains such as ICT hardware, the following problems may arise: Firstly, the model's semantic understanding of specialized terminology may be insufficient, easily leading to the categorization of text within similar technical fields into broad categories, resulting in coarse-grained classification results that are difficult to align with the knowledge hierarchy of the vertical domain itself. Secondly, the correspondence between the model's output and predefined classification labels depends on the guidance of prompts, and its stability and consistency are subject to uncertainty.
[0020] Therefore, to obtain classification results that match the knowledge level of the vertical domain in the above scenarios, we can consider building corresponding text classification models based on the knowledge system of the specific vertical domain. This ensures that the classification granularity and boundaries of the text classification model correspond to the knowledge level of the vertical domain. Compared to directly using large language models pre-trained in a general domain, this type of text classification model can better fit the specific granularity of the vertical domain and directly output corresponding classification labels for business texts.
[0021] However, constructing the aforementioned text classification model relies on obtaining high-quality training samples. In vertical fields such as ICT hardware, obtaining high-quality training samples for this type of text classification model may face the following difficulties in practice: First, sample texts that conform to the semantic boundaries of the vertical field's subcategories are usually distributed in professional materials such as professional specifications, industry standards, and internal technical documents. Moreover, the technical content involved in some subcategories is relatively new or highly specialized, and the publicly available sample texts for reference are relatively limited, making it difficult to obtain a sufficient number of training samples that cover all subcategories in a short period of time. Second, if training samples are supplemented by manual annotation, the annotators usually need to have the corresponding domain background to make relatively accurate judgments on the sample texts. At the same time, for sample texts that span multiple subcategories or are located at the intersection of domains, different annotators may give different classification labels based on their own understanding, resulting in inconsistent labels in the training samples. Third, the knowledge level and technical boundaries of the vertical field itself will continue to adjust with industry development, and the attribution of training samples to classification labels may change accordingly. The adaptability of training samples to the new knowledge level also needs to be updated accordingly.
[0022] The aforementioned issues further impact the retrieval performance of text within vertical domains in practical business applications. Specifically, when the quantity or quality of training samples fails to meet the training requirements of a text classification model, the resulting classification labels output by the model for business text may deviate from the actual subcategories to which the text belongs. Consequently, in subsequent tag-based retrieval processes, relevant business text may not be retrieved, or the retrieval results may not match the query intent satisfactorily. Furthermore, when the knowledge hierarchy within a vertical domain changes, and the classification labels and text classification model fail to adjust accordingly, business text involving new technology branches may be difficult to locate accurately during retrieval. Therefore, in practical vertical domain business applications, the efficiency and accuracy of tag-based retrieval of business text may be affected.
[0023] In view of this, the embodiments of this specification propose a text retrieval method for a vertical domain. See also... Figure 1 and Figure 2 , Figure 1 This is a schematic flowchart illustrating a text retrieval method in a vertical field, as exemplified by an embodiment of this specification. Furthermore, to more intuitively demonstrate the core components and their interrelationships involved in the embodiments of this specification, Figure 2 This is a schematic diagram illustrating the application architecture of the text retrieval method as exemplified in an embodiment of this specification. (Refer to...) Figure 1 and Figure 2 The method may include the following steps.
[0024] S101: Obtain a pre-built domain tree based on the knowledge system of the specified vertical domain.
[0025] The designated vertical domain refers to the specific application field addressed by the embodiments of this specification, such as the aforementioned ICT hardware fields like integrated circuits, computing infrastructure, and communication networks. The knowledge system of the designated vertical domain refers to the overall structure constituted by the various technical concepts within that domain and their hierarchical relationships.
[0026] Each node 2011 in the domain tree 201 has path information and basic description information. Path information is used to characterize the relationship between node 2011 and other nodes 2011 in the knowledge system. In one implementation, path information can be expressed as a complete hierarchical link from the root node 2011 to the node 2011, such as "Semiconductor and Microelectronics → Packaging Form → BGA". The path information determines the hierarchical position of node 2011 in the domain tree 201 and its relationship with other nodes 2011. Basic description information is used to characterize the semantic scope of node 2011 within the knowledge system. Basic description information can be in the form of text descriptions, explaining the connotation, extension, and main characteristics of the technical concept corresponding to node 2011. For example, the basic description information corresponding to the "BGA" node 2011 could explain that BGA packaging is a surface mount packaging technology, its solder balls are arranged in a grid at the bottom of the package, and electrical connections to the printed circuit board are achieved through reflow soldering, etc.
[0027] It should be noted that the domain tree 201 is pre-constructed before being applied to the methods provided in the embodiments of this specification, and its specific construction method can be varied. In one implementation, the domain tree 201 can be constructed by personnel with relevant vertical domain backgrounds, referring to publicly available technical standards, product classification manuals, and technical evolution logic in that domain; in another implementation, the domain tree 201 can also be obtained by organizing and supplementing an existing classification system within that vertical domain. The embodiments of this specification do not limit the specific initialization construction method of the domain tree 201.
[0028] S102: For each node, use the large language model to generate at least one text content describing the node based on the path information and basic description information of the node.
[0029] Among them, the large language model 202 refers to a model that has been pre-trained on a large-scale corpus and has the ability to generate natural language.
[0030] For any node 2011 in the domain tree 201, the path information and basic description information of node 2011 can be provided as input information to the large language model 202. The large language model 202 generates at least one text content describing node 2011 based on this input information. The generated text content is used to describe the technical concept corresponding to node 2011, and its content can cover the technical principles, application scenarios, technical features, or question-and-answer expressions related to the technical concept involved in node 2011. For example, for a node 2011 with path information of "semiconductor and microelectronics → packaging form → BGA" and basic description information of BGA packaging-related explanations, the large language model 202 can generate at least one text content describing the technical characteristics, applicable scenarios, and differences between BGA packaging and other packaging forms. The specific number of text contents generated for each node 2011 can be set according to actual needs, and this embodiment does not limit this.
[0031] By performing the above generation process on each node 2011 in the domain tree 201, multiple text contents corresponding to each node 2011 in the domain tree 201 can be obtained. These text contents can be used as the initial data source for training the text classification model 203.
[0032] S103: For each text content, input the text content into a text classification model containing multiple levels of text classifiers, so that the text classification model classifies the text content layer by layer through the text classifiers at each level, and obtains at least one classification label for the text content and the confidence level corresponding to each classification label.
[0033] The hierarchical structure of the text classification model 203 is constructed based on the hierarchical structure of the domain tree 201. The text classification model 203 contains multiple levels of text classifiers 2031, and each level of text classifier 2031 corresponds to a corresponding level in the domain tree 201.
[0034] For any given text content, after inputting it into the text classification model 203, the model classifies the text content sequentially using text classifiers 2031 at each level, from highest to lowest hierarchy. The classification results output by higher-level text classifiers 2031 can be used to instruct lower-level text classifiers 2031 to participate in subsequent classifications, allowing them to further refine the classification of the text content. Through this hierarchical classification process, at least one classification label 2042 corresponding to the node 2011 to which the text content belongs in the domain tree 201 can be obtained.
[0035] In this context, the classification label 2042 represents the node 2011 to which the text content belongs in the domain tree 201. The confidence level represents the degree to which the text classifier 2031 believes in the corresponding classification label 2042, and typically takes a value in the range [0,1]. A higher confidence level indicates that the text classifier 2031 is more inclined to classify the result corresponding to the corresponding classification label 2042; a lower confidence level indicates that the text classifier 2031 is less inclined to classify the result corresponding to the corresponding classification label 2042. For example, when a text content is classified layer by layer to the node 2011 "Semiconductor and Microelectronics → Packaging Forms → BGA", the classification labels 2042 corresponding to the three nodes 2011 ("Semiconductor and Microelectronics", "Packaging Forms", and "BGA"), and the confidence levels corresponding to each classification label 2042, can be obtained respectively.
[0036] After obtaining the category label 2042 and the confidence score, it is determined whether the confidence score corresponding to each category label 2042 is not lower than a preset threshold, and the corresponding processing branch is entered based on the determination result. The preset threshold is a pre-set reference value used to determine the confidence score, and its specific value can be configured according to actual application requirements.
[0037] If the confidence scores of each category label 2042 corresponding to the text content are not lower than the preset threshold, proceed to step S104: use the text content as input features and each category label of the text content as ground truth labels to construct samples for training the text classification model.
[0038] When the confidence scores of each classification label 2042 are not lower than a preset threshold, the text classification model 203 can be considered to have a relatively high overall credibility in its layer-by-layer classification results of the text content. Therefore, the text content can be combined with its corresponding classification labels 2042 to construct samples 205 for training the text classification model 203. Specifically, the text content can be used as the input feature of sample 205, and the corresponding classification labels 2042 can be used as the ground truth labels for the corresponding levels in sample 205. The constructed sample 205 can be added to the training dataset for use in subsequent steps to train the text classification model 203.
[0039] If the confidence level of at least one category label 2042 corresponding to the text content is lower than a preset threshold, proceed to step S105: optimize the target node in the domain tree using the text content, so as to regenerate the text content and construct samples based on the optimized domain tree.
[0040] In this context, the target node 2011 is the node 2011 represented by the classification label 2042 corresponding to the given confidence level. When the confidence level corresponding to at least one classification label 2042 is lower than a preset threshold, the text classification model 203 can be considered to have low reliability in classifying the text content at the corresponding level. This situation may be related to a deviation between the definition boundary of the corresponding node 2011 in the domain tree 201 and the semantics of the text content, or a deviation between the attribution relationship of the node 2011 and other nodes 2011. Therefore, the node 2011 represented by the classification label 2042 corresponding to the given confidence level can be determined as the target node 2011, and the relevant attributes of the target node 2011 in the domain tree 201 can be adjusted accordingly using the text content, thereby improving the correspondence between the definition of the target node 2011 and the text content. The specific optimization method for the target node 2011 will be further explained in subsequent embodiments.
[0041] After optimizing the target node 2011, step S102 can be executed again for the optimized nodes 2011 in the domain tree 201 (including the aforementioned target node 2011). This involves using the large language model 202 to regenerate the text content describing node 2011 based on the path information and basic description information of the optimized node 2011. The regenerated text content then undergoes the layer-by-layer classification process described in step S103 to obtain corresponding classification labels 2042 and confidence scores. Then, when the above judgment conditions are met, samples 205 are constructed in accordance with the method described in step S104. Through this process, samples 205 for training the text classification model 203 can be continuously obtained while the domain tree 201 is gradually optimized.
[0042] S106: Use samples to train a text classification model, and use the trained text classification model to generate classification labels for business texts in the vertical domain, so as to retrieve business texts based on the classification labels.
[0043] After obtaining sample 205, the text classification model 203 can be trained based on sample 205. The training process can adopt a supervised learning approach: input features from sample 205 are input into text classification model 203, and the classification outputs of text classifiers 2031 at each level of text classification model 203 for the input features are obtained. Each classification output is compared with the ground truth labels of the corresponding levels in sample 205, and the model parameters of text classification model 203 are adjusted accordingly based on the comparison results. After several iterations of training, the trained text classification model 203 can be obtained.
[0044] It should be noted that the above process of training the text classification model 203 based on sample 205 can be carried out uniformly after the sample 205 has accumulated to a certain scale, or it can be carried out in batches as the sample 205 is gradually obtained. This specification does not limit this process.
[0045] After obtaining the trained text classification model 203, business text 2041 from the vertical domain can be provided as input to the text classification model 203. Here, business text 2041 refers to text data generated in the actual business process of the vertical domain that needs to be organized and retrieved. For example, it can be chip specifications, printed circuit board design documents, packaging process descriptions, fault work orders, etc. in the aforementioned ICT hardware field.
[0046] The text classification model 203 classifies the business text 2041 according to the aforementioned hierarchical classification method, and outputs at least one classification label 2042 corresponding to the business text 2041. The node 2011 represented by the output classification label 2042 is the node 2011 to which the business text 2041 belongs in the domain tree 201. After obtaining the classification label 2042 corresponding to the business text 2041, the business text 2041 can be retrieved based on the classification label 2042. In one implementation, the business text 2041 along with its corresponding classification label 2042 can be stored in... Figure 2 The database 204 shown is used to establish an index relationship between category tags 2042 and business text 2041. During actual retrieval, the category tag 2042 corresponding to the query conditions input by the retrieval user can be determined. Then, the business text 2041 corresponding to the target category tag 2042 can be found through the aforementioned index relationship, and the found business text 2041 is output as the retrieval result.
[0047] Therefore, this embodiment of the specification introduces the knowledge system of a vertical domain into the training process of the text classification model in a structured form through a pre-constructed domain tree. The path information and basic description information of the nodes in this domain tree define the semantic boundaries and technical connotations of various professional terms in the vertical domain. Based on this, when generating text content for training, the large language model can generate text content for training within the domain scope defined by the aforementioned path information and the semantic boundaries defined by the basic description information. This avoids the situation where a general large language model generates semantically ambiguous or deviating text content from the actual business scenario due to a lack of vertical domain expertise, and helps to alleviate the difficulties faced when relying on manual annotation to obtain training samples.
[0048] Furthermore, by judging and processing the confidence level of the text classification model output, the target node in the domain tree is optimized when the confidence level is lower than a preset threshold. The optimized domain tree then guides the subsequent training of the text classification model, thereby forming a mutually driving iterative optimization mechanism between the domain tree and the text classification model. This improves the accuracy of the text classification model in classifying business text outputs, thus enhancing the efficiency and accuracy of the vertical domain text retrieval process.
[0049] Specifically, a domain tree is a data format for structurally representing the aforementioned knowledge system. (See also...) Figure 3 , Figure 3 This is a schematic diagram illustrating the hierarchical structure of a domain tree, as exemplified in embodiments of this specification. Figure 3 As shown, the domain tree 201 comprises multiple levels, each level including at least one node. Nodes are connected by lines to represent direct parent-child relationships. Nodes at higher levels typically correspond to broader technology categories, while nodes at lower levels typically correspond to more specific technology categories. The various levels in the domain tree 201 form a complete path from root to leaf. For any given node, its path information consists of a complete hierarchical link starting from the first-level node, passing through each parent node, and finally reaching that node. This path information uniquely determines the node's position in the domain tree 201 and clearly characterizes its affiliation with other nodes.
[0050] It should be noted that, Figure 3 In the structure shown, the hierarchical depth under different branches does not necessarily need to be consistent. Some branches may terminate at a third-level node, while others may continue to extend to a fourth level or even deeper, depending on whether the technical field corresponding to the branch needs further subdivision. Furthermore, child nodes under the same parent node are semantically mutually exclusive in principle to ensure the exclusivity and accuracy of the classification. Those skilled in the art will understand that... Figure 3 This is merely an illustrative representation of the domain tree 201 used in the embodiments of this specification. In actual construction, the number of nodes, the depth of the hierarchy, and the attribute information of each node can be dynamically adjusted according to the business needs and technological evolution of the vertical domain.
[0051] For example, in the ICT hardware field, nodes at the first level could be "semiconductors and microelectronics," "network communication equipment," etc. The next level down from "semiconductors and microelectronics" could be "packaging forms," "digital circuits," etc., and the next level down from "packaging forms" could be specific packaging categories such as "BGA" and "QFN." Similarly, in the smart manufacturing field, nodes at the first level could be "smart equipment," "smart production lines," "smart workshops," "smart factories," etc., and the next level down from "smart equipment" could be "industrial robots," "smart sensors," "additive manufacturing equipment," etc., and the next level down from "industrial robots" could be specific models such as "articular robots," "SCARA robots," and "collaborative robots." It is understood that the hierarchical divisions and node naming in the above vertical fields are merely illustrative examples. In practical applications, those skilled in the art can adaptively adjust and expand the domain tree 201 according to specific business scenarios and technological evolution.
[0052] In one implementation, each text classifier in the text classification model corresponds to a node in the domain tree, and the text classifier is used to identify the child node to which the text content belongs from the various child nodes of the corresponding node.
[0053] See Figure 4 , Figure 4 This is a schematic diagram illustrating the hierarchical structure of a text classification model, as exemplified in embodiments of this specification. Figure 4 As shown, the text classification model 203 in this embodiment adopts a hierarchical tree structure corresponding to the aforementioned domain tree 201 in its overall architecture. This model contains multiple levels of text classifiers, specifically including a first-level text classifier, a second-level text classifier, a third-level text classifier, and a fourth-level text classifier. Each level of text classifier corresponds to a node in the corresponding level of the domain tree 201—the first-level text classifier corresponds to a first-level node in the domain tree 201, used to identify the first-level category to which the text content belongs from the top-level major category; the second-level text classifier corresponds to a second-level node in the domain tree 201, used to further identify the second-level category to which the text content belongs within the scope determined by the first-level category; the third-level and fourth-level text classifiers follow the same pattern, refining the classification granularity layer by layer downwards.
[0054] For example, refer to Figure 3 and Figure 4In conjunction with the domain tree 201 exemplified in the aforementioned embodiments, the text classifier corresponding to the root node at the first level can be used to identify the first-level sub-node to which the text content belongs from first-level sub-nodes such as "semiconductor and microelectronics" and "network communication equipment"; the text classifier corresponding to the "semiconductor and microelectronics" node can be used to identify the second-level sub-node to which the text content belongs from each second-level sub-node (e.g., "package form", "digital circuit"); the text classifier corresponding to the "package form" node can be used to identify the third-level sub-node to which the text content belongs from each third-level sub-node (e.g., "BGA", "QFN"); and so on.
[0055] Based on this, the text classification model 203 performs layer-by-layer classification of text content as follows: The text content is first input to the text classifier corresponding to a higher-level node in the domain tree 201. This text classifier identifies the child node to which the text content belongs from each of the node's child nodes, thus obtaining the classification label for the text content at that level. After identifying the child node to which the text content belongs, if there are child nodes at a lower level below that child node, the text content is further input to the text classifier corresponding to that child node. This text classifier identifies the next-level child node to which the text content belongs from each of the next-level child nodes below that child node, thus obtaining the classification label for the text content at the next level. If there are no next-level child nodes below that child node, i.e., the child node is a leaf node in the domain tree 201, then the classification at the next level is not continued. The classification labels corresponding to each level node along a complete path in the domain tree 201, along with the confidence scores for each classification label, are output.
[0056] For example, each text classifier can be based on a BERT (Bidirectional Encoder Representations from Transformers) classifier. BERT is a language model pre-trained on a large corpus that captures the contextual semantic information of the text through a bidirectional Transformer structure.
[0057] When using a BERT-based architecture, any text classifier can include two parts: a backbone network and a classification head. The backbone network employs BERT's Transformer encoder to encode the input text content, obtaining a vector that represents the semantic features of the text content. For example, the vector corresponding to the [CLS] marker in the BERT output can be used as the semantic representation vector of the text content, and this semantic representation vector is provided to the classification head. The classification head, located after the backbone network, is used to classify the text content based on the semantic representation vector output by the backbone network. For example, the classification head can include a Dropout layer and a fully connected layer, where the output dimension of the fully connected layer corresponds to the number of child nodes under the node corresponding to the text classifier; that is, the fully connected layer outputs a value for each child node under that node. Based on the values output by the fully connected layer for each child node, the child node to which the text content belongs can be determined from the various child nodes of that node. The specific method for determining the child node to which the text content belongs from each child node based on the output of the fully connected layer will be further explained in subsequent embodiments.
[0058] For each text classifier in the text classification model 203, the text classifiers corresponding to different nodes in the domain tree 201 maintain mutual independence in terms of parameters. That is, each text classifier has its own backbone network parameters and classification head parameters, and the text classifiers corresponding to different nodes in the domain tree 201 do not share backbone network parameters and classification head parameters. When organizing the text classification model 203 according to the above implementation method, the objects that each text classifier needs to distinguish are only the child nodes under the corresponding node, rather than all nodes at all levels in the domain tree 201. In this way, the number of categories that a single text classifier needs to identify is relatively limited, and the categories are similar in semantic granularity, which is beneficial to improving the classification results of a single text classifier. In addition, since each text classifier corresponds to a specific node in the domain tree 201, the text classifiers can maintain relative independence in terms of training and parameter storage. That is, the training data and model parameters used by the text classifier corresponding to a node in the domain tree 201 do not affect each other.
[0059] It should be noted that the specific model structure used by each text classifier is not limited to the BERT-based structure described above. In other embodiments, each text classifier may also adopt other model structures suitable for text classification tasks, and the embodiments in this specification do not limit the specific structure of the text classifier.
[0060] In one implementation, a text classifier determines the probability that text content belongs to each child node of a corresponding node, and selects the child node with the highest probability as the child node to which the text content belongs. For ease of explanation, this implementation can be referred to as the single-label mode. In the single-label mode, the text classifier, based on the numerical values output for each child node, uses normalization processing methods such as the Softmax function to convert each output value into a probability that the text content belongs to the corresponding child node, with the sum of the probabilities corresponding to each child node being 1. After obtaining the probabilities corresponding to each child node, the text classifier selects the child node with the highest probability from among the child nodes of the corresponding node as the child node to which the text content belongs. For example, for the text classifier corresponding to the "Semiconductor and Microelectronics" node, if the probabilities corresponding to the three secondary child nodes "Packaging Form," "Digital Circuit," and "Passive Device" for a certain text content are 0.65, 0.20, and 0.15 respectively, then the child node with the highest probability, "Packaging Form," can be determined as the child node to which the text content belongs.
[0061] The single-label mode is suitable for scenarios where the semantic boundaries between the child nodes of a corresponding node are relatively clear, and the text content usually semantically belongs to only one child node. When using the single-label mode, calling the child nodes to which the text content belongs, obtained by calling the text classifier corresponding to the corresponding node in the domain tree, can form a path extending downward from the root node of the domain tree.
[0062] In another implementation, the text classifier determines the probability that text content belongs to each child node of a corresponding node, and identifies at least one child node with a probability greater than a preset probability threshold as the child node to which the text content belongs. For ease of explanation, this implementation can be referred to as multi-label mode. In multi-label mode, the text classifier, based on the values output for each child node, obtains the probability that the text content belongs to each child node through mapping processing methods such as the Sigmoid function. The probabilities corresponding to each child node are independent of each other and there are no normalization constraints between them. After obtaining the probability corresponding to each child node, the text classifier compares the probability corresponding to each child node with the preset probability threshold, and identifies at least one child node with a probability greater than the preset probability threshold as the child node to which the text content belongs. The specific value of the preset probability threshold can be configured according to actual application requirements. For example, it can be configured to 0.5, or different preset probability thresholds can be configured for different text classifiers. This embodiment does not limit this.
[0063] Multi-label mode is suitable for scenarios where there is some semantic overlap between the child nodes of a corresponding node, or where the text content may semantically belong to multiple child nodes simultaneously. For example, a technical text describing a power supply filter circuit may simultaneously involve the classification of capacitors under passive devices and the classification of ceramic capacitors under basic electronic components and interconnections. In this case, if a single-label mode is used to assign the text content to only one child node, there may be a deviation from the actual semantics of the text content; however, using a multi-label mode, which assigns multiple child nodes with a probability greater than a preset probability threshold as the child nodes to which the text content belongs, can better align with the actual semantics of the text content. When using multi-label mode, calling the child nodes to which the text content belongs, obtained by calling the text classifier corresponding to the corresponding node in the domain tree layer by layer, can form one or more paths extending downwards from the root node of the domain tree.
[0064] It should be noted that the single-label and multi-label modes described above can be used independently or separately in different text classifiers within the text classification model. For example, some nodes may use the single-label mode for their text classifiers, while others may use the multi-label mode. The specific mode used by each text classifier can be pre-set before training, thus coordinating with the parameter training process of the backbone network and classification head of that text classifier.
[0065] In practical applications, the knowledge system of a vertical domain may change as business develops. For example, newly emerging technical branches may need to be reflected in the domain tree, or the relevant attributes of existing nodes may need to be adjusted accordingly. Furthermore, the process of optimizing target nodes in the domain tree using text content, as described in the aforementioned embodiments, may also trigger changes in the attributes or structural relationships of related nodes in the domain tree. After these changes occur, the text classifiers in the text classification model corresponding to the domain tree need to be adjusted accordingly to adapt to the changes in the domain tree.
[0066] Adapting the text classification model to changes in the domain tree by retraining the entire model could incur relatively high training costs and might also affect the parameters of the text classifiers corresponding to the unchanged branches of the domain tree. Therefore, based on the architecture described in the preceding implementation where each text classifier corresponds to a node in the domain tree and the parameters of each text classifier are independent, the following implementation method can be used to adjust the text classification model.
[0067] In one implementation, when the target node changes, each node in the path information of the target node is taken as the affected node, and the text classifier corresponding to the affected node is fine-tuned using samples.
[0068] A change in the target node can be understood as a change in the target node's basic description information or attribution relationship within the domain tree. When a target node changes, the nodes on the complete hierarchical link corresponding to the target node's path information can be identified as the affected nodes.
[0069] After identifying the affected nodes, the text classifiers corresponding to those nodes can be fine-tuned. Specifically, the text classifiers corresponding to the affected nodes can be further trained using samples. These samples can be the samples constructed for training the text classification model as described in the preceding embodiments, which may include samples constructed by regenerating text content based on the optimized neighborhood tree. The training method used during fine-tuning can be similar to that used during the initial training of the text classifier, i.e., inputting the input features from the samples into the text classifier corresponding to the affected nodes, comparing the output of the text classifier with the corresponding ground truth labels in the samples, and adjusting the corresponding parameters of the text classifier based on the comparison results.
[0070] For text classifiers corresponding to other nodes not within the affected node range, their parameters can remain unchanged. For example, in the case where the aforementioned "neural network processor" node changes as the target node, the text classifier corresponding to the "network communication device" node and the text classifiers corresponding to its subordinate child nodes are not within the affected node range, so the parameters of these text classifiers do not need to be adjusted.
[0071] Through the above implementation method, when a local change occurs in the domain tree, only the relevant text classifiers located on the changed path in the text classification model need to have their parameters adjusted, while the parameters of text classifiers located on other branches can remain unchanged. Compared to retraining the entire text classification model, this approach helps reduce the computational resources and time overhead required for the training process. Simultaneously, the classification results of text classifiers not within the affected node range will not change due to this adjustment, which helps the text classification model maintain relatively stable classification results for business texts not affected by the domain tree change.
[0072] As described in the foregoing embodiments, for each node in the domain tree, a large language model can be used to generate at least one text content describing that node based on its path information and basic description information. In practical applications, directly submitting the path information and basic description information to the large language model in their raw form usually requires consideration of the domain scope, the technical levels to be covered, and the expressive form to be presented in the generated text content. Therefore, the generation process of the large language model can be organized using the following implementation method.
[0073] In one implementation, the aforementioned method of generating at least one text content describing the node based on the node's path information and basic description information using a large language model may include the following steps.
[0074] First, the path information and basic description information are embedded into a preset prompt word template to obtain the input text.
[0075] The prompt word template is a pre-defined text template used to guide the large language model in performing generation tasks. The prompt word template can contain several fixed text fragments and several replaceable fields. The fixed text fragments describe the generation task of the large language model, such as the goal of the generation task, the formal requirements that the generated result must meet, and the constraints that must be followed during the generation process. The replaceable fields carry information related to specific nodes in the domain tree, and the specific content of the replaceable fields can change accordingly when the prompt word template is applied to different nodes in the domain tree.
[0076] The prompt word template can contain at least two replaceable fields, one for path information and the other for basic description information. When generating text content for a node in the domain tree, the path information and basic description information of that node can be filled into the corresponding replaceable fields to obtain complete input text that embeds the path information and basic description information of that node. In one implementation, the prompt word template can also contain a description of the role positioning of the large language model. This role positioning can switch according to the domain scope corresponding to the embedded path information. For example, when the path information points to the semiconductor packaging domain, the prompt word template positions the large language model as a "semiconductor packaging process expert," and when the path information points to the network communication domain, it switches to "network communication technology expert." The prompt word template also contains a task objective description, which requires the large language model to "generate high-quality technical question-and-answer pairs (QA pairs) related to the node based on the domain scope defined by the 'path information' and the technical boundaries defined by the 'basic description information.'" The system employs a QA pair format, presenting the generated text content in "question-answer" pairs. The "question" portion of each QA pair guides a deeper discussion of a specific technical aspect of that node, while the "answer" portion provides the corresponding technical explanation or clarification. Furthermore, the prompt word template includes positive and negative constraints set by experts: positive constraints specify "what the large language model should do" during generation, such as requiring the generated text content to adhere to industry standards, delve into technical details, and reflect domain expertise; negative constraints specify "what the large language model should not do," such as prohibiting the generation of overly superficial or general domain content, and prohibiting content from other domains unrelated to the current node. Path information defines the domain scope of the generation task, while basic descriptive information defines the semantic boundaries of that domain scope; together, they constitute the constraints for the large language model's generation.
[0077] Then, the input text is fed into the large language model to obtain at least one text content describing the node.
[0078] After obtaining the input text, it can be used as input to the large language model. The large language model performs generation processing based on the input text and outputs the generated result corresponding to the input text. The generated result contains at least one text content describing the node.
[0079] The quantity of at least one generated text content can be set according to actual needs. In one implementation, the number of text contents to be output in this generation task can be specified in the prompt word template, and the large language model outputs the corresponding number of text contents according to the specification. In another implementation, the large language model can be called multiple times, and the text contents obtained from each call can be summarized to obtain multiple text contents corresponding to the node. The embodiments in this specification do not limit the specific implementation method of the number of generated text contents.
[0080] Through the above implementation method, on the one hand, the path information is used to constrain the domain scope of the generation task, and on the other hand, the basic description information is used to supplement the semantic boundary of the domain scope. This is conducive to forming a closer correspondence between the text content generated by the large language model and the semantics of the corresponding nodes in the domain tree.
[0081] Furthermore, in the above scenarios, the phenomenon of confidence levels falling below the preset threshold may be related to deviations between the definition boundaries of the corresponding nodes in the domain tree in some cases; while in other cases, it may be related to deviations in the guidance methods for the generation task in the prompt word template. For example, if the prompt word template does not adequately describe the constraints on the generation task in terms of technical depth and professionalism, the text content generated by the large language model based on the prompt word template may semantically deviate from the technical boundaries corresponding to the nodes in the domain tree, resulting in lower confidence levels when the text classification model performs layer-by-layer classification of the text content. Therefore, the prompt word template can be further optimized using the following implementation methods.
[0082] Therefore, in one implementation, if any confidence level is lower than a preset threshold, the prompt word template is optimized using the text content, so as to regenerate the text content and construct a sample based on the optimized prompt word template.
[0083] Optimizing the prompt word template using text content can be understood as referring to the text content and its corresponding classification results, and making corresponding adjustments to the relevant content in the prompt word template. The specific objects of adjustment can include fixed text segments in the prompt word template, or the organization of replaceable fields in the prompt word template; the specific direction of adjustment can be refining the target description of the generation task, supplementing the formal requirements that the generated results must meet, or adjusting the constraints that need to be followed during the generation process, etc. This specification does not limit the specific method of optimizing the prompt word template in the embodiments.
[0084] After optimizing the prompt word template, the text content generation process described in the previous embodiment can be re-executed based on the optimized prompt word template. If the confidence scores of each category label corresponding to the new text content are not lower than the preset threshold, the new text content can be used as input features and each category label of the new text content can be used as ground truth labels to construct samples for training the text classification model. If the confidence scores of at least one category label corresponding to the new text content are still lower than the preset threshold, the prompt word template can be further adjusted based on the new text content, forming an iterative optimization process for the prompt word template.
[0085] It should be noted that the optimization of target nodes in the domain tree described in the foregoing embodiments and the optimization of prompt word templates described in this embodiment can be used in conjunction with each other in scenarios where the confidence level is lower than a preset threshold. For example, the corresponding optimization target can be selected based on the specific reason why the confidence level is lower than the preset threshold: if the lower confidence level is mainly due to the deviation between the definition boundary of the corresponding node in the domain tree and the text content, then the focus can be on optimizing the target nodes in the domain tree; if the lower confidence level is mainly due to the deviation between the prompt word template's guidance method for generating the task and the text content, then the focus can be on optimizing the prompt word template. This specification does not limit the selection method of the above two optimization methods in the embodiments.
[0086] As described in the foregoing embodiments, the text content can be used to optimize the target nodes in the domain tree. For each piece of text with a confidence level below a preset threshold, determining how to optimize the corresponding target node from scratch could require a relatively large investment of time and resources. Furthermore, in the long-term use of the text classification model, text content with confidence levels below the preset threshold may exhibit semantic similarities, and their corresponding target nodes may share certain commonalities in their optimization methods. Therefore, similar cases that have been processed previously can be preserved and reused in subsequent processing. Specifically, the following implementation method is used to optimize the target nodes in the domain tree.
[0087] See below. Figure 5 This section provides an overall explanation of the methods for optimizing confidence feedback. Figure 5 This is a data interaction diagram illustrating the feedback optimization process as exemplified in the embodiments of this specification.
[0088] Regarding the optimization method for target node 2011, in one embodiment, the aforementioned optimization of target node 2011 in the domain tree 201 using text content may include the following steps.
[0089] First, match historical text content 2071 that is semantically similar to the text content from the historical correction library 207.
[0090] The historical correction library 207 is a pre-established and continuously maintained data storage structure, containing several historical text contents 2071 and pre-defined correction strategies 2072 for each historical text content 2071. At least one classification label of each historical text content 2071 contained in the historical correction library 207 has a confidence level lower than a preset threshold. In other words, each historical text content 2071 stored in the historical correction library 207 belongs to the text content whose classification results, output by the text classification model 203 during previous processing, contain low-confidence classification labels.
[0091] For text content with a confidence level below a preset threshold, semantic similarity can be compared with various historical text content 2071 in the historical correction library 207. There are various specific methods for semantic similarity comparison. For example, a pre-trained text representation model can be used to encode both the text content and the historical text content 2071 in the historical correction library 207 to obtain corresponding semantic vectors. The similarity between these semantic vectors can then be used to measure their semantic similarity. This specification does not limit the specific implementation of semantic similarity comparison in its embodiments. Based on the above semantic similarity comparison results, historical text content 2071 semantically similar to the current text content can be determined from the historical correction library 207.
[0092] Then, based on the correction strategy 2072 corresponding to the semantically similar historical text content 2071, the target node 2011 in the domain tree 201 is optimized.
[0093] After matching historical text content 2071 that is semantically similar to the text content from the historical correction library 207, the correction strategy 2072 corresponding to the historical text content 2071 can be obtained, and the target node 2011 in the domain tree 201 can be adjusted accordingly based on the correction strategy 2072.
[0094] The correction strategy 2072 is a pre-defined processing method for adjusting the corresponding target node 2011 in the domain tree 201, based on the historical text content 2071. The correction strategy 2072 illustrated below will be explained in detail.
[0095] First, adjust the basic description information of target node 2011. This strategy is applicable when the hierarchical position and affiliation of target node 2011 are basically reasonable, but its semantic boundaries deviate from the actual text content. The adjustment method is to supplement or refine the basic description information to more accurately define the technical boundaries of node 2011. For example, when the semantic boundaries between "box switch" node 2011 and other similar nodes 2011 are not clear enough, distinguishing features such as fixed number of ports and lack of support for expansion cards can be added to its basic description information.
[0096] Secondly, adjust the position of the target node 2011 in the domain tree 201. This strategy is applicable when there is a discrepancy between the affiliation of the target node 2011 and the actual knowledge level. The adjustment method is to move the target node 2011 to a more suitable parent node 2011, or adjust its arrangement relationship with nodes 2011 at the same level, and its path information will be automatically updated accordingly.
[0097] Third, add a child node 2011 under the parent node 2011 of the target node 2011. This strategy is applicable when the existing domain tree 201 fails to fully cover the new technical branches reflected in the historical correction library 207. The adjustment method is to add a child node 2011 under the parent node 2011 of the target node 2011. The path information of the new child node 2011 is determined by its location, and the basic description information is set based on the technical boundaries reflected in the corresponding text content. For example, when there are already two child nodes 2011, "box switch" and "frame switch", under the path "network communication equipment → forwarding device", and the historical text content 2071 reflects that there is general switch technology content that does not distinguish between box switches and frame switches that is not covered, a child node 2011 can be added under the "forwarding device" node 2011 to take over this type of content.
[0098] After adjusting the target node 2011 in the domain tree 201 according to the correction strategy 2072, the optimized domain tree 201 is obtained. Then, the text content can be regenerated and the sample 205 can be constructed based on the optimized domain tree 201. The specific process can be referred to the relevant description in the foregoing implementation.
[0099] Furthermore, the historical correction library 207 can be continuously expanded during the use of the text classification model 203. For example, when new text content with a confidence level lower than a preset threshold appears, and the optimization process of the target node 2011 corresponding to the text content is completed, the text content and the corresponding correction strategy 2072 can be added to the historical correction library 207 as a reference for subsequent processing.
[0100] Through the above implementation method, when the confidence of the classification label output by the text classification model 203 is lower than the preset threshold, the target node 2011 in the domain tree 201 can be adjusted accordingly by using the historical text content 2071 accumulated in the historical correction library 207 and the corresponding correction strategy 2072. This helps to reduce the processing overhead caused by re-judging the target node 2011 optimization method for each text content with a confidence lower than the preset threshold.
[0101] Continue reading Figure 5 In this feedback optimization process, when the text classification model 203 classifies a certain text content, if the confidence level of the output classification label is lower than the preset threshold, it indicates that there is a fuzzy area in the technical classification of the text content or the model judgment is uncertain. At this time, the text content and its corresponding low-confidence classification label are sent to the historical correction library 207 together.
[0102] The historical correction library 207 pre-stores several historically low-confidence text contents and their corresponding correction strategies 2072. These correction strategies 2072 are standardized processing schemes for specific types of fuzzy samples 205, confirmed through business review processes. The system performs semantic similarity matching between the current text content and historical text content 2071 in the historical correction library 207. If a semantically similar historical text content 2071 is matched, the correction strategy 2072 corresponding to that historical text content 2071 is obtained, and the target node 2011 in the domain tree 201 is optimized based on the correction strategy 2072. Specifically, the correction strategy 2072 includes adjusting the basic description information of the target node 2011 to accurately define its technical boundaries, adjusting the hierarchical position of the target node 2011 in the domain tree 201 to correct attribution bias, or adding new child nodes 2011 under the parent node 2011 of the target node 2011 to cover emerging technology branches not fully represented by the existing structure. The specific implementation methods of the above three strategies have been described in [the document]. Figure 5The relevant steps are explained in detail in the instructions and will not be repeated here. Meanwhile, if analysis reveals that the low confidence level is related to the imprecise constraint guidance method for the generated task in the prompt word template 206 corresponding to node 2011, then the role positioning, task objective, positive or negative constraints in the prompt word template 206 are adjusted accordingly to ensure that the subsequently generated text content is more closely aligned with the definition boundary of node 2011 in terms of technical depth and professional granularity. After the target node 2011 is optimized, the optimized domain tree 201 interacts again with the large language model 202 and the prompt word template 206. The large language model 202 regenerates the text content based on the path information and basic description information of the affected node 2011 in the optimized domain tree 201, generates sample 205 based on the regenerated text content, and fine-tunes the text classifier 2031 corresponding to the affected node 2011.
[0103] In one implementation, this application employs a progressive review strategy to grade the matching results in the historical correction database. Specifically, the review process consists of three stages: In the initial stage, domain experts manually verify the matched historical correction strategies in accordance with industry standards to ensure that the initial operation complies with business specifications.
[0104] During the alignment phase, the system adjusts the constraints in the prompt word template (such as the wording of positive / negative constraints and the granularity of node description information) in reverse based on the confirmation or rejection results of expert review, so that the distribution generated by the model gradually moves closer to the expert review standards.
[0105] In the mature stage, when the system's accuracy rate in reviewing and recommending specific nodes or paths continuously exceeds a preset threshold, the system switches the correction mode of that path to automatic execution, and only retains a manual review channel for samples that fail automatic verification.
[0106] By employing the aforementioned gradual strategy, data quality was ensured during the system's cold start phase, and the review process was automated once the system matured, further reducing the cost of manual intervention.
[0107] See below. Figure 6 , Figure 6 This is a flowchart illustrating a training method for a text classification model in a vertical domain, as provided in the embodiments of this specification, including: S601: Obtain a pre-built domain tree based on the knowledge system of the specified vertical domain. The domain tree includes multiple levels, each level includes at least one node, and each node has path information and basic description information. The path information is used to characterize the node's affiliation with other nodes in the knowledge system, and the basic description information is used to characterize the semantic scope of the node in the knowledge system. S602: For each node, use a large language model to generate at least one text content describing the node based on the node's path information and basic description information.
[0108] S603: For each piece of text content, input the text content into a text classification model containing multiple levels of text classifiers. This allows the text classification model to classify the text content layer by layer through the text classifiers at each level, obtaining at least one classification label for the text content and the confidence score corresponding to each classification label. The hierarchical structure of the text classification model is constructed based on the hierarchical structure of a domain tree, and the classification labels are used to characterize the node to which the text content belongs in the domain tree. Then, determine whether the confidence level corresponding to each category label is not lower than the preset threshold.
[0109] If all confidence levels are not lower than the preset threshold, proceed to step S604: use the text content as input features and the classification labels of the text content as ground truth labels to construct samples for training the text classification model.
[0110] If any confidence level is lower than a preset threshold, proceed to step S605: optimize the target node in the domain tree using the text content, and regenerate the text content and construct the sample based on the optimized domain tree.
[0111] S606: Use samples to train a text classification model.
[0112] This specification also provides a text classification model for a vertical domain, the hierarchical structure of which can be referred to... Figure 4 Corresponding to the description of the above embodiments, the model is used to generate classification labels for received business text, which is text in a specified vertical domain. The text classification model can be trained using the training method provided in the above embodiments.
[0113] In the above description of specific embodiments, the technical features of each embodiment can be combined with each other without conflict. Those skilled in the art can select appropriate embodiments to implement the technical solution of this application according to actual needs.
[0114] This specification also provides a computer-readable storage medium storing a computer program that is executed by a processor to implement any of the methods provided in this specification.
[0115] The embodiments of this specification may take the form of a computer program product implemented on one or more storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing program code. Computer-usable storage media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. Information may be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to: phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.
[0116] For each protected subject, multiple implementation methods have been provided above. Without conflict or contradiction, those skilled in the art can freely combine these various implementation methods according to the actual situation to form various different technical solutions. Due to space limitations, this application document has not elaborated on all the combined technical solutions; however, it is understood that these omitted technical solutions also fall within the scope of the embodiments disclosed in this specification.
Claims
1. A text retrieval method for a vertical domain, characterized in that, include: Obtain a pre-constructed domain tree based on a knowledge system of a specified vertical domain. The domain tree includes multiple levels, each level includes at least one node, and each node has path information and basic description information. The path information is used to characterize the relationship between the node and other nodes in the knowledge system, and the basic description information is used to characterize the scope of the semantics of the node in the knowledge system. For each node, at least one text content describing the node is generated using a large language model based on the node's path information and basic description information. For each piece of text content, the text content is input into a text classification model containing multiple levels of text classifiers, so that the text classification model classifies the text content layer by layer through the text classifiers at each level, and obtains at least one classification label of the text content and the confidence level corresponding to each classification label. The hierarchical structure of the text classification model is constructed based on the hierarchical structure of the domain tree, and the classification label is used to characterize the node to which the text content belongs in the domain tree. If each confidence level is not lower than a preset threshold, the text content is used as the input feature, and each classification label of the text content is used as the ground truth label to construct a sample for training the text classification model. If any of the confidence levels is lower than a preset threshold, the target node in the domain tree is optimized using the text content. The target node is the node represented by the classification label corresponding to the confidence level, so as to regenerate the text content and construct samples based on the optimized domain tree. The text classification model is trained using the samples, and the trained text classification model is used to generate classification labels for the business text in the vertical domain, so as to retrieve the business text based on the classification labels.
2. The method according to claim 1, characterized in that, Each text classifier corresponds to a node in the domain tree, and the text classifier is used to identify the child node to which the text content belongs from each of the child nodes of the corresponding node.
3. The method according to claim 2, characterized in that, The method further includes: If the target node changes, each node in the path information of the target node will be considered as the affected node. The text classifier corresponding to the affected node is fine-tuned using the sample.
4. The method according to claim 2, characterized in that, The text classifier is used to determine the probability that the text content belongs to each child node, and takes the child node with the highest probability as the child node to which the text content belongs; and / or, the text classifier is used to determine the probability that the text content belongs to each child node, and takes at least one child node with a probability greater than a preset probability threshold as the child node to which the text content belongs.
5. The method according to claim 1, characterized in that, The step of generating at least one text content describing the node based on the node's path information and basic descriptive information using a large language model includes: The path information and the basic description information are embedded into a preset prompt word template to obtain the input text; The input text is input into the large language model to obtain at least one text content describing the node.
6. The method according to claim 5, characterized in that, The method further includes: If any of the confidence levels is lower than a preset threshold, the prompt word template is optimized using the text content, and the text content is regenerated and a sample is constructed based on the optimized prompt word template.
7. The method according to claim 1, characterized in that, The optimization of the target node in the domain tree using the text content includes: Match historical text content that is semantically similar to the text content from the historical correction library. The historical correction library contains several historical text contents and a preset correction strategy for each historical text content. The confidence level of at least one classification label of the historical text content is lower than the preset threshold. Based on the correction strategy corresponding to the semantically similar historical text content, the target node in the domain tree is optimized; The correction strategy includes at least one of the following: adjusting the basic description information of the target node, adjusting the position of the target node in the domain tree, and adding child nodes under the parent node of the target node.
8. A training method for a text classification model in a vertical domain, characterized in that, include: Obtain a pre-constructed domain tree based on a knowledge system of a specified vertical domain. The domain tree includes multiple levels, each level includes at least one node, and each node has path information and basic description information. The path information is used to characterize the relationship between the node and other nodes in the knowledge system, and the basic description information is used to characterize the scope of the semantics of the node in the knowledge system. For each node, at least one text content describing the node is generated using a large language model based on the node's path information and basic description information. For each piece of text content, the text content is input into a text classification model containing multiple levels of text classifiers, so that the text classification model classifies the text content layer by layer through the text classifiers at each level, and obtains at least one classification label of the text content and the confidence level corresponding to each classification label. The hierarchical structure of the text classification model is constructed based on the hierarchical structure of the domain tree, and the classification label is used to characterize the node to which the text content belongs in the domain tree. If each confidence level is not lower than a preset threshold, the text content is used as the input feature, and each classification label of the text content is used as the ground truth label to construct a sample for training the text classification model. If any of the confidence levels is lower than a preset threshold, the target node in the domain tree is optimized using the text content. The target node is the node represented by the classification label corresponding to the confidence level, so as to regenerate the text content and construct samples based on the optimized domain tree. The text classification model is trained using the samples.
9. A text classification model for a vertical domain, characterized in that, Used to generate classification labels for received business text, wherein the business text is text from a specified vertical domain; The text classification model is trained by the method described in claim 8.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method according to any one of claims 1-8.
