Corpus information processing method and device, electronic equipment and readable storage medium

Abstract

The embodiment of the invention discloses a corpus information processing method and device, electronic equipment and a readable storage medium. The method comprises the steps that information extraction is conducted on multiple pieces of corpus information through a large language model, label information of all the corpus information is determined, and a feature model is trained based on the label information of part of the corpus information and a clustering result after clustering; clustering the label information of each piece of corpus information based on the feature model, and determining a clustering tree; and finally, classifying the to-be-analyzed corpus input into the large language model according to the clustering tree, and determining a corresponding category label. Therefore, according to the embodiment, an automatic corpus analysis link can be established in the large language model, and the corpus information is processed by utilizing rich knowledge in the large language model, so that the corpus analysis efficiency and accuracy are improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more particularly, to a method, device, electronic device, and readable storage medium for processing corpus information. Background Art

[0002] In some scenarios, it is necessary to analyze corpus information. For example, in a huge e-commerce system, there are various products and multiple complex links, so that a large number of potential problems cannot be discovered from the perspective of the builder, and it is more necessary to rely on the original voice feedback of users (i.e., corpus information) to optimize the experience.

[0003] However, due to the huge volume of user original voices, diverse expression methods, and a large amount of low-quality information in the original voice content, the analysis of user original voices is restricted, and it is impossible to perform efficient and accurate corpus analysis. Summary of the Invention

[0004] In view of this, embodiments of the present invention aim to provide a method, device, electronic device, and readable storage medium for processing corpus information to improve the efficiency and accuracy of corpus analysis.

[0005] In a first aspect, an embodiment of the present invention aims to provide a method for processing corpus information, the method including:

[0006] Obtain corpus data, where the corpus data includes multiple pieces of corpus information;

[0007] Input each piece of the corpus information into a pre-trained large language model for information extraction to determine the label information corresponding to the corpus information, where the label information is a general expression of the corresponding corpus information;

[0008] Cluster the label information corresponding to some of the corpus information based on the large language model to determine a clustering result;

[0009] Train and determine a feature model according to the clustering result and the label information corresponding to the clustering result;

[0010] Cluster the label information of each piece of the corpus information based on the feature model to determine a clustering tree;

[0011] Classify the to-be-analyzed corpus input into the large language model according to the clustering tree to determine the corresponding class label.

[0012] Further, the inputting each piece of the corpus information into a pre-trained large language model for information extraction to determine the label information corresponding to the corpus information includes:

[0013] Perform context learning, few-shot learning, and chain-of-thought analysis on the corpus information based on a large language model to determine the label information corresponding to the corpus information.

[0014] Further, clustering the label information of each corpus information based on the feature model, and determining that the clustering tree includes:

[0015] Perform multi-level clustering on the label information of each corpus information based on the feature model to determine at least one clustering level and the clustering categories corresponding to each clustering level;

[0016] Input the corpus information belonging to the clustering category into the large language model for analysis to determine the category label corresponding to the clustering category;

[0017] Generate a clustering tree according to the level relationship and corresponding category labels of each clustering category.

[0018] Further, the classifying the to-be-analyzed corpus input into the large language model according to the clustering tree to determine the corresponding category label includes:

[0019] Input the to-be-analyzed corpus into the large language model for information extraction to determine the corresponding label information;

[0020] Classify the label information according to the clustering tree to determine the category label of the to-be-analyzed corpus.

[0021] Further, the method further includes:

[0022] Send the category label and / or label information of the to-be-analyzed corpus to the target terminal.

[0023] Further, the method further includes:

[0024] Periodically collect the category labels of different to-be-analyzed corpora;

[0025] Perform statistical analysis on the category labels of each to-be-analyzed corpus to generate a corresponding analysis report;

[0026] Send the analysis report to the target terminal.

[0027] Further, the content of the corpus information includes the feedback information of merchant users on the operation platform and / or the supplementary information of the operation platform on the feedback information, and the category label is used to characterize the category of the corresponding feedback information.

[0028] Further, the feature model uses a BERT model.

[0029] In a second aspect, an embodiment of the present invention aims to provide a corpus information processing device, and the device includes:

[0030] An acquisition module, configured to acquire corpus data, where the corpus data includes multiple pieces of corpus information;

[0031] An expression module, configured to respectively input each piece of the corpus information into a pre-trained large language model for information extraction, and determine the label information corresponding to the corpus information, where the label information is a general expression of the corresponding corpus information;

[0032] A clustering module, configured to cluster the label information corresponding to some of the corpus information based on the large language model to determine a clustering result; train and determine a feature model according to the clustering result and the label information corresponding to the clustering result; cluster the label information of each piece of the corpus information based on the feature model to determine a clustering tree;

[0033] A classification module, configured to classify the corpus to be analyzed input into the large language model according to the clustering tree, and determine the corresponding category label.

[0034] In a third aspect, an embodiment of the present invention aims to provide a computer program product, where the computer program product includes computer programs / instructions, and when the computer programs / instructions are executed by a processor, the method described in any one of the above is implemented.

[0035] In a fourth aspect, an embodiment of the present invention aims to provide an electronic device, including a memory and a processor, where the memory is used to store one or more computer program instructions, and wherein the one or more computer program instructions are executed by the processor to implement the method described in any one of the above.

[0036] In a fifth aspect, an embodiment of the present invention aims to provide a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method described in any one of the above is implemented.

[0037] The technical solution of the embodiment of the present invention extracts information from multiple pieces of corpus information through a large language model, determines the label information of each piece of corpus information, and trains a feature model based on the label information of some of the corpus information and the clustering result after clustering; then clusters the label information of each piece of corpus information based on the feature model to determine a clustering tree; finally, classifies the corpus to be analyzed input into the large language model according to the clustering tree, and determines the corresponding category label. Thus, this embodiment can build an automated corpus analysis link in the large language model and use the rich knowledge in the large language model to process the corpus information, thereby improving the efficiency and accuracy of corpus analysis. Description of the Drawings

[0038] Through the following description of the embodiments of the present invention with reference to the accompanying drawings, the above and other objects, features, and advantages of the present invention will become clearer. In the drawings:

[0039] Figure 1 is a flowchart of the corpus information processing method according to an embodiment of the present invention;

[0040] Figure 2 is a flowchart of determining a clustering tree according to an embodiment of the present invention;

[0041] Figure 3 is a flowchart of presenting the corpus analysis results according to an embodiment of the present invention;

[0042] Figure 4 is a schematic diagram of the corpus analysis process according to an embodiment of the present invention;

[0043] Figure 5 is a data flow diagram of the corpus information processing according to an embodiment of the present invention;

[0044] Figure 6 is a schematic diagram of the corpus information processing apparatus according to an embodiment of the present invention;

[0045] Figure 7 is a schematic diagram of the electronic device according to an embodiment of the present invention. Detailed Embodiments

[0046] The following describes the present application based on embodiments, but the present application is not limited to these embodiments. In the following detailed description of the present application, some specific details are described in detail. Those skilled in the art can fully understand the present application without the description of these details. To avoid obscuring the essence of the present application, well-known methods, processes, flows, elements, and circuits are not described in detail.

[0047] In addition, those of ordinary skill in the art should understand that the accompanying drawings provided herein are for illustrative purposes only, and the drawings are not necessarily drawn to scale.

[0048] Unless the context clearly requires otherwise, words such as "including" and "comprising" in the entire application document should be interpreted as having an inclusive meaning rather than an exclusive or exhaustive meaning; that is, it is the meaning of "including but not limited to".

[0049] In the description of the present application, it should be understood that terms such as "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance. In addition, in the description of the present application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0050] In the solutions described in this specification and the embodiments, if personal information processing is involved, it will be processed on the premise of having a legal basis (such as obtaining the consent of the personal information subject, or being necessary for performing a contract, etc.), and will only be processed within the specified or agreed scope. If a user refuses to process personal information other than the necessary information required for basic functions, it will not affect the user's use of basic functions.

[0051] In this embodiment, taking the voice analysis of merchant users of specific merchant products in an e-commerce platform as an example to illustrate the corpus information processing method. By analyzing the voice corpus information of merchant users, the intention or problem classification of the corpus information can be determined, which can assist operation and product personnel in decision-making and improvement. However, it should be understood that the corpus information processing method in this embodiment can also be used in other application scenarios that require analyzing corpus information, such as a customer service system, etc., and no limitation is imposed here.

[0052] In the existing e-commerce voice analysis, it mainly focuses on the analysis of corpus content such as industry content and consumer experience feedback, but pays less attention to the users on the merchant side (i.e., merchant users), lacking the processing of the voice analysis of merchant users, and thus unable to comprehensively optimize the platform based on the feedback from the merchant side. At the same time, in the existing voice analysis, due to the huge volume of user voices, it is almost impossible to manually confirm each user voice one by one; and the expressions of user voices are diverse, with large differences in length, language organization, and expression content, and there will also be a large amount of low-quality information such as noise and emotional venting mixed in the voices, all of which will make the difficulty of analyzing and classifying user voices extremely large, resulting in the inability to achieve efficient, accurate, and comprehensive voice corpus analysis, and further determining the category of problems reflected by the voices.

[0053] In view of this, the embodiments of the present invention aim to provide a corpus information processing method, which combines the currently rapidly developing large language model to construct an automated analysis link, and applies the powerful understanding and reasoning capabilities of the large language model to analyze user voices, thereby improving the efficiency and accuracy of corpus analysis.

[0054] Figure 1 is a flowchart of the corpus information processing method according to the embodiments of the present invention. As Figure 1 shown, the corpus information processing method in this embodiment includes the following steps.

[0055] In step S110, corpus data is obtained, and the corpus data includes multiple pieces of corpus information.

[0056] In this embodiment, the corpus information may be text corpus, audio corpus, image corpus, or a corpus combining different information modalities, such as a corpus containing both audio corpus and image corpus; the information modality of the corpus information can be determined according to the processing capabilities of the large language model used. For example, when the large language model can only process text corpus, the corpus information here is text corpus or text corpus generated by converting corpus of other information modalities (such as text corpus generated after text conversion of audio corpus).

[0057] Optionally, the content of the corpus information in this embodiment includes feedback information of merchant users on the operation platform and / or supplementary information of the operation platform on the feedback information. The feedback information can be difficulties encountered by merchants on the platform (such as being unable to unfreeze the margin), confusions (such as what the margin is), demands (such as needing to unfreeze the margin), etc., which are corpus contents related to merchants and the platform. The supplementary information can be corpus contents obtained after the platform customer service or relevant personnel query, verify, etc. the merchant's feedback information (such as querying that the margin is frozen, and the reason for the freeze is delayed delivery).

[0058] It should be noted that the above-given corpus information is only an example, and the specific content of the corpus information is selected according to actual corpus analysis requirements, and no limitation is imposed here.

[0059] In step S120, each corpus information is respectively input into a pre-trained large language model for information extraction to determine the label information of the corresponding corpus information. The label information is a general expression of the corresponding corpus information.

[0060] In this embodiment, since the corpus information output by users usually includes a large amount of corpus content and the expression ways of corpus content output by different users are significantly different, in order to improve the accuracy of analysis of different corpus information, when processing the corpus information, each corpus information will be analyzed and summarized first to accurately generalize and express different corpus information.

[0061] Furthermore, in this embodiment, each corpus information is subjected to information extraction through a pre-trained large language model to determine the label information of each corpus information, and the general expression of the corresponding corpus information is represented by the label information, so as to subsequently determine the category information of each corpus information based on the accurate general expression of the corpus information, thereby overcoming the limitations of the analysis difficulties of the corpus information itself and improving the accuracy of corpus analysis.

[0062] For example, if the corpus information is about difficulties or confusions encountered during operations by merchants, the label information provided is the merchant's current situation, incomprehensible, or unsolvable problems; if the corpus information is about demands reflected by merchants, the label information provided is the summarized merchant demands. Furthermore, to highlight the content of the merchant corpus information, the label information provided in this embodiment is expressed from the merchant's perspective and focuses on using verbs such as "inquire" and "urge" to summarize the merchant corpus information. This makes the label information more closely aligned with the user's actual needs, thereby improving the accuracy of the overall corpus information analysis results.

[0063] Optionally, the large language model in this embodiment is determined by a training method based on multi-angle knowledge modeling of product entities, focusing on discovering and modeling various risk knowledge, violation information, and governance logic related to e-commerce to meet the needs of information analysis and governance in the e-commerce domain. At the same time, to meet the needs of merchant corpus analysis, the large language model in this embodiment will build on existing knowledge to conduct comprehensive knowledge modeling of various compliant data that does not involve privacy information, such as product categories, product reviews, product live broadcasts, product governance rules, and merchant business logic. It also uses pre-training methods such as multimodal alignment and multi-task knowledge injection to inject new knowledge into the large language model, enabling the large language model to comprehensively and deeply understand various knowledge about e-commerce and e-commerce risks from multiple perspectives. Thus, by using multiple knowledge modeling, the modeling concept of the "people-goods-market" model in the e-commerce domain is embedded in the modeling process, so that the final large language model is not only familiar with the information of e-commerce itself, but also has a richer understanding of the products in the e-commerce system, the operational methods during the operation process, industry rules, and the field of play, especially the risk knowledge and industry judgments on the product governance side, thereby improving the accuracy of corpus information processing.

[0064] Furthermore, the large language model in this embodiment can generate label information for each corpus information using methods such as extractive summarization methods, generative summarization methods, contextual learning, few-shot learning methods, and thought chain analysis, or a combination of these methods. Specifically, to further improve the accuracy and efficiency of label information generation, this embodiment uses the large language model to perform contextual learning, few-shot learning, and thought chain analysis on the acquired corpus information to summarize the content of the corresponding corpus information and determine the label information for the corresponding corpus information.

[0065] Optionally, when determining the label information of the corpus information, the large language model in this embodiment first uses the context learning method to understand the structure and context of the corpus information; then uses few-shot learning to learn and generalize in the case of a small number of labeled samples (such as examples similar to the information to be summarized), so that the large language model can learn how to extract key information from the input text and generate a summary expression, quickly adapting to the summary task under the original voice analysis; finally, through the chain of thought, gradually reason and think about the core content and logical relationship of the text, so as to generate a concise, clear and accurate summary expression. Among them, in the context learning method, by constructing a specific Prompt (prompt word), the large language model can use the context information of the input corpus information to understand the structure and context of the corpus information. In the few-shot learning method, by providing the large language model with a preset number of corpus instances similar to the corpus information and the summary expressions of the corpus instances as the labeled data for model training, it helps the large language model learn to extract key information from the corpus information and generate a summary expression. Specifically, the model will analyze the relationship between the input and output of the corpus instance, so as to learn a mapping or transformation rule. The chain of thought is an improved prompting strategy, which guides the model to perform deeper and more coherent reasoning and thinking by introducing intermediate reasoning steps in the Prompt. Therefore, when performing a chain of thought analysis on the corpus information, the model will perform step-by-step reasoning on the corpus information based on the rules learned in the context learning and few-shot learning stages of the corpus information. Each reasoning step will form an intermediate result, and the intermediate results are connected to form a complete thinking process. Finally, the model will generate a summary expression based on these intermediate results. Specifically, the model will identify the key information points in the corpus information, then summarize and conclude these information points, and finally form the summary expression of the corpus information, that is, the label information corresponding to the corpus information.

[0066] Optionally, after determining the label information corresponding to each corpus information, in this embodiment, clustering can be performed based on the label information corresponding to the corpus information of the large language model to determine the clustering result, so as to generate a clustering tree for analyzing the corpus information of the merchant's original voice based on the clustering result, thereby accurately classifying the corpus information to be analyzed through the clustering tree and determining the category label of the corpus information to be analyzed.

[0067] Another option is that considering that directly using the large language model to cluster all the corpus information requires a large amount of processing resources, high deployment and inference costs, in this embodiment, sample distillation technology can be used to generate a clustering tree based on the label information of part of the corpus information, so as to reduce the computational complexity and improve the overall corpus information processing efficiency while ensuring the performance of the clustering tree.

[0068] Furthermore, in this embodiment, sample distillation technology will be used to generate a clustering tree based on the label information of part of the corpus information.

[0069] In step S130, cluster the label information corresponding to the partial corpus information based on the large language model to determine the clustering result.

[0070] In this embodiment, the clustering result is determined by using the large language model to cluster the label information corresponding to the partial corpus information. Among them, the partial corpus information used in the large language model clustering in this embodiment can be a certain number of corpus information arbitrarily selected from the obtained all corpus information, or a part of the corpus information screened from all corpus information based on a preset selection rule. For example, multiple corpus information with a semantic similarity less than a preset similarity threshold to each other are selected from all corpus information according to semantic similarity and input into the large language model for clustering to obtain the corresponding clustering result.

[0071] In step S140, train and determine the feature model according to the clustering result and the label information corresponding to the clustering result.

[0072] In this embodiment, the clustering result obtained by clustering the label information of the partial corpus information by the large language model and the label information corresponding to the clustering result are used as training samples to train and determine the feature model, so as to accurately represent the text of the labels of each corpus information through the feature model.

[0073] In step S150, cluster the label information of each corpus information based on the feature model to determine the clustering tree.

[0074] In this embodiment, when clustering the label information of each corpus information based on the feature model to generate the clustering tree, the label information of each corpus information will be first input into the feature model for processing to obtain the text representation of the corresponding label information; then a preset clustering algorithm is used to cluster each text representation, so as to determine the clustering tree.

[0075] Optionally, the feature model in this embodiment can adopt models that can be used to generate text representations, such as the BERT model, convolutional neural network CNN, bag of words model, word embedding model, etc.; the clustering algorithm can adopt algorithms that can be used for clustering, such as the K-means algorithm, hierarchical clustering, DBSCAN clustering, etc.

[0076] Furthermore, in this embodiment, the feature model uses the BERT model, and the clustering algorithm uses the K-means algorithm. By combining the BERT model and the K-means algorithm, the BERT model is used to capture the context information of the text, and the text representation generated by the BERT model is used as the input of the K-means algorithm for clustering, so as to generate a clustering tree for corpus analysis. Since the BERT model has been pre-trained on a large-scale corpus, it has strong generalization ability, can adapt to various types of text data, and generate text representations with higher accuracy and more comparability; while the K-means algorithm, as a general clustering algorithm, can be applied to various types of data sets and has strong flexibility in use; therefore, by combining the BERT model and the K-means algorithm, the clustering process can have stronger semantic understanding ability, and the clustering results are more accurate, more applicable, and more interpretable, thus improving the accuracy and reliability of the generated clustering tree.

[0077] Figure 2 is the flow chart for determining the clustering tree in the embodiment of the present invention. As Figure 2 shown, in this embodiment, the clustering tree for corpus information analysis is determined by the following method.

[0078] In step S210, multi-level clustering is performed on the label information of each corpus information based on the feature model to determine at least one clustering level and the clustering categories corresponding to each clustering level.

[0079] In this embodiment, when performing multi-level clustering on the label information of each corpus information based on the feature model, first, the label information of each corpus information is represented in text based on the feature model to determine the corresponding text vectors; then, a preset clustering algorithm is used to perform multi-level clustering on the text vectors corresponding to each corpus information to determine at least one clustering level and the clustering categories corresponding to each clustering level.

[0080] Optionally, when performing multi-level clustering, in this embodiment, top-down hierarchical clustering or bottom-up hierarchical clustering can be used. Among them, top-down hierarchical clustering means starting from all samples, specifying a cluster, and then dividing it into two sub-clusters, and then continuing to operate on the sub-clusters until the required number of clusters is reached. Bottom-up hierarchical clustering means starting from a single sample, regarding each sample as a cluster, and then merging these clusters into larger clusters until the required number of clusters is finally reached.

[0081] Furthermore, in this embodiment, top-down hierarchical clustering method can be used for clustering to generate a clustering tree; and the generated clustering tree includes three levels of clustering categories, and the clustering categories at each level respectively correspond to the root node, the first-level node, and the second-level node of the clustering tree, and each node represents a clustering category.

[0082] When generating a clustering tree for clustering, first, the text vectors corresponding to all corpus information are used as an overall data set, and this overall data set is used as the root node of the clustering tree; then, the data in the root node is input into a clustering algorithm, and the clustering algorithm divides the data set into a preset number of subsets according to the similarity of the text vectors, and each subset corresponds to a first-level node; after that, for each first-level node, the data corresponding to the first-level node is used as a new whole, and the above clustering process is repeated to obtain multiple new subsets, and each new subset corresponds to a second-level node. Finally, the clustering nodes are arranged in descending order of node level to generate a clustering tree including a root node, at least one first-level node, and at least one second-level node.

[0083] It should be understood that the structure of the clustering tree given in this embodiment is only an example. Specifically, the level tree of the clustering tree and the corresponding number of clustering times can be selected according to the actual application scenario to generate a clustering tree that meets the requirements, and this is not limited here.

[0084] In step S220, the corpus information belonging to the clustering category is input into a large language model for analysis to determine the category label corresponding to the clustering category.

[0085] Optionally, after determining the clustering categories corresponding to the clustering tree, for each clustering category, in this embodiment, some corpus information is randomly selected from all the corpus information corresponding to the clustering category and input into the large language model for analysis and parsing to generate the category name corresponding to the clustering category, that is, the category label.

[0086] In step S230, a clustering tree is generated according to the level relationship of each clustering category and the corresponding category label.

[0087] In this embodiment, after the category labels of all clustering categories are determined, according to the clustering level relationship of each clustering category, the category labels of each clustering category are combined, and a clustering tree containing general semantics will be generated. Since this clustering tree is very decoupled and there is a clear distinction between different clustering categories, it can be used to comprehensively analyze the corpus information reflected by the merchant's original voice, thereby further improving the accuracy of corpus information processing.

[0088] Optionally, the process of generating the clustering tree in this embodiment can be completed offline before the start of the corpus information processing process, or it can be completed in real time. Further, considering that the process of generating a clustering tree according to the existing merchant corpus information takes a certain amount of time, in this embodiment, when processing the corpus information, the offline-generated clustering tree is used, so that when analyzing the merchant's original voice corpus, the pre-generated clustering tree can be directly used to process the corpus information to be analyzed, reducing the overall processing time of the corpus information processing and being beneficial to further improving the processing efficiency of the corpus information.

[0089] Further optionally, the corpus analysis link in this embodiment will be dynamically updated. By using the latest corpus information and corpus analysis results, parts such as corpus knowledge, large language models, and clustering models involved in the corpus analysis link are upgraded and optimized, so that the performance of the corpus analysis link in this embodiment is continuously improved, and the efficiency and accuracy of corpus information processing are higher.

[0090] In step S160, the corpus to be analyzed input into the large language model is classified according to the clustering tree to determine the corresponding category label.

[0091] In this embodiment, after determining the clustering tree for corpus information processing, the corpus to be analyzed will be classified according to the category labels in the generated clustering tree, and the category labels of the corpus to be analyzed will be generated, so that relevant personnel can effectively handle the problems feedback by the corpus to be analyzed according to the category labels of the corpus to be analyzed, improve the performance of the operation platform, and enhance the merchant user experience.

[0092] Optionally, to further improve the efficiency and accuracy of corpus information processing, when processing the corpus to be analyzed in this embodiment, the large language model is still combined to determine the category label of the corpus to be analyzed. Specifically, when determining the category label of the corpus information to be analyzed, the corpus to be analyzed will be first input into the large language model for information extraction to determine the corresponding label information; then, the category label information in the clustering tree is used as a prompt word to control the large language model to classify the label information corresponding to the corpus information according to the category labels in the clustering tree, so as to determine the category label of the corpus to be analyzed.

[0093] Further, in this embodiment, after determining the category label of the corpus to be analyzed, in order to facilitate relevant personnel (such as operation personnel or product personnel, etc.) to timely understand and process the problems feedback by merchants, the analysis result of the corpus to be analyzed will be sent to the target terminal of the relevant personnel. Optionally, the analysis result of the corpus to be analyzed can be the category label of the corpus to be analyzed, or the label information of the corpus to be analyzed, or can also include both the label information and the category label of the corpus to be analyzed. Correspondingly, the corpus information processing method in this embodiment further includes: sending the category label and / or label information of the corpus information to be analyzed to the target terminal, so that the relevant personnel of the target terminal can understand the problem content feedback by the merchant according to the category label or label information of the corpus information to be analyzed, and make targeted improvements and optimizations based on the problem content, thereby improving the operation user experience.

[0094] Optionally, in this embodiment, the method shown in Figure 3 is adopted to display the analysis result of the corpus to be analyzed, so as to facilitate relevant personnel to batch process the problems feedback by merchants.

[0095] In step S310, category labels of different corpora to be analyzed are collected periodically.

[0096] In this embodiment, considering that the problems feedback by merchants are individual problems and the processing capabilities of the relevant personnel handling the problems feedback by merchants are limited, problems feedback by merchants will be collected periodically and / or in batches, and then centralized processing will be carried out after a certain number of problems are collected. Therefore, in this embodiment, category labels of different corpora or these corpora to be analyzed feedback by different users or the same user will be collected periodically and / or in batches, so that the relevant personnel can conduct centralized processing on the actual problems corresponding to these category labels.

[0097] Optionally, in this embodiment, by collecting the category labels of different corpora to be analyzed periodically to conduct batch and centralized processing on the problems reflected by the category labels, the processing efficiency of the problems reflected by the category labels can be improved, and the user experience can be enhanced.

[0098] In step S320, statistical analysis is performed on the category labels of each corpus to be analyzed, and a corresponding analysis report is generated.

[0099] In this embodiment, to facilitate the relevant personnel to understand the existing problems, statistical analysis will be performed on the category labels of each corpus to be analyzed, and a corresponding analysis report will be generated. Among them, the statistical analysis includes determining the proportion of each category label in all the collected category labels, the change parameters of each category label in different periods and other category label-related parameters, so that the relevant personnel can quickly understand the main problems in the problems feedback by merchants and the solution progress of the problems feedback by merchants from the analysis report.

[0100] Optionally, in this embodiment, when collecting the category labels of the corpora to be analyzed, the label information of the corpora to be analyzed can be collected at the same time, so that the relevant personnel can intuitively understand the general expression of the corpora feedback by merchants from the analysis report, thereby quickly understanding the user's intention and conducting targeted processing, and improving the problem processing efficiency.

[0101] It should be understood that the analysis report given in this embodiment is only an example, and the content and display method of the analysis report can be set according to the actual application scenario, and no limitation is imposed here.

[0102] In step S330, the analysis report is sent to the target terminal.

[0103] In this embodiment, after generating an analysis report according to the above method, the analysis report will be sent to the target terminal, which can be a terminal device (such as a laptop computer, a smart phone, etc.) held by the corresponding person or other electronic devices that can facilitate relevant personnel to view. The type of the target terminal is not limited here. Thus, in this embodiment, an analysis report is generated according to the analysis result of the corpus information to be analyzed by the above method, and the analysis report is sent to relevant personnel, which can facilitate relevant personnel to quickly discover and solve problems from the perspective of merchants, thereby improving the usage performance of the operation platform and enhancing the user experience.

[0104] The technical solution of this embodiment extracts information from multiple pieces of corpus information through a large language model, determines the tag information of each piece of corpus information, and trains a feature model based on the tag information of some corpus information and the clustering results after clustering; then clusters the tag information of each piece of corpus information based on the feature model to determine a clustering tree, and finally classifies the corpus to be analyzed input into the large language model according to the clustering tree to determine the corresponding category label; it can use the large language model to build a general and automated corpus analysis link with the lowest construction cost, utilize the rich knowledge in the large language model to process the corpus information to determine the corresponding analysis result, while greatly reducing the human analysis cost, improving the efficiency and accuracy of corpus analysis. Further, by analyzing the category label and analysis result corresponding to the corpus to be analyzed of the merchant and generating an analysis report to convey to relevant personnel, it can help the platform quickly and accurately understand and sort out the merchant's demands, thereby empowering the construction and iteration of the e-commerce platform, iteratively optimizing the product and operation strategies, improving the merchant experience, and avoiding merchant loss.

[0105] Figure 4 It is a schematic diagram of the corpus analysis process of the embodiment of the present invention. As Figure 4 shown, in this embodiment, before analyzing the corpus information, a clustering tree will be generated offline based on the existing corpus information fed back by merchants; then the category label of the corpus to be analyzed will be generated based on the clustering tree.

[0106] In the process of generating the clustering tree, in this embodiment, first, each corpus information and the prompting words for information extraction are input into the large language model, so that the large prediction model performs information extraction on each corpus information based on the prompting words for information extraction, and outputs a summary expression of each corpus information, that is, the label information corresponding to the corpus information. After determining the label information of each corpus information, the large language model is used to cluster the label information corresponding to some corpus information to obtain a clustering result, and this part of the corpus information and the corresponding clustering result are used to train and determine the feature model. Then, the feature model is used to perform text representation on each corpus information respectively, and combined with a preset clustering algorithm, multi-level clustering is performed on the label information of each corpus information to determine at least one clustering level and the clustering categories corresponding to each clustering level. Finally, the corpus information belonging to each clustering category is respectively input into the large language model for category naming to generate the category labels of each clustering category, and the category labels of each clustering category are combined according to the clustering level relationship of each clustering category, thus generating a clustering tree for corpus information analysis.

[0107] After generating the clustering tree, combined with Figure 5 the data flow diagram of corpus information processing shown in

[0108] In this embodiment, after receiving the to-be-analyzed corpus 51 newly fed back by the merchant, the to-be-analyzed corpus 51 is first input into the large language model for information extraction to determine the corresponding label information 52; then, the category label information in the clustering tree is used as a prompting word to control the large language model to classify the label information 52 corresponding to the analyzed corpus 51 according to the category labels in the clustering tree, so as to determine the category label 53 of the to-be-analyzed corpus.

[0109] Figure 6 is a schematic diagram of the corpus information processing device according to an embodiment of the present invention. As Figure 6As shown in the figure, the corpus information processing device in this embodiment includes: an acquisition module 61, an expression module 62, a clustering module 63, and a classification module 64. Among them, the acquisition module 61 is used to acquire corpus data, and the corpus data includes multiple pieces of corpus information. The expression module 62 is used to input each piece of corpus information into a pre-trained large language model for information extraction, and determine the label information corresponding to the corpus information. The label information is a general expression of the corresponding corpus information. The clustering module 63 is used to cluster the label information corresponding to some of the corpus information based on the large language model to determine the clustering result; train and determine a feature model according to the clustering result and the label information corresponding to the clustering result; cluster the label information of each piece of corpus information based on the feature model to determine a clustering tree. The classification module 64 is used to classify the corpus to be analyzed input into the large language model according to the clustering tree to determine the corresponding category label.

[0110] Optionally, the expression module 62 is further used to perform context learning, few-shot learning, and chain-of-thought analysis on the corpus information based on the large language model to determine the label information corresponding to the corpus information.

[0111] Optionally, when the clustering module 63 clusters the label information of each piece of corpus information based on the feature model to determine a clustering tree, it is further used to perform multi-level clustering on the label information of each piece of corpus information based on the feature model to determine at least one clustering level and the clustering categories corresponding to each clustering level; input the corpus information belonging to the clustering category into the large language model for analysis to determine the category label corresponding to the clustering category; generate a clustering tree according to the level relationship of each clustering category and the corresponding category label.

[0112] When the clustering module 63 classifies the corpus to be analyzed input into the large language model according to the clustering tree to determine the corresponding category label, it is further used to input the corpus to be analyzed into the large language model for information extraction to determine the corresponding label information; classify the label information according to the clustering tree to determine the category label of the corpus to be analyzed.

[0113] Optionally, the corpus information processing device in this embodiment is further used to send the category label and / or label information of the corpus to be analyzed to the target terminal. Further, the corpus information processing device is further used to periodically collect the category labels of different corpora to be analyzed; perform statistical analysis on the category labels of each corpus to be analyzed to generate a corresponding analysis report; send the analysis report to the target terminal.

[0114] Figure 7 It is a schematic diagram of an electronic device according to an embodiment of the present invention. In this embodiment, as Figure 7As shown, the electronic device 7 includes at least one processor 71; and a memory 72 communicatively connected to the at least one processor 71; and a communication component 73 communicatively connected to the scanning device, and the communication component 73 receives and transmits data under the control of the processor 71; wherein, the memory 72 stores instructions executable by the at least one processor 71, and the instructions are executed by the at least one processor 71 to implement the above-mentioned corpus information processing method.

[0115] Specifically, the electronic device includes one or more processors 71 and a memory 72. Figure 7 Here, one processor 71 is taken as an example. The processor 71 and the memory 72 can be connected by a bus or other means. Figure 7 Here, the connection by a bus is taken as an example. As a non-volatile computer-readable storage medium, the memory 72 can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. The processor 71 executes various functional applications and data processing of the device by running the non-volatile software programs, instructions, and modules stored in the memory 72, that is, to implement the above-mentioned corpus information processing method.

[0116] The memory 72 can include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store an option list, etc. In addition, the memory 72 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 72 optionally includes a memory remotely set relative to the processor 71, and these remote memories can be connected to an external device through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0117] One or more modules are stored in the memory 72, and when executed by one or more processors 71, they implement the corpus information processing method in any of the above method embodiments.

[0118] The above product can execute the method provided in the embodiments of the present application, and has corresponding functional modules and beneficial effects for executing the method. For technical details not described in detail in this embodiment, reference can be made to the method provided in the embodiments of the present application.

[0119] The technical solution of this embodiment extracts information from multiple pieces of corpus information through a large language model, determines the tag information of each corpus information, and trains a feature model based on the tag information of some corpus information and the clustering results after clustering; then clusters the tag information of each corpus information based on the feature model to determine a clustering tree; finally, classifies the corpus to be analyzed input into the large language model according to the clustering tree to determine the corresponding class label. Thus, in this embodiment, an automated corpus analysis link can be built using the large language model, and the rich knowledge in the large language model can be used to process the corpus information, thereby improving the efficiency and accuracy of corpus analysis.

[0120] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program, and the computer-readable program is used for a computer to execute the above-mentioned partial or all method embodiments.

[0121] That is, those skilled in the art can understand that all or part of the steps in implementing the above-mentioned embodiment methods can be completed by instructing relevant hardware through a program, and this program is stored in a storage medium, including several instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of the present application. And the aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs and other various media that can store program codes.

[0122] The above are only the preferred embodiments of the present application and are not used to limit the present application. For those skilled in the art, various changes and modifications can be made to the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for processing corpus information, characterized in that The method includes: Obtaining corpus data, where the corpus data includes multiple pieces of corpus information; Inputting each piece of the corpus information into a pre-trained large language model for information extraction, and determining the label information corresponding to the corpus information, where the label information is a summary expression of the corresponding corpus information; Clustering the label information corresponding to some of the corpus information based on the large language model to determine a clustering result; Training and determining a feature model according to the clustering result and the label information corresponding to the clustering result; Clustering the label information of each piece of the corpus information based on the feature model to determine a clustering tree; Classifying the corpus to be analyzed input into the large language model according to the clustering tree, and determining the corresponding category label.

2. The method according to claim 1, wherein The inputting each piece of the corpus information into a pre-trained large language model for information extraction, and determining the label information corresponding to the corpus information includes: Based on the large language model, performing context learning, few-shot learning, and chain of thought analysis on the corpus information to determine the label information corresponding to the corpus information.

3. The method according to claim 1, wherein The clustering the label information of each piece of the corpus information based on the feature model to determine a clustering tree includes: Performing multi-level clustering on the label information of each piece of the corpus information based on the feature model to determine at least one clustering level and the clustering categories corresponding to each clustering level; Inputting the corpus information belonging to the clustering category into the large language model for analysis to determine the category label corresponding to the clustering category; Generating a clustering tree according to the level relationship of each clustering category and the corresponding category label.

4. The method according to claim 1, characterized in that The classifying the corpus to be analyzed input into the large language model according to the clustering tree, and determining the corresponding category label includes: Inputting the corpus to be analyzed into the large language model for information extraction to determine the corresponding label information; Classifying the label information according to the clustering tree to determine the category label of the corpus to be analyzed.

5. The method according to claim 4, wherein The method further includes: Sending the category label and / or label information of the corpus to be analyzed to a target terminal.

6. The method according to claim 1, wherein The method further includes: Periodically collecting the category labels of different corpora to be analyzed; Performing statistical analysis on the category labels of each corpus to be analyzed to generate a corresponding analysis report; Sending the analysis report to a target terminal.

7. The method according to claim 1, wherein The content of the corpus information includes feedback information of merchant users on the operation platform and / or supplementary information of the operation platform on the feedback information, and the category label is used to characterize the category of the corresponding feedback information.

8. The method according to claim 1, characterized in that, The feature model adopts a BERT model.

9. A corpus information processing device, characterized in that, The device includes: An acquisition module, configured to acquire corpus data, where the corpus data includes multiple pieces of corpus information; An expression module, configured to input each piece of the corpus information into a pre-trained large language model for information extraction, and determine the label information corresponding to the corpus information, where the label information is a summary expression of the corresponding corpus information; A clustering module, configured to cluster label information corresponding to some of the corpus information based on the large language model to determine a clustering result; train and determine a feature model according to the clustering result and the label information corresponding to the clustering result; cluster the label information of each corpus information based on the feature model to determine a clustering tree; A classification module, configured to classify the to-be-analyzed corpus input into the large language model according to the clustering tree to determine a corresponding category label.

10. A computer program product, characterized in that, The computer program product includes a computer program / instructions, and when the computer program / instructions are executed by a processor, the method according to any one of claims 1-8 is implemented.

11. An electronic device, comprising a memory and a processor, characterized in that, The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method according to any one of claims 1-8.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1-8 is implemented.

Images