A script classification method, apparatus and electronic device

Abstract

Embodiments of this application disclose a script classification method, apparatus, and electronic device. The method includes: constructing a script knowledge base, a script intent base, and a script category base based on internal documents and script datasets; obtaining relevant documents from the script knowledge base, relevant intents from the script intent base, and relevant candidate classes from the category base based on the script to be classified; constructing a thought chain based on the relevant documents, relevant intents, and candidate classes; and inputting the thought chain into a target large language model for analysis to obtain the target category corresponding to the script to be classified. This application achieves intelligent classification of scripts generated by enterprise operations and maintenance. When a script is input, some candidate classes are provided, and an automated intent description is generated for the input script. The reasoning ability of the large language model itself and relevant documents are used to identify the intent description associated with the input script, thereby improving both the efficiency and accuracy of script classification.

Description

Technical Field

[0001] This application relates to the field of production operation and maintenance technology; specifically, it relates to a script classification method, apparatus, and electronic device. Background Technology

[0002] Scripts typically consist of multiple parts, including commands, parameters, function definitions, and explanations for a given scenario. In the production and operation of internet companies, script classification is a key step in standardization and plays a vital role in ensuring code security and privacy of permissions.

[0003] Traditional script classification mainly relies on manual annotation, depending on the experience of staff to annotate and categorize scripts. This is not only costly and inefficient, but also subject to subjectivity and instability, affecting the accuracy of script classification and management.

[0004] It is clear that improving the efficiency and accuracy of script classification is an urgent problem to be solved. Summary of the Invention

[0005] To address the aforementioned technical problems, embodiments of this application provide a script classification method, apparatus, and electronic device, which solve the problems of low efficiency and low accuracy in script classification in related technologies.

[0006] According to one aspect of the embodiments of this application, a script classification method is provided. The method includes: constructing a script knowledge base, a script intent base, and a script category base based on internal documents and a script dataset; obtaining relevant documents from the script knowledge base, relevant intents from the script intent base, and relevant candidate classes from the category base based on the script to be classified; constructing a thought chain based on the relevant documents, the relevant intents, and the candidate classes; and inputting the thought chain into a target large language model for analysis to obtain the target category corresponding to the script to be classified.

[0007] Optionally, when the script dataset includes a closed-source script dataset and an open-source script dataset, the step of constructing a script intent library based on the script dataset includes: obtaining a first statistical feature of the closed-source script dataset and a second statistical feature of the open-source script dataset, wherein the statistical features include script length, script language, and commonly used commands of the script; obtaining script data related to the closed-source script dataset from the open-source script dataset based on the distribution differences of the first statistical feature and the second statistical feature to obtain an external script dataset; obtaining a matching intent recognition model based on the first statistical feature; performing intent recognition on the closed-source script dataset and the external script dataset respectively based on the intent recognition model to obtain internal commonly used command intent pairs and external general command intent pairs; and constructing the script intent library based on the internal commonly used command intent pairs and external general command intent pairs.

[0008] Optionally, when there is a special command with an empty intent in the internal commonly used command intent pair, the method further includes: preprocessing the special command to obtain a target command; obtaining the target intent and Internet document corresponding to the target command by searching the Internet, and inserting the target intent into the script intent library and updating the Internet document to the internal knowledge base.

[0009] Optionally, based on the script to be classified, obtaining relevant documents from the script knowledge base, relevant intents from the script intent library, and relevant candidate classes from the category library includes: vectorizing the script knowledge base, the script intent library, and the script category library to obtain vector clusters corresponding to each script category; obtaining at least two candidate vector clusters that match the target vector based on the distance from the target vector corresponding to the script to be classified to the center of each vector cluster; and obtaining relevant documents, relevant intents, and relevant candidate classes corresponding to the script to be classified from the at least two candidate vector clusters.

[0010] Optionally, the script knowledge base, the script intent base, and the script category base are vectorized to obtain vector clusters corresponding to each script category. This includes: extracting vectors from the data in the script knowledge base, the script intent base, and the script category base to obtain vector datasets corresponding to different categories; selecting K initial cluster centers based on the mean and standard deviation of the vector datasets; calculating the distance from each vector data to each cluster and assigning each vector data to the cluster with the minimum distance; calculating new cluster centers, inter-cluster dispersion, and intra-cluster dispersion based on the adjusted new clusters; obtaining a distance criterion function based on the new cluster centers, the inter-cluster dispersion, and the intra-cluster dispersion; selecting the value that minimizes the distance criterion function as the number of clusters; and assigning the vector datasets to the corresponding clusters.

[0011] Optionally, after constructing the script category library, the method further includes: updating the script category library with the new category when a new category exists; retrieving enhanced knowledge related to the new category from an external knowledge graph and updating the enhanced knowledge in the internal knowledge base.

[0012] Optionally, retrieving augmented knowledge related to the new category from an external knowledge graph includes: obtaining the external knowledge graph and the target entity corresponding to the new category; wherein the external knowledge graph includes an entity layer and an instance layer, the entity layer includes several entities with related relationships, and the instance layer includes instances corresponding to each entity; obtaining the first-order nearest neighbor entities of the target entity based on the external knowledge graph; wherein the first-order nearest neighbor entities are entities that are directly related to the target entity; and using the instances corresponding to the target entity and the instances corresponding to the first-order nearest neighbors as augmented knowledge related to the new category.

[0013] Optionally, the method further includes: calculating the vector distance between the target entity and each entity in the external knowledge graph; calculating the similarity between the target entity and each entity in the external knowledge graph based on the vector distance; obtaining the extended entity of the target entity based on the similarity between the target entity and each entity in the external knowledge graph; obtaining the first-order nearest neighbor entities of the extended entity based on the external knowledge graph; and using the instance corresponding to the extended entity and the instance corresponding to the first-order nearest neighbor entity of the extended entity as the enhanced knowledge related to the new category.

[0014] According to one aspect of the embodiments of this application, a script classification apparatus is provided, the apparatus comprising: a database construction module, configured to construct a script knowledge base, a script intent base, and a script category base based on an internal document and script dataset; a candidate class acquisition module, configured to acquire relevant documents from the script knowledge base, relevant intents from the script intent base, and relevant candidate classes from the category base based on the script to be classified; a thought chain construction module, configured to construct a thought chain based on the relevant documents, the relevant intents, and the candidate classes; and a category analysis module, configured to input the thought chain into a target large language model for analysis to obtain the target category corresponding to the script to be classified.

[0015] According to one aspect of the embodiments of this application, a computer-readable medium is provided, on which a computer program is stored, which, when executed by a processor, implements the script classification method as described in the above technical solutions.

[0016] According to one aspect of the embodiments of this application, an electronic device is provided, the electronic device comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions to cause the electronic device to implement the script classification method as described in the above technical solutions.

[0017] According to one aspect of the embodiments of this application, a computer program product is provided, including a computer program that, when executed by a processor, implements the script classification method as described in the above technical solutions.

[0018] The technical solution provided in this application includes at least the following beneficial effects: This application constructs a thought chain by extracting relevant intents, relevant documents, and candidate classes that match the script to be classified from the script intent library, script knowledge library, and script category library, respectively. This enables the large language model to identify the script category using the relevant information in the thought chain. Therefore, this application achieves intelligent classification of scripts generated by enterprise operations and maintenance. When a script is input, some candidate classes will be provided. At the same time, the input script will be automatically generated with intent descriptions. The reasoning ability of the large language model itself and relevant documents will be used to identify the intent descriptions associated with the input script, thereby improving the efficiency and accuracy of script classification. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0020] Figure 1 The diagram shown is a flowchart of a script classification method provided in an embodiment of this application;

[0021] Figure 2 As shown Figure 1 An exemplary flowchart of step S10;

[0022] Figure 3 The diagram shown is a logical framework diagram of a script intent library and a script knowledge base provided in an embodiment of this application;

[0023] Figure 4 The diagram shown is a sample illustration of a thought chain provided in an embodiment of this application;

[0024] Figure 5 The diagram shown is a schematic representation of the output of a target large language model provided in an embodiment of this application.

[0025] Figure 6 The diagram shown is a flowchart of another script classification method provided in an embodiment of this application;

[0026] Figure 7 The diagram shown is a flowchart of a vector clustering method provided in an embodiment of this application.

[0027] Figure 8 The diagram shown is a structural schematic of a script classification device provided in an embodiment of this application;

[0028] Figure 9 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown. Detailed Implementation

[0029] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0030] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0031] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0032] It should also be noted that "multiple" as mentioned in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0033] In the production operations and maintenance (O&M) field of internet companies, the classification of O&M scripts is a crucial step in standardization. O&M scripts typically include multiple parts such as commands, parameters, function definitions, and explanations for a given scenario. Even a single line of command can constitute a script. Companies need to standardize, classify, and review the source code of their products and the scripts running in their information systems. Specifically, frontline O&M personnel in the industry spend nearly half their time each day manually writing execution scripts containing multiple commands to operate production equipment. However, this process carries operational risks. Auditing these operations requires labeling and categorizing the scripts according to the operational action categories in the management specifications to conduct the appropriate audit. Therefore, clearly defining the user permissions and risk levels involved in the scripts is crucial for ensuring code security and access privacy.

[0034] However, in real production environments, script classification still relies primarily on manual annotation. Skilled personnel annotate scripts, categorizing them into a series of specific and clearly defined categories. These category tags are essentially operation IDs in the management specifications and are updated over time and according to business needs. For example, an operations and maintenance script can be categorized by auditors into operation ID-45 according to management specifications. This operation consists of tuples composed of multiple subcategories, corresponding to different content. For instance, a category could be an application server / container, a system log, or a query. A category triple could be represented as (application server / container, system log, query) or application server / container-system log-query. Then, auditing is conducted based on these tags to determine the applicable job position, department, and operation level of the operations and maintenance script, ensuring its security and compliance. Finally, the script is submitted to downstream tasks by business personnel or the audit department.

[0035] However, manual annotation for script classification is not only costly and inefficient, but also results in subjective and unstable classifications based on individual experience, affecting the accuracy of script classification. To address these issues, this application provides a script classification method based on a large language model, specifically including the following embodiments:

[0036] Figure 1 The diagram shown is a flowchart illustrating a script classification method provided in an embodiment of this application; as follows: Figure 1 As shown, the method specifically includes the following steps:

[0037] Step S10: Based on the internal documents and script dataset, construct a script knowledge base, a script intent library, and a script category library.

[0038] Specifically, this includes: building a script category library, building a script knowledge base based on internal documentation, and building a script intent library based on script datasets.

[0039] In this embodiment, the internal documents include resources such as code writing documents, specification documents of different systems, and user manuals of commonly used tools that may be encountered by operation and maintenance personnel. Sensitive information, duplicate values, and blank values ​​are removed through data engineering to obtain cleaned internal documents. A script knowledge base based on document context is constructed with documents as nodes and the relationships between documents as edges. The script knowledge base can improve the understanding and effective utilization of script knowledge by the large language model.

[0040] Optionally, to enhance the semantic connection between code and text, this application can also generate knowledge fragments that combine code and text, such as code fragments containing text comments and code fragments containing script intent descriptions, from the script knowledge base based on document context obtained in the above steps through punctuation processing, unit length segmentation, document clustering, and other means. These fragments can be stored in markdown (lightweight markup language) format. This method can yield a script knowledge base with rich script code knowledge, thereby improving the classification effect.

[0041] Figure 2 As shown Figure 1 An exemplary flowchart of step S10; as shown Figure 2 As shown, building a script intent library based on a script dataset specifically includes the following steps:

[0042] Step S11: Obtain the first statistical features of the closed-source script dataset and the second statistical features of the open-source script dataset. The statistical features include script length, script language, and commonly used commands of the script.

[0043] It should be noted that the script dataset includes both closed-source and open-source script datasets. The closed-source script dataset consists of real scripts written by internal operations and maintenance personnel, representing a low-sample scenario. The open-source script dataset is crawled and filtered from developer code hosting platforms like GitHub, online code Q&A communities like Stack Overflow and Stack Exchange, and code tutorial websites such as Runoob. This data includes knowledge about script commands, Q&A-explanation data sample pairs, and operations-related knowledge, which is then converted into Markdown format for context storage. The open-source dataset is then used to filter the command data of Python and shell scripts, remove irrelevant field features, and use some summary and explanation fields as pseudo-labels to construct the open-source script dataset used in this application as a high-sample scenario.

[0044] In this embodiment, statistical methods are used to calculate the distribution of surface information such as the length of the internal real script, the language system to which the script belongs, and the commonly used commands of the script (referring to basic commands that frequently appear in the internal script but are not basic commands of the Shell and Python language systems). Specifically, the script length is calculated by using an editing tool to measure the file size, number of lines of code, and character length of the real script; the script language (Python, Shell) is determined by the script file header, file extension, and keywords in the script content; and the commonly used commands of the operation and maintenance script are determined by calculating word frequency, keyword extraction, and rule-based methods. Combining the above methods, this embodiment can obtain the first statistical characteristics of closed-source script data in the internal environment and the second statistical characteristics of open-source script data in the external environment; wherein the first and second statistical characteristics respectively include script length, script language, and commonly used commands of the script.

[0045] Step S12: Based on the distribution differences of the first statistical feature and the second statistical feature, obtain script data related to the closed-source script dataset from the open-source script dataset to obtain the external script dataset.

[0046] It should be noted that although a large amount of open-source script data is obtained in the above steps, it may not be related to the internal script data. In order to reduce data interference and improve processing efficiency, this embodiment measures the difference in script data distribution by calculating the KL divergence between the first statistical feature and the second statistical feature, and extracts script data with smaller distribution differences from the closed-source script dataset as the external script dataset.

[0047] Step S13: Obtain the matching intent recognition model based on the first statistical feature.

[0048] It should be noted that, in order to improve the accuracy and efficiency of intent recognition, this embodiment matches the script length in the first statistical feature with the context length supported by the language model, and evaluates whether the script language in the first statistical feature has relevant language data in the language model pre-training, and selects a suitable intent recognition model.

[0049] Step S14: Perform intent recognition on the closed-source script dataset and the external script dataset respectively according to the intent recognition model to obtain the internal commonly used command intent pairs and the external general command intent pairs.

[0050] Specifically, internal commonly used command intent pairs include commonly used commands and their intents, while external general command intent pairs include general commands and their intents.

[0051] It should be noted that this stage requires extracting the common intent from the open-source scripts, i.e., the common commands and their intents across different Linux systems. Inference can be performed using an intent recognition model or by calling the API of a commercial large-scale model for common intent extraction. After completing the common intent extraction operation, the corresponding open-source script-intent pairs are obtained. Due to the large volume of open-source data, this embodiment also needs to perform dimensionality reduction and clustering on the common intents based on features such as commands and templates, such as sampling machine learning clustering algorithms and topic models. After this operation, representative external common command-intent pairs are obtained, upon which a basic intent knowledge base can be built.

[0052] Meanwhile, by deploying an intent recognition model in the internal testing environment to perform intent recognition on internal scripts (i.e., closed-source scripts), and using prompting engineering to extract frequently used internal commands with high operation frequency, high degree of repetition, and industry user preferences from a massive amount of internal operation and maintenance scripts, we can obtain intent pairs of frequently used internal commands.

[0053] Step S15: Construct a script intent library based on commonly used internal command intent pairs and common external command intent pairs.

[0054] In one embodiment, a basic intent library is constructed based on external general command intent pairs, and the basic intent library is improved by internal commonly used command intent pairs. That is, the internal commonly used command intent pairs are matched with the basic intent library. If there are commonly used commands that are different from general commands, the commonly used command intent pairs are inserted into the basic intent library, thereby forming a script intent library.

[0055] In another embodiment, the internal script contains internal special commands, such as common commands with the wx- prefix, whose intents may not be extractable, and only the case where the intent is empty can be obtained; therefore, when there are special commands with empty intents in the internal common command intent pairs, the method further includes: preprocessing the special commands to obtain the target command; obtaining the target intent and Internet documents corresponding to the target command based on Internet search, and inserting the target intent into the script intent library and updating the script knowledge base with Internet documents.

[0056] It should be noted that the preprocessing of special commands includes removing predefined prefixes or suffixes, then obtaining the intent corresponding to the preprocessed target command through internet search, and inserting the intent into the script intent library to improve the script intent library; in addition, this application also uses internet search to obtain internet documents corresponding to the target command in the same way as obtaining open source script datasets, and updates them to the script knowledge base.

[0057] In summary, the logical framework for constructing the script intent library and script knowledge base in this application is as follows: Figure 3As shown, the collected open-source script data is parsed using an intent recognition model to obtain an initial script intent library composed of external general command intent pairs. Simultaneously, the closed-source script data is analyzed using the intent recognition model to obtain internal common command intent pairs. These internal common command intent pairs are then matched against all external general command intent pairs in the script intent library. If a match is found, the internal common command intent pair is considered an external general command intent pair; otherwise, it is inserted into the script intent library. If a special command with an empty intent exists, the corresponding intent is searched online and added to the script intent library. Finally, professionals conduct a quality assessment of the intent pairs corresponding to the internal common commands, manually correcting their true intents to ultimately improve the script knowledge base and script intent library, providing knowledge support for the subsequent construction of the retrieval enhancement framework.

[0058] Step S20: Based on the script to be classified, extract relevant documents from the script knowledge base, relevant intents from the script intent library, and relevant candidate classes from the category library.

[0059] In this embodiment, the process of extracting relevant documents from the script knowledge base, relevant intents from the script intent library, and relevant candidate classes from the category library based on the script to be classified specifically includes: vectorizing the script knowledge base, script intent library, and script category library to obtain vector clusters corresponding to each script category; obtaining at least two candidate vector clusters that match the target vector based on the distance from the target vector corresponding to the script to be classified to the center of each vector cluster; and obtaining the relevant documents, relevant intents, and relevant candidate classes corresponding to the script to be classified from the at least two candidate vector clusters.

[0060] It should be noted that a fine-tuning module is designed. This module applies existing lightweight language models (such as BERT) to learn task- and data-related features, such as key instructions and comments in the script, which are of high value for classifying the script. The fine-tuning module projects the relevant intents, relevant documents, and candidate class information of different categories of scripts into a vector space using a lightweight language model trained by contrastive learning. Then, the k-means method is used to generate clusters corresponding to each category in the vector space. When classifying scripts, this invention first projects the script to be classified into the vector space using a lightweight language model trained by contrastive learning. Then, referring to the distance between the script and the cluster centers corresponding to each category in the vector space, some clusters with the closest distances are selected. The relevant intents, relevant documents, and candidate class information of the scripts corresponding to these clusters are then used as the relevant documents, relevant intents, and relevant candidate classes of the script to be classified.

[0061] Step S30: Construct a thought chain based on relevant documents, relevant intents, and candidate classes.

[0062] This embodiment will utilize relevant intents extracted from the script intent library, combined with relevant documents extracted from the enhanced script knowledge base, and candidate classes selected from the script category library to construct a thought chain example. Figure 4 As shown, the thought chain includes task description, input script (i.e., the script to be classified), relevant intent, relevant documents, candidate classes, and reasoning steps. Using the thought chain as input to the large language model generates the classification ideas and reasoning process of the large language model, thereby enabling the large language model to achieve better performance in subsequent classification of similar data and with reliable explanations.

[0063] Furthermore, when constructing the thought chain, the order in which script-related intents, related documents, and candidate class information belonging to different clusters are arranged in the instructions will be determined by the distance between the input script and the corresponding clusters of these information. The closer the input script is to a certain cluster, the higher the ranking of the related intents, related documents, and candidate class information corresponding to that cluster in the instructions; for example: <Related command 1, intent...> Figure 1 The cluster corresponding to "ID 1, Document Context 1" and Category 1 is closest to the vector space of the input script.

[0064] Step S40: Input the thought chain into the target large language model for analysis to obtain the target category corresponding to the script to be classified.

[0065] This invention combines CoT (Cooperation of Thought) technology to enable a target large language model to read information in a pre-constructed CoT instruction, and based on this information, to think and judge independently, outputting the CoT of the judgment process. This allows the large model to better utilize various relevant information for script classification tasks. For data input, the large model constructs its own CoT and completes the specified task according to the CoT, as shown in the example below. Figure 5 As shown.

[0066] The technical solution provided in this embodiment has at least the following beneficial effects:

[0067] This application constructs a thought chain by extracting relevant intents, related documents, and candidate classes that match the script to be classified from the script intent library, script knowledge base, and script category library, respectively. This enables the large language model to identify the script category using the relevant information in the thought chain. Therefore, this application achieves intelligent classification of scripts generated by enterprise operations and maintenance. When a script is input, some candidate classes are provided. At the same time, the input script is automatically generated with intent descriptions. By utilizing the reasoning ability of the large language model itself and related documents, the intent descriptions associated with the input script are identified, thereby improving both the efficiency and accuracy of script classification.

[0068] In another embodiment of this application, after constructing the script category library, the method further includes: updating the script category library with the new category when a new category exists; retrieving enhanced knowledge related to the new category from an external knowledge graph and updating the enhanced knowledge to the internal knowledge base.

[0069] It should be noted that after the script knowledge base and script intent base are built, this solution will use Retrieval Enhanced Generation (RAG) technology to assist in the prediction of newly added categories. Since the data for new categories is often less than that for existing categories, the prediction effect for newly added categories will be poor. Therefore, the main purpose of this embodiment is to efficiently utilize and update the script knowledge base and script intent base built in the previous steps, thereby enhancing the rationality of the thought chain generation; such as... Figure 6 As shown, firstly, the category library will be dynamically updated during category updates. Simultaneously, relevant knowledge will be retrieved from the external knowledge graph by calculating the term frequency-inverse text frequency index (TF-IDF) and code vector dot product similarity, providing a knowledge foundation for subsequent script knowledge base enhancement. Then, knowledge from the external knowledge graph will be used to supplement information related to new categories, adding additional knowledge from the external knowledge graph to the script knowledge base, thereby enhancing its knowledge and providing richer information related to the new categories. Finally, relevant intents extracted from the script intent library, combined with relevant documents extracted from the enhanced script knowledge base and candidate classes from the script category library, will be used to construct instruction inputs. Few-sample and zero-sample scenarios will be set up, allowing the large language model to construct its own thought chain (CoT), generating its classification ideas and reasoning process. This will enable the large language model to achieve better performance and provide reliable explanations when classifying similar data subsequently.

[0070] Since the input length is displayed, the thought chain itself cannot contain all relevant information in some cases. Therefore, this application can also inject the extracted relevant intents, relevant documents, and candidate classes into the fine-tuning module through parameter fine-tuning and contrastive learning. The fine-tuning module will project this information into the vector space to form clusters. Then, when classifying scripts, the fine-tuning module will first encode the script into a vector in the vector space, and then find the clusters that are close to it in the vector space, thereby extracting the relevant intents, relevant documents, and candidate classes in these clusters, and generating their corresponding confidence scores. Instructions are constructed using this information to assist the large model in class prediction.

[0071] like Figure 6As shown, this application designs a training mechanism that uses only the large model as an information source and predictor during training, thereby bypassing the fine-tuning of the large language model and training only the fine-tuning module. After each category update, this application employs a contrastive learning method to project the obtained relevant intents and document information into the vector space. Specifically, during training, it maximizes the distance between the relevant intents and document information of different categories of scripts in the vector space. During each update, this application maximizes the distance between the new category and other different, irrelevant old categories in the vector space. Using this differential contrastive learning method, the new category information is incrementally trained into the fine-tuning module. This allows the fine-tuning module to retain old category information while receiving new category information. Thus, after obtaining relevant data from multiple sources, it can efficiently integrate multi-source relevant data knowledge into the fine-tuning module without updating the parameters of the large model itself, saving time spent fine-tuning a large number of parameters and avoiding the catastrophic forgetting problem of the large model.

[0072] In one embodiment of this application, such as Figure 7 As shown, the script knowledge base, script intent base, and script category base are vectorized to obtain vector clusters corresponding to each script category. This process includes the following steps:

[0073] Step S21: Extract vectors from the data in the script knowledge base, script intent base, and script category base to obtain vector datasets corresponding to different categories;

[0074] Step S22: Select K initial cluster centers based on the mean and standard deviation of the vector dataset;

[0075] Step S23: Calculate the distance from each vector data to each cluster, and divide each vector data into the cluster with the minimum distance;

[0076] Step S24: Calculate the new cluster centers, inter-cluster dispersion, and intra-cluster dispersion based on the adjusted new clusters;

[0077] Step S25: Obtain the distance criterion function based on the new cluster centers, inter-cluster dispersion, and intra-cluster dispersion. Select the K value that minimizes the distance criterion function as the number of clusters to divide the data, and assign the vector dataset to the corresponding clusters.

[0078] It should be noted that, assuming the vector dataset includes N three-dimensional vector data to be clustered, the specific steps for performing vector clustering on adjacent datasets are as follows: (a) Calculate the mean of all vector data. and standard deviation In other words, the data is mainly distributed in ( - , + (a) Select K initial cluster centers within this interval; (b) Calculate the distance from each vector data to each cluster according to the Euclidean distance formula, and assign each data to the cluster with the minimum distance; (c) Calculate new cluster centers for the adjusted new clusters. If there is no change in the cluster centers between two adjacent clusters, it indicates that the data object adjustment is complete; (d) Since the K value is given in advance, it may not be the optimal solution. Based on the principle of maximizing inter-cluster dispersion and minimizing intra-cluster dispersion, the K value is further optimized using a criterion function. Using the determined cluster centers as the initial cluster centers, calculate the distance between each data object and the initial cluster centers, and calculate the distance criterion function. The K value that minimizes the criterion function value is used as the final number of clusters. (e) Reassign the spatial objects to the corresponding clusters according to the Euclidean distance formula, update each cluster center, until the clustering result remains unchanged.

[0079] In one embodiment of this application, retrieving augmented knowledge related to a new category from an external knowledge graph includes: obtaining the external knowledge graph and the target entity corresponding to the new category. Based on the external knowledge graph, the first-order nearest neighbor entities of the target entity are obtained; where the first-order nearest neighbor entities are entities that are directly associated with the target entity; the instances corresponding to the target entity and the instances corresponding to the first-order nearest neighbors are used as enhanced knowledge related to the new category.

[0080] It should be noted that the external knowledge graph in this embodiment is constructed using known technologies. The external knowledge graph includes an entity layer and an instance layer. The entity layer includes several entities with relationships, and the instance layer includes instances corresponding to each entity. Let the set of all entities in the entity layer be E = ( The trained entity recognition model is used to extract the target entity e from the new category retrieval statement. The target entity e is then matched with entities in the external knowledge graph to identify the entities. , so that e = In other words, it means finding entities that are identical to the target entity in an external knowledge graph. Then, based on the relationship of the entity layer, obtain the entity... Directly connected first-order nearest neighbor entities; entities The corresponding instance and the instance corresponding to the first-order nearest neighbor entity are used as augmented knowledge related to the new category.

[0081] Furthermore, to further expand the retrieval scope and obtain richer enhanced knowledge, this application also includes calculating the vector distance between the target entity and each entity in the external knowledge graph; calculating the similarity between the target entity and each entity in the external knowledge graph based on the vector distance; obtaining the extended entity of the target entity based on the similarity between the target entity and each entity in the external knowledge graph; obtaining the first-order nearest neighbor entities of the extended entity based on the external knowledge graph; and using the instances corresponding to the extended entities and the instances corresponding to the first-order nearest neighbor entities of the extended entities as enhanced knowledge related to the new category.

[0082] It should be noted that the Manhattan distance is used to calculate the distance between the target entity e and the j-th entity in the external knowledge graph. Vector distance between The specific calculation formula is as follows:

[0083]

[0084] Then use vector distance Calculate the j-th entity from target entity e to E. The formula for calculating the similarity is:

[0085]

[0086] Calculate the target entity e and the entity After determining the similarity, we can find several extended entities that have the highest similarity to the target entity e. These extended entities have a strong semantic similarity to the target entity, thus completing the associative expansion of the entity in the new category retrieval statement.

[0087] Figure 8 The diagram shown is a structural schematic of a script classification device provided in an embodiment of this application; as follows: Figure 8 As shown, the device includes:

[0088] Database building module 810 is used to build a script knowledge base, a script intent library, and a script category library based on internal documents and script datasets;

[0089] The candidate class acquisition module 820 is used to acquire relevant documents from the script knowledge base, relevant intents from the script intent library, and relevant candidate classes from the category library based on the script to be classified.

[0090] The Mind Chain Building Module 830 is used to build mind chains based on relevant documents, relevant intents, and candidate classes.

[0091] The category analysis module 840 is used to input the thought chain into the target large language model for analysis to obtain the target category corresponding to the script to be classified.

[0092] In one embodiment, the database construction module 810 includes:

[0093] The statistical feature acquisition unit is used to acquire the first statistical features of the closed-source script dataset and the second statistical features of the open-source script dataset. The statistical features include script length, script language, and commonly used commands of the script.

[0094] The external script dataset acquisition unit is used to obtain script data related to the closed-source script dataset from the open-source script dataset based on the distribution differences of the first statistical feature and the second statistical feature, thus obtaining the external script dataset.

[0095] The intent recognition model acquisition unit is used to acquire a matching intent recognition model based on the first statistical feature.

[0096] The command intent pair acquisition unit is used to perform intent recognition on the closed-source script dataset and the external script dataset respectively according to the intent recognition model, and obtain the internal commonly used command intent pairs and the external general command intent pairs.

[0097] The intent library building unit is used to build a script intent library based on commonly used internal command intent pairs and common external command intent pairs.

[0098] In one embodiment, the database construction module 810 further includes:

[0099] The preprocessing unit is used to preprocess special commands to obtain the target command;

[0100] The search unit is used to retrieve the target intent and internet documents corresponding to the target command based on internet searches, and insert the target intent into the script intent library and update the internet documents to the internal knowledge base.

[0101] In one embodiment, the candidate class acquisition module 820 includes:

[0102] The vector clustering unit is used to vectorize the script knowledge base, script intent base, and script category base to obtain the vector cluster corresponding to each script category;

[0103] The center distance matching unit is used to obtain at least two candidate vector clusters that match the target vector based on the center distance from the target vector to each vector cluster corresponding to the script to be classified.

[0104] The candidate class acquisition unit is used to obtain the relevant documents, relevant intents and relevant candidate classes corresponding to the script to be classified from at least two candidate vector clusters.

[0105] In one embodiment, the vector clustering unit is further used to extract vectors from data in the script knowledge base, script intent base, and script category base to obtain vector datasets corresponding to different categories; select K initial cluster centers based on the mean and standard deviation of the vector datasets; calculate the distance from each vector data to each cluster, and assign each vector data to the cluster with the minimum distance; calculate new cluster centers, inter-cluster dispersion, and intra-cluster dispersion based on the adjusted new clusters; obtain a distance criterion function based on the new cluster centers, inter-cluster dispersion, and intra-cluster dispersion, select the minimum value of the distance criterion function as the number of clusters, and assign the vector datasets to the corresponding clusters.

[0106] In one embodiment, the apparatus further includes:

[0107] The new category update module is used to update the script category library when a new category exists;

[0108] The knowledge graph retrieval module is used to retrieve enhanced knowledge related to new categories from external knowledge graphs and update the enhanced knowledge into the internal knowledge base.

[0109] In one embodiment, the knowledge graph retrieval module includes:

[0110] The target entity acquisition unit is used to acquire the target entities corresponding to the external knowledge graph and the new category; wherein, the external knowledge graph includes an entity layer and an instance layer, the entity layer includes several entities with related relationships, and the instance layer includes instances corresponding to each entity;

[0111] The first nearest neighbor entity acquisition unit is used to acquire the first-order nearest neighbor entities of the target entity based on the external knowledge graph; wherein, the first-order nearest neighbor entities are entities that are directly associated with the target entity;

[0112] The first instance acquisition unit is used to take the instance corresponding to the target entity and the instance corresponding to the first-order nearest neighbor as the new category-related augmented knowledge.

[0113] In one embodiment, the knowledge graph retrieval module further includes:

[0114] The vector distance calculation unit is used to calculate the vector distance between the target entity and each entity in the external knowledge graph;

[0115] The similarity calculation unit is used to calculate the similarity between the target entity and each entity in the external knowledge graph based on the vector distance.

[0116] The extended entity acquisition unit is used to acquire extended entities of the target entity based on the similarity between the target entity and each entity in the external knowledge graph.

[0117] The second nearest neighbor entity acquisition unit is used to acquire the first-order nearest neighbor entities of the extended entity based on the external knowledge graph.

[0118] The second instance acquisition unit is used to take the instance corresponding to the extended entity and the instance corresponding to the first-order nearest neighbor entity of the extended entity as the new category-related augmented knowledge.

[0119] Figure 9 A schematic diagram of the structure of a computer system suitable for implementing the electronic device of the present application is shown.

[0120] It should be noted that, Figure 9 The computer system 1000 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0121] like Figure 9 As shown, the computer system 1000 includes a Central Processing Unit (CPU) 1001, which can perform various appropriate actions and processes based on programs stored in Read-Only Memory (ROM) 1002 or programs loaded from storage portion 1008 into Random Access Memory (RAM) 1003, such as performing the methods described in the above embodiments. Various programs and data required for system operation are also stored in RAM 1003. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via bus 1004. An Input / Output (I / O) interface 1005 is also connected to bus 1004.

[0122] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1010 as needed so that computer programs read from them can be installed into storage section 1008 as needed.

[0123] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs various functions defined in the system of this application.

[0124] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0125] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0126] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0127] The above description is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.

Claims

1. A script classification method, characterized in that, The method includes: A script category library is constructed, a script knowledge base is constructed based on internal documents, and a script intent library is constructed based on a script dataset; wherein, the internal documents include code writing documents, system specification documents, and user manuals, and a script knowledge base based on document context is constructed with documents as nodes and the relationships between documents as edges; When the script dataset includes both closed-source and open-source script datasets, the step of constructing a script intent library based on the script datasets includes: obtaining a first statistical feature of the closed-source script dataset and a second statistical feature of the open-source script dataset, wherein the statistical features include script length, script language, and commonly used commands of the script; obtaining script data related to the closed-source script dataset from the open-source script dataset based on the distribution differences between the first and second statistical features to obtain an external script dataset; obtaining a matching intent recognition model based on the first statistical feature; performing intent recognition on the closed-source script dataset and the external script dataset respectively based on the intent recognition model to obtain internal commonly used command intent pairs and external general command intent pairs; and constructing the script intent library based on the internal commonly used command intent pairs and external general command intent pairs. Based on the script to be classified, relevant documents are obtained from the script knowledge base, relevant intents are obtained from the script intent base, and relevant candidate classes are obtained from the category base. Construct a thought chain based on the relevant documents, the relevant intents, and the candidate classes; The thought chain is input into the target large language model for analysis to obtain the target category corresponding to the script to be classified.

2. The method according to claim 1, characterized in that, When there is a special command with an empty intent in the internal commonly used command intent pair, the method further includes: The special command is preprocessed to obtain the target command; The target intent and internet documents corresponding to the target command are obtained by searching the internet, and the target intent is inserted into the script intent library and the internet documents are updated to the internal knowledge base.

3. The method according to claim 1, characterized in that, Based on the script to be classified, relevant documents are obtained from the script knowledge base, relevant intents are obtained from the script intent base, and relevant candidate classes are obtained from the category base, including: The script knowledge base, the script intent base, and the script category base are vectorized to obtain vector clusters corresponding to each script category; Based on the distance from the target vector corresponding to the script to be classified to the center of each vector cluster, obtain at least two candidate vector clusters that match the target vector; Obtain the relevant documents, relevant intents, and relevant candidate classes corresponding to the script to be classified from the clustering of the at least two candidate vectors.

4. The method according to claim 3, characterized in that, The script knowledge base, the script intent base, and the script category base are vectorized to obtain vector clusters corresponding to each script category, including: Vector extraction is performed on the data in the script knowledge base, the script intent base, and the script category base to obtain vector datasets corresponding to different categories; K initial cluster centers are selected based on the mean and standard deviation of the vector dataset; Calculate the distance from each vector data point to each cluster, and assign each vector data point to the cluster with the minimum distance; Calculate the new cluster centers, inter-cluster dispersion, and intra-cluster dispersion based on the adjusted new clusters; The distance criterion function is obtained based on the new cluster centers, the inter-cluster dispersion, and the intra-cluster dispersion. The minimum value of the distance criterion function is selected as the number of clusters to divide the data, and the vector dataset is assigned to the corresponding clusters.

5. The method according to any one of claims 1-4, characterized in that, After constructing the script category library, the method further includes: When a new category exists, update the script category library with the new category; Retrieve enhanced knowledge related to the new category from an external knowledge graph and update the enhanced knowledge into the internal knowledge base.

6. The method according to claim 5, characterized in that, Retrieve enhanced knowledge related to the new category from an external knowledge graph, including: Obtain an external knowledge graph and the target entity corresponding to the new category; wherein, the external knowledge graph includes an entity layer and an instance layer, the entity layer includes several entities with related relationships, and the instance layer includes instances corresponding to each entity; Based on the external knowledge graph, the first-order nearest neighbor entities of the target entity are obtained; wherein, the first-order nearest neighbor entities are entities that are directly associated with the target entity; The instance corresponding to the target entity and the instance corresponding to the first-order nearest neighbor are used as the enhanced knowledge related to the new category.

7. The method according to claim 6, characterized in that, The method further includes: Calculate the vector distance between the target entity and each entity in the external knowledge graph; The similarity between the target entity and each entity in the external knowledge graph is calculated based on the vector distance. Based on the similarity between the target entity and each entity in the external knowledge graph, the extended entity of the target entity is obtained; Based on the external knowledge graph, obtain the first-order nearest neighbor entities of the extended entity; The instances corresponding to the extended entity and the instances corresponding to the first-order nearest neighbors of the extended entity are used as the enhanced knowledge related to the new category.

8. A script classification device, characterized in that, The device includes: The database construction module is used to build a script category library, a script knowledge base based on internal documents, and a script intent library based on script datasets. The internal documents include code writing documents, system specification documents, and user manuals. A script knowledge base based on document context is constructed with documents as nodes and relationships between documents as edges. The script datasets include closed-source script datasets and open-source script datasets. The candidate class acquisition module is used to acquire relevant documents from the script knowledge base, relevant intents from the script intent library, and relevant candidate classes from the category library based on the script to be classified. The thought chain construction module is used to construct a thought chain based on the relevant documents, the relevant intents, and the candidate classes; The category analysis module is used to input the thought chain into the target large language model for analysis to obtain the target category corresponding to the script to be classified. The database construction module includes: The statistical feature acquisition unit is used to acquire the first statistical features of the closed-source script dataset and the second statistical features of the open-source script dataset. The statistical features include script length, script language, and commonly used commands of the script. The external script dataset acquisition unit is used to obtain script data related to the closed-source script dataset from the open-source script dataset based on the distribution differences of the first statistical feature and the second statistical feature, thus obtaining the external script dataset. The intent recognition model acquisition unit is used to acquire a matching intent recognition model based on the first statistical feature. The command intent pair acquisition unit is used to perform intent recognition on the closed-source script dataset and the external script dataset respectively according to the intent recognition model, and obtain the internal commonly used command intent pairs and the external general command intent pairs. The intent library building unit is used to build a script intent library based on commonly used internal command intent pairs and common external command intent pairs.

9. An electronic device, characterized in that, include: processor; as well as A memory for storing executable instructions of the processor; wherein the processor is configured to execute the executable instructions to enable the electronic device to implement the script classification method as described in any one of claims 1 to 7.

Images