Corpus processing method and apparatus based on paradigm evolution, and device, medium, and product

Abstract

The present invention relates to artificial intelligence technology, and disclosed are a corpus processing method and apparatus based on paradigm evolution, and a device, a storage medium, and a program product. The method comprises: acquiring a knowledge graph of an instruction sample set of a target domain, wherein the knowledge graph comprises task networks of instruction samples in the instruction sample set; extracting a task paradigm network from the knowledge graph, wherein the task paradigm network is a common network among task networks of a same task type; evolving the task paradigm network to obtain an evolved task paradigm network; and generating a learning corpus on the basis of the evolved task paradigm network, wherein the learning corpus is used for learning of a target domain model. A learning corpus is processed by means of an evolution mechanism based on a task paradigm network, thereby improving the granularity of the learning corpus, reducing dependence on a large amount of annotated data, and reducing data preparation costs and difficulty of model learning; and the learning corpus is used for learning of a target domain model, thereby enhancing an understanding capability of the model for highly specialized and highly focused domain tasks.

Description

Paradigm Evolution-Based Corpus Processing Methods, Devices, Equipment, Media, and Products

[0001] Cross-references to related applications

[0002] This application claims priority to Chinese Patent Application No. 2025100831959, filed on January 20, 2025, entitled "Corpus Processing Method, Apparatus, Device, Medium and Product Based on Paradigm Evolution", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This invention relates to the field of artificial intelligence technology, and in particular to a corpus processing method, apparatus, device, medium and product based on paradigm evolution. Background Technology

[0004] In the development of artificial intelligence, models have become a research hotspot due to their powerful learning capabilities and wide range of applications. However, the lack of sufficient adaptability and generalization ability of models when performing domain-specific tasks limits their application in diverse and complex industries.

[0005] Existing methods for improving model capabilities primarily focus on enhancing the model itself, such as direct fitting, guided learning, and distillation. However, imprecise corpus management often results in insufficient granularity and depth of understanding for specialized and highly focused domain tasks, and the need for large amounts of domain-labeled data leads to high model training costs. Summary of the Invention

[0006] This invention provides a corpus processing method, apparatus, device, medium, and product based on paradigm evolution to address the problems of existing model capability improvement methods that are mainly model-centric and lack fine-grained corpus management, resulting in insufficient granularity and depth of understanding for highly specialized and focused domain tasks, and requiring a large amount of domain-labeled data, leading to high model training costs.

[0007] In a first aspect, embodiments of the present invention provide a corpus processing method based on paradigm evolution, including:

[0008] Obtain a knowledge graph of a set of instruction samples in a target domain; the knowledge graph includes the task network of each instruction sample in the instruction sample set.

[0009] Extract task paradigm networks from the knowledge graph, wherein the task paradigm networks are common networks among task networks of the same task type;

[0010] The task paradigm network is evolved to obtain the task paradigm evolution network;

[0011] The learning corpus is generated based on the task paradigm evolution network, and the learning corpus is used for learning the target domain model.

[0012] Secondly, embodiments of the present invention provide a corpus processing method based on paradigm evolution, including:

[0013] The knowledge graph acquisition module is used to acquire a knowledge graph of the instruction sample set in the target domain; the knowledge graph includes the task network of each instruction sample in the instruction sample set.

[0014] The network extraction module is used to extract task paradigm networks from the knowledge graph, wherein the task paradigm network is a common network among task networks of the same task type.

[0015] The network evolution module is used to evolve the task paradigm network to obtain a task paradigm evolution network.

[0016] The corpus generation module is used to generate learning corpus based on the task paradigm evolution network, and the learning corpus is used for learning the target domain model.

[0017] Thirdly, embodiments of the present invention provide an electronic device, the electronic device comprising:

[0018] At least one processor; and

[0019] A memory communicatively connected to the at least one processor; wherein,

[0020] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the paradigm evolution-based corpus processing method according to any embodiment of the present invention.

[0021] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer instructions, which are used to cause a processor to execute the corpus processing method based on paradigm evolution as described in any embodiment of the present invention.

[0022] Fifthly, embodiments of the present invention provide a computer program product including a computer program, which, when executed by a processor, implements the corpus processing method based on paradigm evolution as described in any embodiment of the present invention.

[0023] The technical solution of this invention involves acquiring a knowledge graph of a set of instruction samples in a target domain. The knowledge graph includes task networks for each instruction sample in the set. A task paradigm network is extracted from the knowledge graph; this network is a common network among task networks of the same task type. The task paradigm network is then evolved to obtain a task paradigm evolution network. Learning corpus is generated based on this evolution network and used for learning the target domain model. By processing instruction samples using an evolutionary mechanism based on the task paradigm network to obtain the learning corpus, the granularity of the learning corpus is improved. Existing model capability enhancement methods are primarily model-centric, with imprecise corpus management, leading to insufficient granularity and depth of understanding for highly specialized and focused domain tasks. Furthermore, these methods require a large amount of domain-annotated data, resulting in high model training costs. This invention reduces reliance on large amounts of labeled data, lowers the data preparation cost and difficulty for model learning, and, by using the learning corpus for learning the target domain model, improves the model's understanding of highly specialized and focused domain tasks and enhances its generalization ability.

[0024] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0025] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0026] Figure 1 is a flowchart of a corpus processing method based on paradigm evolution provided by an embodiment of the present invention;

[0027] Figure 2 is a schematic diagram of a corpus processing device based on paradigm evolution provided in an embodiment of the present invention;

[0028] Figure 3 is a schematic diagram of the structure of an electronic device that implements the paradigm evolution-based corpus processing method of this invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] Figure 1 is a flowchart of a corpus processing method based on paradigm evolution provided by an embodiment of the present invention. This embodiment is applicable to the processing of learning corpora for model learning based on paradigm evolution. The method can be executed by a corpus processing device based on paradigm evolution, which can be implemented in hardware and / or software and can be configured in an electronic device. As shown in Figure 1, the method includes:

[0032] S110. Obtain the knowledge graph of the instruction sample set in the target domain; the knowledge graph includes the task network of each instruction sample in the instruction sample set.

[0033] The target domain can be one or more practical application domains, such as finance, technology, electricity, and life services, etc., and this embodiment of the invention does not limit this. The instruction sample set can be understood as a collection of instruction samples, which may include sample questions and sample answers. To enable large models to better understand the intent of the instructions, the instruction samples may also include a Chain of Thought (CoT). A Chain of Thought can be understood as inserting intermediate reasoning steps between the question and the answer, simulating the step-by-step thinking process of humans when solving problems, thereby improving the performance of large language models in complex reasoning tasks.

[0034] A knowledge graph can be understood as a relational network that connects various types of information contained in a set of instruction samples. A knowledge graph includes task networks for each instruction sample in the set. These task networks can be understood as relational networks based on task events within the instruction samples and the relationships between these task events. Specifically, a task network can be represented by the task execution workflow represented by nodes and edges. Nodes can be task events, and edges can be the relationships between events. Nodes and edges constitute the network structure of the task network.

[0035] Specifically, the instruction sample set of the target domain is analyzed, the task execution information of each instruction sample is extracted, the task network of the instruction sample is constructed based on the task execution information, and the task network of each instruction sample is stored in a knowledge graph.

[0036] S120. Extract task paradigm networks from knowledge graphs; task paradigm networks are common networks among task networks of the same task type.

[0037] In this context, a task paradigm network can be understood as a network composed of task events that can perform specific tasks or have certain specific execution capabilities.

[0038] Specifically, the task networks in the knowledge graph are classified and analyzed to obtain task networks of multiple task types; the common network among task networks of the same task type is identified as the task paradigm network.

[0039] For example, techniques such as frequent subgraph mining or GNN and subgraph matching are used to compare the nodes, edges and network structures of different task networks to identify common networks in task networks of the same task type, and the common networks are identified as task paradigm networks.

[0040] Understandably, in the process of constructing a knowledge graph, only one node needs to be built for the same event, eliminating the need to repeatedly build nodes. Essentially, this is a compression process of the model's ability to perform different tasks. Determining the task paradigm network based on the task network in the knowledge graph represents another round of clustering and compression of the model's capabilities.

[0041] S130. Evolve the task paradigm network to obtain the task paradigm evolution network.

[0042] In this context, the task paradigm evolution network can be understood as a task paradigm network obtained through capability evolution. The task paradigm network can be considered as the core capability operator of a large model to complete different tasks, and is the "meta-capability" of the large model. Capability evolution of the task paradigm network is a method that can improve the capabilities of large models from the essence, core and fundamental aspects, enabling the model to have better generalization ability.

[0043] Specifically, the task paradigm network can be evolved from at least one dimension, including node operations, node connections, and network structure, to further improve the model's capabilities and performance accuracy. Evolution methods can include rule-based evolution or evolution using a large model. To ensure the correctness of the evolution results, a verification mechanism can be introduced to validate the results.

[0044] S140. Learning corpus is generated based on task paradigm evolution network, and the learning corpus is used for learning the target domain model.

[0045] The learning corpus can be understood as the dataset used to fine-tune the basic large model. The target domain model can be understood as the model learned by fine-tuning the learning corpus of the target domain. The target domain model can be a deep learning model or a large model.

[0046] For example, the training corpus can be generated using rule-based generation methods, corpus-based template generation methods, or large-scale model generation techniques. The generated training corpus can be processed and stored in a structured corpus to facilitate subsequent model learning and validation, improve the model's generalization ability, enable the target domain model to form a systematic and complete domain task processing capability, and enhance the model's specialization and efficiency in a specific domain.

[0047] The technical solution of this invention involves acquiring a knowledge graph of a set of instruction samples in a target domain. The knowledge graph includes task networks for each instruction sample in the set. A task paradigm network is extracted from the knowledge graph; this network is a common network among task networks of the same task type. The task paradigm network is then evolved to obtain a task paradigm evolution network. Learning corpus is generated based on this evolutionary network and used for learning a model in the target domain. By processing the instruction samples using an evolutionary mechanism based on the task paradigm network to obtain the learning corpus, the precision of the learning corpus is improved, and the dependence on large amounts of labeled data is reduced, lowering the data preparation cost and difficulty for model learning. Furthermore, using the learning corpus for learning a model in the target domain enhances the model's understanding of specialized and focused domain tasks, thereby improving the model's generalization ability.

[0048] As an optional embodiment of the present invention, S130, the evolution of the task paradigm network to obtain a task paradigm evolution network includes: evolving at least one of the node operations, node links and network structure of the task paradigm network to obtain a task paradigm evolution network.

[0049] The task paradigm network consists of nodes and node links. Nodes represent task events, which can be understood as the execution steps of a task. Node attributes include the event's input object, output object, and operation. An operation (OP) describes the specific processing method and process of the event; it can be a text description, an API interface, or a tool call. Node links represent the relationship between a node's input and output objects, i.e., link = {input->output}.

[0050] Specifically, the evolution of a task paradigm network can include the evolution of at least one of the following: node operations, node links, and network structure. The evolution of node operations in a task paradigm network can be called OP evolution, the evolution of node links can be called LINK evolution, and the evolution of the network structure can be called structural evolution.

[0051] Both OP and LINK evolution alter the network structure of the task paradigm network, and can be considered passive structural changes. The purpose of structural evolution is to actively modify the network structure to generate new task paradigm networks, thereby endowing them with new functions. Of course, structural evolution inevitably leads to corresponding changes in the OP and LINK components of the task paradigm network.

[0052] In an optional embodiment, after evolving the task paradigm network to obtain the evolved task paradigm network, the evolved task paradigm network can be input into a network verification module to verify its feasibility. Verification includes, but is not limited to, checking the network's consistency, completeness, and execution efficiency, and outputting the evolved and verified task paradigm network. For networks that pass the feasibility verification, they are output as valid results after evolution. For networks that fail the verification, the reasons for failure need to be recorded, and it may be necessary to return to the corresponding evolution step for adjustment. The technical means of implementing verification in this process include, but are not limited to: using automated testing tools for batch verification to improve verification efficiency and accuracy; employing model verification methods to ensure the logical correctness of each node and task flow; and using performance evaluation tools to measure the network's execution efficiency and identify potential performance bottlenecks.

[0053] As an optional embodiment of the present invention, the evolution of node operations in the task paradigm network includes: performing at least one evolution operation among node operation splitting, node operation merging, and node operation modification on the node operations in the task paradigm network without changing the input and output objects of the nodes.

[0054] Specifically, the evolution of node operations in task paradigm networks mainly involves evolving node operations without changing the node link relationship (link = {input->output}), that is, keeping the node's input and output objects unchanged and without affecting upstream and downstream nodes. OP evolution includes: node operation splitting (OP splitting), node operation merging (OP merging), and node operation modification (OP modification); OP modification can include OP deletion. Node operation splitting is used to split the original node operation into multiple intermediate node operations without changing the node's input and output objects. Node operation merging is used to merge multiple original node operations into a single, comprehensive node operation without changing the node's input and output objects. Node operation modification is used to modify the original node operation into the target node operation without changing the node's input and output objects.

[0055] It should be noted that there is no requirement for consistency in the functionality of the task paradigm network before and after OP evolution; that is, the task paradigm network can generate new capabilities through OP evolution.

[0056] In a specific example, for node operation splitting, the node in the task paradigm network is represented as {input->OP}. org Without changing the input and output objects of the node, the node after the OP split can be represented as {input->[…OP}. i …->OP org ->…OP j …]->output}. OP splitting can be implemented in several ways: rule-based or based on large model generation. To ensure the correctness of the OP splitting results, an auditing mechanism can be introduced to verify them. By splitting a node operation in the task paradigm network into multiple intermediate node operations, the input-output relationship of the nodes remains unchanged while enhancing the detail and granularity of the task workflow.

[0057] For example, the prompt message used to generate the OP splitting result using large model techniques can include: task description information, description information of the original node operations, description information of the OP splitting requirements, description information of the intermediate node operations, and output result format requirements. The description information of the OP splitting requirements can include consistency, integrity, and invariance of input and output objects. For example, the prompt message can be expressed as: "Please play the role of a problem-solving expert. Your task is to complete the node operation (OP) evolution of the task paradigm network, keeping the original node input and output objects unchanged, and splitting the OP into multiple intermediate OPs."

[0058] Original node operations (OP)org The operation is: [Extracting the original node (OP)] org (Name, description, input objects, and output objects).

[0059] OP splitting must adhere to the following requirements:

[0060] (1) Define intermediate node operations (OP) i and OP j ):OP i : Describe the OP org Previous node operation section; OP j : Describe the OP org The following section covers node operations.

[0061] The generation logic is suitable for intermediate node operations (OPs) between the original input and output. i and OP j This ensures that the final input->output relationship remains unchanged.

[0062] (2) Ensure consistency and integrity:

[0063] Verify intermediate node operations (OP) i and OP j Does the task flow maintain consistency and integrity?

[0064] Ensure the new sequence {input->[...OP i ...->OP org ->...OP j ...]->output} conforms to logic and supports the original workflow.

[0065] (3) Detailed description of node operations:

[0066] Provide intermediate node operations (OP) i and OP j A detailed description of the method and process, including its implementation. The description can be in text format, API interface, tool call, etc.

[0067] Output format requirements: List the name, description, input object, and output object of each node operation in separate sections.

[0068] Please perform the OP splitting evolution operation according to the above rules.

[0069] In another specific example, for node operation merging, the nodes in the task paradigm network can be represented as {input1->OP}. org1 ->output1}, {input2->OP org2 ->output2}, {input3->OPorg3 ->output3}, ..., Operations on each original node (OP) org Interconnected, meaning nodes are linked by edges, and the output of an upstream node becomes the input of its connected downstream node. Without altering the node's input and output objects, each node's operations (OPs) are defined. org1 ,OP org2 OPorg3, ... can be merged into a more streamlined and efficient OP. i The nodes after the OP split can be represented as {input} i ->OP i =[OP org1 ,OP org2 ,OP org3 ,…]->output i},input i ∈{input1,input2,input3,…},output i ∈{output1,output2,output3…}. Operation merging can be implemented using rule-based methods or methods based on large model generation. To ensure the correctness of the merging results, an auditing mechanism can be introduced to verify them. By merging multiple node operations in the task paradigm network into a single comprehensive node operation, the input-output relationship of the nodes remains unchanged while improving task processing efficiency.

[0070] For example, the prompt message generated using large model techniques to produce OP merging results may include: task description information, description information of the original node operations, description information of the OP merging requirements, description information of the synthesized node operations, and output result format requirements, etc. The OP merging requirements may include consistency, completeness, and invariance of input and output objects. For example, the prompt message can be represented as:

[0071] "Please take on the role of a problem-solving expert. Your task is to complete the node operation evolution of the task paradigm network, keeping the input and output objects of the nodes unchanged, and transforming the interconnected original node operations (OPs)..." org 1,OP org2 O Porg3 ...) are merged into a more streamlined and efficient integrated node operation (OP). i Now, to handle relationships between multiple nodes, please follow these steps:

[0072] (1) Analyze the original node operations:

[0073] Confirm all original node operations (OPs) to be merged. org1 ,OPorg2 ,OP org3 Each node operation (...) includes its own name, description, input object, and output object; verify that these node operations are indeed interconnected, i.e., the output of one node operation serves as the input of the next node operation.

[0074] (2) Define the integrated node operation (OP) i ):

[0075] Generate a new integrated node operation (OP) i This is used to replace multiple original node operations while maintaining the same input->output relationship.

[0076] Ensure OP i The design takes into account the functionality of all original node operations and enables these functions to be implemented in a more efficient way.

[0077] (3) Ensure consistency:

[0078] Verify the merged OP i Whether the consistency and integrity of the task process have been maintained.

[0079] Ensure the new integrated node operation {input->OP i =[OP org1 OP org2 OP org3 The output of the output conforms to the logic and supports the original workflow.

[0080] (4) Detailed description of the operation:

[0081] Provides comprehensive node operation (OP) i A detailed description of the method and process, which may be in text format, API interface, or tool call, etc.

[0082] The output format requirements are as follows: list the name, description, input object, and output object of each node operation in segments, as well as the name, description, input object, output object, and combined node operation of the combined operation node.

[0083] Please perform the OP merging evolution operation according to the rules above.

[0084] In another specific example, for node operation modifications, a node in the task paradigm network can be represented as {input->OP}. org Without changing the node's input and output objects, the modified node operation can be represented as {input->output}. new->output}. OP modifications can be implemented in ways that include rule-based approaches or large-model generation. To ensure the correctness of OP modification results, an auditing mechanism can be introduced to verify the results. By modifying multiple node operations in the task paradigm network to target node operations, the network structure is optimized while ensuring that the input-output relationship of the nodes remains unchanged, thus improving task execution efficiency.

[0085] For example, the prompt message generated using large model techniques to indicate OP modification results may include: task description information, description information of the original node operation, description information of the OP modification requirements, description information of the target node operation, and output result format requirements. The description information of the OP modification requirements may include consistency, completeness, and invariance of input and output objects. For example, the prompt message can be represented as:

[0086] "Please take on the role of a problem-solving expert. Your task is to complete the evolution of the operation (OP) of the nodes in the task paradigm network, while keeping the input and output objects of the nodes unchanged, and performing operations (OP) on the original nodes." org Modify the target node operation (OP) to form the target node operation. new Now we need to handle the specific operations within a single step. Please follow these rules:

[0087] (1) Extract and analyze the original node operations (OPs) in detail. org ): [Name, Description, Input Object, and Output Object]. Identify the role of this node operation in the task flow and its relationship to other node operations.

[0088] (2) Define target node operations (OP) new ):

[0089] Generate a target node operation (OP) that is logically suitable to replace the original node operation. new This ensures that the final input->output relationship remains unchanged. OP new It should be able to improve efficiency, reduce complexity, or enhance functionality without changing the original input / output links.

[0090] (3) Ensure consistency:

[0091] Verify target node operation (OP) new Whether the consistency and integrity of the task flow have been maintained.

[0092] Ensure {input->OP new The ->output} property conforms to logic and supports the original workflow.

[0093] (4) Detailed description of the operation:

[0094] Provide target node operations (OP) new A detailed description of the method and process, including its implementation. The description can be in text format, API interface, tool call, etc.

[0095] The output format should be as follows: List the name, description, input object, and output object of both the original node operation and the target node operation in separate sections.

[0096] As an optional embodiment of the present invention, the node links in the task paradigm network are evolved, including: modifying the descriptions of the input and output objects of the nodes in the task paradigm network, and establishing the evolved node links.

[0097] Specifically, by modifying the descriptions of the input and output objects of nodes in the task paradigm network, new relationships can be created between nodes, corresponding to new execution steps in the business logic. For the evolution of node links, nodes undergoing type transformation can be added to achieve data matching between input and output objects. Furthermore, based on the changed data matching relationships between input and output objects, evolved LINK relationships are established, and new edges are added to the graph accordingly.

[0098] For example, the prompt message generated using large model techniques to describe the node link evolution results may include: descriptions of the target node, requirements for LIN evolution, and output format requirements. The requirements for LIN evolution may include consistency and completeness. For instance, the prompt message could be represented as:

[0099] "Please assume the role of a problem-solving expert. Your task is to complete the link evolution of nodes in a task paradigm network. The goal is to modify the input and output descriptions of the nodes in the task paradigm network to create new relationships between the nodes. Please follow these rules:"

[0100] (1) Identify the target node:

[0101] Extract the nodes in the task paradigm network that require modification of input or output, including node name, description, current input, and output.

[0102] (2) Define the evolved link:

[0103] Generate new input and output descriptions for the target node, ensuring that these links can form new logical relationships between the nodes.

[0104] (3) Ensure consistency:

[0105] Verify that the evolved links maintain the consistency and integrity of the task flow; ensure that the evolved link relationships are logical and support improvements to the original workflow.

[0106] (4) Detailed description link:

[0107] Provide a detailed description of the evolved link, including its methods and processes. The description can be in text format, API interface, tool call, etc.

[0108] The output format requirements are as follows: list the name, description, input objects, and output objects of the target node in segments, as well as the input objects, output objects, and descriptions of the evolved links.

[0109] Please perform LINK evolution according to the rules above.

[0110] As an optional embodiment of the present invention, the network structure in the task paradigm network is evolved, including:

[0111] The task paradigm network is segmented to obtain multiple task stages;

[0112] Without changing the input and output objects of the nodes, the network structure of the task paradigm network is optimized by adopting the alternative scheme of the task stage.

[0113] Specifically, following the principles of computational graph partitioning, the task paradigm network is divided into multiple task stages, each representing an independent step in the task flow. Without altering the input and output objects of the nodes, a simpler alternative that achieves the same input and output is found for each task stage, thereby optimizing the internal structure of the task paradigm network and improving task execution efficiency.

[0114] For example, network segmentation and network structure optimization can be achieved based on rules or large model techniques.

[0115] The prompt for segmenting a task paradigm network using large model techniques can include: a description of the task paradigm network segmentation task, a description of the task paradigm network analysis method, a description of the segmentation points, a description of the task stages, and output format requirements. An example of a prompt could be:

[0116] "Please assume the role of a computational graph design expert. Your task is to partition the task paradigm network, dividing it into multiple task stages, each representing an independent step in the task process. Please follow these rules:"

[0117] (1) Analyze the task paradigm network:

[0118] Extract all nodes and edges from the task paradigm network, identify the input objects, output objects, and operation descriptions for each node, and determine the overall structure and task flow of the task paradigm network.

[0119] (2) Determine the dividing point:

[0120] Following the conventional partitioning principles of computational graphs, identify key partitioning points in the task flow; partitioning points should be able to divide the task flow into multiple independent task stages.

[0121] (3) Task-segmented paradigm network:

[0122] The task paradigm network is divided according to the determined split points to ensure that each task stage is independent and the connection relationship between each task stage is clear.

[0123] (4) Describe the task phases after segmentation:

[0124] Provide a detailed description of each task stage, including its name, included nodes, input objects, output objects, and operation descriptions, to ensure that the description of each task stage is clear and facilitates subsequent analysis and optimization.

[0125] The output format should be as follows: List each task stage in separate sections, including its name, included nodes, input objects, output objects, and operation description.

[0126] The prompt for optimizing the network structure of a task paradigm network using large model techniques can include: a description of the network structure optimization task, a description of the task phase analysis, a description of the optimization scheme, a description of the network optimization process, a description of the optimized task phase, and output format requirements. For example, the prompt could be:

[0127] "Please assume the role of a task flow optimization expert. Your task is to optimize the internal structure of the task paradigm network without affecting the network's input and output, thereby improving the efficiency of task execution. Please follow these rules:"

[0128] (1) Analyze each task phase:

[0129] Extract all nodes and edges in each task stage, identify their input objects, output objects, and operation descriptions; determine the overall structure and task flow of each task stage.

[0130] (2) Finding an optimization solution:

[0131] For each task stage, find alternative solutions with a simpler structure that can achieve the same input and output. These alternative solutions should simplify the task process and improve execution efficiency.

[0132] (3) Optimize the task paradigm network:

[0133] Based on the found optimization scheme, the internal structure of the task paradigm network was adjusted to ensure that the optimized structure improves the efficiency of task execution without affecting the input and output.

[0134] (4) Describe the optimized task phase:

[0135] Provide a detailed description of each optimized task stage, including its name, included nodes, input objects, output objects, and operation descriptions, to ensure that the optimized task stage descriptions are clear and facilitate subsequent analysis and verification.

[0136] The output format should be as follows: For each task stage in the optimized task paradigm network, list the name of the task stage, the nodes it contains, the input objects, the output objects, and the operation description.

[0137] As an optional embodiment of the present invention, the evolution of the network structure in the task paradigm network further includes:

[0138] Identify target task stages from the task paradigm network whose input objects are of the same type as the output objects of the task stages; add the target task stages to the task stages of the task paradigm network one by one according to their structural complexity.

[0139] Specifically, target task stages with the same input object type as the output object of the task stage are identified from the task paradigm network. These target task stages that meet the criteria are then sorted according to their structural complexity and added to the network's task stages one by one. During task execution, the optimal solution can be selected from the task paradigm network based on performance and effectiveness evaluations, thereby extending the network's functionality.

[0140] As an optional embodiment of the present invention, S110, obtaining the knowledge graph of the instruction sample set of the target domain, includes:

[0141] S111. Extract task event information from the thought chain of each instruction sample in the instruction sample set of the target domain; the task event information includes: event name, input object, output object, operation, and the relationship between events.

[0142] Among them, task event information can be understood as the information of events included in the execution steps of a task, which may include: event name, input object, output object, operation, and the relationship between events.

[0143] Specifically, for each instruction sample in the instruction sample set of the target domain, the event name, input object, output object, operation, and relationship between events corresponding to each step are extracted from the CoT of the instruction sample's thought chain.

[0144] For example, task event information can be extracted from the thought chain of each instruction sample by using techniques such as Named Entity Recognition (NER), Event Extraction (EE), and Relation Extraction (RE).

[0145] S112. Create a node for each event. The name of the node is the event name of the event. The attributes of the node include: the input object, the output object, and the operation of the event. The input object and the output constitute a node link.

[0146] S113. Construct directed edges based on the relationships between the events described.

[0147] Specifically, directed edges are constructed between nodes based on the relationships between events. These directed edges represent the data flow and control flow from one event to another.

[0148] S114. Construct a task network based on each node and each directed edge, and store the task network in a knowledge graph.

[0149] Specifically, all nodes and directed edges are combined into a task execution flow graph, which serves as the task network for sample instructions. The task network fully represents the task's workflow. All sample task networks are stored in a knowledge graph.

[0150] As an optional embodiment of the present invention, before extracting task event information from the thought chain of each sample in the instruction sample set, the method further includes:

[0151] The instruction samples in the instruction sample library are classified into instruction samples containing thought chains and instruction samples not containing thought chains.

[0152] For instruction samples that do not contain a thought chain, the sample questions and sample answers from the instruction samples that do not contain a thought chain are input into the thought chain generation model to obtain the thought chain.

[0153] Specifically, based on whether the instruction sample contains a thought chain (CoT), the instruction samples in the instruction sample library are divided into two types: instruction samples containing a thought chain (CoT), denoted as S. wCoT ={Q,CoT,A}; Instruction samples that do not contain CoT are denoted as S. woCoT＝ {Q,A}; where Q represents the sample question, A represents the sample answer, and CoT represents the thought process chain. For SwoCoT Instruction samples can be converted into S by supplementing the corresponding CoT description. wCoT Samples. For example, the classification method for quality samples can be based on rules, large model analysis, or deep learning classification models, etc., and the embodiments of the present invention do not limit or elaborate on these methods.

[0154] For instruction samples that do not contain CoT, a thought chain for the instruction sample is generated based on the thought chain generation model. For example, the thought chain generation model can use open-source large models, online large model services, or fine-tuned domain large models. Taking the instruction sample S1 = {Q1, A1} as an example, the thought chain generation prompt is S1Prompt. Inputting S1Prompt into the thought chain generation model, the output is the thought chain CoT1 corresponding to the instruction sample S1, thus forming a new sample S1 = {Q1, CoT1, A1} containing CoT.

[0155] This embodiment generates a thought chain from instruction samples that do not contain a thought chain, providing a necessary foundation for extracting task event information from the thought chain of the instruction samples.

[0156] As an optional embodiment of this application, after inputting the thought chain generation prompt information into the thought chain model to obtain the thought chain of the instruction samples that do not contain the thought chain, and before extracting task event information from the thought chain of each instruction sample in the instruction sample set, the method further includes:

[0157] A1. Input the sample questions from the thought chain and the instruction samples that do not contain the thought chain into the thought chain generation model to obtain the output answer.

[0158] The output answer is the answer generated by the mind chain generation model for the sample question.

[0159] Specifically, based on the sample questions in the instruction samples that do not contain the thought chain and the generated thought chain, the question-and-answer prompts are input into the thought chain generation model to obtain the output answer corresponding to the sample question.

[0160] A2. Verify the thought chain of the instruction sample that does not contain a thought chain based on the comparison results between the output answer and the sample answer.

[0161] Specifically, the output answer is compared with the sample answer, and the thought chain of the instruction sample that does not contain a thought chain is verified based on the comparison result. If the comparison result shows that the output answer and the sample answer are the same, then the thought chain of the instruction sample that does not contain a thought chain is considered to have passed the verification; if the comparison result shows that the output answer and the sample answer are different, then the thought chain of the instruction sample that does not contain a thought chain is considered to have failed the verification.

[0162] For example, the methods for comparing output answers and sample answers may include one or more of the following: evaluation methods based on expert experience, evaluation methods based on approximate metrics, evaluation methods based on verification platforms and test cases, evaluation methods based on verifier engineering, internal consistency and self-feedback methods, confidence estimation and illusion detection methods, and combinations thereof.

[0163] Evaluation methods based on expert experience mainly rely on the experience of domain experts to compare and analyze sample answers and output answers, assess whether the output answers meet the expected correctness standards, identify biases, and provide detailed feedback and correction suggestions.

[0164] Evaluation methods based on approximation metrics primarily assess the correctness of the generated results by calculating the similarity index between the sample answer and the output answer. Natural language processing techniques can be used to compare two text sequences, calculate their overlap, and thus derive a quantified similarity score.

[0165] Evaluation methods based on verification platforms and test cases primarily assess the correctness of the output answers by running a series of predefined test cases on a dedicated verification platform. The test cases are designed to cover various expected input and output scenarios. By comparing sample answers with the output answers, the output of the large-scale mind chain generation model can be automatically verified to ensure it meets predetermined standards.

[0166] Evaluation methods based on verifier engineering primarily use automated verifiers to perform verification tasks and provide feedback to the base model. They combine automation and human feedback, optimize model performance through a closed-loop feedback mechanism, analyze sample answers and output answers, identify inconsistencies, and provide improvement suggestions.

[0167] Rating methods based on internal consistency and self-feedback primarily assess the correctness of an output answer by analyzing its internal consistency during the generation process. It checks whether the model maintains logical coherence during decoding and whether it produces consistent results under different sampling strategies. Through a self-feedback mechanism, the model can self-evaluate the consistency and correctness of its output, thereby improving its reasoning ability and reducing errors.

[0168] Confidence estimation and hallucination detection are primarily achieved by having a large model that generates thought chains evaluate the correctness of the output answer. The model provides the output answer and is asked whether it is true or false; the model's log probability P(True) is used as the confidence score. Simultaneously, hallucination detection techniques, such as SelfCheckGPT, are used to determine whether the output answer contains factual errors.

[0169] A3. If the verification result is that the thought chain of the instruction sample that does not contain a thought chain fails the verification, then obtain the difference information between the output answer and the sample answer.

[0170] Specifically, if the thought chain of the instruction sample that does not contain the thought chain fails the verification, the difference information between the output answer and the sample answer is determined. The difference information may include: missing information, redundant information, incorrect information, inconsistent expression, etc.

[0171] A4. Input the generated thought chain, the output answer, the instruction sample that does not contain the thought chain, and the difference information into the thought chain generation model to obtain the corrected thought chain of the instruction sample that does not contain the thought chain.

[0172] Specifically, the output answer and thought chain of the thought chain generation model, the sample questions and sample answers contained in the instruction sample, and the difference information between the output answer and the sample answer are used to generate thought chain correction prompts. The thought chain correction prompts are then input into the thought chain generation model to obtain the thought chain corrected for instruction samples that do not contain thought chains.

[0173] For example, the thought chain correction prompt information may include: a description of the thought chain correction task, the output answer and thought chain of the thought chain generation model, sample questions and sample answers included in the instruction sample, and information on the differences between the output answer and the sample answers. For instance, the thought chain correction prompt information could be:

[0174] "Please play the role of a solution expert. Your task is to understand the problem, analyze the differences between the current answer to the problem and the standard answer, analyze the reasons for the differences, and revise the specific steps to solve the problem."

[0175] Given the original problem (Q): [the problem in the sample];

[0176] Standard answer (A): [The answer in the sample];

[0177] Current CoT: [Currently used CoT];

[0178] Current answer (A'): [Newly generated answer];

[0179] Information on the difference between the current answer and the standard answer (ΔA): [Detailed description of the difference];

[0180] Based on the information above, please revise the original problem-solving steps to ensure that the problem is solved accurately and completely.

[0181] The revised thought chain is (CoT): [Content to be generated].

[0182] As an optional embodiment of this application, the generation of learning corpus based on the task paradigm evolution network includes:

[0183] The task paradigm evolution network is decomposed into multiple task units;

[0184] Each of the aforementioned task units is input into the corpus to generate a large model, thereby obtaining the learning corpus.

[0185] Specifically, decomposing the task paradigm evolution network into multiple task units and inputting them into the corpus to generate a large model can yield richer and more targeted learning data. Using this learning data to fine-tune the large model can significantly enhance its ability to solve complex problems.

[0186] For example, a way to decompose a task paradigm evolution network into multiple task units is to introduce a task extraction module, which decomposes the complex task paradigm evolution network into multiple more detailed task units.

[0187] Figure 2 is a schematic diagram of a corpus processing device based on paradigm evolution provided in an embodiment of the present invention. As shown in Figure 2, the device includes: a knowledge graph acquisition module 210, a task paradigm network extraction module 220, a network evolution module 230, and a corpus generation module 240; wherein,

[0188] The knowledge graph acquisition module 210 is used to acquire a knowledge graph of the instruction sample set in the target domain; the knowledge graph includes the task network of each instruction sample in the instruction sample set.

[0189] The network extraction module 220 is used to extract task paradigm networks from the knowledge graph, wherein the task paradigm network is a common network among task networks of the same task type.

[0190] The network evolution module 230 is used to evolve the task paradigm network to obtain a task paradigm evolution network.

[0191] The corpus generation module 240 is used to generate learning corpus based on the task paradigm evolution network, and the learning corpus is used for learning the target domain model.

[0192] The technical solution of this invention involves acquiring a knowledge graph of a set of instruction samples in a target domain. The knowledge graph includes task networks for each instruction sample in the set. A task paradigm network is extracted from the knowledge graph; this network is a common network among task networks of the same task type. The task paradigm network is then evolved to obtain a task paradigm evolution network. Learning corpus is generated based on this evolutionary network and used for learning a model in the target domain. By processing the instruction samples using an evolutionary mechanism based on the task paradigm network to obtain the learning corpus, the precision of the learning corpus is improved, and the dependence on large amounts of labeled data is reduced, lowering the data preparation cost and difficulty for model learning. Furthermore, using the learning corpus for learning a model in the target domain enhances the model's understanding of specialized and focused domain tasks, thereby improving the model's generalization ability.

[0193] Optionally, the network evolution module 230 is used for:

[0194] The task paradigm evolution network is obtained by evolving at least one of the node operations, node links, and network structure of the task paradigm network.

[0195] Optionally, the network evolution module 230 includes:

[0196] The node operation evolution unit is used to perform at least one evolution operation among node operation splitting, node operation merging, and node operation modification on the node operations in the task paradigm network without changing the input and output objects of the nodes.

[0197] The node operation splitting is used to classify the original node operation into multiple intermediate node operations; the node operation merging is used to merge multiple original node operations into a comprehensive node operation; and the node operation modification is used to modify the original node operation into a target node operation.

[0198] Optionally, the network evolution module 230 includes:

[0199] The node link evolution unit is used to modify the descriptions of the input and output objects of the nodes in the task paradigm network and establish the evolved node links.

[0200] Optionally, the network evolution module 230 includes:

[0201] The network structure evolution unit is used to perform network segmentation on the task paradigm network to obtain multiple task stages; without changing the input and output objects of the nodes, it optimizes the network structure of the task paradigm network by adopting alternative solutions for the task stages.

[0202] Optionally, the network structure evolution unit further includes:

[0203] Identify target task stages from the task paradigm network whose input objects are of the same type as the output objects of the task stages; add the target task stages to the task stages of the task paradigm network one by one according to their structural complexity.

[0204] Optionally, the knowledge graph acquisition module 210 includes:

[0205] The event information extraction module is used to extract task event information from the thought chain of each instruction sample in the instruction sample set of the target domain; the task event information includes: event name, input object, output object, operation, and the relationship between events;

[0206] A node creation unit is used to create a node for each event. The name of the node is the event name, and the attributes of the node include: the input object, the output object, and the operation of the event; the input object and the output constitute a node link.

[0207] An edge construction unit is used to construct directed edges based on the relationships between the events;

[0208] The task network construction unit is used to construct a task network based on each of the nodes and each of the directed edges, and to store the task network in a knowledge graph.

[0209] Optional, also includes:

[0210] The sample classification module is used to classify the instruction samples in the instruction sample library into instruction samples containing thought chains and instruction samples not containing thought chains before extracting task event information from the thought chain of each sample in the instruction sample set.

[0211] The thought chain generation module is used to take instruction samples that do not contain thought chains, input the sample questions and sample answers from the instruction samples that do not contain thought chains into the thought chain generation model, and obtain the thought chain of the instruction samples that do not contain thought chains.

[0212] Optional, also includes:

[0213] The answer output module is used to input the thought chain and the sample questions from the instruction samples that do not contain thought chains into the thought chain generation model after inputting the thought chain generation prompt information into the thought chain model to obtain the thought chain of the instruction sample that does not contain thought chains, and before extracting task event information from the thought chain of each instruction sample in the instruction sample set, to obtain the output answer.

[0214] The thought chain verification module is used to verify the thought chain of the instruction sample that does not contain a thought chain based on the comparison result between the output answer and the sample answer.

[0215] The difference information determination module is used to obtain the difference information between the output answer and the sample answer if the verification result is that the thought chain of the instruction sample that does not contain a thought chain fails the verification.

[0216] The thought chain correction module is used to input the generated thought chain, the output answer, the instruction sample that does not contain the thought chain, and the difference information into the thought chain generation model to obtain the corrected thought chain of the instruction sample that does not contain the thought chain.

[0217] Optionally, the corpus generation module 240 is specifically used for:

[0218] The task paradigm evolution network is decomposed into multiple task units;

[0219] Each of the aforementioned task units is input into the corpus to generate a large model, thereby obtaining the learning corpus.

[0220] The corpus processing device based on paradigm evolution provided in the embodiments of the present invention can execute the method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0221] Figure 3 illustrates a schematic diagram of an electronic device 10 that can be used to implement embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0222] As shown in Figure 3, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer programs stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0223] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0224] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as paradigm evolution-based corpus processing methods.

[0225] In some embodiments, the paradigm evolution-based corpus processing method can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the paradigm evolution-based corpus processing method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the paradigm evolution-based corpus processing method by any other suitable means (e.g., by means of firmware).

[0226] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0227] In some embodiments, the paradigm evolution-based corpus processing method can be implemented as a computer program, which is implicitly included in a computer program product. When executed by a processor, the computer program implements the paradigm evolution-based corpus processing method of the present invention. The computer program product can be understood as a software product that primarily implements its solution through a computer program. The computer program used to implement the method of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program can be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0228] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0229] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0230] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0231] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0232] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0233] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A corpus processing method based on paradigm evolution, characterized in that, include: Obtain a knowledge graph of a set of instruction samples in a target domain; the knowledge graph includes the task network of each instruction sample in the instruction sample set. Extract task paradigm networks from the knowledge graph, wherein the task paradigm networks are common networks among task networks of the same task type; The task paradigm network is evolved to obtain the task paradigm evolution network; The learning corpus is generated based on the task paradigm evolution network, and the learning corpus is used for learning the target domain model.

2. The method according to claim 1, characterized in that, The process of evolving the task paradigm network to obtain a task paradigm evolution network includes: The task paradigm evolution network is obtained by evolving at least one of the node operations, node links, and network structure of the task paradigm network.

3. The method according to claim 2, characterized in that, Evolving the node operations in the task paradigm network includes: Without changing the input and output objects of the nodes, perform at least one evolutionary operation among node operation splitting, node operation merging, and node operation modification on the node operations in the task paradigm network. The node operation splitting is used to classify the original node operation into multiple intermediate node operations; the node operation merging is used to merge multiple original node operations into a comprehensive node operation; and the node operation modification is used to modify the original node operation into a target node operation.

4. The method according to claim 2, characterized in that, Evolving the node links in the task paradigm network includes: Modify the descriptions of the input and output objects of the nodes in the task paradigm network to establish the evolved node links.

5. The method according to claim 2, characterized in that, Evolving the network structure in the task paradigm network includes: The task paradigm network is segmented to obtain multiple task stages; Without changing the input and output objects of the nodes, the network structure of the task paradigm network is optimized by adopting the alternative scheme of the task stage.

6. The method according to claim 5, characterized in that, Evolving the network structure in the aforementioned task paradigm network also includes: Identify target task stages from the task paradigm network whose input objects are of the same type as the output objects of the task stages; The target task stages are added one by one to the task stages of the task paradigm network according to their structural complexity.

7. The method according to claim 1, characterized in that, The knowledge graph for obtaining the instruction sample set of the target domain includes: Task event information is extracted from the thought chain of each instruction sample in the instruction sample set of the target domain; the task event information includes: event name, input object, output object, operation, and the relationship between events; Create a node for each event. The name of the node is the event name. The attributes of the node include: the input object, the output object, and the operation of the event. The input object and the output form a node link. Construct directed edges based on the relationships between the events; A task network is constructed based on each node and each directed edge, and the task network is stored in a knowledge graph.

8. The method according to claim 7, characterized in that, Before extracting task event information from the thought chain of each sample in the instruction sample set, the following steps are also included: The instruction samples in the instruction sample library are classified into instruction samples containing thought chains and instruction samples not containing thought chains. For instruction samples that do not contain a thought chain, the sample questions and sample answers from the instruction samples that do not contain a thought chain are input into the thought chain generation model to obtain the thought chain of the instruction samples that do not contain a thought chain.

9. The method according to claim 8, characterized in that, After inputting the thought chain generation prompt information into the thought chain model to obtain the thought chain of the instruction samples that do not contain a thought chain, and before extracting task event information from the thought chain of each instruction sample in the instruction sample set, the method further includes: Input the sample questions from the thought chain and the instruction samples that do not contain the thought chain into the thought chain generation model to obtain the output answer; The thought process of the instruction sample that does not contain a thought process is verified based on the comparison results between the output answer and the sample answer. If the verification result is that the thought chain of the instruction sample that does not contain a thought chain fails the verification, then obtain the difference information between the output answer and the sample answer; The generated thought chain, the output answer, the instruction sample that does not contain the thought chain, and the difference information are input into the thought chain generation model to obtain the corrected thought chain of the instruction sample that does not contain the thought chain.

10. The method according to claim 1, characterized in that, The learning corpus generated based on the task paradigm evolution network includes: The task paradigm evolution network is decomposed into multiple task units; Each of the aforementioned task units is input into the corpus to generate a large model, thereby obtaining the learning corpus.

11. A corpus processing device based on paradigm evolution, comprising: The knowledge graph acquisition module is used to acquire the knowledge graph of the instruction sample set in the target domain. The knowledge graph includes the task network of each instruction sample in the instruction sample set; The network extraction module is used to extract task paradigm networks from the knowledge graph, wherein the task paradigm network is a common network among task networks of the same task type. The network evolution module is used to evolve the task paradigm network to obtain a task paradigm evolution network. The corpus generation module is used to generate learning corpus based on the task paradigm evolution network, and the learning corpus is used for learning the target domain model.

12. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the corpus processing method based on paradigm evolution as described in any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause a processor to execute the corpus processing method based on paradigm evolution as described in any one of claims 1-10.

14. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the corpus processing method based on paradigm evolution as described in any one of claims 1-10.

Images