Construction method of dialogue data set, training method and device of multi-turn dialogue model

Abstract

Embodiments of the present application provide a method for constructing a dialogue dataset, a method and device for training a multi-turn dialogue model. The method comprises: identifying target text data to obtain an identification result; constructing a knowledge tree according to the identification result, the knowledge tree comprising a plurality of nodes; determining, for each node, a calling probability of each next-level node of the node; taking a target node as a starting node, selecting nodes in a loop according to a target turn corresponding to the target node, generating a plurality of business paths, and when generating a business path, selecting a next node from the target node according to the calling probability of each next-level node of the current node until the number of nodes of the business path reaches a target number of nodes; and for each business path, generating a set of dialogue data according to the dialogue intent and entity information corresponding to each node in the business path to obtain a dialogue dataset, which can avoid logical discontinuity in dialogue data and improve the comprehensiveness of scene coverage.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, and particularly relates to a dialog data set construction method and a multi-turn dialog model training method and device. BACKGROUND

[0002] With the rapid development of artificial intelligence technology, large language models are increasingly widely used in multi-turn dialog scenarios in vertical fields. For example, in a business collaboration scenario, a large language model is used to understand the context dependency relationship in a business process to realize multi-turn dialog interaction across departments.

[0003] In order to improve the prediction accuracy and interaction reliability of the large language model, the large language model needs to be trained using dialog data sets in the vertical field.

[0004] In related technologies, the dependency relationship of nodes is defined by using manual annotation, and the dialog data set is constructed based on the dependency relationship between the nodes. However, for a relatively complex business scenario, the number of nodes is large, and the dependency relationship between the nodes grows exponentially. The manual annotation method is difficult to maintain, cannot cover complex dialog scenarios, and leads to logical discontinuity of the data, thereby affecting the accuracy of model training.

[0005] In addition, when constructing the dialog data set, a fixed order generation or a random combination of nodes is usually used, which cannot reflect the strength distribution of the logical relationship in the real dialog, and also affects the accuracy of model training. SUMMARY

[0006] The embodiments of the present application provide a dialog data set construction method, a multi-turn dialog model training method and device, an electronic device and a storage medium. The dialog data set is constructed based on a knowledge tree, the flow relationship of each node can be structured and maintained, logical discontinuity of the dialog data is avoided, the comprehensiveness of scene coverage is improved, the dialog data set construction complexity is reduced, and the construction efficiency is improved.

[0007] In a first aspect, an embodiment of the present application provides a method for constructing a dialogue dataset, the method comprising: identifying target text data to obtain an identification result, the identification result comprising a plurality of dialogue intents, an association relationship between the plurality of dialogue intents, and entity information corresponding to each dialogue intent; constructing a knowledge tree according to the identification result, the knowledge tree comprising a plurality of nodes, one node corresponding to one dialogue intent, and one node being associated with entity information corresponding to the dialogue intent; for each node, determining a calling probability of each next-level node of the node, the calling probability of the next-level node being related to a coverage value and a logical association of the next-level node; taking a target node as a starting node, and selecting nodes in a loop according to a target round corresponding to the target node to generate a plurality of business paths, wherein, when generating one business path, starting from the target node, selecting a next node according to the calling probability of each next-level node of the current node until the number of nodes of the business path reaches a target number of nodes; for each business path, generating a group of dialogue data according to the dialogue intent and the entity information corresponding to each node in the business path; and constructing a plurality of groups of dialogue data corresponding to the plurality of business paths into a dialogue dataset, the dialogue dataset being used to train a multi-turn dialogue model.

[0008] In a possible implementation, the determining, for each node, of the calling probability of each next-level node of the node comprises: determining the calling probability of the second node according to the historical co-occurrence frequency of the first node and the second node and attribute information of the second node, wherein the first node is any node in the knowledge tree, the second node is a next-level node of the first node, and the attribute information is used to represent the coverage value of the second node and the degree of logical association between the second node and the first node.

[0009] In a possible implementation, the determining, according to the historical co-occurrence frequency of the first node and the second node and the attribute information of the second node, of the calling probability of the second node comprises: determining an association score of the second node according to the attribute information of the second node; and determining the calling probability of the second node according to the historical co-occurrence frequency and the association score of the second node.

[0010] In a possible implementation, the attribute information of the second node comprises first attribute information and second attribute information, and the determining, according to the attribute information of the second node, of the association score of the second node comprises: determining a first score according to the first attribute information of the second node, the first attribute information comprising at least one of dependency strength information, logical coherence information, and data quality information; determining a second score according to the second attribute information of the second node, the second attribute information comprising at least one of irreplaceability information, scarcity information, and historical coverage gap information; and performing weighted average processing on the first score and the second score according to a logical weight corresponding to the first score and a coverage weight corresponding to the second score to obtain the association score of the second node.

[0011] In a possible implementation, the dialogue dataset includes a first dialogue dataset and a second dialogue dataset; when the first dialogue dataset is constructed, the coverage weight is greater than the logic weight; and when the second dialogue dataset is constructed, the logic weight is greater than the coverage weight.

[0012] In a possible implementation, the determining of the calling probability of the second node according to the historical co-occurrence frequency and the relevance score of the second node includes: determining a hierarchy weight parameter of a hierarchy where the second node is located; adjusting the relevance score of the second node according to the hierarchy weight parameter of the hierarchy where the second node is located to obtain an adjusted relevance score; and determining the calling probability of the second node according to the historical co-occurrence frequency and the adjusted relevance score.

[0013] In a possible implementation, in the plurality of service paths, the proportion of the fourth node located in the core layer is greater than the proportion of the fifth node located in the extension layer, and the proportion of the fifth node is greater than the proportion of the sixth node located in the edge layer.

[0014] In a possible implementation, after the plurality of service paths are generated, the method further includes: in a case where the proportion of the seventh node is not the fourth preset proportion, taking a ratio of the fourth preset proportion and the proportion of the seventh node as a correction parameter; the seventh node is the fourth node, the fourth preset proportion is the first preset proportion, or the seventh node is the fifth node, the fourth preset proportion is the second preset proportion, or the seventh node is the sixth node, and the fourth preset proportion is the third preset proportion; the first preset proportion is greater than the second preset proportion, and the second preset proportion is greater than the third preset proportion; for each next-level node, the calling probability of the next-level node is corrected according to the correction parameter to obtain a corrected calling probability of the next-level node; the target node is taken as a starting node, and the node selection is performed cyclically downwards according to the target round corresponding to the target node to regenerate the plurality of service paths, wherein when a service path is generated, the next node is selected from the target node according to the corrected calling probability of each next-level node of the current node, and the node number of the service path is made to reach the target node number.

[0015] In a possible implementation, the method includes: obtaining a dialogue dataset, the dialogue dataset being constructed based on the method for constructing a dialogue dataset according to the first aspect; and training an initial multi-turn dialogue model based on the dialogue dataset to obtain a multi-turn dialogue model.

[0016] In a possible implementation, the method includes: obtaining a dialogue dataset, the dialogue dataset being constructed based on the method for constructing a dialogue dataset according to the first aspect; and training an initial multi-turn dialogue model based on the dialogue dataset to obtain a multi-turn dialogue model.

[0017] The recognition module is configured to recognize the target text data to obtain a recognition result, the recognition result including a plurality of dialogue intents, an association relationship between the plurality of dialogue intents, and entity information corresponding to each dialogue intent.

[0018] The first construction module is configured to construct a knowledge tree according to the recognition result, the knowledge tree including a plurality of nodes, one node corresponding to one dialogue intent, and one node being associated with entity information corresponding to the dialogue intent.

[0019] The determination module is configured to determine, for each node, a calling probability of each next-level node of the node, the calling probability of the next-level node being related to a coverage value and a logical association of the next-level node.

[0020] The first generation module is configured to take a target node as a starting node, and perform node selection in a loop according to a target round corresponding to the target node to generate a plurality of business paths, wherein, when generating one business path, the next node is selected from the target node according to the calling probability of each next-level node of the current node, until the number of nodes of the business path reaches a target number of nodes.

[0021] The second generation module is configured to generate, for each business path, a group of dialogue data according to dialogue intents and entity information corresponding to each node in the business path.

[0022] The second construction module is configured to construct a plurality of groups of dialogue data corresponding to the plurality of business paths into a dialogue data set, the dialogue data set being used to train the multi-turn dialogue model.

[0023] In a fourth aspect, an embodiment of the present application provides a training device of a multi-turn dialogue model, including:

[0024] The acquisition module is configured to acquire the dialogue data set.

[0025] The training module is configured to train an initial multi-turn dialogue model by using the dialogue data set to obtain the multi-turn dialogue model.

[0026] In a fifth aspect, an embodiment of the present application provides an electronic device, including a memory and a processor.

[0027] The memory stores computer execution instructions.

[0028] The processor executes the computer execution instructions stored in the memory, so that the processor executes the first aspect and / or various possible implementation manners of the first aspect, or executes the second aspect and / or various possible implementation manners of the second aspect.

[0029] In a sixth aspect, an embodiment of the present application provides a computer readable storage medium, which stores computer execution instructions. When the computer execution instructions are executed by a processor, the computer execution instructions are used to implement the first aspect and / or various possible implementation manners of the first aspect, or are used to implement the second aspect and / or various possible implementation manners of the second aspect.

[0030] In a seventh aspect, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the computer program implements the first aspect and / or various possible implementation manners of the first aspect, or implements the second aspect and / or various possible implementation manners of the second aspect.

[0031] In the embodiment of the present application, the target text data is recognized, and a knowledge tree is constructed according to a recognition result. The knowledge tree includes a plurality of nodes, one node corresponds to one dialogue intent, and one node is associated with entity information of the corresponding dialogue intent. After the knowledge tree is constructed, the target node is taken as a starting node, and node selection is performed in a loop downward according to a target round corresponding to the target node, to generate a plurality of business paths. Then, for each business path, a set of dialogue data is generated, to obtain a dialogue data set. In this way, the dialogue data set is constructed based on the knowledge tree, the flow relationship of each node can be structured and maintained, logical discontinuity of dialogue data can be avoided, the comprehensiveness of scene coverage can be improved, the dialogue scenario is suitable for complex dialogue scenarios, the complexity of construction of the dialogue data set is reduced, and the construction efficiency is improved. Furthermore, when the dialogue data set is constructed, the calling probability of each next-level node is determined in combination with the coverage value and logical association of the next-level node of each node, and then the business path is generated by using the calling probability of each next-level node of the current node, so that the logical relationship between each node is guaranteed, the user's real dialogue habits can be better met, and the accuracy of model training can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0032] The accompanying drawings, which are incorporated herein and form a part of the specification, illustrate embodiments consistent with the present application and, together with the description, further serve to explain the principles of the application.

[0033] Figure 1 A schematic diagram of an application scenario of a dialogue data set construction method according to an embodiment of the present application;

[0034] Figure 2 One of the flowcharts of the dialogue data set construction method provided by an embodiment of the present application;

[0035] Figure 3 A schematic diagram of a knowledge tree provided by an embodiment of the present application;

[0036] Figure 4Another schematic diagram of a knowledge tree provided by an embodiment of the present application is shown in FIG. 6.

[0037] Figure 5 Another schematic diagram of a knowledge tree provided by an embodiment of the present application is shown in FIG. 6.

[0038] Figure 6 A flowchart of a method for constructing a dialogue dataset provided by an embodiment of the present application is shown in FIG. 7.

[0039] Figure 7 A flowchart of a method for constructing a dialogue dataset provided by an embodiment of the present application is shown in FIG. 7.

[0040] Figure 8 A flowchart of a method for constructing a dialogue dataset provided by an embodiment of the present application is shown in FIG. 7.

[0041] Figure 9 A flowchart of a method for constructing a dialogue dataset provided by an embodiment of the present application is shown in FIG. 7.

[0042] Figure 10 A flowchart of a method for constructing a dialogue dataset provided by an embodiment of the present application is shown in FIG. 7.

[0043] Figure 11 A flowchart of a method for training a multi-turn dialogue model provided by an embodiment of the present application is shown in FIG. 8.

[0044] Figure 12 A structural diagram of a device for constructing a dialogue dataset provided by an embodiment of the present application is shown in FIG. 9.

[0045] Figure 13 A structural diagram of a device for training a multi-turn dialogue model provided by an embodiment of the present application is shown in FIG. 10.

[0046] Figure 14 Another structural diagram of an electronic device provided by an embodiment of the present application is shown in FIG. 11. DETAILED DESCRIPTION

[0047] The embodiments will be described in detail with reference to the drawings, of which examples are shown. The following description is with reference to the drawings, where like numerals indicate like elements throughout the several figures. The description of embodiments in this document including exemplary methods should not be taken as limiting—tacitly or otherwise—of the examples of the systems and methods disclosed in this patent document. Many modifications and variations of the examples described and illustrated herein are possible and are contemplated and included in the scope of the described and illustrated examples. Therefore, the described and illustrated examples of methods should not be taken as limiting—tacitly or otherwise—of the scope of the exemplary systems and methods disclosed in this patent document.

[0048] It should be noted that the brief description of terms in this application is only for the convenience of understanding the following described embodiments, and is not intended to limit the embodiments of the present application. Unless otherwise specified, these terms should be understood according to their ordinary and general meanings.

[0049] The terms "first", "second", "third", and the like in the description and in the claims of the present application and above-described drawings are used for distinguishing between similar or identical objects or entities without necessarily implying a specific order or sequence, unless otherwise specified. It is to be understood that the terms so used are interchangeable under appropriate circumstances.

[0050] The terms "comprise", "comprising", "include", "including", "have", "having", "contain", "containing", "characterized by" and the like are used in the detailed description and in the claims of this disclosure to mean including, but not limited to, the stated components or elements unless otherwise indicated.

[0051] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software codes that can perform the function related to the element that it is associated with.

[0052] With the rapid development of artificial intelligence technology, large language models are increasingly applied in multi-turn dialogue scenarios in vertical fields. For example, in a business collaboration scenario, a large language model is used by an intelligent assistant within an enterprise to understand the context dependency relationship in a business process to achieve multi-turn dialogue interaction across departments. Also, in a test management scenario, a test tool implements multi-turn dialogue interaction for test management according to the dependency relationship of each node in a test process.

[0053] To improve the prediction accuracy and interaction reliability of a large language model, the large language model needs to be trained using dialogue datasets in vertical fields.

[0054] In related technologies, the dependency relationship of nodes is defined using manual annotation, and dialogue datasets are constructed based on the dependency relationship between nodes. However, for more complex business scenarios, the number of nodes is large, and the dependency relationship between nodes grows exponentially. The manual annotation method is difficult to maintain, cannot cover complex dialogue scenarios, and leads to logical incoherence of data, thereby affecting the accuracy of model training. For example, for some scenarios, the frequency of occurrence is low, and cross-department collaboration processes are involved. The manual annotation method requires special research, which is time-consuming and labor-intensive. Therefore, in actual operation, in order to save costs, data of these scenarios is discarded, resulting in incomplete coverage of scenarios.

[0055] In addition, when constructing dialogue datasets, a fixed order generation or random combination of nodes is usually used, which cannot reflect the strength distribution of logical relationships in real dialogue, and also affects the accuracy of model training.

[0056] Therefore, an embodiment of the present application provides a method for constructing a dialogue dataset. The target text data is recognized, and a knowledge tree is constructed according to the recognition result. The knowledge tree includes multiple nodes, one node corresponds to one dialogue intent, and one node is associated with entity information of the corresponding dialogue intent. After constructing the knowledge tree, the target node is taken as a starting node, and the node selection is performed in a loop downward according to the target round corresponding to the target node, to generate multiple business paths. Then, for each business path, a set of dialogue data is generated, to obtain the dialogue dataset. In this way, the dialogue dataset is constructed based on the knowledge tree, the flow relationship of each node can be structured and maintained, the logical discontinuity of the dialogue data can be avoided, the comprehensiveness of the scene coverage is improved, it is suitable for complex dialogue scenarios, and the complexity of constructing the dialogue dataset is reduced and the construction efficiency is improved. Furthermore, when constructing the dialogue dataset, the coverage value and the logical association of the next level node of each node are combined to determine the calling probability of each next level node, and then the calling probability of each next level node of the current node is used to generate a business path, which can be more consistent with the real dialogue habits of users while ensuring the logical relationship between each node, and thus the accuracy of model training can be improved.

[0057] Before introducing the method for constructing the dialogue dataset provided by an embodiment of the present application, the application scenario of the method for constructing the dialogue dataset is described.

[0058] Figure 1 An application scenario of the method for constructing the dialogue dataset provided by an embodiment of the present application is shown in the following figure.

[0059] An embodiment of the present application is applied to a multi-round dialogue platform. As shown in the figure, Figure 1 In actual application, first, the multi-round dialogue dataset is obtained; then, the multi-round dialogue model is trained by using the multi-round dialogue dataset; finally, the multi-round dialogue platform 10 receives the question of the user, and outputs the corresponding answer by using the trained multi-round dialogue model, to realize the multi-round dialogue interaction with the user.

[0060] In an embodiment of the present application, the multi-round dialogue platform 10 can be used for multi-round dialogue in a certain vertical field. For example, the multi-round dialogue platform 10 can be an intelligent assistant in an enterprise, a test management platform, etc.

[0061] Of course, in some examples, the pre-trained multi-round dialogue model can also be fine-tuned by using the multi-round dialogue dataset.

[0062] The method for constructing the dialogue dataset provided by an embodiment of the present application is described in detail in combination with the figures and the application scenario.

[0063] Figure 2A flowchart of a method for constructing a dialogue dataset is provided for an embodiment of the present application. As shown in Figure 2 the method for constructing the dialogue dataset can include the following steps:

[0064] S201, identifying target text data to obtain an identification result, the identification result including multiple dialogue intents, association relationships between the multiple dialogue intents, and entity information corresponding to each dialogue intent.

[0065] The target text data refers to technical documents in a certain vertical field, which is the data source of the dialogue dataset. For example, in a business collaboration scenario, the target text data can be a business operation document. For another example, in a test management scenario, the target text data can be a test management document.

[0066] From the target text data, multiple dialogue intents, association relationships between the multiple dialogue intents, and entity information corresponding to each dialogue intent can be extracted. The entity information refers to specific objects involved in executing the dialogue intent. It can be understood that one dialogue intent corresponds to one or more entity information.

[0067] For example, continuing with the test management scenario, the extracted dialogue intents include "submit defect", "supplement screenshot", and "assign handler". The entity information corresponding to "submit defect" can include defect identification (i.e., defect ID), defect description information, defect detail information, defect level, etc. The entity information corresponding to "supplement screenshot" can include screenshot information. The entity information corresponding to "assign handler" can include personnel information.

[0068] S202, constructing a knowledge tree according to the identification result, the knowledge tree including multiple nodes, one node corresponding to one dialogue intent, and one node associating entity information corresponding to the dialogue intent.

[0069] Specifically, for each dialogue intent, a node is created, the association relationships between the nodes are constructed according to the association relationships between the dialogue intents, and the entity information corresponding to each dialogue intent is associated as an attribute to the corresponding node, to obtain a knowledge tree corresponding to the target text data.

[0070] In a specific implementation, a pre-trained knowledge tree construction model can be used. The target text data is input into the knowledge tree construction model. The knowledge tree construction model is used to perform semantic analysis on the target text data, identify multiple dialogue intents, entity information corresponding to each dialogue intent, and association relationships between different dialogue intents. Then, for each dialogue intent, a node is created. According to the association relationships between different dialogue intents, association relationships between the nodes are constructed. The entity information corresponding to each dialogue intent is associated as an attribute to the corresponding node. A knowledge tree corresponding to the target text data is obtained and output. In this way, the pre-trained model can automatically construct a vertical knowledge tree without the need for a large amount of manual intervention, and the processing efficiency is relatively high.

[0071] For example, continuing with the test management scenario, as shown in FIG. 6, for the dialogue intent of “submitting a defect”, a node 1 is created. For the dialogue intent of “supplementing a screenshot”, a node 2 is created. For the dialogue intent of “confirming a repair”, a node 3 is created. For the dialogue intent of “allocating a handler”, a node 4 is created. According to the association relationships between the four dialogue intents, association relationships between the node 1, the node 2, the node 3, and the node 4 are constructed. The entity information corresponding to each dialogue intent is associated as an attribute to the corresponding node. A knowledge tree is obtained. Figure 3 Also, continuing with the test management scenario, as shown in FIG. 7, the node 1 is “submitting a defect”, the node 2 is “supplementing a screenshot”, the node 3 is “confirming a repair”, the node 4 is “allocating a handler”, the node 5 is “test environment description”, and the node 6 is “emergency test suspension”. The association relationships between the nodes are as shown in FIG. 7.

[0072] Figure 4 Figure 4

[0073] In some examples, after the knowledge tree is constructed, for each node in the knowledge tree, a node level corresponding to the node is determined. Exemplarily, the node level can include a core layer, an extension layer, and an edge layer. The node level corresponding to the node can be defined according to the scene covered by the node and / or the importance of the node.

[0074] Exemplarily, a node covering more scenes and being more important is taken as a node in the core layer. For example, “submitting a defect” is a node in the core layer (which can be referred to as a core node). A node covering fewer scenes is taken as a node in the extension layer. For example, “test environment description” is a node in the extension layer (which can be referred to as an extension node). A node covering an edge scene (a few special scenes) is taken as a node in the edge layer. For example, “test interruption handling” is a node in the edge layer (which can be referred to as an edge node). For ease of description, a node in the core layer in the knowledge tree is referred to as a preset core node. A node in the extension layer in the knowledge tree is referred to as a preset extension node. A node in the edge layer in the knowledge tree is referred to as a preset edge node.​​​

[0075] Specifically, the node level of each node in the knowledge tree can be determined according to the target text data (business document), actual operation data and / or process specification document (such as test specification document, quality specification document).

[0076] For example, after the knowledge tree is constructed, for each node in the knowledge tree, the level corresponding to each node is determined according to the frequency of occurrence of each node in the target text data (business document). Specifically, if the frequency of occurrence of a node is greater than or equal to a first frequency threshold, the level corresponding to the node is determined to be the core layer, i.e., the node is the first node located in the core layer. If the frequency of occurrence of a node is less than the first frequency threshold and greater than a second frequency threshold, the level corresponding to the node is determined to be the extension layer, i.e., the node is the second node located in the extension layer. If the frequency of occurrence of a node is less than or equal to the second frequency threshold, the level corresponding to the node is determined to be the edge layer, i.e., the node is the third node located in the edge layer. It can be understood that the specific values of the first frequency threshold and the second frequency threshold can be set according to actual needs, and the embodiments of the present application do not limit this.

[0077] For example, in the test management scenario, the nodes of "submit defect", "confirm repair" and "assign handler" have a high frequency of occurrence and are located in the core layer. The nodes of "supplement screenshot", "test environment description" and "defect priority annotation" have a low frequency of occurrence and are located in the extension layer. The nodes of "emergency pause parameter", "test interruption processing" and "cross-version defect traceability" have the lowest frequency of occurrence and are located in the edge layer. It can be understood that in actual application, the levels corresponding to various nodes can be set for different scenarios.

[0078] After determining the node level corresponding to each node in the knowledge tree, the level weight parameter of each node can be determined based on the node level corresponding to each node, the relevance score of the node is adjusted according to the level weight parameter of each node, and then the calling probability of the node is determined based on the adjusted relevance score. For specific implementation modes of S501 to S503, please refer to the following.

[0079] S203, for each node, determine the calling probability of each next level node of the node, and the calling probability of the next level node is related to the coverage value and logical relevance of the next level node.

[0080] The calling probability of the next level node refers to the conditional probability of selecting one of the next level nodes as the next node starting from the current node (such as the target node).

[0081] In the embodiments of the present application, the calling probability of the next level node is determined based on the coverage value of the next level node, the historical co-occurrence frequency between the next level node and the current node, and the logical association degree (i.e., logical association) between the next level node and the current node. The determination process of the calling probability of the next level node can be referred to below, which will not be described here.

[0082] In S204, the target node is taken as a starting node, and the node selection is performed cyclically downward according to the target round corresponding to the target node, to generate a plurality of service paths. When generating a service path, the next node is selected from the target node according to the calling probability of each next level node of the current node, until the number of nodes of the service path reaches the target number of nodes.

[0083] The target node is the starting point of the service path. The target node can be pre-set according to actual needs. When constructing the dialogue dataset, one target node can be pre-set, or a plurality of target nodes can be pre-set.

[0084] For example, the target node can be the starting node (i.e., parent node) of the knowledge tree. Of course, the target node can also be any node in the knowledge tree. For example, continuing with the test management scenario, the target node can be "submit defect".

[0085] After obtaining the target node, the path generation parameters corresponding to each target node can be obtained, wherein the path generation parameters can include the target round and the target number of nodes described above. The target round refers to the number of times of generating a service path starting from the target node, i.e., the total number of service paths with the target node as the starting node. The target number of nodes refers to the number of nodes selected when generating a service path starting from the target node, i.e., the total number of nodes included in a service path.

[0086] Specifically, starting from the target node, the next node is selected according to the calling probability of each next level node of the current node, until the number of nodes of the service path reaches the target number of nodes, to obtain a service path. Next, starting from the target node again, the next node is selected according to the calling probability of each next level node of the current node, until the number of nodes of the service path reaches the target number of nodes, to obtain the next service path, and the number of cycles reaches the target round, i.e., a plurality of service paths are obtained.

[0087] The following will be described in combination with Figure 5 The generation process of the service path will be described schematically. As shown in FIG. 2, the target node is taken as the starting node, and the node selection is performed cyclically downward according to the target round corresponding to the target node, to generate a plurality of service paths. Figure 5As shown, taking node A as the target node, node A's next-level nodes include nodes B and C, and node B's next-level nodes include nodes D and E. Therefore, starting from node A, based on the calling probabilities of nodes B and C, either node B or node C is selected as the next node. Then, starting from the next node, the next node is selected based on the calling probabilities of its next-level nodes. For example, taking node B as the next node, starting from node B, based on the calling probabilities of nodes D and E, either node D or node E is selected as the next node.

[0088] It is understandable that the proportion of different business paths in multiple business paths is related to the call probability of each level of node.

[0089] For example, taking node A as the target node, the target round as 10, the target number of nodes as 3, and the probability of calling node B (i.e., calling node B from node A) as 80%, the probability of calling node C (i.e., calling node C from node A) as 20%, the probability of calling node D (i.e., calling node D from node B) as 60%, and the probability of calling node E (i.e., calling node E from node B) as 40%, there are three possible business paths starting from the target node: business path a: node A → node B → node D, business path b: node A → node B → node E, and business path c: node A → node C → node F. Among the selected business paths, business path a accounts for 48%, business path b accounts for 32%, and business path c accounts for 20%.

[0090] S205 generates a set of dialogue data for each business path based on the dialogue intent and entity information corresponding to each node in the business path.

[0091] For example, with Figure 4 Taking the knowledge tree shown as an example, the business path can include business path 1, business path 2, and business path 3. Business path 1 is "Submit Defect" → "Supplement Screenshot" → "Confirm Repair" → "Assign Processor"; business path 2 is "Submit Defect" → "Supplement Screenshot" → "Test Environment Description" → "Assign Processor"; and business path 3 is "Submit Defect" → "Supplement Screenshot" → "Test Environment Description" → "Emergency Pause Testing". For each business path, based on the dialogue intent of each node and the corresponding entity information, a set of dialogue data is generated.

[0092] For example, dialogue data 1: "Submit defect B001 (login button click unresponsive)" → "Provide a screenshot of the button click without a pop-up window"; dialogue data 2: "Submit defect B001" → "Explain that the test environment is V2.1"; dialogue data 3: "Emergency test pause (system data anomaly)" → "Please provide the start time of the downtime and the exception log".

[0093] It should be noted that when generating corresponding dialogue data along the business path, slot information (i.e., attribute information) is filled for each node. For example, taking the business path 1 as "Submit defect" → "Supplement screenshot" → "Confirm repair" → "Assign handler" as an example, when the process enters the "Supplement screenshot" node, the system checks whether the slot of "defect picture" is filled (i.e., whether the user uploads the defect picture). If the slot of "defect picture" is filled, the "Supplement screenshot" node is completed, and the process enters the next "Confirm repair" node. If the slot of "defect picture" is not filled, the system will stay in the "Supplement screenshot" node and prompt the user to upload the defect picture. During this period, the user's dialogue intention is always "Supplement screenshot" through context judgment, and the system will not create a new sub-node (such as information missing) or jump to the next node. That is, through the above slot filling mechanism, based on a single business path, dialogue data of multiple interaction scenarios can be automatically covered, for example, interaction scenarios that the user completes the operation as expected (such as directly uploading the defect picture), and interaction scenarios that the system needs to prompt and complete (such as the user not uploading the defect picture). In this way, while ensuring that the process does not jump abnormally, the diversity and authenticity of the dialogue data are greatly enriched.

[0094] S206, a plurality of groups of dialogue data corresponding to a plurality of business paths are constructed into a dialogue data set, and the dialogue data set is used to train a multi-turn dialogue model.

[0095] Specifically, the initial multi-turn dialogue model is trained using the multi-turn dialogue data set to obtain a multi-turn dialogue model. After receiving the user's question, the trained multi-turn dialogue model is used to output the corresponding answer to realize multi-turn dialogue interaction with the user.

[0096] In the embodiments of the present application, the target text data is recognized, and a knowledge tree is constructed according to the recognition result. The knowledge tree includes a plurality of nodes, one node corresponds to one dialogue intent, and one node is associated with entity information of the corresponding dialogue intent. After constructing the knowledge tree, the target node is taken as a starting node, and node selection is performed in a loop downward according to the target round corresponding to the target node, to generate a plurality of business paths. Then, for each business path, a set of dialogue data is generated to obtain a dialogue data set. In this way, the dialogue data set is constructed based on the knowledge tree, which can structurally maintain the flow relationship of each node, can avoid logical discontinuity of dialogue data, can improve the comprehensiveness of scene coverage, is suitable for complex dialogue scenarios, and can reduce the complexity of constructing the dialogue data set and improve the construction efficiency. Furthermore, when constructing the dialogue data set, the coverage value and logical association of the next level node of each node are combined to determine the calling probability of each next level node, and then the calling probability of each next level node of the current node is used to generate a business path, which can be more in line with the real dialogue habits of users while ensuring the logical relationship between each node, and can improve the accuracy of model training.

[0097] The determination process of the calling probability of the next level node will be described below taking any node (i.e., a first node) in the knowledge tree and any next level node (a second node) of the first node as an example.

[0098] In some embodiments, the determination process of the calling probability of the next level node can include determining the calling probability of the second node according to the historical co-occurrence frequency of the first node and the second node and attribute information of the second node. The first node is any node in the knowledge tree, the second node is a next level node of the first node, and the attribute information is used to represent the coverage value of the second node and the logical association between the second node and the first node.

[0099] Optionally, the attribute information of the node includes first attribute information and second attribute information. The first attribute information is used to represent the logical association of the node. The second attribute information is used to represent the coverage value of the node.

[0100] When constructing the dialogue data set, the coverage value and logical association of the next level node of each node are combined to determine the calling probability of each next level node, and then the calling probability of each next level node of the current node is used to generate a business path, which can be more in line with the real dialogue habits of users while ensuring the logical relationship between each node.

[0101] In some examples, as shown in Figure 6 The determination of the calling probability of the second node according to the historical co-occurrence frequency of the first node and the second node and the attribute information of the second node can include the following steps:

[0102] S301, determining the relevance score of the second node according to the attribute information of the second node.

[0103] As shown in the example, Figure 7 S301 can include the following steps:

[0104] S401, determining a first score (also referred to as a basic score) according to the first attribute information of the second node.

[0105] The first attribute information is used to represent the logical relevance of the node. As an example, the first attribute information can include at least one of dependency strength information, logical coherence information, and data quality information.

[0106] The dependency strength information can represent the dependency strength of the node (i.e., the second node) on the parent node (the first node) of the node. For example, as shown in the example, Figure 4 Taking the second node as "supplementary screenshot" and the first node as "submit defect" as an example, the dependency strength of the second node on the first node is strong. Taking the second node as "test environment description" and the first node as "submit defect" as an example, the dependency strength of the second node on the first node is weak.

[0107] The logical coherence information can represent the logical coherence from the parent node (the first node) to the child node (i.e., the second node). For example, as shown in the example, Figure 4 Taking the second node as "supplementary screenshot" and the first node as "submit defect" as an example, the logical coherence from "submit defect" to "supplementary screenshot" is smooth. Taking the second node as "test environment description" and the first node as "submit defect" as an example, the logical coherence from "test environment description" to "supplementary screenshot" is poor.

[0108] The data quality information can represent the data quality of the entity information associated with the node. As an example, the data quality information can be the signal-to-noise ratio of the entity information associated with the node. For example, taking the "defect screenshot" node as an example, 90% of the defect screenshot is valid defect evidence, and the signal-to-noise ratio is 0.9.

[0109] As an example, the dependency strength information can be represented by a dependency strength score. The higher the dependency strength of the node (i.e., the second node) on the parent node (the first node) of the node, the higher the dependency strength score. The logical coherence information can also be represented by a logical coherence score. The better the logical coherence from the parent node (the first node) to the child node (i.e., the second node), the higher the logical coherence score. The data quality information can also be represented by a data quality score.

[0110] As an example, in the first score of the second node, the average of the dependency strength score, the logical coherence score, and the data quality score of the second node is taken as the first score.

[0111] For example, taking the second node as "supplemental screenshot" and the first node as "submit defect" as an example, the dependency strength score of "supplemental screenshot" is 0.9, the logical coherence score of "supplemental screenshot" is 0.85, and the data quality score of "supplemental screenshot" is 0.9. The first score of "supplemental screenshot" is (0.9+0.85+0.9) / 3=0.88.

[0112] S402, determining a second score (which can also be referred to as a compensation score) according to the second attribute information of the second node.

[0113] The second attribute information is used to represent the coverage value of the node. For example, the second attribute information can include at least one of irreplaceability information, scarcity information, and historical coverage gap information.

[0114] The irreplaceability information can represent the irreplaceability of the node. For example, taking the "test interruption processing" node as an example, there is no other replaceable node, so the irreplaceability is high, and the irreplaceability score is high. The scarcity information can represent the scarcity degree of the node. For example, for a node with less data in the historical data set, the scarcity degree is high. For the historical coverage gap information, the historical data set does not cover the scene, and the historical coverage gap of the node involved in the scene is large.

[0115] For example, the irreplaceability information can be represented by an irreplaceability score. The higher the irreplaceability of the node (i.e., the second node), the higher the irreplaceability score. The scarcity information can also be represented by a scarcity score. The higher the scarcity degree of the node (i.e., the second node), the higher the scarcity score. The historical coverage gap information can also be represented by a historical coverage gap score. For a low-frequency scene, the historical coverage gap score of the node (i.e., the second node) included in the scene is higher.

[0116] For example, in the second score of the second node, the average value of the irreplaceability score, the scarcity score, and the historical coverage gap score of the second node is taken as the second score.

[0117] For example, taking the second node as "supplemental screenshot" and the first node as "submit defect" as an example, the irreplaceability score of "supplemental screenshot" is 0.3, the scarcity score of "supplemental screenshot" is 0.2, and the historical coverage gap score of "supplemental screenshot" is 0.1. The second score of "supplemental screenshot" is (0.3+0.2+0.1) / 3=0.2.

[0118] S403, performing weighted average processing on the first score and the second score according to the logical weight corresponding to the first score and the coverage weight corresponding to the second score, to obtain the relevance score of the second node.

[0119] Exemplarily, taking the second node as an example, the relevance score of the second node can be determined according to formula (1) as follows.

[0120] (1)

[0121] For example, taking the second node as “supplemental screenshot” and the first node as “submit defect” as an example, the logical weight and the coverage weight of the “supplemental screenshot” are both 0.5, and then the relevance score of the “supplemental screenshot” is 0.88*0.5+0.2*0.5=0.54. The relevance score is medium to high, reflecting the basic role of the node in the core process.

[0122] In the embodiment of the application, the first score representing the logical relevance of the node is calculated by using the first attribute of the node, and the second score representing the coverage value of the node is calculated by using the second attribute of the node, and then the relevance score of the node is obtained by combining the first score and the second score, so that the quantitative evaluation of the logical relevance and the coverage value of the node can be realized, and then the calling probability of each node can be more accurately determined by using the relevance score of the node.

[0123] In some examples, the dialogue data set includes a first dialogue data set and a second dialogue data set, the coverage weight is greater than the logical weight when the first dialogue data set is constructed, and the logical weight is greater than the coverage weight when the second dialogue data set is constructed.

[0124] That is, at the initial stage of construction of the dialogue data set, the coverage weight is greater than the logical weight. Exemplarily, at the initial stage of construction of the dialogue data set, the coverage weight is valued at 0.6, and the logical weight is valued at 0.4. The logical weight at the optimization stage of the dialogue data set is higher than the logical weight at the initial stage. Exemplarily, at the optimization stage of the dialogue data set, the logical weight is greater than the coverage weight. For example, at the optimization stage of the dialogue data set, the logical weight is valued at 0.6, and the coverage weight is valued at 0.4.

[0125] In the embodiment of the application, by means of quantitative scoring and dynamic weight adjustment, at the initial stage of construction of the dialogue data set, a larger coverage weight is adopted to ensure the full coverage of the node and avoid missing key scenarios. And at the optimization stage of the dialogue data set, the logical weight is improved to improve the logical coherence of the subsequent generated business path. In this way, in the test management scenario, process jamming can be reduced and test efficiency can be improved.

[0126] After S30, S302 is performed to determine the calling probability of the second node according to the historical co-occurrence frequency and the relevance score of the second node.

[0127] The historical co-occurrence probability refers to the proportion of the number of times that the parent node (the first node) and the child node (i.e. the second node) appear continuously in the historical data set.

[0128] As shown in the example of FIG. 3, S302 can include the following steps: Figure 8

[0129] S501, determining a hierarchical weight parameter of a level in which the second node is located.

[0130] In the embodiment of the present application, after the knowledge tree is constructed, for each node in the knowledge tree, the node level corresponding to the node is determined, that is, the node level corresponding to the second node can be determined. After the node level corresponding to the second node is determined, the hierarchical weight parameter of the level in which the second node is located can be determined.

[0131] The more important the level in which the node is located, the greater the value of the hierarchical weight parameter of the level in which the node is located. As an example, the level in which the node (such as the second node) is located is the core layer, and the corresponding hierarchical weight parameter value is 1; the level in which the node (such as the second node) is located is the expansion layer, and the corresponding hierarchical weight parameter value is 0.7; the level in which the node (such as the second node) is located is the edge layer, and the corresponding hierarchical weight parameter value is 0.5. In this way, the core weight is higher than the expansion weight, which reflects the priority of the core node.

[0132] S502, adjusting the relevance score of the second node according to the hierarchical weight parameter of the level in which the second node is located, to obtain an adjusted relevance score.

[0133] S503, determining the calling probability of the second node according to the historical co-occurrence frequency and the adjusted relevance score.

[0134] As an example, the calling probability of the second node can be determined according to the following formula (2).

[0135] (2)

[0136] As an example, the decay factor is 0.3-0.5. For example, the decay factor takes a value of 0.4, which is used to balance the influence of the co-occurrence frequency, avoid over-dominance of high-frequency co-occurrence nodes, and reduce the influence of accidental co-occurrence.

[0137] For example, taking the second node as "supplementary screenshot" and the first node as "submit defect" as an example, the relevance score of "supplementary screenshot" is 0.54, "supplementary screenshot" is in the expansion layer, and the hierarchical weight parameter thereof is 0.7. The historical co-occurrence frequency of "supplementary screenshot" and "submit defect" is 0.7, and the decay factor is 0.4. Therefore, the score of "supplementary screenshot" is 0.82.

[0138] ​Taking the second node as "test environment description" and the first node as "submit defect" as an example, the relevance score of "test environment description" is 0.4, "test environment description" is in the extension layer, the hierarchical weight parameter of "test environment description" is 0.7, the historical co-occurrence frequency of "test environment description" and "submit defect" is 0.3, and the decay factor is 0.4. Therefore, the score of "test environment description" is 0.4.

[0139] The normalization coefficient is 0.82+0.4=1.22. Based on this, the calling probability of "supplementary screenshot" is 0.82 / 1.22≈67%, and the calling probability of "test environment description" is 0.4 / 1.22≈33%. In this way, the calling probability of the "supplementary screenshot" node matches the high-frequency attribute of the core node, and the calling probability of the "test environment description" matches the auxiliary position of the extension layer.

[0140] In the embodiments of the present application, by dividing the nodes into different levels such as core layer, extension layer and edge layer, the business priority of the corresponding scene (such as high-frequency scene or edge scene) can be effectively quantified. When calculating the calling probability, the hierarchical weight and the decay factor are combined, so that the calling probability (67%) of the core node (such as "supplementary screenshot") can be significantly higher than that of the extension layer node (such as "environment description" of 33%). This mechanism ensures that the dialogue flow conforms to the strong and weak relationship of the business logic, thereby significantly improving the naturalness and logical coherence of the dialogue.

[0141] In some examples, after obtaining the plurality of business paths, according to the node level of each node in the business path, a proportion of the fourth node located in the core layer in the nodes included in the plurality of business paths (i.e., the proportion of the fourth node located in the core layer), a proportion of the fifth node located in the extension layer in the nodes included in the plurality of business paths (i.e., the proportion of the fifth node located in the extension layer), and a proportion of the sixth node located in the edge layer in the nodes included in the plurality of business paths (i.e., the proportion of the sixth node located in the edge layer) are determined.

[0142] Optionally, in the plurality of business paths, the proportion of the fourth node located in the core layer is greater than the proportion of the fifth node located in the extension layer, and the proportion of the fifth node is greater than the proportion of the sixth node located in the edge layer.

[0143] For example, in the plurality of business paths, the proportion of the fourth node is a first preset proportion, the proportion of the fifth node is a second preset proportion, and the proportion of the sixth node is a third preset proportion. The first preset proportion is greater than the second preset proportion, and the second preset proportion is greater than the third preset proportion. Of course, the first preset proportion can be a proportion range. The second preset proportion can be a proportion range. The third preset proportion can be a proportion range.

[0144] For example, the first preset proportion is 50%-60%, the second preset proportion is 30%-40%, and the third preset proportion is 10%-20%.

[0145] For example, the first preset proportion is 50%-60%, the second preset proportion is 30%-40%, and the third preset proportion is 10%-20%.

[0146] Of course, the first preset proportion, the second preset proportion and the third preset proportion can be set according to actual conditions, and the embodiments of the application are not limited thereto.

[0147] In the embodiments of the application, after the knowledge tree is constructed, the nodes are divided into the core layer, the expansion layer and the edge layer, and the quantification of the importance of the scene is realized. This division directly guides the data allocation in model training: on the one hand, sufficient core scene data is ensured to consolidate the model's grasp of the core business; on the other hand, certain expansion data is covered to improve the completeness of the data. In addition, necessary edge scene data is reserved to prevent it from being excessively diluted and avoid the absence of extreme scenes, thereby ensuring the generalization ability of the system to cope with special situations.

[0148] In some examples, after the plurality of business paths are generated, if the plurality of business paths do not satisfy the first preset condition, for each node, the calling probability of each next level node of the node is re-determined to obtain a corrected calling probability. The target node is taken as a starting node, and the node selection is performed cyclically downward according to the target round corresponding to the target node, and the plurality of business paths are re-generated, wherein when a business path is generated, the next node is selected from the target node according to the corrected calling probability of each next level node of the current node, and the node number of the business path reaches the target node number.

[0149] In some examples, after the plurality of business paths are generated, if the plurality of business paths do not satisfy the first preset condition, for each node, the calling probability of each next level node of the node is re-determined to obtain a corrected calling probability. The target node is taken as a starting node, and the node selection is performed cyclically downward according to the target round corresponding to the target node, and the plurality of business paths are re-generated, wherein when a business path is generated, the next node is selected from the target node according to the corrected calling probability of each next level node of the current node, and the node number of the business path reaches the target node number.

[0150] For example, when the calling probability is corrected, a correction coefficient is determined according to a fourth preset proportion (i.e. a target proportion) and a proportion of the seventh node (i.e. a current actual proportion). The seventh node is the fourth node, and the fourth preset proportion is the first preset proportion; or the seventh node is the fifth node, and the fourth preset proportion is the second preset proportion; or the seventh node is the sixth node, and the fourth preset proportion is the third preset proportion; the first preset proportion is greater than the second preset proportion, and the second preset proportion is greater than the third preset proportion.

[0151] It can be understood that when the proportion of the fourth node located in the core layer is not the first preset proportion, the seventh node is the fourth node, and the fourth preset proportion is the first preset proportion; when the proportion of the fifth node located in the extension layer is not the second preset proportion, the seventh node is the fifth node, and the fourth preset proportion is the second preset proportion; when the proportion of the sixth node located in the edge layer is not the third preset proportion, the seventh node is the sixth node, and the fourth preset proportion is the third preset proportion.

[0152] After the correction parameter is determined, for each next level node (i.e., the second node), the calling probability of the next level node is corrected according to the correction parameter, to obtain the corrected calling probability of the next level node.

[0153] For example, the corrected calling probability of the next level node is calculated according to the following formula (3).

[0154] (3)

[0155] In the embodiments of the present application, when the proportions of the nodes of each level in the plurality of service paths do not satisfy the first preset condition, the calling probabilities of the nodes can be corrected by the current actual proportion and the target proportion, so as to regenerate the service paths by using the corrected calling probabilities, and the coverage of the generated dialogue data set can be ensured.

[0156] In some embodiments, after the dialogue data set is constructed, it is judged whether the dialogue data set satisfies a second preset condition. If the dialogue data set does not satisfy the second preset condition, the calling probability is determined again. The second preset condition includes at least one of the following: the proportion of the preset core node is greater than or equal to a fifth preset proportion, or the proportion of the preset extension node is greater than or equal to a sixth preset proportion, or the occurrence number of the preset edge node is greater than or equal to a preset number.

[0157] As described above with respect to determining the node level corresponding to each node in the knowledge tree, it can be known that the node located in the core layer in the knowledge tree is referred to as a preset core node, the node located in the extension layer in the knowledge tree is referred to as a preset extension node, and the node located in the edge layer in the knowledge tree is referred to as a preset edge node. That is, the preset core node is a node involved in a pre-set core scenario. For example, "submit defect". The preset extension node is a node involved in a pre-set extension scenario. For example, "test environment description". The preset edge node is a node involved in a pre-set edge scenario. For example, "test interruption processing".

[0158] The preset proportion of the core node can be understood as the proportion of the preset core node in the fourth node of the core layer in the plurality of service paths. The preset proportion of the extension node can be understood as the proportion of the preset extension node in the fifth node of the extension layer in the plurality of service paths. The occurrence number of the preset edge node can be understood as the occurrence number of the preset edge node in the plurality of service paths.

[0159] For example, the fifth preset proportion is 100%, the sixth preset proportion is 90%, and the preset number is 5.

[0160] That is, the proportion of the preset core node is 100%, the proportion of the preset extension node is greater than or equal to 90%, and the occurrence number of the preset edge node (i.e., the preset number) is greater than or equal to 5.

[0161] When the calling probability is re-determined, the calling probability can be adjusted by adjusting the logical weight and the coverage weight to adjust the relevance score of the node. The calling probability can also be adjusted by the hierarchical weight and the decay factor. For example, the decay factor is increased.

[0162] The construction method of the dialogue data set provided by the embodiments of the present application will be described in detail below with a specific example.

[0163] Figure 9 A flowchart of the construction method of the dialogue data set provided by the embodiments of the present application is shown in FIG. 6. As shown in FIG. 6, the construction method of the dialogue data set includes the following steps: Figure 9

[0164] S601, identifying the target text data to obtain an identification result, the identification result including a plurality of dialogue intents, an association relationship between the plurality of dialogue intents, and entity information corresponding to each dialogue intent.

[0165] S602, constructing a knowledge tree according to the identification result, the knowledge tree including a plurality of nodes, one node corresponding to one dialogue intent, and one node being associated with entity information corresponding to the dialogue intent.

[0166] S603, determining, for each node, a relevance score of each next-level node of the node.

[0167] When determining the relevance score of each next-level node, the coverage weight is greater than the logical weight.

[0168] The specific implementation of S603 can refer to the specific implementation of S401 to S403 described above, which will not be described here.

[0169] S604, determining a calling probability of each next-level node according to the relevance score of each next-level node.

[0170] ​The specific implementation of S604 can refer to the specific implementation of S501 to S503, which will not be described here.

[0171] S605, taking the target node as a starting node, and according to the target round corresponding to the target node, the node selection is performed in a loop to generate a plurality of service paths.

[0172] In the generation of a service path, the next node is selected from the target node according to the call probability of each next level node of the current node, and the number of nodes of the service path reaches the target node number.

[0173] S606, for each service path, a set of dialogue data is generated according to the dialogue intent and entity information corresponding to each node in the service path, and a plurality of sets of dialogue data are obtained.

[0174] S607, judging whether the dialogue data set meets the verification condition, if yes, executing S608, otherwise, executing S603.

[0175] That is, after generating the dialogue data set, it is judged whether the dialogue data set meets the verification condition, if yes, the dialogue data set is output, otherwise, returning to S603, that is, the relevance score and the call probability of each next level node are recalculated, and then the service path is regenerated based on the recalculated call probability.

[0176] The verification process of S604 is described below S701 to S705, which will not be described here.

[0177] S608, outputting the dialogue data set.

[0178] It should be noted that the dialogue data set can include a first dialogue data set and a second dialogue data set, and the coverage weight is greater than the logic weight when constructing the first dialogue data set. After constructing the first dialogue data set, the second dialogue data set is constructed according to the processes of S601 to S608, and the logic weight is greater than the coverage weight when constructing the second dialogue data set.

[0179] The verification condition includes a coverage verification condition, a proportion verification condition and a parameter stability condition. The verification process of the dialogue data set will be described below. Figure 10

[0180] As shown in Figure 10 The verification process of the dialogue data set can include the following steps:

[0181] S701, judging whether the dialogue data set meets the coverage verification condition (i.e. the second preset condition described above), if yes, executing S703, otherwise, executing S702.

[0182] ​Specifically, after the dialogue dataset is constructed, the nodes at each level in the dialogue dataset are determined, the proportion of the preset core nodes in the nodes at the core level is determined, it is judged whether the proportion of the preset core nodes in the nodes at the core level is greater than or equal to a fifth preset proportion, the proportion of the preset extension nodes in the nodes at the extension level is determined, it is judged whether the proportion of the preset extension nodes in the nodes at the extension level is greater than or equal to a sixth preset proportion, and it is determined whether the occurrence number of the preset edge nodes is greater than or equal to a preset number, that is, it is judged whether the dialogue dataset satisfies the coverage verification condition (i.e., the second preset condition described above).

[0183] For example, the fifth preset proportion is 100%, the sixth preset proportion is 90%, and the preset number is 5 times.

[0184] For example, it is determined that the proportion of the preset core nodes in the dialogue dataset is 100%, the proportion of the preset extension nodes is 92%, and the occurrence number of the preset edge nodes is 5 times, which satisfies the coverage verification condition (i.e., the second preset condition described above).

[0185] S702, increase the node dataset.

[0186] Specifically, if the proportion of the preset core nodes is less than the fifth preset proportion (such as 100%), the missing preset core nodes are determined, and the business path related to the missing preset core nodes is added. If the proportion of the preset extension nodes is less than the sixth preset proportion (such as 90%), the missing preset core nodes are determined, and the business path related to the missing preset extension nodes is added. If the occurrence number of the preset edge nodes is less than the preset number (such as 5 times), the occurrence number of the preset edge nodes in the business path is increased.

[0187] It can be understood that the correlation score of the node can be adjusted by adjusting the logic weight and the coverage weight, and then the calling probability is adjusted. The calling probability can also be adjusted by the level weight and the attenuation factor. Based on the adjusted calling probability, the business path can be regenerated, and then the node dataset is increased.

[0188] After S702 is performed, S701 is performed, that is, the verification is performed again.

[0189] S703, judge whether the dialogue dataset satisfies the proportion verification condition (i.e., the first preset condition described above), if yes, perform S705, otherwise, perform S704.

[0190] Specifically, a proportion of the nodes in the core layer (i.e., the fourth node above) in the dialogue data set, a proportion of the nodes in the extension layer (i.e., the fifth node above), and a proportion of the nodes in the edge layer (i.e., the sixth node above) are determined. It is determined whether the proportion of the nodes in the core layer is a first preset proportion, whether the proportion of the nodes in the extension layer is a second preset proportion, and whether the proportion of the nodes in the edge layer is a third preset proportion. If yes, it is determined that the dialogue data set satisfies the proportion verification condition (i.e., the first preset condition above).

[0191] For example, the first preset proportion is 50% to 60%, the second preset proportion is 30% to 40%, and the third preset proportion is 10% to 20%.

[0192] For example, the proportion of the nodes in the core layer is 55%, the proportion of the nodes in the extension layer is 36%, and the proportion of the nodes in the edge layer is 9%. The error with respect to the preset range (i.e., the first preset proportion, the second preset proportion, and the third preset proportion) is less than a preset error threshold (e.g., 1%). Therefore, it is determined that the dialogue data set satisfies the proportion verification condition (i.e., the first preset condition above).

[0193] S704, the proportions of the nodes in each layer are adjusted.

[0194] Specifically, the relevance scores of the nodes can be adjusted by adjusting the logical weights and the coverage weights, and then the calling probabilities are adjusted. Alternatively, the calling probabilities can be adjusted by adjusting the layer weights and the decay factors. Based on the adjusted calling probabilities, the business paths can be regenerated, and then the proportions of the nodes in each layer are adjusted until the dialogue data set satisfies the proportion verification condition.

[0195] Specifically, reference can be made to the description of formula (3) above.

[0196] After S704 is performed, S701 is performed, i.e., the verification is performed again.

[0197] S705, it is determined whether the dialogue data set satisfies the parameter stability condition. If yes, S707 is performed. Otherwise, S706 is performed.

[0198] Specifically, after the dialogue data set is constructed, for each node, an actual calling rate of each next-level node is determined, and it is determined whether a difference between the actual calling rate and the calling probability of the next-level node is less than a preset threshold (e.g., 5%). If the difference between the actual calling rate and the calling probability of the next-level node is less than the preset threshold, it is determined that the dialogue data set satisfies the parameter stability condition, and the dialogue data set is output. Otherwise, the decay factors and the layer weights are adjusted.

[0199] S706, the decay factors and the layer weights are adjusted, and the calling probabilities of each next-level node are determined again.

[0200] After S706 is performed, S701 is performed, that is, re-verification is performed.

[0201] S707, outputting the dialogue dataset.

[0202] The embodiment of the application realizes multi-dimensional technical improvement by integrating knowledge tree, double-algorithm regulation, hierarchical proportion and correction factor mechanism, and closed-loop optimization, as follows.

[0203] 1. Significant improvement in logical coherence: The knowledge tree structurally maintains the node flow relationship, avoiding dialogue context breaks from the source. Taking the test field as an example, the fixed process of “submitting a defect → supplementing a screenshot → assigning a handler” is forcibly associated through the knowledge tree, making more than 95% of dialogues form a complete business closed loop, effectively solving the problem of logical discontinuity.

[0204] 2. Full compliance of business coverage: By setting a minimum quota for edge nodes and combining a compensation scoring mechanism, it is ensured that low-frequency but critical business scenarios will not be ignored by the system. This mechanism effectively meets the demand for full-scenario coverage in the business domain.

[0205] 3. High matching of resource allocation and business priority: By setting the proportion of core layer nodes to 50%-60%, it is ensured that high-frequency core scenarios (such as “submitting a defect” and “confirming a repair”) dominate the dataset, with their data volume being 5-6 times that of edge nodes. This allocation strategy perfectly matches the business importance, significantly improving the model's recognition accuracy for core scenarios.

[0206] The embodiment of the application also provides a training method of a multi-turn dialogue model. As shown in Figure 11 the training method of the multi-turn dialogue model includes the following steps:

[0207] S801, obtaining a dialogue dataset.

[0208] S802, training an initial multi-turn dialogue model using the dialogue dataset to obtain a multi-turn dialogue model.

[0209] In some examples, the dialogue dataset includes a first dialogue dataset and a second dialogue dataset; wherein, when constructing the first dialogue dataset, the coverage weight is greater than the logical weight; when constructing the second dialogue dataset, the logical weight is greater than the coverage weight; training the initial multi-turn dialogue model using the dialogue dataset to obtain the multi-turn dialogue model includes: training the initial multi-turn dialogue model using the first dialogue dataset to obtain a first multi-turn dialogue model; training the first multi-turn dialogue model using the second dialogue dataset to obtain the multi-turn dialogue model.

[0210] The embodiment of the application also provides a dialogue dataset construction device. As shown inFigure 12 As shown in the figure, the constructing device 1200 of the dialogue data set includes an identifying module 1201, a first constructing module 1202, a determining module 1203, a first generating module 1204, a second generating module 1205, and a second constructing module 1206. The identifying module 1201 is configured to identify target text data to obtain an identification result, the identification result including a plurality of dialogue intents, an association relationship between the plurality of dialogue intents, and entity information corresponding to each dialogue intent. The first constructing module 1202 is configured to construct a knowledge tree according to the identification result, the knowledge tree including a plurality of nodes, one node corresponding to one dialogue intent, and one node being associated with entity information corresponding to the dialogue intent. The determining module 1203 is configured to determine, for each node, a calling probability of each next-level node of the node, the calling probability of the next-level node being related to a coverage value and a logical association of the next-level node. The first generating module 1204 is configured to take a target node as a starting node, and perform node selection in a loop according to a target round corresponding to the target node to generate a plurality of business paths. In generating one business path, the next node is selected from the target node according to the calling probability of each next-level node of the current node until the number of nodes of the business path reaches a target number of nodes. The second generating module 1205 is configured to generate, for each business path, a group of dialogue data according to the dialogue intent and the entity information corresponding to each node in the business path. The second constructing module 1206 is configured to construct a plurality of groups of dialogue data corresponding to the plurality of business paths into a dialogue data set, the dialogue data set being used to train a multi-turn dialogue model.

[0211] The embodiment of the present application also provides a training device of a multi-turn dialogue model. As shown in the figure, Figure 13 The training device 130 of the multi-turn dialogue model includes an obtaining module 1301 and a training module 1302. The obtaining module 1301 is configured to obtain a dialogue data set. The training module 1302 is configured to train an initial multi-turn dialogue model by using the dialogue data set to obtain a multi-turn dialogue model.

[0212] The electronic device provided by the embodiment can execute the method provided by the method embodiment, and has similar implementation principles and technical effects. Therefore, the electronic device will not be described here.

[0213] Figure 14 The electronic device provided by the embodiment is a structural schematic diagram of the electronic device. As shown in the figure, Figure 14 The electronic device 140 provided by the embodiment includes at least one processor 1401 and a memory 1402. Optionally, the electronic device 140 further includes a communication component 1403. The processor 1401, the memory 1402, and the communication component 1403 are connected through a bus.

[0214] In the implementation process, the at least one processor 1401 executes computer execution instructions stored in the memory 1402, so that the at least one processor 1401 executes the above-mentioned method.

[0215] The specific implementation process of the processor 1401 can refer to the method embodiments described above, which have similar implementation principles and technical effects, and details are not described here.

[0216] In the above embodiments, it should be understood that the processor can be a central processing unit (English: Central Processing Unit, CPU for short), and can also be other general-purpose processors, digital signal processors (English: Digital Signal Processor, DSP for short), application specific integrated circuits (English: Application Specific Integrated Circuit, ASIC for short), etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the application can be directly embodied as hardware processor execution or combined with hardware and software modules in the processor for execution.

[0217] The memory can include a random access memory (RAM), and can also include a non-volatile memory (NVM), such as at least one disk memory.

[0218] The bus can be an industry standard architecture (ISA) bus, a peripheral component (PCI) bus, or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, the bus in the drawings of the present application does not limit only one bus or one type of bus.

[0219] The present application also provides a computer program product, comprising a computer program, which is executed by a processor to implement the above-mentioned method.

[0220] The present application also provides a computer readable storage medium, which stores computer execution instructions, and when the processor executes the computer execution instructions, the above-mentioned method is implemented.

[0221] The above-mentioned readable storage medium can be realized by any type of volatile or nonvolatile storage devices or their combinations, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general or special purpose computer.

[0222] An exemplary readable storage medium is coupled to the processor, so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in the device.

[0223] The division of units is only a logical functional division, and in actual implementation, there can be another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0224] The units described as separate components can or can not be physically separated, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on multiple network units. According to actual needs, part or all of the units can be selected to achieve the purpose of the embodiment scheme.

[0225] In addition, the functional units in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0226] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the embodiments of the method of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0227] It can be understood by those skilled in the art that all or part of the steps of the above-mentioned method embodiments can be completed by program instruction related hardware. The aforementioned program can be stored in a computer readable storage medium. When the program is executed, the steps of the above-mentioned method embodiments are executed; and the aforementioned storage medium includes: ROM, RAM, magnetic disk or optical disk, and various media that can store program codes.

[0228] Finally, it should be noted that: those skilled in the art will easily think of other embodiments of the present application after considering the specification and practicing the application disclosed herein. The present application is intended to cover any variations, uses or adaptations of the present application that follow the general principles of the present application and include common knowledge or conventional technical means in the art that are not disclosed in the present application, and is not limited to the precise structure described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the present application is only limited by the appended claims.

Claims

1. A method of constructing a dialog data set, characterized by, The method comprises: performing recognition on target text data to obtain a recognition result, the recognition result comprising a plurality of dialogue intents, an association relationship between the plurality of dialogue intents, and entity information corresponding to each dialogue intent; constructing a knowledge tree according to the recognition result, the knowledge tree comprising a plurality of nodes, one node corresponding to one dialogue intent, and one node being associated with entity information corresponding to the dialogue intent; for each node, adjusting an association score of a next-level node according to a hierarchical weight parameter of a level in which the next-level node is located, obtaining an adjusted association score, determining a calling probability of the next-level node based on a historical co-occurrence frequency of the node and the next-level node corresponding to the node and the adjusted association score, and the association score of the next-level node being used to represent coverage value of the next-level node and logical association between the node and the next-level node; taking a target node as a starting node, performing node selection in a loop according to a target round corresponding to the target node, and generating a plurality of business paths, wherein, when generating one business path, a next node is selected according to a calling probability of each next-level node of a current node starting from the target node, and the node number of the business path reaches a target node number; for each business path, a group of dialogue data is generated according to dialogue intents and entity information corresponding to each node in the business path; a plurality of groups of dialogue data corresponding to the plurality of business paths are constructed into a dialogue data set, and the dialogue data set is used to train a multi-turn dialogue model.

2. The method of claim 1, wherein, Any node in the knowledge tree is a first node, and the adjusting of the association score of the next-level node according to the hierarchical weight parameter of the level in which the next-level node is located to obtain the adjusted association score comprises: determining a hierarchical weight parameter of a level in which a second node is located, the second node being a next-level node of the first node; adjusting the association score of the second node according to the hierarchical weight parameter of the level in which the second node is located to obtain the adjusted association score.

3. The method of claim 2, wherein, The method further comprises: determining the association score of the second node according to attribute information of the second node, wherein the attribute information is used to represent coverage value of the second node and a logical association degree between the second node and the first node.

4. The method of claim 3, wherein, The attribute information of the second node comprises first attribute information and second attribute information, and the determining of the association score of the second node according to the attribute information of the second node comprises: determining a first score according to the first attribute information of the second node, the first attribute information comprising at least one of dependency strength information, logical coherence information and data quality information; determining a second score according to the second attribute information of the second node, the second attribute information comprising at least one of irreplaceability information, scarcity information and historical coverage gap information. According to the logical weight corresponding to the first score and the coverage weight corresponding to the second score, the first score and the second score are weighted and averaged to obtain the relevance score of the second node.

5. The method of claim 4, wherein, The dialogue dataset includes a first dialogue dataset and a second dialogue dataset; wherein, when the first dialogue dataset is constructed, the coverage weight is greater than the logical weight; when the second dialogue dataset is constructed, the logical weight is greater than the coverage weight.

6. The method of claim 1, wherein, In the plurality of service paths, the proportion of a fourth node located in the core layer is greater than the proportion of a fifth node located in the expansion layer, and the proportion of the fifth node is greater than the proportion of a sixth node located in the edge layer. The historical appearance frequency of the fourth node is greater than or equal to a first frequency threshold; the historical appearance frequency of the fifth node is less than the first frequency threshold and greater than a second frequency threshold; and the historical appearance frequency of the sixth node is less than or equal to the second frequency threshold.

7. The method of claim 6, wherein, After the plurality of service paths are generated, the method further includes: In the case that the proportion of the seventh node is not the fourth preset proportion, the ratio of the fourth preset proportion to the proportion of the seventh node is taken as a correction parameter; wherein, the seventh node is the fourth node, the fourth preset proportion is a first preset proportion, or the seventh node is the fifth node, the fourth preset proportion is a second preset proportion, or the seventh node is the sixth node, and the fourth preset proportion is a third preset proportion; the first preset proportion is greater than the second preset proportion, and the second preset proportion is greater than the third preset proportion; For each next level node, the calling probability of the next level node is corrected according to the correction parameter to obtain the corrected calling probability of the next level node; The target node is taken as a starting node, and node selection is performed cyclically downward according to the target round corresponding to the target node to regenerate a plurality of service paths; wherein, when a service path is generated, the next node is selected from the target node according to the corrected calling probability of each next level node of the current node until the number of nodes of the service path reaches a target node number.

8. A method for training a multi-turn dialogue model, comprising: The method includes: Obtaining a dialogue dataset, which is constructed based on the dialogue dataset construction method in any one of claims 1-7; Training an initial multi-turn dialogue model using the dialogue dataset to obtain a multi-turn dialogue model.

9. The method of claim 8, wherein, The dialogue dataset includes a first dialogue dataset and a second dialogue dataset; wherein, when the first dialogue dataset is constructed, the coverage weight is greater than the logical weight; when the second dialogue dataset is constructed, the logical weight is greater than the coverage weight; and the training of the initial multi-turn dialogue model using the dialogue dataset to obtain a multi-turn dialogue model includes: Training an initial multi-turn dialogue model using the first dialogue dataset to obtain a first multi-turn dialogue model; Training the first multi-turn dialogue model using the second dialogue dataset to obtain the multi-turn dialogue model.

10. An apparatus for constructing a dialog data set, characterized by It includes: The recognition module is configured to recognize target text data to obtain a recognition result, the recognition result including a plurality of dialogue intents, an association relationship between the plurality of dialogue intents, and entity information corresponding to each dialogue intent. The first construction module is configured to construct a knowledge tree according to the recognition result, the knowledge tree including a plurality of nodes, one node corresponding to one dialogue intent, and one node being associated with entity information corresponding to the dialogue intent. The determination module is configured to, for each node, adjust an association score of a next-level node according to a hierarchical weight parameter of a level in which the next-level node is located, to obtain an adjusted association score, determine a calling probability of the next-level node based on a historical co-occurrence frequency of the node and the next-level node corresponding to the node and the adjusted association score, and use the association score of the next-level node to represent coverage value of the next-level node and logical association between the node and the next-level node. The first generation module is configured to take a target node as a starting node, select nodes in a loop according to a target round corresponding to the target node, and generate a plurality of business paths, wherein, when generating one business path, a next node is selected according to a calling probability of each next-level node of a current node, starting from the target node, until a number of nodes of the business path reaches a target number of nodes. The second generation module is configured to, for each business path, generate a group of dialogue data according to dialogue intents and entity information corresponding to each node in the business path. The second construction module is configured to construct a plurality of groups of dialogue data corresponding to the plurality of business paths into a dialogue data set, the dialogue data set being used to train a multi-turn dialogue model. 11.A training device of a multi-turn dialogue model, characterized in that, The method comprises: The acquisition module is configured to acquire a dialogue data set, the dialogue data set being constructed based on the method for constructing a dialogue data set according to any one of claims 1-7. The training module is configured to train an initial multi-turn dialogue model using the dialogue data set to obtain a multi-turn dialogue model.

12. An electronic device, comprising: The method comprises: A memory and a processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the method according to any one of claims 1-7, or executes the method according to claim 8 or 9.

13. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method according to any one of claims 1-7, or to implement the method according to claim 8 or 9.

Images