Quantitative verification method, device and equipment for task disassembly and computer storage medium

Abstract

The invention discloses a quantitative verification method and device for task disassembly, equipment and a computer storage medium, and relates to the technical field of task disassembly, and the method comprises the steps: obtaining an original task text, and carrying out the task disassembly of the original task text through a preset task disassembly model, and outputting a sub-task text set; calculating semantic similarity between the original task text and each subtask text in the subtask text set, and constructing a semantic similarity matrix; calculating a semantic coverage degree between the original task text and the subtask text set, and determining a quantitative evaluation index; and analyzing the semantic similarity matrix and the quantitative evaluation index, and determining the outlier sub-tasks in the sub-task text set to obtain a quantitative verification result of task disassembly, so that the interpretability of the task disassembly process is improved.

Description

Technical Field

[0001] This application relates to the field of task breakdown technology, and in particular to a quantitative verification method, apparatus, device and computer storage medium for task breakdown. Background Technology

[0002] With the continuous development of task decomposition technology and its increasing application in various fields, users have also raised higher requirements for the task decomposition process.

[0003] The traditional task decomposition process involves inputting a task description into a large model to obtain the decomposed task steps. This process has significant drawbacks. Because the reasoning process within the large model is a black box, it is difficult to verify the correctness of the decomposed task steps. In other words, this task decomposition process suffers from insufficient interpretability due to the difficulty in verifying the correctness of the decomposed task steps.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main objective of this application is to provide a quantitative verification method, apparatus, device, and computer storage medium for task decomposition, aiming to solve the technical problem of insufficient interpretability in the task decomposition process.

[0006] To achieve the above objectives, this application proposes a quantitative verification method for task decomposition, the quantitative verification method for task decomposition comprising:

[0007] Obtain the original task text, and the set of subtask texts output by performing task decomposition on the original task text using a preset task decomposition model;

[0008] Calculate the semantic similarity between the original task text and each subtask text in the subtask text set, and construct a semantic similarity matrix;

[0009] Calculate the semantic coverage between the original task text and the set of subtask texts, and determine the quantitative evaluation index;

[0010] The semantic similarity matrix and the quantitative evaluation index are analyzed to identify outlier subtasks in the subtask text set, so as to obtain the quantitative verification results of task decomposition.

[0011] In one embodiment, the step of calculating the semantic similarity between the original task text and each subtask text in the subtask text set, and constructing a semantic similarity matrix, includes:

[0012] Determine the output vector information of the original task text and the subtask text set in the preset semantic coding model, wherein the output vector information includes the original embedding vector corresponding to the original task text and the vector list corresponding to the subtask text set, and the vector list includes the subtask embedding vector corresponding to each subtask in the subtask text set;

[0013] Calculate the semantic similarity between the original embedding vector and the vector list, and construct a semantic similarity matrix between the original task text and the subtask text set.

[0014] In one embodiment, the step of calculating the semantic coverage between the original task text and the set of subtask texts, and determining the quantitative evaluation metric, includes:

[0015] Based on the original task text and the set of subtask texts, a vector list and original embedding vectors are determined, and the average semantic vector corresponding to the subtask embedding vectors in the vector list is calculated.

[0016] The cosine similarity between the average semantic vector and the original embedding vector is determined as the cosine evaluation index, and a quantitative evaluation index of semantic coverage between the cosine evaluation index and the embedding vector of each subtask in the vector list is calculated.

[0017] In one embodiment, the step of analyzing the semantic similarity matrix and the quantitative evaluation index to determine outlier subtasks in the subtask text set, in order to obtain the quantitative verification result of task decomposition, includes:

[0018] Determine the perturbation-generated text corresponding to the original task text, and determine the output prediction text of the perturbation-generated text in the preset task decomposition model;

[0019] The preset interpretation proxy model is trained based on the perturbation-generated text and the output prediction text. Based on the preset interpretation proxy model after training, the keyword feature heatmap of the original task text is determined. The semantic similarity matrix, the quantitative evaluation index and the keyword feature heatmap are analyzed to identify outlier subtasks in the subtask text set, so as to obtain the quantitative verification results of task decomposition.

[0020] In one embodiment, the step of analyzing the semantic similarity matrix, the quantitative evaluation index, and the keyword feature heatmap to determine outlier subtasks in the subtask text set, in order to obtain the quantitative verification results of task decomposition, includes:

[0021] The first keyword basis is determined in the keyword feature heatmap, the second keyword basis is determined in the semantic similarity matrix, and the third keyword basis is determined in the quantitative evaluation index;

[0022] If the first keyword basis, the second keyword basis, and the third keyword basis do not meet the preset semantic similarity range, then the target keyword that does not meet the preset semantic similarity range is determined, and the quantitative verification result of the task decomposition is determined to be an improved decomposition, wherein the improved decomposition includes outlier subtasks in the subtask text set as the target keyword.

[0023] In one embodiment, after the step of determining the keyword feature heatmap of the original task text based on the preset interpretation proxy model after training is completed, the quantitative verification method of task decomposition further includes:

[0024] A deviation heatmap between the original task text and the subtask text set is generated based on the semantic similarity matrix, and a decision path map between the original task text and the subtask text set is generated based on the quantitative evaluation index.

[0025] Based on analysis of at least two of the aforementioned deviation heatmap, decision path map, and keyword feature heatmap, outlier subtasks in the subtask text set are identified to obtain quantitative verification results of task decomposition.

[0026] In one embodiment, after the step of analyzing the semantic similarity matrix and the quantitative evaluation index to determine outlier subtasks in the subtask text set to obtain the quantitative verification result of task decomposition, the method further includes:

[0027] If the quantitative verification result of the task decomposition is an improved decomposition, the target keywords in the outlier subtasks are determined, and the preset task decomposition model is updated based on the target keywords.

[0028] Furthermore, to achieve the above objectives, this application also proposes a quantitative verification device for task decomposition, the quantitative verification device for task decomposition comprising:

[0029] The information acquisition module is used to acquire the original task text and the set of subtask texts output by performing task decomposition on the original task text through a preset task decomposition model.

[0030] The first processing module is used to calculate the semantic similarity between the original task text and each subtask text in the subtask text set, and to construct a semantic similarity matrix;

[0031] The second processing module is used to calculate the semantic coverage between the original task text and the set of subtask texts, and to determine the quantitative evaluation index.

[0032] The task verification module is used to analyze the semantic similarity matrix and the quantitative evaluation index to identify outlier subtasks in the subtask text set, so as to obtain the quantitative verification results of task decomposition.

[0033] In addition, to achieve the above objectives, this application also proposes a quantitative verification device for task decomposition, the quantitative verification device for task decomposition comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the quantitative verification method for task decomposition as described above.

[0034] In addition, to achieve the above objectives, this application also proposes a computer storage medium storing a computer program, which, when executed by a processor, implements the steps of the quantitative verification method for task decomposition as described above.

[0035] This application provides a quantitative verification method for task decomposition. It involves acquiring the original task text and a set of sub-task texts output by performing task decomposition on the original task text using a preset task decomposition model; calculating the semantic similarity between the original task text and each sub-task text in the sub-task text set to construct a semantic similarity matrix; calculating the semantic coverage between the original task text and the sub-task text set to determine a quantitative evaluation index; and analyzing the semantic similarity matrix and the quantitative evaluation index to identify outlier sub-tasks in the sub-task text set, thereby obtaining the quantitative verification result of the task decomposition. This quantitative verification method for task decomposition constructs a semantic similarity matrix by calculating the semantic similarity between the original task text and the sub-task text set (the set of texts output by performing task decomposition on the original task text). Simultaneously, the semantic coverage between the original task text and the sub-task text set is calculated to determine the quantitative evaluation index. This process addresses the black-box phenomenon of internal reasoning during task decomposition by transforming it into a semantic similarity matrix and quantitative evaluation index. The semantic similarity matrix, quantitative evaluation index, and original task text can then be analyzed to identify outlier sub-tasks within the sub-task text set, thus obtaining quantitative verification results for task decomposition. This avoids the problem that relying solely on the internal reasoning process of a large model makes it difficult to verify the correctness of the decomposed task steps. This quantitative verification method for task decomposition can analyze the semantic similarity matrix and quantitative evaluation index to identify outlier sub-tasks within the sub-task text set, obtaining quantitative verification results for task decomposition. This allows for visual verification of the task decomposition process, thereby improving its interpretability. Attached Figure Description

[0036] Figure 1 A flowchart illustrating the first embodiment of the quantitative verification method for task breakdown in this application;

[0037] Figure 2 A flowchart illustrating the first embodiment of the quantitative verification method for task breakdown in this application;

[0038] Figure 3 A schematic diagram of the implementation process of the quantitative verification method for the task breakdown in this application;

[0039] Figure 4 A schematic diagram of the modules of the quantitative verification device disassembled for the purpose of this application;

[0040] Figure 5 This is a schematic diagram of the hardware operating environment involved in the device in this application.

[0041] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0042] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0043] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0044] Traditional task decomposition processes are logically black-boxed, only able to obtain the decomposed task steps after inputting the task description into a large model. This addresses the pain points of subtasks deviating from requirements and lacking interpretability, ensuring the rationality and traceability of complex task decomposition. However, because the internal reasoning process of the large model is a black box, it is difficult to verify the correctness of its decomposed task steps, resulting in insufficient interpretability of the task decomposition process.

[0045] Therefore, based on the shortcomings of the above task decomposition schemes, this application proposes a quantitative verification method for task decomposition. The solution of this application is as follows: a semantic similarity matrix is ​​constructed by calculating the semantic similarity between the original task text and the set of sub-task texts (the text set output by decomposing the original task text). Simultaneously, the semantic coverage between the original task text and the set of sub-task texts is calculated to determine a quantitative evaluation index. This process transforms the black-box phenomenon of internal reasoning during task decomposition into a semantic similarity matrix and a quantitative evaluation index. The semantic similarity matrix, quantitative evaluation index, and original task text can be used for analysis to identify outlier sub-tasks in the sub-task text set, thereby obtaining a quantitative verification result for task decomposition. This avoids the problem that it is difficult to verify the correctness of the decomposed task steps using only the internal reasoning process of a large model. This quantitative verification method for task decomposition can analyze the semantic similarity matrix and quantitative evaluation index to identify outlier sub-tasks in the sub-task text set, thereby obtaining a quantitative verification result for task decomposition. This allows for visual verification of the task decomposition process, thus improving the interpretability of the task decomposition process.

[0046] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or a device capable of performing the above functions, such as a task decomposition quantitative verification device. The following description uses a task decomposition quantitative verification device as an example to illustrate this embodiment and the subsequent embodiments.

[0047] Based on this, embodiments of this application provide a quantitative verification method for task decomposition, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the quantitative verification method for task breakdown in this application.

[0048] Reference Figure 1 This application provides a quantitative verification method for task decomposition, which includes:

[0049] Step S10: Obtain the original task text and the set of subtask texts output by performing task decomposition on the original task text using a preset task decomposition model.

[0050] In this embodiment, when verifying the task decomposition process, the verification information is obtained during the task decomposition process. This information requires interpretability verification of the task decomposition process. For example, the verification information includes the original task text and a set of sub-task texts output by a preset task decomposition model. The original task text refers to the text initially decomposed, and the set of sub-task texts refers to the set of texts output after the original task text is decomposed by the preset task decomposition model. Therefore, the interpretability of the preset task decomposition model needs to be verified. The preset task decomposition model is a model that decomposes a complex task or goal into multiple smaller, manageable sub-tasks or sub-goals. It can be a work breakdown model, a critical path model, etc., and is not limited here.

[0051] Step S20: Calculate the semantic similarity between the original task text and each subtask text in the subtask text set, and construct a semantic similarity matrix;

[0052] Step S30: Calculate the semantic coverage between the original task text and the subtask text set, and determine the quantitative evaluation index;

[0053] In this embodiment, after determining the original task text and sub-task text sets in the task decomposition process, a semantic similarity matrix is ​​constructed based on the semantic similarity between the original task text and the sub-task text sets, and a quantitative evaluation index for the semantic coverage between the two task texts is determined. The semantic similarity matrix refers to the matrix of semantic similarity between the original task text and the sub-task text sets. A numerical similarity value is defined to interpret the text similarity. The quantitative evaluation index refers to the set of values ​​obtained by calculating the cosine similarity between the sub-task embedding vectors of each sub-task text set and the original embedding vectors, i.e., a numerical representation of the semantic coverage between the two task texts. At this point, the task decomposition process can be verified based on the semantic similarity matrix and the quantitative evaluation index to avoid the black-box problem of the task decomposition process and ensure its interpretability.

[0054] Step S40: Analyze the semantic similarity matrix and quantitative evaluation indicators to identify outlier subtasks in the subtask text set, so as to obtain the quantitative verification results of task decomposition.

[0055] In this embodiment, after determining the semantic similarity matrix and quantitative evaluation index, analysis can be performed directly based on these metrics. This allows for the identification of outlier subtasks within the subtask text set, yielding quantitative verification results for task decomposition. This quantitative verification of task decomposition determines whether the current task decomposition process is reasonable or meets preset requirements. Outlier subtasks refer to subtask texts outside the scope of task decomposition. For example, if the task is a route and the subtask text is beef, then that subtask text is identified as an outlier. The method for identifying outlier subtasks can be to directly check whether there is a set of subtask texts that differ significantly from the original task text based on the semantic similarity matrix and quantitative evaluation index. This can be directly manifested as a point in the semantic similarity matrix that differs significantly from the overall value, or a value in the quantitative evaluation index that differs significantly from other values. A threshold for this difference can be set for further judgment. Alternatively, the original task text can be processed simultaneously. In this case, the semantic relationships determined by the original task text need to be similar to the semantic similarity matrix and the quantitative evaluation indicators. Specifically, the similarity of the semantic similarity matrix cannot contradict the semantic relationships determined by the original task text, thus allowing for the quantitative verification of the task decomposition. Furthermore, heatmaps can be created for the similarity matrix, quantitative evaluation indicators, and the semantic relationships determined by the original task text. This visualization approach, combining these three indicators, provides an interpretable path from the macro (quantitative evaluation indicators) to the micro (semantic similarity matrix and semantic relationships determined by the original task text), making the decision-making process analyzable and ensuring the interpretability of the task decomposition process.

[0056] In this embodiment, a quantitative verification method for task decomposition is provided. This method involves acquiring the original task text and the set of sub-task texts output by performing task decomposition on the original task text using a preset task decomposition model; calculating the semantic similarity between the original task text and each sub-task text in the sub-task text set to construct a semantic similarity matrix; calculating the semantic coverage between the original task text and the sub-task text set to determine a quantitative evaluation index; and analyzing the semantic similarity matrix and the quantitative evaluation index to identify outlier sub-tasks in the sub-task text set, thereby obtaining the quantitative verification result of the task decomposition. This quantitative verification method for task decomposition constructs a semantic similarity matrix by calculating the semantic similarity between the original task text and the sub-task text set (the set of texts output by performing task decomposition on the original task text). This method calculates the semantic coverage between the original task text and the set of subtask texts to determine a quantitative evaluation index. It then treats the black-box phenomenon of internal reasoning during task decomposition as a semantic similarity matrix and a quantitative evaluation index. By analyzing the semantic similarity matrix, the quantitative evaluation index, and the original task text, outlier subtasks in the subtask text set are identified, resulting in a quantitative verification result for task decomposition. This avoids the problem that relying solely on the internal reasoning process of a large model makes it difficult to verify the correctness of the decomposed task steps. This quantitative verification method for task decomposition uses semantic similarity matrix and quantitative evaluation index to analyze and identify outlier subtasks in the subtask text set, thus obtaining a quantitative verification result for task decomposition. This allows for visual verification of the task decomposition process, thereby improving its interpretability.

[0057] Furthermore, based on the first embodiment of this application described above, a second embodiment of the quantitative verification method for task decomposition of this application is proposed. In this embodiment, step S20, which calculates the semantic similarity between the original task text and each subtask text in the subtask text set, and constructs a semantic similarity matrix, includes:

[0058] Step S21: Determine the output vector information of the original task text and the subtask text set in the preset semantic coding model. The output vector information includes the original embedding vector corresponding to the original task text and the vector list corresponding to the subtask text set. The vector list includes the subtask embedding vector corresponding to each subtask in the subtask text set.

[0059] Step S22: Calculate the semantic similarity between the original embedding vector and the vector list, and construct the semantic similarity matrix between the original task text and the subtask text set.

[0060] In this embodiment, the original task text and the set of subtask texts are input into a preset semantic encoding model to obtain output vector information. This output vector information includes a single original embedding vector of the original task text and a vector list of the subtask text sets. The vector list includes the subtask embedding vector corresponding to each subtask in the subtask text set. The preset semantic encoding model can be a Sentence-BERT pre-trained model, etc. A semantic similarity matrix can then be constructed using the original embedding vectors and the vector list. This is achieved by calculating the cosine similarity between the original embedding vectors and the vector list. The cosine similarity can be calculated using common methods for calculating the cosine similarity between two vectors, which will not be elaborated here. For example, mean pooling can be used to process the vector list and compare it with the original embedding vectors. The semantic similarity matrix between the original task text and the subtask text sets must be obtained using the embedded vectors. This allows for the creation of deviation heatmaps based on the semantic similarity matrix, visualizing the semantic similarity between each subtask and the original task, as well as other subtasks. This enables the rapid identification of subtasks with significant deviations. The method for creating these deviation heatmaps from the semantic similarity matrix can be either direct creation or using relevant software; no specific method is specified here. Further details can be found in [reference needed]. Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the quantitative verification method for task decomposition in this application. The entire process involves processing the input original task and subtask set (i.e., the original task text and the subtask text set) in a pre-trained model to obtain a semantic similarity matrix. At this point, a deviation heatmap can be directly generated based on the semantic similarity matrix, thereby visualizing the semantic similarity between each subtask and the original task and other subtasks. This allows for the rapid identification of subtasks with significant deviations, enabling targeted improvements to the preset task decomposition model to ensure the accuracy of the preset task decomposition model's output.

[0061] In one embodiment, the step of calculating the semantic coverage between the original task text and the set of subtask texts, and determining the quantitative evaluation metric, includes:

[0062] Step S31: Determine the vector list and the original embedding vector based on the original task text and the subtask text set, and calculate the average semantic vector corresponding to the subtask embedding vector in the vector list;

[0063] Step S32: Determine the cosine similarity between the average semantic vector and the original embedding vector as the cosine evaluation index, and calculate the quantitative evaluation index of semantic coverage between the cosine evaluation index and the embedding vector of each subtask in the vector list.

[0064] In this embodiment, while determining the semantic similarity matrix, a quantitative evaluation index can also be determined based on the original task text and the subtask text set. That is, a vector list and original embedding vectors are determined based on the original task text and the subtask text set, and the process can follow the control flow described in the previous embodiment. Further, the average semantic vector corresponding to the vector list is determined, i.e., mean pooling is performed on the vector list `subtask_embeds`. For example, assuming there are N D-dimensional vectors, the average value is taken for each dimension of these N vectors, resulting in a single D-dimensional vector. At this point, mean pooling yields an aggregated vector `subtask_embed_mean` that represents the average semantics of all subtasks. Of course, other mean pooling methods can also be used, and are not limited here. At this point, the cosine similarity between the average semantic vector and the original embedding vector can be determined as the cosine evaluation metric. Specifically, the cosine similarity between the original embedding vector `origin_embed` and the previously calculated average semantic vector `subtask_embed_mean` yields the cosine evaluation metric `semantic_score`. The cosine similarity between two vectors measures their directional similarity. It assesses their similarity by calculating the cosine of the angle between the two vectors in space. Cosine similarity primarily concerns the alignment of the two vectors in direction and is insensitive to their absolute length (size). This means that two vectors pointing in the same (or opposite) directions, even if their lengths differ, will have a high (or low) similarity. Having determined the cosine evaluation metric, we can then calculate a quantitative evaluation metric (typically the difference) for the semantic coverage between the cosine evaluation metric and the embedding vectors of each subtask in the vector list. Each subtask embedding vector in the vector list is first evaluated using a first assessment metric, `semantic_matrix`. `semantic_matrix` typically refers to the set of values ​​obtained by calculating the cosine similarity between each subtask embedding vector and the original embedding vector. If there are multiple subtasks, the set will contain multiple similarity scores. Furthermore, the difference between the cosine evaluation metric and each first assessment metric can be used as a quantitative evaluation metric to determine the similarity between each subtask text set and the original task text. Figure 2 The process of calculating semantic coverage index based on semantic similarity matrix, i.e., quantitative evaluation index, is used to show the task decomposition process and thus ensure the visualization of the task decomposition process.

[0065] It is worth noting that the quantitative evaluation indicators can also be determined by directly identifying dimensions such as target alignment, semantic similarity, structural matching, difference features, and comprehensive evaluation between the original task text and the set of subtask texts, transforming the abstract task completion status into comparable and optimizable numerical results. Of course, other methods can also be used, which will not be described in detail here.

[0066] Furthermore, based on the first and / or second embodiments of this application described above, a third embodiment of the quantitative verification method for task decomposition of this application is proposed. In this embodiment, step S40, which involves analyzing the semantic similarity matrix and quantitative evaluation indicators to determine outlier subtasks in the subtask text set in order to obtain the quantitative verification result of task decomposition, includes:

[0067] Step S41: Determine the perturbation-generated text corresponding to the original task text, and determine the output prediction text of the perturbation-generated text in the preset task decomposition model.

[0068] Step S42: Train the preset interpretation proxy model based on the perturbation-generated text and the output prediction text. Based on the preset interpretation proxy model after training, determine the keyword feature heatmap of the original task text, and analyze the semantic similarity matrix, quantitative evaluation index and keyword feature heatmap to identify outlier subtasks in the subtask text set, so as to obtain the quantitative verification results of task decomposition.

[0069] In this embodiment, while determining the semantic similarity matrix and quantitative evaluation indicators, a keyword feature heatmap can also be determined based on the original task text. The process for determining the keyword feature heatmap can be as follows: determine the perturbation-generated text corresponding to the original task text, and then determine the keyword feature heatmap based on the perturbation-generated text. The perturbation-generated text refers to a large number of new samples that are similar to but not exactly the same as the original text, generated by making minor, understandable modifications to the original text. For example, the LIME (Local Interpretable Model-agnostic Explanations) algorithm analyzes the decision boundary of the task decomposition model. The purpose of the LIME algorithm is to find the key factors affecting task decomposition. For example, if the original task is "applying for a business license," LIME can understand whether the model bases its decomposition decision on keywords such as "business" and "business license." In the LIME algorithm-based analysis of the task decomposition model's decision-making process, samples are generated by locally perturbing the original input task text. These samples are then predicted by a black-box model and used to train an interpretable surrogate model (such as linear regression). The output is a keyword feature weight heatmap. The perturbed text is fed into the black-box prediction model, and the model's prediction for each perturbed text is recorded. This results in a dataset consisting of "perturbed samples" and their corresponding "black-box model prediction results," which is used to train the interpretable surrogate model. Based on this, a keyword feature weight heatmap is then generated, as shown below. Figure 2The document describes the LIME boundary decision parsing, and then obtains a keyword feature weight heatmap based on the trained interpretable surrogate model. The keyword feature weight heatmap contains the weight (contribution value) of each key feature (keyword) in the complex task description. This heatmap allows visualization of keywords in the original text, thus visualizing the entire task decomposition process. Finally, the semantic similarity matrix, quantitative evaluation indicators, and keyword feature heatmap are combined for analysis to identify outlier subtasks in the subtask text set, obtaining quantitative verification results of the task decomposition. This allows for an interpretable path from macroscopic (quantitative evaluation indicators) to microscopic (semantic similarity matrix and semantic relationships determined by the original task text), making the decision-making process analyzable and ensuring the interpretability of the task decomposition process. It is worth noting that, at this point, outlier subtasks in the subtask text set can also be determined based on the semantic similarity matrix and quantitative evaluation indicators. That is, subtask texts that show differences or dissimilarities in the semantic similarity matrix and quantitative evaluation indicators (large differences in vector values ​​or other criteria) are considered outlier subtasks. Of course, other judgment methods can also be used, such as the first keyword in the quantitative evaluation indicators but a slightly similar keyword in the semantic similarity matrix. Then, based on the definition of excessive differences in the semantic similarity matrix and quantitative evaluation indicators, the subtask text can be determined as an outlier subtask in the task text set.

[0070] For example, training an interpretable proxy model can be done by configuring parameters such as classifier_fn (the prediction function of the task decomposition model to be interpreted), num_features (controlling the number of key features returned), and num_samples (the number of perturbation samples) in the interpreter, so as to provide a human-understandable explanation for a specific prediction of the black box model.

[0071] In one embodiment, the step of analyzing the semantic similarity matrix, quantitative evaluation indicators, and keyword feature heatmaps to identify outlier subtasks in the subtask text set, in order to obtain the quantitative verification results of task decomposition, includes:

[0072] Step S421: Determine the first keyword basis in the keyword feature heatmap, the second keyword basis in the semantic similarity matrix, and the third keyword basis in the quantitative evaluation index;

[0073] For example, if the first keyword basis, the second keyword basis, and the third keyword basis all meet the preset semantic similarity range, then the quantitative verification result of the task decomposition is determined to be a reasonable decomposition (i.e., it meets the reasonable standard defined by humans).

[0074] Step S322: If the first keyword basis, the second keyword basis, and the third keyword basis do not meet the preset semantic similarity range, then the target keywords that do not meet the preset semantic similarity range are determined, and the quantitative verification result of the task decomposition is determined to be an improved decomposition. The improved decomposition includes outlier subtasks in the subtask text set as target keywords (i.e., they do not meet the reasonable standards defined by humans and need to be improved).

[0075] In this embodiment, after determining the semantic similarity matrix, quantitative evaluation index, and keyword feature heatmap, the first keyword basis in the keyword feature heatmap, the second keyword basis in the semantic similarity matrix, and the third keyword basis in the quantitative evaluation index can be determined sequentially (the keywords in the semantic similarity matrix, quantitative evaluation index, and keyword feature heatmap can be determined based on importance or similarity, or different basis can be determined for the same keyword in the semantic similarity matrix, quantitative evaluation index, and keyword feature heatmap, such as keyword A being an important keyword in the quantitative evaluation index, while A is a general keyword in the quantitative evaluation index). Then, if the first, second, and third keyword basis all satisfy the preset semantic similarity range (e.g., all are important keywords, all are general keywords), the quantitative verification result of the task decomposition is determined to be a reasonable decomposition. Conversely, if the first, second, and third keyword basis do not satisfy the preset semantic similarity range, the quantitative verification result of the task decomposition is determined to be an improved decomposition. The improved decomposition includes outlier subtasks in the subtask text set as target keywords. In other words, the rationality of the entire task decomposition process can be determined based on these three keywords. Of course, at this point, each keyword in the semantic similarity matrix, quantitative evaluation indicators, and keyword feature heatmap can be evaluated one by one. Alternatively, the keyword with the highest contribution value in the semantic similarity matrix, quantitative evaluation indicators, and keyword feature heatmap can be selectively evaluated. A semantic comparison can be performed based on the keywords in the semantic similarity matrix, quantitative evaluation indicators, and keyword feature heatmap to determine the rationality of the entire task decomposition process. The first, second, and third keyword criteria can be the keyword with the highest frequency or the highest contribution value, or other keywords of the same type; this is not limited here. Alternatively, the rationality of the task decomposition process can be determined directly after comparing the keywords with the highest frequency or the highest contribution value, or further evaluation can be performed. Rationality can be determined when the percentage of reasonable keyword evaluations is above A%. Alternatively, a weight can be assigned to each keyword, and the sum of these weights can be used to determine the rationality of the verification results. In this case, the weight of a keyword is related to its contribution value and heatmap percentage (i.e., the frequency or percentage of the keyword). The contribution value is the frequency of each keyword's occurrence. The preset semantic similarity range refers to a defined range of semantic similarity. For example, "route" and "navigation" are defined as one semantic similarity, while "route" and "bread" are defined as different semantic similarity ranges. This can be designed in advance by the user or obtained based on model training, and is not limited here. At this time, the quantitative verification results of task decomposition can be identified based on keyword judgment, thereby ensuring the visualization and interpretability of the task decomposition process.

[0076] For example, the key outputs of all preceding steps are aggregated, including: LIME feature weights, similarity matrix, and quantitative evaluation metrics. Using the LIME feature weights, a LIME feature weight heatmap or highlighted text is generated to visually demonstrate the decision-making basis. Using the similarity matrix, a heatmap is drawn to visualize the semantic similarity between each subtask and the original task, as well as other subtasks, quickly locating subtasks with significant deviations.

[0077] In one embodiment, reference is made to Figure 3 , Figure 3 This is a schematic diagram of the implementation process of the quantitative verification method for task decomposition in this application. After determining the keyword feature heatmap of the original task text based on the pre-trained pre-defined interpretation proxy model, the quantitative verification method for task decomposition further includes:

[0078] Step S43: Generate a deviation heatmap between the original task text and the sub-task text set based on the semantic similarity matrix, and generate a decision path map between the original task text and the sub-task text set based on the quantitative evaluation index.

[0079] Step S44: Analyze at least two of the deviation heatmap, decision path map, and keyword feature heatmap to identify outlier subtasks in the subtask text set, so as to obtain the quantitative verification results of task decomposition.

[0080] In this embodiment, a keyword feature heatmap is determined. Simultaneously, a deviation heatmap between the original task text and the sub-task text set can be generated based on the semantic similarity matrix. A decision path map between the original task text and the sub-task text set is generated based on quantitative evaluation indicators. Then, analysis is performed based on at least two of the deviation heatmap, decision path map, and keyword feature heatmap to identify outlier sub-tasks in the sub-task text set, thus obtaining quantitative verification results of the task decomposition. The deviation heatmap is a visualization tool used to intuitively display the systematic deviations (such as prediction errors, data distribution shifts, model biases, etc.) between data or model prediction results and a reference benchmark. It maps the magnitude and direction of the deviation in spatial or structural dimensions (such as images, geographic grids, feature matrices) using color coding (such as red-blue gradients), helping users quickly locate problem areas. The semantic similarity matrix can directly generate the relationship between each keyword and its contribution value, thereby determining the heatmap. When analyzing based on two of the three, we can consider the feature heatmap of the original text or the keyword feature heatmap. Then, we can compare the features of the three with those of the original text to obtain the quantitative verification results of the task decomposition. If there are differences between the decision path diagram and the keyword feature heatmap, it also indicates that the quantitative verification results need improvement. The decision path diagram is a visualization tool used to show the decision logic chain from input features to the final result in a machine learning model (especially a tree-structured model) during prediction. It reveals the inference steps inside the model graphically, helping users understand how the model makes a final decision based on a specific input. It can be directly determined based on a defined quantitative evaluation index, since the quantitative evaluation index is actually a set of differences, and the decision path diagram can be determined based on this set of differences. Further details are not provided here. This approach integrates LIME's visualization capabilities with semantic similarity calculation, pioneering a logical mapping analysis for task decomposition. By comparing the embedding vectors (such as BERT encoding) of the original task and subtask sets, semantic coverage is quantified, and a multimodal verification report including a decision path diagram and a bias heatmap is generated, thus ensuring the visualization of the task decomposition process.

[0081] In one embodiment, determining the quantitative verification result of task decomposition based on at least one of the deviation heatmap, decision path map, and keyword feature heatmap can be used to determine the rationality of the task decomposition process based on the other map. (See reference...) Figure 2Users generate a multimodal verification report based on deviation heatmaps, decision path maps, and keyword feature heatmaps, which is then manually verified. If the manual review is successful, the task decomposition process is deemed reasonable. Conversely, when a deviated subtask is identified, modification suggestions are provided based on the keywords or other decomposition criteria of that subtask. For example, if the keyword for decomposing a deviated subtask S is S1, the default task decomposition model will be modified to exclude keyword S1, resulting in an updated task decomposition model and ensuring the accuracy of subsequent task decompositions.

[0082] Furthermore, based on the first, second, and / or third embodiments of this application described above, a fourth embodiment of the quantitative verification method for task decomposition of this application is proposed. In this embodiment, after the step of analyzing the semantic similarity matrix and quantitative evaluation indicators to determine outlier subtasks in the subtask text set to obtain the quantitative verification result of task decomposition, the method includes:

[0083] Step a: If the quantitative verification result of the task decomposition is an improved decomposition, determine the target keywords in the outlier subtasks and update the preset task decomposition model based on the target keywords.

[0084] In this embodiment, after determining the verification result, if the verification result indicates an improved decomposition (meaning the preset task decomposition model needs improvement), then at least two of the following will be compared and analyzed: semantic similarity matrix, quantitative evaluation index, and keyword feature heatmap. This will identify outlier subtasks, which can refer to a single keyword. The target keyword corresponding to the outlier subtask will then be determined, and the preset task decomposition model will be updated based on the target keyword. The advantage of using two methods is that they can be compared to determine the differences between them. For example, if subtask text set D1 has a contribution value of 0 on the quantitative evaluation index, but a contribution value of 100 on the keyword feature heatmap determined from the original task text, then subtask text set D1 is determined to have a difference between the two and is therefore identified as an outlier subtask. At this point, the target keyword corresponding to the outlier subtask can be identified, such as D11, from the keyword feature heatmap. The pre-defined task decomposition model can then be updated based on this target keyword. The update process involves adding the keyword D11 and its corresponding decomposition rules to the updated model. This is because the keyword feature heatmap, obtained directly from the original task text, can be used as a standard. However, the semantic similarity matrix and quantitative evaluation metrics, derived from the previous model, are generally not considered reliable. When using the semantic similarity matrix and quantitative evaluation metrics, the target keyword must first be identified. Then, it must be determined whether the target keyword contributes a certain threshold value to the keyword feature heatmap. Only when the target keyword demonstrates this threshold value should it be added to the updated model along with its corresponding decomposition rules to ensure the accuracy of subsequent task decomposition processes.

[0085] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the quantitative verification method of task decomposition in this application. Any simple transformations based on this technical concept are within the protection scope of this application.

[0086] This application also provides a quantitative verification device for task decomposition; please refer to [reference needed]. Figure 4 The quantitative verification device for task breakdown includes:

[0087] The information acquisition module 10 is used to acquire the original task text and the set of subtask texts output by performing task decomposition on the original task text through a preset task decomposition model.

[0088] The first processing module 20 is used to calculate the semantic similarity between the original task text and each subtask text in the subtask text set, and to construct a semantic similarity matrix;

[0089] The second processing module 30 is used to calculate the semantic coverage between the original task text and the set of subtask texts and to determine the quantitative evaluation index.

[0090] The task verification module 40 is used to analyze the semantic similarity matrix and quantitative evaluation indicators to identify outlier subtasks in the subtask text set in order to obtain the quantitative verification results of task decomposition.

[0091] The task decomposition quantitative verification device provided in this application, employing the task decomposition quantitative verification method in the above embodiments, can solve the technical problem of insufficient interpretability in the task decomposition process. Compared with the prior art, the beneficial effects of the task decomposition quantitative verification device provided in this application are the same as those of the task decomposition quantitative verification method provided in the above embodiments, and other technical features in the task decomposition quantitative verification device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0092] This application provides a quantitative verification device for task breakdown, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the quantitative verification method for task breakdown in the first embodiment described above.

[0093] The following is for reference. Figure 5 The diagram illustrates a structural schematic of a quantitative verification device suitable for implementing task breakdown in the embodiments of this application. The quantitative verification device for task breakdown in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The quantitative verification device for task breakdown shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0094] like Figure 5As shown, the task disassembly quantization verification device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 1002 or a program loaded from storage device 1003 into random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the task disassembly quantization verification device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following devices can be connected to I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the task-disassembled quantitative verification equipment to communicate wirelessly or wiredly with other devices to exchange data. Although a task-disassembled quantitative verification equipment with various devices is shown in the figure, it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0095] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0096] The task decomposition quantitative verification device provided in this application, employing the task decomposition quantitative verification method described in the above embodiments, can solve the technical problem of insufficient interpretability in the task decomposition process. Compared with the prior art, the beneficial effects of the task decomposition quantitative verification device provided in this application are the same as those of the task decomposition quantitative verification method described in the above embodiments, and other technical features in this task decomposition quantitative verification device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0097] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0098] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0099] This application provides a computer storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the quantitative verification method for task decomposition in the above embodiments.

[0100] The computer storage medium provided in this application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor devices, apparatuses, or components, or any combination thereof. More specific examples of computer storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution apparatus, device, or device. The program code contained on the computer storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0101] The aforementioned computer storage medium may be included in the quantitative verification equipment disassembled from the task; or it may exist independently and not be assembled into the quantitative verification equipment disassembled from the task.

[0102] The aforementioned computer storage medium carries one or more programs. When these programs are executed by the task-decomposition quantization verification device, the task-decomposition quantization verification device:

[0103] Obtain the original task text, and the set of subtask texts output by performing task decomposition on the original task text using a preset task decomposition model;

[0104] Calculate the semantic similarity between the original task text and each subtask text in the subtask text set, and construct a semantic similarity matrix;

[0105] Calculate the semantic coverage between the original task text and the set of subtask texts, and determine the quantitative evaluation metrics;

[0106] By analyzing the semantic similarity matrix and quantitative evaluation indicators, outlier subtasks in the subtask text set are identified to obtain quantitative verification results of task decomposition.

[0107] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0108] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using dedicated hardware-based apparatus to perform the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0109] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0110] The computer storage medium provided in this application stores computer-readable program instructions (i.e., a computer program) for executing the quantitative verification method for task decomposition described above, which can solve the technical problem of insufficient interpretability in the task decomposition process. Compared with the prior art, the beneficial effects of the computer storage medium provided in this application are the same as the beneficial effects of the quantitative verification method for task decomposition provided in the above embodiments, and will not be repeated here.

[0111] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the quantitative verification method for task decomposition as described above.

[0112] The computer program product provided in this application can solve the technical problem of insufficient interpretability in the task decomposition process. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the quantitative verification method for task decomposition provided in the above embodiments, and will not be repeated here.

[0113] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for quantitatively verifying task decomposition, characterized in that, The task decomposition quantitative verification method comprises the following steps: Obtain the original task text and the subtask text set output by the original task text after task decomposition by a preset task decomposition model; Calculate the semantic similarity between the original task text and each subtask text in the subtask text set, and construct a semantic similarity matrix; Calculate the semantic coverage between the original task text and the subtask text set, and determine the quantitative evaluation index; Analyze the semantic similarity matrix and the quantitative evaluation index to determine the outlier subtask in the subtask text set, and obtain the quantitative verification result of task decomposition.

2. The method of claim 1, wherein, The step of calculating the semantic similarity between the original task text and each subtask text in the subtask text set, and constructing a semantic similarity matrix, comprises the following steps: Determine the output vector information of the original task text and the subtask text set in a preset semantic encoding model, wherein the output vector information comprises an original embedding vector corresponding to the original task text and a vector list corresponding to the subtask text set, and the vector list comprises a subtask embedding vector corresponding to each subtask in the subtask text set; Calculate the semantic similarity between the original embedding vector and the vector list, and construct the semantic similarity matrix between the original task text and the subtask text set.

3. The task-decomposition quantification verification method of claim 2, wherein, The step of calculating the semantic coverage between the original task text and the subtask text set, and determining the quantitative evaluation index, comprises the following steps: Determine the vector list and the original embedding vector based on the original task text and the subtask text set, and calculate the average semantic vector corresponding to the subtask embedding vector in the vector list; Determine the cosine similarity between the average semantic vector and the original embedding vector as a cosine evaluation index, and calculate the quantitative evaluation index of the semantic coverage between the cosine evaluation index and each subtask embedding vector in the vector list.

4. The method of claim 1, wherein, The step of analyzing the semantic similarity matrix and the quantitative evaluation index to determine the outlier subtask in the subtask text set, and obtaining the quantitative verification result of task decomposition, comprises the following steps: Determine the disturbance generation text corresponding to the original task text, and determine the output prediction text of the disturbance generation text output by the preset task decomposition model; Train a preset explanation agent model based on the disturbance generation text and the output prediction text, determine the keyword feature heat map of the original task text based on the trained preset explanation agent model, and analyze the semantic similarity matrix, the quantitative evaluation index and the keyword feature heat map to determine the outlier subtask in the subtask text set, and obtain the quantitative verification result of task decomposition.

5. The method of claim 4, wherein the task decomposition is quantitatively verified by: The step of analyzing the semantic similarity matrix, the quantitative evaluation index and the keyword feature heat map to determine the outlier subtask in the subtask text set, and obtaining the quantitative verification result of task decomposition, comprises the following steps: determine a first keyword basis in the keyword feature heat map, determine a second keyword basis in the semantic similarity matrix, and determine a third keyword basis in the quantitative evaluation index; in a case where the first keyword basis, the second keyword basis, and the third keyword basis do not satisfy a preset semantic similarity range, determine a target keyword that does not satisfy the preset semantic similarity range, and determine a quantitative verification result of task decomposition as improvement decomposition, wherein the improvement decomposition includes an outlier subtask in the subtask text set as the target keyword.

6. The method of claim 4, wherein the task decomposition is quantitatively verified by: After the step of determining the keyword feature heat map of the original task text based on the preset explanation agent model after training is completed, the quantitative verification method of task decomposition further includes: generating a deviation heat map between the original task text and the subtask text set based on the semantic similarity matrix, and generating a decision path graph between the original task text and the subtask text set based on the quantitative evaluation index; analyzing at least two of the deviation heat map, the decision path graph, and the keyword feature heat map to determine an outlier subtask in the subtask text set to obtain a quantitative verification result of task decomposition.

7. The task-decomposition quantification verification method of any one of claims 1 to 6, wherein, After the step of analyzing the semantic similarity matrix and the quantitative evaluation index to determine an outlier subtask in the subtask text set to obtain a quantitative verification result of task decomposition, the step includes: in a case where the quantitative verification result of task decomposition is improvement decomposition, determining a target keyword in the outlier subtask, and updating the preset task decomposition model based on the target keyword.

8. A device for quantitatively verifying task decomposition, characterized in that, The quantitative verification device of task decomposition includes: an information acquisition module configured to acquire an original task text and a subtask text set output by performing task decomposition on the original task text by a preset task decomposition model; a first processing module configured to calculate semantic similarity between the original task text and each subtask text in the subtask text set, and construct a semantic similarity matrix; a second processing module configured to calculate semantic coverage between the original task text and the subtask text set, and determine a quantitative evaluation index; a task verification module configured to analyze the semantic similarity matrix and the quantitative evaluation index to determine an outlier subtask in the subtask text set to obtain a quantitative verification result of task decomposition.

9. A task decomposition quantification verification device, comprising: The quantitative verification device of task decomposition includes a memory, a processor, and a computer program stored on the memory and executable on the processor, the computer program being configured to implement the steps of the quantitative verification method of task decomposition according to any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The computer storage medium stores a computer program, and the computer program is executed by a processor to implement the steps of the quantitative verification method of task decomposition according to any one of claims 1 to 7.

Images