A Workflow Similarity Evaluation Method and System for Simulated Expert Scoring

Abstract

This invention provides a method and system for evaluating workflow similarity by simulating expert scoring, comprising the following steps: Evaluation index acquisition step: obtaining a comprehensive workflow structural similarity evaluation index through simulated expert scoring; Similarity evaluation step: evaluating the similarity between workflows based on the comprehensive workflow structural similarity evaluation index. This invention generates a set of workflow similarity evaluation criteria using data from simulated expert scoring, and can automatically add labels to workflow similarities, facilitating the generation of large amounts of data for downstream tasks and reducing the manpower and resources consumed by manual annotation.

Description

Technical Field

[0001] This invention relates to the technical field of data mining and analysis, specifically to a workflow similarity evaluation method and system that simulates expert scoring. Background Technology

[0002] As business processes and scientific research become increasingly complex, automated workflow technologies are playing an increasingly important role. A workflow is a general description of a work process and the rules governing the relationships between its various operational steps. It is mainly divided into business workflows and scientific workflows. It defines many task units for process handling and analysis, which are connected and driven by data flow, and can usually be represented as a directed graph.

[0003] Staff can search the database for workflows that meet their needs for reuse or modification, eliminating the need for redesigning from scratch and saving significant time to focus on business or scientific problems. Related technologies include workflow clustering, workflow retrieval, and workflow recommendation, all of which address the core issue of measuring workflow similarity. Existing methods are mainly divided into text-based and structure-based similarity calculations: text-based methods primarily utilize workflow titles and descriptions for textual similarity calculations but do not consider the workflow's structural semantics, while structure-based methods calculate similarity based on the correspondence between workflow topological units of different granularities, including modules, workflow substructures, and the entire graph structure. However, current methods each have their own emphasis on extracting structural features from workflows, necessitating the design of a comprehensive similarity calculation method that considers the workflow's structural semantics. To evaluate the effectiveness of various similarity methods, we can invite domain experts to score workflow similarity.

[0004] Chinese invention patent document CN106991006A discloses a cloud workflow task clustering method that supports dependency and time balance. When clustering tasks at the same level in the flowchart, it prioritizes the data dependency relationship between tasks and considers the time balance between clusters. When clustering tasks at the same level, it does not simply group parent tasks with the same sub-tasks together, but comprehensively considers the sub-tasks common to these tasks as well as their unique sub-tasks. Based on this, it proposes the concept and calculation formula of task correlation degree to characterize the degree of dependency between tasks.

[0005] Regarding the aforementioned technologies, the inventors believe that while inviting domain experts to score workflow similarity has high credibility, the data collection process is time-consuming and labor-intensive, and the data volume is relatively small. Furthermore, experts often score using coarse-grained levels, which cannot meet the evaluation requirements for high precision. Therefore, if it were possible to simulate expert scoring data and generate an automatic evaluation criterion for workflow similarity, accurately calculating the similarity between any two workflows, it would possess rationality, universality, and practical application value. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this invention is to provide a workflow similarity evaluation method and system that simulates expert scoring.

[0007] A workflow similarity evaluation method for simulated expert scoring provided by the present invention includes the following steps:

[0008] Steps for obtaining evaluation indicators: Obtain the overall workflow structure similarity evaluation indicator by simulating expert scoring;

[0009] Similarity evaluation steps: Evaluate the similarity between workflows based on the overall workflow structure similarity evaluation index.

[0010] Preferably, the evaluation index acquisition step includes the following steps:

[0011] Dataset construction steps: Obtain the expert rating dataset, which includes several workflow query models and workflow comparison models;

[0012] Comparison pair construction steps: For each workflow query model, arbitrarily extract comparison pairs (x) from the corresponding workflow comparison models. 1 ,x 2 ); where x 1 Let x represent the first workflow comparison model. 2 This represents the second workflow comparison model;

[0013] Structural similarity calculation steps: Calculate the structural similarity between the workflow comparison model and the workflow query model in each comparison pair;

[0014] Evaluation index calculation steps: Calculate x based on structural similarity. 1 and x 2 The similarity difference between them, x 1 and x 2 Whether the workflow comparison models belong to the same group is used as labels, and logistic regression is used to fit and obtain the overall workflow structure similarity evaluation index.

[0015] Preferably, in the dataset construction step, each workflow query model corresponds to a predetermined number of workflow comparison models, and the predetermined number of workflow comparison models are sorted and grouped according to the similarity of expert ratings; the workflow comparison models in the same group have the same similarity to the workflow query model.

[0016] Preferably, in the structural similarity calculation step, the workflow comparison model and workflow query model in each comparison pair are regarded as activity sets, and the maximum weight matching between the two activity sets is obtained using a bipartite graph matching algorithm to obtain x. 1 Similarity of activity set to workflow query model Get x 2 Similarity of activity set to workflow query model

[0017] Treating the workflow comparison model and workflow query model in each comparison pair as path sets, and using bipartite graph matching and non-crossing bipartite graph matching algorithms, the maximum weight matching between the two path sets is obtained, thus yielding x. 1 Similarity of path sets with workflow query model Get x 2 Similarity of path sets with workflow query model

[0018] Treating the workflow comparison model and workflow query model in each comparison pair as directed acyclic graphs (DAGs), and using the graph edit distance algorithm to calculate the edit distance between the two DAGs, we obtain x. 1 Global similarity with workflow query model Get x 2 Global similarity with workflow query model

[0019] Preferably, in the evaluation index calculation step, x 1 and x 2 Difference in similarity between As a feature, x 1 and x 2 Using the comparison models of whether they belong to the same group as labels, logistic regression is used to fit and obtain the weight coefficients w1 for activity set similarity, w2 for path set similarity, and w3 for global similarity, thus obtaining the comprehensive workflow structural similarity evaluation index:

[0020] Sim COM =W l Sim AS +W2Sim PS +W3Sim GS

[0021] Among them, Sim COM Sim represents the overall similarity between any two workflows; AS Sim represents the similarity of activity sets between any two workflows; PS Sim represents the path set similarity between any two workflows; GS This represents the global similarity between any two workflows.

[0022] A workflow similarity evaluation system for simulated expert scoring, provided by the present invention, includes the following modules:

[0023] Evaluation index acquisition module: Obtains the overall workflow structure similarity evaluation index by simulating expert scoring;

[0024] Similarity evaluation module: Evaluates the similarity between workflows based on the overall workflow structure similarity evaluation index.

[0025] Preferably, the evaluation index acquisition module includes the following modules:

[0026] Dataset building module: Obtains the expert rating dataset, which includes several workflow query models and workflow comparison models;

[0027] Comparison Pair Building Module: For each workflow query model, a comparison pair (x) is arbitrarily extracted from the corresponding workflow comparison model. 1 ,x 2 ); where x 1 Let x represent the first workflow comparison model. 2 This represents the second workflow comparison model;

[0028] The structural similarity calculation module calculates the structural similarity between the workflow comparison model and the workflow query model in each comparison pair.

[0029] Evaluation index calculation module: Calculates x based on structural similarity. 1 and x 2 The similarity difference between them, x 1 and x 2 Whether the workflow comparison models belong to the same group is used as labels, and logistic regression is used to fit and obtain the overall workflow structure similarity evaluation index.

[0030] Preferably, in the dataset construction module, each workflow query model corresponds to a predetermined number of workflow comparison models, and the predetermined number of workflow comparison models are sorted and grouped according to the similarity of expert ratings; the workflow comparison models in the same group have the same similarity to the workflow query model.

[0031] Preferably, in the structural similarity calculation module, the workflow comparison model and workflow query model in each comparison pair are regarded as activity sets, and the maximum weight matching between the two activity sets is obtained using a bipartite graph matching algorithm to obtain x. 1 Similarity of activity set to workflow query model Get x 2 Similarity of activity set to workflow query model

[0032] Treating the workflow comparison model and workflow query model in each comparison pair as path sets, and using bipartite graph matching and non-crossing bipartite graph matching algorithms, the maximum weight matching between the two path sets is obtained, thus yielding x. 1 Similarity of path sets with workflow query model Get x 2 Similarity of path sets with workflow query model

[0033] Treating the workflow comparison model and workflow query model in each comparison pair as directed acyclic graphs (DAGs), and using the graph edit distance algorithm to calculate the edit distance between the two DAGs, we obtain x. 1 Global similarity with workflow query model Get x 2 Global similarity with workflow query model

[0034] Preferably, in the evaluation index calculation module, x 1 and x 2 Difference in similarity between As a feature, x 1 and x 2 Using the comparison models of whether they belong to the same group as labels, logistic regression is used to fit and obtain the weight coefficients w1 for activity set similarity, w2 for path set similarity, and w3 for global similarity, thus obtaining the comprehensive workflow structural similarity evaluation index:

[0035] Sim COM =W l Sim AS +W2Sim PS +W3Sim GS

[0036] Among them, Sim COM Sim represents the overall similarity between any two workflows; AS Sim represents the similarity of activity sets between any two workflows; PS Sim represents the path set similarity between any two workflows;GS This represents the global similarity between any two workflows.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] 1. This invention simulates expert scoring data to generate a set of evaluation criteria for workflow similarity, which can automatically add labels to workflow similarity, making it easier to generate a large amount of data for downstream tasks and reducing the manpower and material resources consumed by manual annotation.

[0039] 2. Based on various workflow structure similarity methods, this invention designs a comprehensive structure similarity calculation method, which not only considers the multi-granularity structural features of the workflow, but also utilizes the execution semantics of the workflow, and combines the advantages of various methods through a weighted average.

[0040] 3. This invention utilizes comprehensive evaluation indicators to more reasonably and accurately calculate the similarity between any two workflows, which facilitates the generation of a large amount of data for downstream tasks, reduces the manpower and material resources consumed by manual annotation, and has universality and practical value. Attached Figure Description

[0041] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0042] Figure 1 The diagram shown is a flowchart of the method of the present invention;

[0043] Figure 2 The diagram shown is an example of the workflow model for which this invention is applied;

[0044] Figure 3 The diagram shown is a schematic diagram of the workflow structure similarity calculation of the present invention. Detailed Implementation

[0045] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0046] This invention discloses a workflow similarity evaluation method for simulating expert scoring, such as... Figure 1 and Figure 2 As shown, it includes the following steps:

[0047] Steps for obtaining evaluation indicators: Obtain the overall workflow structure similarity evaluation indicators by simulating expert scoring.

[0048] The evaluation metric acquisition steps include the following: Dataset construction steps: Obtain the expert rating dataset, which includes several workflow query models and workflow comparison models. Each workflow query model corresponds to a predetermined number of workflow comparison models, and the predetermined number of workflow comparison models are sorted and grouped according to the similarity of expert ratings; the workflow comparison models in the same group have the same similarity to the workflow query models.

[0049] Specifically, to evaluate the effectiveness of various workflow similarity algorithms, we can invite domain experts to score the workflow similarity. Typically, the experiments involve query workflows and comparison workflows. Domain experts score the similarity of the comparison workflow and query workflow according to a rating (e.g., "very similar," "similar," "moderately similar," "dissimilar," and "uncertain"). Both query and comparison workflows are derived from an online workflow database.

[0050] After processing and aggregation, all comparison workflows are sorted and grouped by similarity (tie) to form the dataset. Workflows (comparison workflows) within the same tie can be considered to have the same similarity to the query workflow. A tie represents a group, and comparison workflows within the same group have the same similarity to the query workflow.

[0051] Each row of the dataset contains one query workflow and a certain number of comparison workflows that group and sort by expert score similarity from high to low, for a total of several rows.

[0052] Comparison pair construction steps: For each workflow query model, arbitrarily extract comparison pairs (x) from the corresponding workflow comparison models. 1 x 2 ); where x 1 Let x represent the first workflow comparison model. 2 This represents the second workflow comparison model.

[0053] Specifically, using a dataset of expert-scored workflow similarity, for each workflow query model, two workflow comparison pairs (x, x) are randomly selected from all expert-scored comparison models (comparison workflows). 1 x 2 ), where x 1 Ranked in x 2 This is to ensure consistency in the calculation. Because the workflow comparison is sorted from highest to lowest based on the similarity between the expert scores and the query workflow, generally, those ranked higher have higher similarity scores across their three structures than those ranked lower. Therefore, if x... 1 and x 2 If they are not in the same group, their label is 1. In this case, if x in a sample...1 Ranked in x 2 Previously, the differences between their three structural similarities were all positive; in another sample, x 1 Ranked in x 2 Afterwards, the difference between their three structural similarities is negative, which will affect the linear regression fit. Therefore, x is uniformly defined here. 1 Ranked in x 2 (It can also be placed before or after.)

[0054] That is, for each workflow query model in the expert rating dataset, two arbitrary pairs are drawn from its comparison model to form a comparison pair (x). 1 x 2 ). x 1 Indicates workflow comparison model 1 (first workflow comparison model), x 2 This represents workflow comparison model 2 (second workflow comparison model).

[0055] Structural similarity calculation steps: Calculate the structural similarity between the workflow comparison model and the workflow query model in each comparison pair.

[0056] like Figure 2 and Figure 3 As shown, for each workflow model in the comparison pair, their structural similarity with the query model is calculated in three ways: Activity Set (AS), Path Set (PS), and Global Set (GS). For each workflow model in the comparison pair, their structural similarity with the query model is calculated in the following three ways:

[0057] (1) Treat the workflow comparison model and workflow query model in each comparison pair as activity sets, and use the bipartite graph matching algorithm to find the maximum weight matching between the two activity sets, thus obtaining x. 1 Similarity of activity set to workflow query model Get x 2 Similarity of activity set to workflow query model

[0058] Activity Set (AS) similarity: The workflow model is regarded as an activity set (that is, each workflow comparison model and the workflow query model within the comparison pair are regarded as activity sets respectively). The maximum weight matching between the two activity sets is obtained by using the bipartite graph matching algorithm, thereby obtaining the similarity between the two workflow models.

[0059] (2) Treat the workflow comparison model and workflow query model in each comparison pair as path sets, and use bipartite graph matching and non-crossing bipartite graph matching algorithms to find the maximum weight matching between the two path sets, and obtain x. 1 Similarity of path sets with workflow query model Get x 2 Similarity of path sets with workflow query model

[0060] The path set (PS) similarity is calculated by treating the workflow model as a path set (i.e., treating each workflow comparison model and the workflow query model within the comparison pair as path sets), and using bipartite graph matching and its improved algorithm (non-cross bipartite graph matching) to find the maximum weight matching between the two path sets, thereby obtaining the similarity between the two workflow models.

[0061] (3) Treat the workflow comparison model and workflow query model in each comparison pair as directed acyclic graphs, and use the graph edit distance algorithm to calculate the edit distance between the two directed acyclic graphs, thus obtaining x. 1 Global similarity with workflow query model Get x 2 Global similarity with workflow query model

[0062] Global Similarity (GS) is calculated by treating the workflow models as DAG graphs (i.e., treating each workflow comparison model and the workflow query model within the comparison pair as acyclic directed graphs), and using a graph edit distance algorithm to calculate the edit distance between the two directed graphs, thus obtaining the similarity between the two workflow models. DAG stands for Directed Acyclic Graph.

[0063] Specifically, such as Figure 2 The diagram shows a typical workflow model, which can usually be represented as a directed graph G(v,l) with a topological structure, where v represents the activity unit that constitutes the analysis process, and l is the directed edge connecting the vertices, indicating the order in which the activities are executed.

[0064] like Figure 3 As shown, to calculate the structural similarity of a workflow, we need to consider the activity units within the workflow and the relationships between them. Therefore, the calculation process can be divided into two steps: the first step is to calculate the activity similarity, and the second step is to calculate the workflow similarity.

[0065] 1. Activity Similarity Calculation: Each activity (activity unit) must have at least three attributes: name, type, and description. For example... Figure 2 Each letter in each rectangle represents a different activity unit name. Considering the diversity of workflows, we uniformly use the following formula to calculate the activity similarity Sim(act1,act2):

[0066]

[0067] Among them, Sim represents similarity; act1 and act2 are two activities, attr(1), attr(2), and attr(3) are the activity name, activity type, and activity description respectively, balid_num represents the number of valid attribute pairs, and attr(i) represents the activity attribute; act1.attr(i) represents the i-th attribute of activity 1; act2.attr(i) represents the i-th attribute of activity 2. If either act1.attr(i) or act2.attr(i) is empty, then Sim(act1.attr(i), act2.attr(i)) = 0 and attr(i) is invalid. Specifically, we use the Levenshtein edit distance to calculate the similarity of the activity name and activity description, and use exact string matching to calculate the similarity of the activity type. The Chinese translation of Levenshtein is Levinstein.

[0068] We need to find the optimal match between two workflow activities to calculate the workflow similarity, which can be modeled as an optimization problem:

[0069]

[0070]

[0071] Among them, Act A and Act B are the activity sets of workflow A and workflow B respectively, a represents an activity in workflow A, b represents an activity in workflow B, max represents maximization, s represents the value of the subsequent similarity summation; s.t. is the abbreviation of such that, which is a common mathematical symbol in optimization problems, followed by the satisfied conditions; at most one edge in M is incident upon t means that at most only one edge in M is connected to t, and t represents an element in the union of Act A and Act B The union, represents arbitrary.

[0072] Sim(a, b) can be calculated by the activity similarity formula. We try to find a match to maximize s. This problem can be solved by algorithms such as the greedy algorithm, the Kuhn - Munkres algorithm (also called the maximum weight matching algorithm), and the non - crossing maximum weight matching. The maximized similarity and s (i.e., the maximum weight matching) will be applied to the calculation of the activity set similarity and the path set similarity later.

[0073] 2. Workflow similarity calculation: Based on the granularity of the workflow topology from small to large, we can divide the workflow similarity calculation methods into three categories: activity set similarity, path set similarity, and global similarity.

[0074] (1) Activity Set (AS) Similarity: The workflow model is regarded as an activity set. The maximum weight matching algorithm is used to find the maximum weight matching between two activity sets. After regularization, the similarity between the two workflow models is obtained.

[0075] (2) Path Set (PS) Similarity: The workflow model is regarded as a path set. First, the non-cross maximum weight matching algorithm is used to calculate the similarity between any two paths in the two path sets. Then, the maximum weight matching algorithm is used to find the maximum weight matching between the two path sets (that is, the similarity between any two paths is combined with the maximum weight matching algorithm to find the maximum weight matching between the two path sets). After regularization, the similarity between the two workflow models is obtained.

[0076] (3) Global Similarity (GS): The workflow model is regarded as a directed graph. The edit distance between two graphs is calculated using the graph edit distance algorithm. After regularization, the similarity between the two workflow models is obtained.

[0077] Evaluation index calculation steps: Calculate x based on structural similarity. 1 and x 2 The similarity difference between them, x 1 and x 2 Whether the workflow comparison models belong to the same group is used as labels, and logistic regression is used to fit and obtain the overall workflow structure similarity evaluation index.

[0078] x 1 and x 2 Difference in similarity between As a feature, x 1 and x 2 Using the comparison models of whether they belong to the same group as labels, logistic regression is used to fit and obtain the weight coefficients w1 for activity set similarity, w2 for path set similarity, and w3 for global similarity, thus obtaining the comprehensive workflow structural similarity evaluation index:

[0079] Sim COM =w l Sim AS +w2Sim PS +w3Sim GS

[0080] Among them, Sim COMSim represents the overall similarity between any two workflows; AS Sim represents the similarity of activity sets between any two workflows; PS Sim represents the path set similarity between any two workflows; GS This represents the global similarity between any two workflows.

[0081] Specifically, x 1 and x 2 Difference in similarity between As a feature, the similarity between the expert evaluations and the query workflow is the same as the label. Logistic Regression is used to fit the data to obtain the overall workflow structure similarity evaluation index COM.

[0082] That is, x 1 and x 2 The similarity difference x between them is used as a feature, i.e. Whether they belong to the same group (tie) in the comparison model is used as the label (0 for in the same group, 1 for not), indicating whether their similarity to the query workflow is the same, i.e., y = 1 or 0; x and y are the features and labels fitted by the logistic regression model below, respectively. Then, the data is fitted using the logistic regression model to obtain the weight coefficients w1, w2, and w3 of the three structural similarities, thus obtaining the comprehensive structural similarity evaluation index (COM), i.e., Sim. COM =w1Sim AS +w2Sim PS +w3Sim GS (Sim is obtained by replacing the calculations between the query workflow and the comparison workflow with the calculations between any two workflows.) AS Sim PS and Sim GS ).

[0083] Logistic regression is a commonly used classification model that is easy to implement and performs well on linearly separable categorical data. It can be represented as:

[0084]

[0085] Where y(x) represents whether the comparison workflow and the query workflow are the same in similarity, θ represents the weight vector, β is the bias term, and e is the natural constant.

[0086] We put x 1 and x 2 The similarity difference between them is used as a feature, that is

[0087]

[0088] We use whether they are in the same group (tie) in the comparison model as labels (0 for being in the same group, 1 for not being in the same group) to indicate whether they have the same similarity to the query workflow, i.e., y = 1 or 0; then we use the logistic regression model to solve the binary classification problem to obtain the weight coefficients θ1, θ2, θ3, which correspond to the three structural similarities.

[0089] Since these three structural similarity methods each emphasize different aspects of workflow structure feature extraction—AS primarily utilizes fine-grained activity semantics but does not consider relationships between activities, PS considers the semantic features of path substructures but lacks a holistic perspective, and GS calculates based on the entire graph structure but ignores activity semantic features—we combine these three methods and calculate the weight coefficients of the three structural similarities through linear normalization.

[0090]

[0091] Among them, w i The weighting coefficient θ represents the structural similarity. i The weighting coefficients represent the structural similarity.

[0092] This yields the Comprehensive Metrics (COM) for workflow structure similarity, i.e., Sim. COM =w1Sim AS +w2Sim PS +w3Sim GS .

[0093] Similarity evaluation steps: Evaluate the similarity between workflows based on the overall workflow structure similarity evaluation index.

[0094] Using this evaluation metric, we can more reasonably and accurately calculate the similarity between any two scientific workflows, which facilitates the generation of a large amount of data for downstream tasks, reduces the manpower and material resources consumed by manual annotation, and has universality and practical value.

[0095] This invention also discloses a workflow similarity evaluation system that simulates expert scoring, such as... Figure 1 and Figure 2 As shown, it includes the following modules:

[0096] Evaluation index acquisition module: Obtain the overall workflow structure similarity evaluation index by simulating expert scoring.

[0097] The evaluation metric acquisition module includes the following modules: Dataset construction module: acquires the expert rating dataset, which includes several workflow query models and workflow comparison models. Each workflow query model corresponds to a predetermined number of workflow comparison models, and the predetermined number of workflow comparison models are sorted and grouped according to the similarity of expert ratings; workflow comparison models in the same group have the same similarity to workflow query models.

[0098] Comparison Pair Building Module: For each workflow query model, a comparison pair (x) is arbitrarily extracted from the corresponding workflow comparison model. 1 ,x 2 ); where x 1 Let x represent the first workflow comparison model. 2 This represents the second workflow comparison model.

[0099] Structural similarity calculation module: Calculates the structural similarity between the workflow comparison model and the workflow query model in each comparison pair.

[0100] Treating the workflow comparison model and workflow query model in each comparison pair as activity sets, and using a bipartite graph matching algorithm to find the maximum weight matching between the two activity sets, we obtain x. 1 Similarity of activity set to workflow query model Get x 2 Similarity of activity set to workflow query model

[0101] Treating the workflow comparison model and workflow query model in each comparison pair as path sets, and using bipartite graph matching and non-crossing bipartite graph matching algorithms, the maximum weight matching between the two path sets is obtained, thus yielding x. 1 Similarity of path sets with workflow query model Get x 2 Similarity of path sets with workflow query model

[0102] Treating the workflow comparison model and workflow query model in each comparison pair as directed acyclic graphs (DAGs), and using the graph edit distance algorithm to calculate the edit distance between the two DAGs, we obtain x. 1 Global similarity with workflow query model Get x 2 Global similarity with workflow query model

[0103] Evaluation index calculation module: Calculates x based on structural similarity. 1 and x 2 The similarity difference between them, x 1 and x2 Whether the workflow comparison models belong to the same group is used as labels, and logistic regression is used to fit and obtain the overall workflow structure similarity evaluation index.

[0104] x 1 and x 2 Difference in similarity between As a feature, x 1 and x 2 Using the comparison models of whether they belong to the same group as labels, logistic regression is used to fit and obtain the weight coefficients w1 for activity set similarity, w2 for path set similarity, and w3 for global similarity, thus obtaining the comprehensive workflow structural similarity evaluation index:

[0105] Sim COM =W1Sim AS +W2Sim PS +W3Sim GS

[0106] Among them, Sim COM Sim represents the overall similarity between any two workflows; AS Sim represents the similarity of activity sets between any two workflows; PS Sim represents the path set similarity between any two workflows; GS This represents the global similarity between any two workflows.

[0107] This invention provides a method for evaluating the similarity of workflows that simulate expert ratings, comprising: Step 1: For each workflow query model in the expert rating dataset, randomly extract two comparison models to form a comparison pair (x... 1 x 2 Step 2: For each workflow model in the comparison pair, calculate their similarity to the query model's Activity Set (AS), Path Set (PS), and Global Set (GS) structures respectively; Step 3: Put x 1 and x 2 Difference in similarity between As a feature, the similarity between expert evaluations and the query workflow is the same, which is used as a label. Logistic regression is used to fit the data to obtain the comprehensive workflow structural similarity evaluation index (COM). Using this comprehensive evaluation index, we can more reasonably and accurately calculate the similarity between any two workflows, which facilitates the generation of a large amount of data for downstream tasks, reduces the manpower and material resources consumed by manual annotation, and has universality and practical value.

[0108] When workflow users want to query and retrieve similar workflows, we can use a comprehensive similarity evaluation method to calculate the similarity between workflows in the database and user workflows, and then recommend them to users after sorting and filtering.

[0109] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0110] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A workflow similarity evaluation method that simulates expert scoring, characterized by, The method comprises the following steps: An evaluation index obtaining step: obtaining a workflow comprehensive structure similarity evaluation index by simulating expert scoring; A similarity evaluation step: evaluating the similarity between workflows according to the workflow comprehensive structure similarity evaluation index; The evaluation index obtaining step comprises the following steps: A data set constructing step: obtaining an expert scoring data set, the expert scoring data set comprising a plurality of workflow query models and workflow comparison models; Comparing pairs of constructing steps: for each workflow query model, randomly extracting from the workflow comparison model corresponding to the workflow query model to form a comparing pair ; wherein, represents the first workflow comparison model, represents the second workflow comparison model; A structure similarity calculating step: respectively calculating the structure similarity of each workflow comparison model and workflow query model in a comparison pair; Evaluation index calculation step: according to the structural similarity calculation And The difference between the similarity, whether And Is the same group of workflow comparison model as a label, using logistic regression fitting to obtain the workflow comprehensive structural similarity evaluation index; In the structure similarity calculation step, each of the workflow comparison model and the workflow query model in each comparison pair is regarded as an activity set, the maximum weight matching between the two activity sets is calculated by using a bipartite graph matching algorithm, and the following is obtained Activity set similarity with the workflow query model , the following is obtained Activity set similarity with the workflow query model ; The workflow comparison model and the workflow query model in each comparison pair are regarded as path sets respectively, and the maximum weight matching between the two path sets is obtained by using bipartite graph matching and non-crossing bipartite graph matching algorithm, so as to obtain The path set similarity with the workflow query model , so as to obtain The path set similarity with the workflow query model ; The workflow comparison model and the workflow query model in each comparison pair are respectively regarded as directed acyclic graphs, and the edit distance between the two directed acyclic graphs is calculated by using a graph edit distance algorithm to obtain The global similarity of the workflow query model , to obtain The global similarity of the workflow query model ; In the evaluation index calculation step, and Difference in similarity between = As a feature, and Whether the workflows belong to the same group is used as a label, and logistic regression is used to fit and obtain the weight coefficients of the activity set similarity. Weight coefficients of path set similarity Weight coefficients of global similarity Thus, the overall workflow structure similarity evaluation index is obtained: wherein, represents the overall similarity between any two workflows; represents the activity set similarity between any two workflows; represents the path set similarity between any two workflows; represents the global similarity between any two workflows.

2. The workflow similarity evaluation method of analog expert scoring according to claim 1, characterized in that, In the data set constructing step, each workflow query model corresponds to a predetermined number of workflow comparison models, and the predetermined number of workflow comparison models are grouped according to expert scoring similarity ranking; the similarity of the workflow comparison models and the workflow query model in the same group is the same.

3. A workflow similarity evaluation system that simulates expert scoring, characterized by, The method comprises the following modules: An evaluation index obtaining module: obtaining a workflow comprehensive structure similarity evaluation index by simulating expert scoring; A similarity evaluation module: evaluating the similarity between workflows according to the workflow comprehensive structure similarity evaluation index; The evaluation index obtaining module comprises the following modules: A data set constructing module: obtaining an expert scoring data set, the expert scoring data set comprising a plurality of workflow query models and workflow comparison models; Comparative pair of construction modules: for each workflow query model, randomly extract from the workflow comparison model corresponding to the workflow query model to form a comparative pair ; wherein, represents the first workflow comparison model, represents the second workflow comparison model; A structure similarity calculating module: respectively calculating the structure similarity of each workflow comparison model and workflow query model in a comparison pair; Evaluation index calculation module: according to the structural similarity calculation And The difference between the similarity, whether And It is the same group of workflow comparison model as a label, using logistic regression fitting to obtain the workflow comprehensive structural similarity evaluation index; In the structure similarity calculation module, each comparison pair of workflow comparison model and workflow query model is regarded as an activity set respectively, and the maximum weight matching between the two activity sets is calculated by using bipartite graph matching algorithm to obtain the activity set similarity with the workflow query model , obtaining the activity set similarity with the workflow query model ; The workflow comparison model and the workflow query model in each comparison pair are regarded as path sets respectively, and the maximum weight matching between the two path sets is obtained by using bipartite graph matching and non-crossing bipartite graph matching algorithm, so as to obtain The path set similarity with the workflow query model , so as to obtain The path set similarity with the workflow query model ; The workflow comparison model and the workflow query model in each comparison pair are respectively regarded as directed acyclic graphs, and the edit distance between the two directed acyclic graphs is calculated by using a graph edit distance algorithm to obtain The global similarity with the workflow query model , obtaining The global similarity with the workflow query model ; In the evaluation index calculation module, the similarity difference between and is calculated as = As a feature, the workflow comparison model of whether and are the same group is taken as a label, and the weight coefficients of the activity set similarity, the path set similarity and the global similarity are obtained by using logistic regression fitting , and , so as to obtain the workflow comprehensive structure similarity evaluation index. wherein, represents the overall similarity between any two workflows; represents the activity set similarity between any two workflows; represents the path set similarity between any two workflows; represents the global similarity between any two workflows.

4. The workflow similarity evaluation system for analog expert scoring of claim 3, wherein, In the data set constructing module, each workflow query model corresponds to a predetermined number of workflow comparison models, and the predetermined number of workflow comparison models are grouped according to expert scoring similarity ranking; the similarity of the workflow comparison models and the workflow query model in the same group is the same.

Images