Hierarchical workflow automatic generation method and system based on large language model

Abstract

The invention discloses a large language model LLM-based hierarchical workflow automatic generation method. The method comprises the following steps: firstly, processing natural language task description input by a user through a pre-trained large language model to obtain a structured task component; then, related workflow modes and optimal practices are retrieved from a pre-constructed domain knowledge base based on the structured task component, and the workflow modes and the optimal practices are fused to form knowledge-enhanced workflow description; based on the knowledge-enhanced workflow description, automatically converting the knowledge-enhanced workflow description into a standardized directed acyclic graph (DAG) through the steps of node extraction, type classification, dependency analysis, structure verification and the like; and finally, performing quality evaluation and iterative optimization on the generated DAG, and finally realizing end-to-end automatic generation from a natural language to an executable workflow. The technical problems that an existing workflow generation method based on template matching seriously depends on a preset template, is insufficient in flexibility and is difficult to adapt to complex and changeable user requirements can be solved.

Description

Technical Field

[0001] The present invention belongs to the fields of artificial intelligence, workflow automation, scientific computing, and machine learning technology, and more specifically, relates to a method and system for automatically generating hierarchical workflows based on a large language model (LLM). Background Art

[0002] In recent years, with the rapid development of artificial intelligence (AI) and the widespread application of cloud computing and big data, workflow automation has become an important means of improving computing efficiency and reducing development costs. Workflow management systems (such as Apache Airflow, Nextflow, and Kubernetes) have been widely used in scientific computing, data processing, and machine learning. Furthermore, large language models (LLMs) such as GPT, Claude, and DeepSeek have demonstrated remarkable capabilities in code generation and natural language understanding, providing a new technical path for intelligent workflow generation. Therefore, a LLM-assisted workflow automation approach offers a promising option for improving computing efficiency and reducing development costs, and holds significant practical significance.

[0003] Currently, there are three main traditional workflow generation methods. The first method is to use a workflow generation method based on template matching, which generates a workflow model through a predefined workflow template and a simple LLM call; the second method is to generate direct code based on LLM, which is optimized for workflow generation represented by a single code snippet or function; the third method is to use traditional visual workflow design tools to generate workflows, which provides a relatively simple and easy-to-use visual interface and a relatively complete workflow generation function.

[0004] However, the above three methods all have some drawbacks that cannot be ignored:

[0005] First, the workflow generation method based on template matching relies heavily on preset templates, which is insufficiently flexible and difficult to adapt to complex and changing user needs. In addition, the generated workflow lacks domain expertise guidance and is difficult to meet the performance and specification requirements of actual applications.

[0006] Second, the direct code generation method based on LLM is mainly aimed at single-step code completion and lacks the ability to systematically decompose and organize complex multi-step tasks. The generated code is in text form and cannot be directly converted into the structured representation required by workflow management systems.

[0007] Third, the method of generating workflows using traditional visual workflow design tools requires users to have professional workflow modeling skills and domain knowledge, which has a high learning threshold. In addition, the entire process of designing workflows relies entirely on manual operations, with a low degree of automation and low development efficiency. Summary of the Invention

[0008] In response to the above defects or improvement needs of the prior art, the present invention provides a method and system for automatically generating hierarchical workflows based on a large language model, which aims to solve the technical problems that the existing workflow generation method based on template matching relies heavily on preset templates, lacks flexibility, is difficult to adapt to complex and changeable user needs, and the workflows generated lack domain expertise guidance and are difficult to meet the performance and specification requirements of actual applications; and the existing method of directly generating code based on LLM lacks the ability to systematically decompose and organize multi-step complex tasks, and the generated results are text-based codes that cannot be directly converted into the structured representation required by the workflow management system; and the existing method of generating workflows using traditional visual workflow design tools requires users to have professional workflow modeling skills and domain knowledge, has a high learning threshold, and the entire process of designing workflows relies entirely on manual operations, has a low degree of automation, and has low development efficiency.

[0009] To achieve the above object, according to one aspect of the present invention, a method for automatically generating a hierarchical workflow based on a large language model is provided, comprising the following steps:

[0010] (1) The server receives the natural language task description input by the user and processes the natural language task description through a pre-trained large language model to obtain the task component corresponding to the natural language task;

[0011] (2) The server uses the task components corresponding to the natural language task obtained in step (1) and adopts the retrieval enhancement generation technology RAG to retrieve the corresponding professional knowledge from the pre-built domain knowledge base, and integrates the retrieved professional knowledge with the task components corresponding to the natural language task into a knowledge-enhanced workflow description;

[0012] (3) The server uses a graph construction algorithm based on dependency parsing to convert the knowledge-enhanced workflow description obtained in step (2) into a standardized directed acyclic graph (DAG), where the nodes of the DAG represent the individual task steps in the workflow description obtained in step (2), and the directed edges represent the dependency relationships between the individual task steps.

[0013] (4) The server uses a topological sorting algorithm to verify the DAG generated in step (3), and uses a path optimization algorithm to schedule the verified DAG to obtain the final executable workflow.

[0014] Preferably, the large language model is an autoregressive language model based on the Transformer architecture, which includes a word embedding layer, a multi-head self-attention layer, a feedforward neural network layer, and an output decoding layer. The specific structure is as follows:

[0015] The first layer is the word embedding layer, whose input is a word sequence of length S that has been processed by word segmentation. This layer first maps the word sequence to a word embedding matrix of [S, d_model] dimensions through a vocabulary of size V, and processes the word embedding matrix using a sine position encoding function or a cosine position encoding function to generate a position encoding matrix of [S, d_model] dimensions. Finally, the position encoding matrix is ​​added element by element to the word embedding matrix to output an [S, d_model]-dimensional embedding representation matrix containing position information, where S is the number of words generated after the input text is processed by the word segmenter, d_model is the preset feature dimension, the position index refers to the absolute position of each element in the word embedding matrix, which ranges from 0 to S-1, and V is the total number of different words contained in the vocabulary constructed by the word segmenter;

[0016] The second layer is a multi-head self-attention layer, whose input is the [S, d_model]-dimensional embedding representation matrix obtained in the first layer. This layer first processes the embedding representation matrix through three linear transformation matrices W_Q, W_K, and W_V to obtain the query matrix Q, key matrix K, and value matrix V respectively. Then, the attention outputs corresponding to these three matrices are obtained through the attention calculation formula. Subsequently, the three attention outputs are concatenated by dimension to form a [S, d_model]-dimensional matrix. Subsequently, the [S, d_model]-dimensional matrix is ​​linearly transformed by the output projection matrix W_O to output a [S, d_model]-dimensional feature matrix containing global attention information;

[0017] The third layer is a feedforward neural network layer, whose input is the [S, d_model]-dimensional feature matrix obtained in the second layer. This layer first performs a residual connection on the [S, d_model]-dimensional feature matrix and the [S, d_model]-dimensional embedding representation matrix obtained in the first layer, and then normalizes the result of the residual connection to obtain a residual feature matrix. Then, the dimension of the residual feature matrix is ​​expanded to d_ff through the first linear transformation. Thereafter, the dimension-expanded residual feature matrix is ​​processed by the nonlinear activation function GELU. Subsequently, the residual feature matrix after the nonlinear activation processing is processed by the second linear transformation to change its dimension to d_model, thereby obtaining the deep feature matrix after the nonlinear transformation. Finally, the normalized residual feature matrix is ​​added element by element to the deep feature matrix after the nonlinear transformation, and the addition result is normalized to output a coding representation matrix of [S, d_model] dimensions, where d_ff is 4 times of d_model.

[0018] The fourth layer is the output decoding layer, whose input is the [S, d_model]-dimensional encoding representation matrix obtained in the third layer. This layer first maps each feature dimension in the encoding representation matrix to [S, V] dimensions through the linear transformation matrix W (obtained by gradient descent learning during the training process), thereby obtaining the [S, V]-dimensional encoding representation matrix. Then, the Softmax function is used to obtain the probability distribution of the word corresponding to each element in the S dimension of the [S, V]-dimensional encoding representation matrix in the vocabulary. Finally, the [S, V]-dimensional word probability distribution matrix composed of the probability distribution of all word units is output.

[0019] Preferably, the large language model is trained by the following steps:

[0020] (1-1) Obtain a text corpus, use a tokenizer to construct a vocabulary corresponding to the text corpus, assign a unique index to each element in the vocabulary, and set the hyperparameters of the large language model, including feature dimension d_model = 1024, number of attention heads h = 16, feedforward layer hidden dimension d_ff = 4096, and maximum sequence length S = 2048;

[0021] (1-2) Use a tokenizer to process the text corpus obtained in step (1-1) into a set D_standard consisting of multiple word sequences [w_1, w_2, ..., w_S]. Specifically, word sequences with a length less than S are padded to a length of S, and word sequences with a length greater than S are truncated to a length of S. The set D_standard is then divided into a training set D_train and a test set D_test in a ratio of 8:2.

[0022] (1-3) Initializing the parameters of the large language model to obtain the large language model after the parameters are initialized;

[0023] (1-4) For each sample in the training set D_train obtained in step (1-2), the sample is input into the word embedding layer of the large language model after the parameters are initialized in step (1-3) to obtain the [S, d_model]-dimensional embedding representation matrix X corresponding to the sample containing the position information;

[0024] (1-5) For each sample in the training set D_train obtained in step (1-2), the [S, d_model]-dimensional embedding representation matrix X corresponding to the sample obtained in step (1-4) is input into the multi-head self-attention layer of the large language model after the parameters are initialized in step (1-3) to obtain the [S, d_model]-dimensional feature matrix H_1 corresponding to the sample, which contains the global attention information;

[0025] (1-6) For each sample in the training set D_train obtained in step (1-2), the [S, d_model]-dimensional feature matrix H_1 corresponding to the sample obtained in step (1-5) is input into the feedforward neural network layer in the large language model after the parameters are initialized in step (1-3) to obtain the [S, d_model]-dimensional encoding representation matrix H_2 corresponding to the sample.

[0026] (1-7) For each sample in the training set D_train obtained in step (1-2), the [S, d_model]-dimensional encoding representation matrix H_2 corresponding to the sample obtained in step (1-6) is input into the output decoding layer of the large language model after the parameters are initialized in step (1-3) to obtain the [S, |V|]-dimensional vocabulary probability distribution matrix P corresponding to the sample;

[0027] (1-8) For each sample in the training set D_train obtained in step (1-2), the cross entropy loss value is calculated based on the [S,|V|]-dimensional vocabulary probability distribution matrix P corresponding to the sample in the training set D_train obtained in step (1-7) (which is used to measure the difference between the model prediction and the true label);

[0028] (1-9) For each sample in the training set D_train obtained in step (1-2), the large language model is iteratively trained using the back propagation algorithm according to the cross entropy loss value corresponding to the sample obtained in step (1-8) until the large language model reaches a preset number of iterations (100 times in the present invention), and the optimal parameters of the large language model at this time are obtained, thereby obtaining a preliminarily trained large language model;

[0029] Specifically, in this step, the gradient is calculated layer by layer through the back propagation algorithm: from the gradient of the fourth layer Start by calculating the gradient of the third layer The second gradient The first layer of gradient The Adam optimizer is used to update all parameters according to the gradient. The learning rate during the update is adjusted using the cosine annealing strategy. Finally, the training cycle of steps (1-3) to (1-7) is repeated until the loss converges to the minimum value, and a trained large language model is obtained.

[0030] (1-10) Using the test set obtained in step (1-2), the large language model preliminarily trained in step (1-9) is tested until the detection accuracy reaches the optimal level, thereby obtaining the final trained large language model.

[0031] Preferably, steps (1-3) are as follows: using Xavier to initialize the embedding parameter matrix E of the first layer, using the normal distribution N (0, 0.02 2 ) Initialize the attention weight matrices W_Q, W_K, W_V and the output projection matrix W_O of the second layer, use Kaiming initialization to process the feedforward network weight matrices W_1 and W_2 of the third layer, and use zero-mean normal distribution to initialize the linear transformation matrix W of the fourth layer.

[0032] Preferably, steps (1-4) are specifically:

[0033] First, each element w_i in the word-gram sequence in the sample is input into the word embedding layer, and a search operation is performed through the word embedding parameter matrix E to obtain the embedding representation matrix X_embed corresponding to the word-gram sequence; the specific process is as follows: first, the vector E[w_1] corresponding to the element w_1 is taken out from the word embedding parameter matrix E, and the vector E[w_2] corresponding to the element w_2 is taken out from the word embedding parameter matrix, ..., and so on, until the vector E[w_S] corresponding to the last element w_S is taken out; then, these S vectors are stacked in the order of retrieval to form the embedding representation matrix X_embed = [E[w_1], E[w_2], ..., E[w_S]] of the word-gram sequence, where i represents the position index of the element in the word-gram sequence, and i∈[1, S].

[0034] Then, the position encoding matrix P_E of the word sequence is calculated. Specifically, if the position index of the i-th element in the word sequence is an odd number, the cosine function is calculated for the element, and the formula is sin If the position index of the i-th element in the word sequence is an even number, the sine function is calculated for the element, and the formula is

[0035] Afterwards, the embedding representation matrix X_embed is added element by element to the position encoding matrix P_E to obtain the [S, d_model]-dimensional embedding representation matrix X corresponding to the sample, which contains the position information.

[0036] Preferably, steps (1-5) are specifically:

[0037] First, perform matrix multiplication on the embedding representation matrix X with the three weight matrices W_Q, W_K, and W_V to obtain the query matrix Q = X·W_Q, the key matrix K = X·W_K, and the value matrix V = X·W_V.

[0038] Then, according to the number of attention heads h in the multi-head self-attention layer, the query matrix Q, key matrix K and value matrix V are reshaped to obtain the corresponding sub-matrices Q_1, Q_2, ..., Q_h, K_1, K_2, ..., K_h, and V_1, V_2, ..., V_h respectively; the specific process is to reshape the query matrix Q, key matrix K and value matrix V from [S, d_model] dimensions to [S, h, d_k], where Then, from the reshaped query matrix Q, key matrix K and value matrix V, we extract the sub-matrices Q_1, Q_2, ..., Q_h, K_1, K_2, ..., K_h and V_1, V_2, ..., V_h corresponding to the h attention heads, respectively. The dimension of each sub-matrix is ​​[S, d_k];

[0039] Then, for each attention head t (where t∈[1,h]), the corresponding query submatrix Q_t, key submatrix K_t and value submatrix V_t are used to obtain the attention output head_t corresponding to the t-th attention head; the specific process is as follows: first calculate the score matrix Score corresponding to the t-th attention head t =Q t ·K_t T , then, divide the score matrix by the scaling factor To get the scaled score matrix Next, the scaled score matrix Scaled_Score_t is normalized using the softmax function to obtain the attention weight matrix Attention_Weight_t = softamax(Scaled_Score_t, Finally, the attention weight matrix is ​​multiplied by the value matrix V_t to obtain the attention output head_t = Attention_Weight_t V_t corresponding to the t-th attention head;

[0040] Afterwards, the attention outputs corresponding to the h attention heads are concatenated according to the feature dimension to obtain the complete output MultiHead_Output = Concat(head_1, head_2, ..., head_h) containing the global attention information, where the concatenation operation is to sequentially connect the d_k-dimensional feature vectors of the attention outputs head_1, head_2, ..., head_h corresponding to the h attention heads to form a complete d_model-dimensional feature vector;

[0041] Next, perform matrix multiplication on the full output containing global attention information and the output projection matrix W_O to obtain the projected output Projected_Output = MultiHead_Output·W_O;

[0042] Finally, the projected output Projected_Output is residually connected with the embedding representation matrix X, and the result of the residual connection is normalized to obtain the [S, d_model]-dimensional feature matrix H_1 corresponding to the sample, which contains the global attention information.

[0043] Preferably, steps (1-6) are specifically:

[0044] First, the [S, d_model]-dimensional feature matrix H_1 is multiplied by the first feedforward network weight matrix W_1 to obtain the intermediate representation matrix H_temp = H_1·W_1 after dimension expansion. Subsequently, the intermediate representation matrix H_temp is input into the GELU activation function for nonlinear transformation to obtain the activated representation matrix H_activated = GELU(H_temp). Subsequently, the activated representation matrix H_activated is multiplied by the second feedforward network weight matrix W_2 to obtain the feedforward output matrix H_ffn = H_activated·W_2 after dimension restoration. Finally, the feedforward output matrix H_ffn is added element-by-element to the original input [S, d_model]-dimensional feature matrix H_1, and the addition result Residual_Output is normalized H_2 = LayerNorm(Residual_Output) to obtain the [S, d_model]-dimensional encoding representation matrix H_2 corresponding to the sample, where LayerNorm() represents the layer normalization operation.

[0045] The specific calculation process of the layer normalization operation is to first calculate the mean μ and variance σ of the addition result Residual_Output on the feature dimension d_model 2 , and then use the normalization formula where γ∈R^{d_model}, β∈R^{d_model}).

[0046] Preferably, steps (1-7) are specifically:

[0047] First, perform matrix multiplication on the [S, d_model]-dimensional encoding representation matrix H_2 and the linear transformation matrix W to obtain the raw score matrix logits = H_2·W for vocabulary prediction;

[0048] Then, the score vector logits_i corresponding to the position index of each element in the word sequence is extracted from the original score matrix logits, and the score vector logits_i of each position index is input into the softmax function to obtain the probability distribution P_i=softmax(logits_i) of each word corresponding to the position index in the vocabulary;

[0049] Subsequently, all probability distributions are stacked in the order of position index to obtain the complete [S,|V|]-dimensional vocabulary probability distribution matrix P.

[0050] Preferably, step (2) includes the following sub-steps:

[0051] (2-1) converting the task description, function label and constraint conditions in the task component corresponding to the natural language task obtained in step (1) into a task query vector through text encoding;

[0052] (2-2) Using the cosine similarity calculation method, the workflow pattern, best practice specification, and code template closest to the task query vector obtained in step (2-1) are retrieved from the pre-built domain knowledge base. The three together constitute the candidate knowledge set.

[0053] (2-3) Calculate the matching score between each knowledge item in the candidate knowledge set obtained in step (2-2) and the task component corresponding to the natural language task obtained in step (1), sort all the scoring results obtained in descending order, and select the knowledge items corresponding to the top K scoring results from the sorted results to form a selected knowledge set, where the value range of K is 5 to 50, preferably 10; the matching score formula between the mth knowledge item and the nth knowledge item is:

[0054] Score (knowledge item m, task component n) = (vector Km·vector Tn) / (||vector Km||×||vector Tn||),

[0055] Where m∈[1, the total number of knowledge items in the candidate knowledge set obtained in step (2-2)], n∈[1, the total number of knowledge items in the candidate knowledge set obtained in step (2-2)]. The vector Km represents the embedding vector representation of the mth knowledge item, and the vector Tn represents the embedding vector representation of the nth task component.

[0056] (2-4) Using the workflow patterns and best practices in the selected knowledge set obtained in step (2-3) as context information, the context information and the task components corresponding to the natural language task obtained in step (1) are input into a pre-trained large language model to obtain a knowledge-enhanced workflow description.

[0057] According to another aspect of the present invention, a hierarchical workflow automatic generation system based on a large language model is provided, comprising:

[0058] The first module is set on the server and is used to receive a natural language task description input by a user and process the natural language task description through a pre-trained large language model to obtain a task component corresponding to the natural language task;

[0059] The second module is set on the server and is used to use the task components corresponding to the natural language tasks obtained by the first module and the retrieval enhancement generation technology RAG to retrieve the corresponding professional knowledge from the pre-built domain knowledge base, and fuse the retrieved professional knowledge with the task components corresponding to the natural language tasks into a knowledge-enhanced workflow description;

[0060] A third module is provided on the server and is used to convert the knowledge-enhanced workflow description obtained by the second module into a standardized directed acyclic graph (DAG) using a graph construction algorithm based on dependency parsing, wherein the nodes of the DAG represent the individual task steps in the workflow description obtained by the second module, and the directed edges represent the dependency relationships between the individual task steps;

[0061] The fourth module is set on the server and is used to verify the DAG generated by the third module using a topological sorting algorithm, and schedule the verified DAG using a path optimization algorithm to obtain a final executable workflow.

[0062] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0063] (1) The present invention adopts steps (1) and (2), processes the natural language task description through a pre-trained large language model to obtain task components, without relying on preset templates, and adopts retrieval enhancement generation technology (RAG) to retrieve professional knowledge from a pre-built domain knowledge base and integrate it with the task components. Therefore, it can solve the technical problems that the existing workflow generation method based on template matching relies heavily on preset templates, lacks flexibility, is difficult to adapt to complex and changing user needs, and the generated workflow lacks domain expertise guidance and is difficult to meet the performance and specification requirements of actual applications;

[0064] (2) The present invention adopts steps (1-4) to (1-7) and step (2-4), and processes the natural language task description layer by layer through a four-layer large language model architecture (word embedding layer, multi-head self-attention layer, feedforward neural network layer, output decoding layer) and generates a knowledge-enhanced workflow description in combination with a selected knowledge set, thereby achieving a direct conversion from natural language to a structured workflow representation. Therefore, it can solve the technical problem that the existing method of generating direct code based on LLM lacks the systematic decomposition and organization ability of multi-step complex tasks, and the generated result is a text-based code that cannot be directly converted into the structured representation required by the workflow management system;

[0065] (3) Due to the adoption of steps (3) and (4), the present invention only requires the user to input a natural language task description, which is automatically processed by a pre-trained large language model and combined with RAG to generate a workflow description, and then the conversion of the executable workflow is automatically completed with the help of a graph construction algorithm. Therefore, it can solve the technical problems that the existing method of using traditional visual workflow design tools to generate workflows requires users to have professional workflow modeling skills and domain knowledge, which has a high learning threshold, and the entire process of designing workflows relies entirely on manual operation, has a low degree of automation, and has low development efficiency.

[0066] (4) This invention organically combines cutting-edge AI technologies such as large language models, knowledge graphs, and search-enhanced generation with traditional workflow management systems, achieving end-to-end intelligent transformation from natural language understanding to structured process generation. This helps promote the industrial application of AI technology in the field of business process automation and promotes the implementation of artificial intelligence technology from the laboratory to the actual production environment.

[0067] (5) By constructing a domain knowledge base and combining it with RAG technology, the present invention establishes a workflow generation framework that can be continuously learned and optimized. This framework can not only generate customized workflows based on the professional knowledge of different industry fields, but also continuously improve the workflow generation capabilities through continuous updating of the knowledge base and iterative optimization of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] Figure 1 This is a flowchart of the method for automatically generating a hierarchical workflow based on a large language model of the present invention. DETAILED DESCRIPTION

[0069] In order to make the objectives, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely for the purpose of explaining the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below may be combined with each other as long as they do not conflict with each other.

[0070] The basic idea of ​​the present invention is to first process the natural language task description input by the user through a pre-trained large language model to obtain structured task components; then, based on the structured task components, relevant workflow patterns and best practices are retrieved from a pre-built domain knowledge base, and the two are integrated to form a knowledge-enhanced workflow description; then, based on the knowledge-enhanced workflow description, it is automatically converted into a standardized directed acyclic graph (DAG) through steps such as node extraction, type classification, dependency parsing, and structure verification; finally, the generated DAG is quality evaluated and iteratively optimized, ultimately achieving end-to-end automatic generation from natural language to executable workflows.

[0071] like Figure 1 As shown, the present invention provides a method for automatically generating a hierarchical workflow based on a large language model, comprising the following steps:

[0072] (1) The server receives the natural language task description input by the user and processes the natural language task description through a pre-trained large language model to obtain the task component corresponding to the natural language task;

[0073] Specifically, the large language model of the present invention is an autoregressive language model based on the Transformer architecture, which includes a word embedding layer, a multi-head self-attention layer, a feedforward neural network layer, and an output decoding layer. The specific structure is as follows:

[0074] The first layer is the word embedding layer, whose input is a word sequence of length S after word segmentation (S is the number of words generated after the input text is processed by the word segmenter, determined according to the computing resources, and its value range is 512-8192, preferably 2048). This layer first maps the word sequence into a word embedding matrix of [S, d_model] dimensions (where d_model is the preset Feature dimension, its value range is 512-4096, preferably 1024 to ensure sufficient representation capability), and the word unit embedding matrix is ​​processed using the sine position encoding function (when the position index is an even number) or the cosine position encoding function (when the position index is an odd number) to generate a [S, d_model]-dimensional position encoding matrix (where the position index refers to the absolute position of each element in the word unit embedding matrix, which ranges from 0 to S-1). Finally, the position encoding matrix is ​​added element by element to the word unit embedding matrix to output an [S, d_model]-dimensional embedding representation matrix containing position information;

[0075] The second layer is a multi-headed self-attention layer, whose input is the [S, d_model]-dimensional embedding representation matrix obtained in the first layer. This layer first processes the embedding representation matrix through three linear transformation matrices W_Q, W_K, and W_V (these three matrices are learned through the backpropagation algorithm during the training of the large language model, that is, the gradient is calculated according to the loss function, and then the parameter weights are updated by gradient descent) to obtain the query matrix Q, key matrix K, and value matrix V respectively. Then, the attention outputs corresponding to these three matrices are obtained by the attention calculation formula. Subsequently, the three attention outputs are spliced ​​by dimension to form a [S, d_model]-dimensional matrix. Subsequently, the [S, d_model]-dimensional matrix is ​​linearly transformed by the output projection matrix W_O (W_O is obtained by gradient descent optimization during the training of the large language model, that is, the partial derivative of the loss with respect to the embedding representation matrix is ​​calculated, and then the parameters are updated according to the learning rate to obtain the projection matrix W_O) to output the [S, d_model]-dimensional feature matrix containing global attention information;

[0076] The third layer is a feedforward neural network layer, whose input is the [S, d_model]-dimensional feature matrix obtained in the second layer. This layer first performs a residual connection on the [S, d_model]-dimensional feature matrix and the [S, d_model]-dimensional embedding representation matrix obtained in the first layer, and then normalizes the result of the residual connection to obtain a residual feature matrix. Then, the dimension of the residual feature matrix is ​​expanded to d_ff (d_ff is usually 4 times of d_model) through the first linear transformation. Thereafter, the dimension-expanded residual feature matrix is ​​processed by the nonlinear activation function GELU. Subsequently, the residual feature matrix after nonlinear activation processing is processed by the second linear transformation to change its dimension to d_model, thereby obtaining the deep feature matrix after nonlinear transformation. Finally, the residual feature matrix obtained by normalization is added element by element to the deep feature matrix after nonlinear transformation, and the addition result is normalized to output the encoding representation matrix of [S, d_model] dimension.

[0077] The fourth layer is the output decoding layer, whose input is the [S, d_model]-dimensional encoding representation matrix obtained in the third layer. This layer first maps each feature dimension in the encoding representation matrix to [S, V] dimensions through the linear transformation matrix W (obtained by gradient descent learning during the training process), thereby obtaining the [S, V]-dimensional encoding representation matrix. Then, the Softmax function is used to obtain the probability distribution of the word corresponding to each element in the S dimension of the [S, V]-dimensional encoding representation matrix in the vocabulary. Finally, the [S, V]-dimensional word probability distribution matrix composed of the probability distribution of all word units is output.

[0078] Specifically, the large language model of the present invention is trained by the following steps:

[0079] (1-1) Obtain a text corpus, use a tokenizer to construct a vocabulary corresponding to the text corpus (of size |V| = 50,000), assign a unique index to each element in the vocabulary, and set the hyperparameters of the large language model, including feature dimension d_model = 1024, number of attention heads h = 16, feedforward layer hidden dimension d_ff = 4096, and maximum sequence length S = 2048;

[0080] (1-2) Use the word segmenter to process the text corpus obtained in step (1-1) into a set D_standard consisting of multiple word sequences [w_1, w_2, ..., w_S]. Specifically, the word sequences with a length less than S are padded to a length of S (specifically, the end of the word sequence is padded <pad>Mark), truncate the word sequence longer than S to length S (specifically, delete the word at the end), and divide the set D_standard into training set D_train and test set D_test in a ratio of 8:2;

[0081] (1-3) Initializing the parameters of the large language model to obtain the large language model after the parameters are initialized;

[0082] Specifically, this step is to use Xavier to initialize the embedding parameter matrix E of the first layer and use the normal distribution N(0,0.02 2 ) Initialize the attention weight matrices W_Q, W_K, W_V and the output projection matrix W_O of the second layer, use Kaiming initialization to process the feedforward network weight matrices W_1 and W_2 of the third layer, and use zero-mean normal distribution to initialize the linear transformation matrix W of the fourth layer;

[0083] (1-4) For each sample in the training set D_train obtained in step (1-2), the sample is input into the word embedding layer of the large language model after the parameters are initialized in step (1-3) to obtain the [S, d_model]-dimensional embedding representation matrix X corresponding to the sample containing the position information;

[0084] Specifically, this step is as follows: first, each element w_i (where i represents the position index of the element in the word-gram sequence, and i∈[1, S]) in the word-gram sequence in the sample is input into the word embedding layer, and a search operation is performed through the word embedding parameter matrix E to obtain the embedding representation matrix X_embed corresponding to the word-gram sequence (the process is specifically as follows: first, the vector E[w_1] corresponding to the element w_1 is taken out from the word embedding parameter matrix E, and the vector E[w_2] corresponding to the element w_2 is taken out from the word embedding parameter matrix, ..., and so on, until the vector E[w_S] corresponding to the last element w_S is taken out; then, these S vectors are stacked in the order of taking out to form the embedding representation matrix X_embed=[E[w_1],E[w_2],...,E[w_S]] of the word-gram sequence.

[0085] Then, the position encoding matrix P_E of the word sequence is calculated (specifically, if the position index of the i-th element in the word sequence is an odd number, the cosine function is calculated for the element, and the formula is sin If the position index of the i-th element in the word sequence is an even number, the sine function is calculated for the element, and the formula is

[0086] Afterwards, the embedding representation matrix X_embed is added element by element to the position encoding matrix P_E to obtain the [S, d_model]-dimensional embedding representation matrix X corresponding to the sample, which contains the position information.

[0087] The advantage of this step (1-4) is that it combines word embedding with positional encoding to provide a sequence representation containing positional information for the large language model, effectively solving the problem that traditional neural networks cannot perceive word order.

[0088] (1-5) For each sample in the training set D_train obtained in step (1-2), the [S, d_model]-dimensional embedding representation matrix X corresponding to the sample obtained in step (1-4) is input into the multi-head self-attention layer of the large language model after the parameters are initialized in step (1-3) to obtain the [S, d_model]-dimensional feature matrix H_1 corresponding to the sample, which contains the global attention information;

[0089] Specifically, this step involves first performing matrix multiplication on the embedding representation matrix X with the three weight matrices W_Q, W_K, and W_V to obtain the query matrix Q = X·W_Q, the key matrix K = X·W_K, and the value matrix V = X·W_V.

[0090] Then, according to the number of attention heads h in the multi-head self-attention layer, the query matrix Q, key matrix K and value matrix V are reshaped to obtain the corresponding sub-matrices Q_1, Q_2, ..., Q_h, K_1, K_2, ..., K_h, and V_1, V_2, ..., V_h respectively (the process is to reshape the query matrix Q, key matrix K and value matrix V from [S, d_model] dimensions to [S, h, d_k], where Then, from the reshaped query matrix Q, key matrix K and value matrix V, we extract the sub-matrices Q_1, Q_2, ..., Q_h, K_1, K_2, ..., K_h and V_1, V_2, ..., V_h corresponding to the h attention heads, respectively. The dimension of each sub-matrix is ​​[S, d_k]);

[0091] Then, for each attention head t (where t∈[1,h]), the corresponding query submatrix Q_t, key submatrix K_t and value submatrix V_t are used to obtain the attention output head_t corresponding to the t-th attention head (the specific process is to first calculate the score matrix Score corresponding to the t-th attention head t =Q t ·K_t T , then, divide the score matrix by the scaling factor To get the scaled score matrix Next, the scaled score matrix Scaled_Score_t is normalized using the softmax function to obtain the attention weight matrix Attention_Weight_t = softamax(Scaled_Score_t). Finally, the attention weight matrix is ​​multiplied by the value matrix V_t to obtain the attention output head_t = Attention_Weight_t V_t corresponding to the t-th attention head.

[0092] Afterwards, the attention outputs corresponding to the h attention heads are concatenated according to the feature dimension to obtain the complete output MultiHead_Output = Concat(head_1, head_2, ..., head_h) containing the global attention information, where the concatenation operation is to sequentially connect the d_k-dimensional feature vectors of the attention outputs head_1, head_2, ..., head_h corresponding to the h attention heads to form a complete d_model-dimensional feature vector;

[0093] Next, perform matrix multiplication on the full output containing global attention information and the output projection matrix W_O to obtain the projected output Projected_Output = MultiHead_Output·W_O;

[0094] Finally, the projected output Projected_Output is residually connected with the embedding representation matrix X, and the result of the residual connection is normalized to obtain the [S, d_model]-dimensional feature matrix H_1 corresponding to the sample, which contains the global attention information.

[0095] The advantage of this step (1-5) is that the introduction of a multi-head self-attention mechanism in a large language model enables the parallel capture of dependencies between different positions in the sequence. By extracting features from different subspaces through multiple attention heads, the model's ability to understand complex language descriptions is significantly enhanced.

[0096] (1-6) For each sample in the training set D_train obtained in step (1-2), the [S, d_model]-dimensional feature matrix H_1 corresponding to the sample obtained in step (1-5) is input into the feedforward neural network layer in the large language model after the parameters are initialized in step (1-3) to obtain the [S, d_model]-dimensional encoding representation matrix H_2 corresponding to the sample.

[0097] Specifically, this step is as follows: first, the [S, d_model]-dimensional feature matrix H_1 is multiplied by the first feedforward network weight matrix W_1 to obtain the intermediate representation matrix H_temp = H_1·W_1 after dimension expansion. Then, the intermediate representation matrix H_temp is input into the GELU activation function for nonlinear transformation to obtain the activated representation matrix H_activated = GELU (H_temp). Then, the activated representation matrix H_activated is multiplied by the second feedforward network weight matrix W_2 to obtain the feedforward output matrix H_ffn = H_a after dimension restoration. ctivated·W_2, finally, the feedforward output matrix H_ffn is added element by element to the original input [S, d_model]-dimensional feature matrix H_1, and the addition result Residual_Output is normalized H_2=LayerNorm(Residual_Output) to obtain the [S, d_model]-dimensional encoding representation matrix H_2 corresponding to the sample, where LayerNorm() represents the layer normalization operation (the specific calculation process is to first calculate the mean μ and variance σ of the addition result Residual_Output on the feature dimension d_model) 2 , and then use the normalization formula where γ∈R^{d_model}, β∈R^{d_model});

[0098] (1-7) For each sample in the training set D_train obtained in step (1-2), the [S, d_model]-dimensional encoding representation matrix H_2 corresponding to the sample obtained in step (1-6) is input into the output decoding layer of the large language model after the parameters are initialized in step (1-3) to obtain the [S, |V|]-dimensional vocabulary probability distribution matrix P corresponding to the sample;

[0099] Specifically, this step involves first performing a matrix multiplication operation on the [S, d_model]-dimensional encoding representation matrix H_2 and the linear transformation matrix W to obtain the original score matrix logits = H_2 W for vocabulary prediction (where the feature dimension d_model corresponding to the S position index in the [S, d_model]-dimensional encoding representation matrix H_2 is mapped to a vocabulary prediction score vector of vocabulary size |V| dimensions).

[0100] Then, extract the score vector logits_i corresponding to the position index of each element in the word sequence from the original score matrix logits, input the score vector logits_i of each position index into the softmax function to obtain the probability distribution P_i=softmax(logits_i) of each word corresponding to the position index in the vocabulary, and then stack all the probability distributions in the order of position index to obtain the complete [S,|V|]-dimensional vocabulary probability distribution matrix P;

[0101] The advantage of this step (1-7) is that the i-th row of the obtained [S,|V|]-dimensional vocabulary probability distribution matrix P represents the predicted probability distribution of all words in the vocabulary corresponding to the i-th element of the word sequence; the j-th column of the [S,|V|]-dimensional vocabulary probability distribution matrix P represents the predicted probability value of the j-th word in the vocabulary at each position index in the word sequence. The [S,|V|]-dimensional vocabulary probability distribution matrix P can be used for the subsequent loss function to calculate the prediction result.

[0102] (1-8) For each sample in the training set D_train obtained in step (1-2), the cross entropy loss value is calculated based on the [S,|V|]-dimensional vocabulary probability distribution matrix P corresponding to the sample in the training set D_train obtained in step (1-7) (which is used to measure the difference between the model prediction and the true label);

[0103] (1-9) For each sample in the training set D_train obtained in step (1-2), the large language model is iteratively trained using the back propagation algorithm according to the cross entropy loss value corresponding to the sample obtained in step (1-8) until the large language model reaches a preset number of iterations (100 times in the present invention), and the optimal parameters of the large language model at this time are obtained, thereby obtaining a preliminarily trained large language model;

[0104] Specifically, in this step, the gradient is calculated layer by layer through the back propagation algorithm: from the gradient of the fourth layer Start by calculating the gradient of the third layer The second gradient The first layer of gradient The Adam optimizer is used to update all parameters according to the gradient. The learning rate during the update is adjusted using the cosine annealing strategy. Finally, the training cycle of steps (1-3) to (1-7) is repeated until the loss converges to the minimum value, and a trained large language model is obtained.

[0105] (1-10) Using the test set obtained in step (1-2), the large language model preliminarily trained in step (1-9) is tested until the detection accuracy reaches the optimal level, thereby obtaining the final trained large language model.

[0106] The advantage of this step (1) is that it analyzes the user's natural language tasks through a large language model and extracts core elements to generate structured intents. It can decompose complex tasks into independent sub-modules, generate execution steps for each sub-task and analyze the relationship between step dependencies, and finally form structured task components, achieving the purpose of task decomposition and process construction.

[0107] (2) The server uses the task components corresponding to the natural language task obtained in step (1) and adopts the Retrieval-Augmented Generation (RAG) technology to retrieve the corresponding professional knowledge from the pre-built domain knowledge base, and integrates the retrieved professional knowledge with the task components corresponding to the natural language task into a knowledge-enhanced workflow description;

[0108] Specifically, this step includes the following sub-steps:

[0109] (2-1) converting the task description, function label and constraint conditions in the task component corresponding to the natural language task obtained in step (1) into a task query vector through text encoding;

[0110] (2-2) Using the cosine similarity calculation method, the workflow pattern, best practice specification, and code template closest to the task query vector obtained in step (2-1) are retrieved from the pre-built domain knowledge base. The three together constitute the candidate knowledge set.

[0111] Specifically, the domain knowledge base is constructed using vector database technology.

[0112] (2-3) Calculate the matching score between each knowledge item in the candidate knowledge set obtained in step (2-2) and the task component corresponding to the natural language task obtained in step (1), sort all the obtained scoring results in descending order, and select the knowledge items corresponding to the top K scoring results from the sorted results to form a selected knowledge set, where the value of K ranges from 5 to 50, preferably 10;

[0113] Specifically, the matching score formula between the mth knowledge item and the nth knowledge item (where m∈[1, the total number of knowledge items in the candidate knowledge set obtained in step (2-2)], n∈[1, the total number of knowledge items in the candidate knowledge set obtained in step (2-2)]) is:

[0114] Score (knowledge item m, task component n) = (vector Km·vector Tn) / (||vector Km||×||vector Tn||),

[0115] The vector Km represents the embedding vector representation of the mth knowledge item, and the vector Tn represents the embedding vector representation of the nth task component.

[0116] The advantage of the above steps (2-2) to (2-3) is that, by combining the cosine similarity calculation and the matching score between task components, the semantic similarity can be better captured, making the relevant content retrieved from the knowledge base more accurate.

[0117] (2-4) Using the workflow patterns and best practices in the selected knowledge set obtained in step (2-3) as context information, the context information and the task components corresponding to the natural language task obtained in step (1) are input into a pre-trained large language model to obtain a knowledge-enhanced workflow description.

[0118] The purpose of this step is to use the task components corresponding to the natural language task, through text encoding, retrieval screening, matching and sorting, etc., to integrate the professional knowledge of the domain knowledge base and generate knowledge-enhanced workflow descriptions with the help of RAG.

[0119] (3) The server uses a graph construction algorithm based on dependency parsing to convert the knowledge-enhanced workflow description obtained in step (2) into a standardized directed acyclic graph (DAG), where the nodes of the DAG represent the individual task steps in the workflow description obtained in step (2), and the directed edges represent the dependency relationships between the individual task steps;

[0120] The advantage of this step (3) is that the use of a graph construction algorithm based on dependency analysis can make the generated DAG meet the acyclic constraint, ensuring that the final workflow structure generated subsequently is effective and logically clear.

[0121] (4) The server uses a topological sorting algorithm to verify the DAG generated in step (3), and uses a path optimization algorithm to schedule the verified DAG to obtain the final executable workflow.

[0122] Through the above steps, the purpose of using a large language model to accept natural language descriptions and automatically process the natural language descriptions hierarchically, and finally generate an executable workflow is achieved.

[0123] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.< / pad>

Claims

1. A method for automatically generating hierarchical workflows based on a large language model, characterized in that: The following steps are involved: (1) The server receives the natural language task description input by the user and processes the natural language task description through a pre-trained large language model to obtain the task component corresponding to the natural language task; (2) The server uses the task components corresponding to the natural language task obtained in step (1) and adopts the retrieval enhancement generation technology RAG to retrieve the corresponding professional knowledge from the pre-built domain knowledge base, and integrates the retrieved professional knowledge with the task components corresponding to the natural language task into a knowledge-enhanced workflow description; (3) The server uses a graph construction algorithm based on dependency parsing to convert the knowledge-enhanced workflow description obtained in step (2) into a standardized directed acyclic graph (DAG), where the nodes of the DAG represent the individual task steps in the workflow description obtained in step (2), and the directed edges represent the dependency relationships between the individual task steps. (4) The server uses a topological sorting algorithm to verify the DAG generated in step (3), and uses a path optimization algorithm to schedule the verified DAG to obtain the final executable workflow.

2. The method for automatically generating hierarchical workflows based on a large language model according to claim 1, characterized in that: The large language model is an autoregressive language model based on the Transformer architecture. It includes a word embedding layer, a multi-head self-attention layer, a feedforward neural network layer, and an output decoding layer. The specific structure is as follows: The first layer is the word embedding layer, whose input is a word sequence of length S that has been processed by word segmentation. This layer first maps the word sequence to a word embedding matrix of [S, d_model] dimensions through a vocabulary of size V, and processes the word embedding matrix using a sine position encoding function or a cosine position encoding function to generate a position encoding matrix of [S, d_model] dimensions. Finally, the position encoding matrix is ​​added element by element to the word embedding matrix to output an [S, d_model]-dimensional embedding representation matrix containing position information, where S is the number of words generated after the input text is processed by the word segmenter, d_model is the preset feature dimension, the position index refers to the absolute position of each element in the word embedding matrix, which ranges from 0 to S-1, and V is the total number of different words contained in the vocabulary constructed by the word segmenter; The second layer is a multi-head self-attention layer, whose input is the [S, d_model]-dimensional embedding representation matrix obtained in the first layer. This layer first processes the embedding representation matrix through three linear transformation matrices W_Q, W_K, and W_V to obtain the query matrix Q, key matrix K, and value matrix V respectively. Then, the attention outputs corresponding to these three matrices are obtained through the attention calculation formula. Subsequently, the three attention outputs are concatenated by dimension to form a [S, d_model]-dimensional matrix. Subsequently, the [S, d_model]-dimensional matrix is ​​linearly transformed by the output projection matrix W_O to output a [S, d_model]-dimensional feature matrix containing global attention information; The third layer is a feedforward neural network layer, whose input is the [S, d_model]-dimensional feature matrix obtained in the second layer. This layer first performs a residual connection on the [S, d_model]-dimensional feature matrix and the [S, d_model]-dimensional embedding representation matrix obtained in the first layer, and then normalizes the result of the residual connection to obtain a residual feature matrix. Then, the dimension of the residual feature matrix is ​​expanded to d_ff through the first linear transformation. Thereafter, the dimension-expanded residual feature matrix is ​​processed by the nonlinear activation function GELU. Subsequently, the residual feature matrix after the nonlinear activation processing is processed by the second linear transformation to change its dimension to d_model, thereby obtaining the deep feature matrix after the nonlinear transformation. Finally, the normalized residual feature matrix is ​​added element by element to the deep feature matrix after the nonlinear transformation, and the addition result is normalized to output a coding representation matrix of [S, d_model] dimensions, where d_ff is 4 times of d_model. The fourth layer is the output decoding layer, whose input is the [S, d_model]-dimensional encoding representation matrix obtained in the third layer. This layer first maps each feature dimension in the encoding representation matrix to [S, V] dimensions through the linear transformation matrix W (obtained by gradient descent learning during the training process), thereby obtaining the [S, V]-dimensional encoding representation matrix. Then, the Softmax function is used to obtain the probability distribution of the word corresponding to each element in the S dimension of the [S, V]-dimensional encoding representation matrix in the vocabulary. Finally, the [S, V]-dimensional word probability distribution matrix composed of the probability distribution of all word units is output.

3. The method for automatically generating hierarchical workflows based on a large language model according to claim 1 or 2, characterized in that: The large language model is trained through the following steps: (1-1) Obtain a text corpus, use a tokenizer to construct a vocabulary corresponding to the text corpus, assign a unique index to each element in the vocabulary, and set the hyperparameters of the large language model, including feature dimension d_model = 1024, number of attention heads h = 16, feedforward layer hidden dimension d_ff = 4096, and maximum sequence length S = 2048; (1-2) Use a tokenizer to process the text corpus obtained in step (1-1) into a set D_standard consisting of multiple word sequences [w_1, w_2, ..., w_S]. Specifically, word sequences with a length less than S are padded to a length of S, and word sequences with a length greater than S are truncated to a length of S. The set D_standard is then divided into a training set D_train and a test set D_test in a ratio of 8:

2. (1-3) Initializing the parameters of the large language model to obtain the large language model after the parameters are initialized; (1-4) For each sample in the training set D_train obtained in step (1-2), the sample is input into the word embedding layer of the large language model after the parameters are initialized in step (1-3) to obtain the [S, d_model]-dimensional embedding representation matrix X corresponding to the sample containing the position information; (1-5) For each sample in the training set D_train obtained in step (1-2), the [S, d_model]-dimensional embedding representation matrix X corresponding to the sample obtained in step (1-4) is input into the multi-head self-attention layer of the large language model after the parameters are initialized in step (1-3) to obtain the [S, d_model]-dimensional feature matrix H_1 corresponding to the sample, which contains the global attention information; (1-6) For each sample in the training set D_train obtained in step (1-2), the [S, d_model]-dimensional feature matrix H_1 corresponding to the sample obtained in step (1-5) is input into the feedforward neural network layer in the large language model after the parameters are initialized in step (1-3) to obtain the [S, d_model]-dimensional encoding representation matrix H_2 corresponding to the sample. (1-7) For each sample in the training set D_train obtained in step (1-2), the [S, d_model]-dimensional encoding representation matrix H_2 corresponding to the sample obtained in step (1-6) is input into the output decoding layer of the large language model after the parameters are initialized in step (1-3) to obtain the [S, |V|]-dimensional vocabulary probability distribution matrix P corresponding to the sample; (1-8) For each sample in the training set D_train obtained in step (1-2), the cross entropy loss value is calculated based on the [S,|V|]-dimensional vocabulary probability distribution matrix P corresponding to the sample in the training set D_train obtained in step (1-7) (which is used to measure the difference between the model prediction and the true label); (1-9) For each sample in the training set D_train obtained in step (1-2), the large language model is iteratively trained using the back propagation algorithm according to the cross entropy loss value corresponding to the sample obtained in step (1-8) until the large language model reaches a preset number of iterations (100 times in the present invention), and the optimal parameters of the large language model at this time are obtained, thereby obtaining a preliminarily trained large language model; Specifically, in this step, the gradient is calculated layer by layer through the back propagation algorithm: from the gradient of the fourth layer Start by calculating the gradient of the third layer The second gradient The first layer of gradient The Adam optimizer is used to update all parameters according to the gradient. The learning rate during the update is adjusted using the cosine annealing strategy. Finally, the training cycle of steps (1-3) to (1-7) is repeated until the loss converges to the minimum value, and a trained large language model is obtained. (1-10) Using the test set obtained in step (1-2), the large language model preliminarily trained in step (1-9) is tested until the detection accuracy reaches the optimal level, thereby obtaining the final trained large language model.

4. The method for automatically generating hierarchical workflows based on a large language model according to any one of claims 1 to 3, characterized in that: Steps (1-3) are as follows: use Xavier to initialize the embedding parameter matrix E of the first layer, and use the normal distribution N (0, 0.02 2 ) Initialize the attention weight matrices W_Q, W_K, W_V and the output projection matrix W_O of the second layer, use Kaiming initialization to process the feedforward network weight matrices W_1 and W_2 of the third layer, and use zero-mean normal distribution to initialize the linear transformation matrix W of the fourth layer.

5. The method for automatically generating hierarchical workflows based on a large language model according to claim 4, characterized in that: Steps (1-4) are specifically: First, each element w_i in the word sequence in the sample is input into the word embedding layer, and a search operation is performed through the word embedding parameter matrix E to obtain the embedding representation matrix X_embed corresponding to the word sequence; The specific process is as follows: first, take out the vector E[w_1] corresponding to the element w_1 from the word embedding parameter matrix E, take out the vector E[w_2] corresponding to the element w_2 from the word embedding parameter matrix, ..., and so on, until the vector E[w_S] corresponding to the last element w_S is taken out; Subsequently, these S vectors are stacked in the order in which they are taken out to form the embedding representation matrix X_embed = [E[w_1], E[w_2], ..., E[w_S]] of the word sequence, where i represents the position index of the element in the word sequence, and i∈[1, S]. Then, the position encoding matrix P_E of the word sequence is calculated. Specifically, if the position index of the i-th element in the word sequence is an odd number, the cosine function is calculated for the element, and the formula is sin If the position index of the i-th element in the word sequence is an even number, the sine function is calculated for the element, and the formula is Afterwards, the embedding representation matrix X_embed is added element by element to the position encoding matrix P_E to obtain the [S, d_model]-dimensional embedding representation matrix X corresponding to the sample and containing the position information.

6. The method for automatically generating hierarchical workflows based on a large language model according to claim 5, characterized in that: Steps (1-5) are specifically: First, perform matrix multiplication on the embedding representation matrix X with the three weight matrices W_Q, W_K, and W_V to obtain the query matrix Q = X·W_Q, the key matrix K = X·W_K, and the value matrix V = X·W_V. Then, according to the number of attention heads h in the multi-head self-attention layer, the query matrix Q, key matrix K and value matrix V are reshaped to obtain the corresponding sub-matrices Q_1, Q_2, ..., Q_h, K_1, K_2, ..., K_h, and V_1, V_2, ..., V_h respectively; the specific process is to reshape the query matrix Q, key matrix K and value matrix V from [S, d_model] dimensions to [S, h, d_k], where Then, from the reshaped query matrix Q, key matrix K and value matrix V, we extract the sub-matrices Q_1, Q_2, ..., Q_h, K_1, K_2, ..., K_h and V_1, V_2, ..., V_h corresponding to the h attention heads, respectively. The dimension of each sub-matrix is ​​[S, d_k]; Then, for each attention head t (where t∈[1,h]), the corresponding query submatrix Q_t, key submatrix K_t and value submatrix V_t are used to obtain the attention output head_t corresponding to the t-th attention head; the specific process is as follows: first calculate the score matrix Score corresponding to the t-th attention head t =Q t ·K_t T , then, divide the score matrix by the scaling factor To get the scaled score matrix Next, the scaled score matrix Scaled_Score_t is normalized using the softmax function to obtain the attention weight matrix Attention_Weight_t = softmax(Scaled_Score_t, Finally, the attention weight matrix is ​​multiplied by the value matrix V_t to obtain the attention output head_t = Attention_Weight_t V_t corresponding to the t-th attention head; Afterwards, the attention outputs corresponding to the h attention heads are concatenated according to the feature dimension to obtain the complete output MultiHead_Output = Concat(head_1, head_2, ..., head_h) containing the global attention information, where the concatenation operation is to sequentially connect the d_k-dimensional feature vectors of the attention outputs head_1, head_2, ..., head_h corresponding to the h attention heads to form a complete d_model-dimensional feature vector; Next, perform matrix multiplication on the full output containing global attention information and the output projection matrix W_O to obtain the projected output Projected_Output = MultiHead_Output·W_O; Finally, the projected output Projected_Output is residually connected with the embedding representation matrix X, and the result of the residual connection is normalized to obtain the [S, d_model]-dimensional feature matrix H_1 corresponding to the sample, which contains the global attention information.

7. The method for automatically generating hierarchical workflows based on a large language model according to claim 6, characterized in that: Steps (1-6) are specifically: First, the [S, d_model]-dimensional feature matrix H_1 is multiplied by the first feedforward network weight matrix W_1 to obtain the intermediate representation matrix H_temp = H_1·W_1 after dimension expansion. Subsequently, the intermediate representation matrix H_temp is input into the GELU activation function for nonlinear transformation to obtain the activated representation matrix H_activated = GELU(H_temp). Subsequently, the activated representation matrix H_activated is multiplied by the second feedforward network weight matrix W_2 to obtain the feedforward output matrix H_ffn = H_activated·W_2 after dimension restoration. Finally, the feedforward output matrix H_ffn is added element-by-element to the original input [S, d_model]-dimensional feature matrix H_1, and the addition result Residual_Output is normalized H_2 = LayerNorm(Residual_Output) to obtain the [S, d_model]-dimensional encoding representation matrix H_2 corresponding to the sample, where LayerNorm() represents the layer normalization operation. The specific calculation process of the layer normalization operation is to first calculate the mean μ and variance σ of the addition result Residual_Output on the feature dimension d_model 2 , and then use the normalization formula where γ∈R^{d_model}, β∈R^{d_model}).

8. The method for automatically generating hierarchical workflows based on a large language model according to claim 7, characterized in that: Steps (1-7) are specifically: First, perform matrix multiplication on the [S, d_model]-dimensional encoding representation matrix H_2 and the linear transformation matrix W to obtain the raw score matrix logits = H_2·W for vocabulary prediction; Then, the score vector logits_i corresponding to the position index of each element in the word sequence is extracted from the original score matrix logits, and the score vector logits_i of each position index is input into the softmax function to obtain the probability distribution P_i=softmax(logits_i) of each word corresponding to the position index in the vocabulary; Subsequently, all probability distributions are stacked in the order of position index to obtain the complete [S,|V|]-dimensional vocabulary probability distribution matrix P.

9. The method for automatically generating hierarchical workflows based on a large language model according to claim 1, characterized in that: Step (2) includes the following sub-steps: (2-1) converting the task description, function label and constraint conditions in the task component corresponding to the natural language task obtained in step (1) into a task query vector through text encoding; (2-2) Using the cosine similarity calculation method, the workflow pattern, best practice specification, and code template closest to the task query vector obtained in step (2-1) are retrieved from the pre-built domain knowledge base. The three together constitute the candidate knowledge set. (2-3) Calculate the matching score between each knowledge item in the candidate knowledge set obtained in step (2-2) and the task component corresponding to the natural language task obtained in step (1), sort all the scoring results obtained in descending order, and select the knowledge items corresponding to the top K scoring results from the sorted results to form a selected knowledge set, where the value range of K is 5 to 50, preferably 10; the matching score formula between the mth knowledge item and the nth knowledge item is: Score (knowledge item m, task component n) = (vector Km·vector Tn) / (||vector Km||×||vector Tn||), Where m∈[1, the total number of knowledge items in the candidate knowledge set obtained in step (2-2)], n∈[1, the total number of knowledge items in the candidate knowledge set obtained in step (2-2)]. The vector Km represents the embedding vector representation of the mth knowledge item, and the vector Tn represents the embedding vector representation of the nth task component. (2-4) Using the workflow patterns and best practices in the selected knowledge set obtained in step (2-3) as context information, the context information and the task components corresponding to the natural language task obtained in step (1) are input into a pre-trained large language model to obtain a knowledge-enhanced workflow description.

10. A hierarchical workflow automatic generation system based on a large language model, characterized by: include: The first module is set on the server and is used to receive a natural language task description input by a user and process the natural language task description through a pre-trained large language model to obtain a task component corresponding to the natural language task; The second module is set on the server and is used to use the task components corresponding to the natural language tasks obtained by the first module and the retrieval enhancement generation technology RAG to retrieve the corresponding professional knowledge from the pre-built domain knowledge base, and fuse the retrieved professional knowledge with the task components corresponding to the natural language tasks into a knowledge-enhanced workflow description; A third module is provided on the server and is used to convert the knowledge-enhanced workflow description obtained by the second module into a standardized directed acyclic graph (DAG) using a graph construction algorithm based on dependency parsing, wherein the nodes of the DAG represent the individual task steps in the workflow description obtained by the second module, and the directed edges represent the dependency relationships between the individual task steps; The fourth module is set on the server and is used to verify the DAG generated by the third module using a topological sorting algorithm, and schedule the verified DAG using a path optimization algorithm to obtain a final executable workflow.

Images