Assembly line task construction method and system, electronic equipment and storage medium

Abstract

The invention provides an assembly line task construction method and system, electronic equipment and a storage medium. The construction method of the assembly line task comprises the steps that a Tekton framework is constructed, and the Tekton framework is composed of a task reference module and a task inline definition module; a first configuration file and a second configuration file are obtained, the first configuration file is a preset configuration file, and the second configuration file is a self-defined configuration file; and according to the first configuration file, the second configuration file and the Tekton framework, generating an assembly line task running instance. According to the invention, the problem of limited expansibility of the assembly line task is solved, so that the assembly line task can quickly adapt to user requirements.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet, in particular to a pipeline task construction method and system, an electronic device and a storage medium. BACKGROUND

[0002] In the related art, the current DevOps (Development Operations) pipeline has the following problems: when there is a special requirement, the pipeline task cannot be customized and extended, and can only be modified and adapted in the back end; in addition, the special requirement needs an iteration cycle to be put into operation, and cannot meet the user's demand quickly. SUMMARY

[0003] The present application aims to at least solve one of the problems in the prior art or related art.

[0004] To this end, a first aspect of the present application provides a pipeline task construction method.

[0005] A second aspect of the present application provides a pipeline task construction system.

[0006] A third aspect of the present application provides an electronic device.

[0007] A fourth aspect of the present application provides a storage medium.

[0008] Therefore, according to the first aspect of the present application, a pipeline task construction method is provided, which comprises: constructing a Tekton framework, wherein the Tekton framework is composed of a task reference module and a task inline definition module; obtaining a first configuration file and a second configuration file, wherein the first configuration file is a preset configuration file, and the second configuration file is a custom configuration file; and generating a pipeline task running instance according to the first configuration file, the second configuration file and the Tekton framework.

[0009] The application provides a pipeline task construction method, which mainly comprises the following steps: first, a Tekton framework is constructed, wherein the Tekton framework is composed of a task reference (TaskRef) module and a task inline definition module (TaskSpec). The Tekton framework is a cloud-native pipeline engine, and the TaskRef and the TaskSpec respectively point to a plurality of independent tasks. The TaskRef, i.e., the task reference module, can refer to an existing independent task resource through a field, and the TaskSpec, i.e., the task inline definition module, can directly embed task logic without an independent task resource. In the related art, the Tekton framework is composed of a plurality of TaskRefs, while in the application, the Tekton framework is composed of a TaskSpec and a TaskRef, thereby improving the flexibility of the Tekton framework and allowing users to customize tasks. Then, a first configuration file and a second configuration file are obtained, wherein the first configuration file is a preset configuration file, and the second configuration file is a customized configuration file, i.e., the second configuration file is written and added by the user. Further, the format of the second configuration file can be YAML (Yet Another Markup Language, another markup language), which includes a plurality of user-defined nodes. Then, a pipeline task running instance is generated according to the first configuration file, the second configuration file and the Tekton framework. Since the first configuration file is a preset configuration file, the first configuration file can be associated with the task reference module in the Tekton framework, and since the second configuration file is a user-defined configuration file, the second configuration file can be associated with the task inline definition module, thereby constructing a pipeline task running instance (Pipeline Run) according to the associated Tekton framework. The application uses the TaskSpec and the TaskRef to jointly construct the Tekton framework, and then uses the Tekton framework, the preset configuration file and the user-defined configuration file to construct the Pipeline Run, thereby expanding the pipeline task and solving the problem of limited expandability of the pipeline task in the related art, so that the pipeline task can quickly adapt to user needs.

[0010] In some technical solutions, optionally, the step of constructing the Tekton framework comprises the following steps: obtaining a configuration file, wherein the configuration file includes a container image, a variable and running logic; performing a formatting operation on the configuration file to obtain a customized task; associating the customized task with the task inline definition module; associating the preset task with the task reference module; and constructing the Tekton framework according to the task inline definition module and the task reference module.

[0011] In the technical scheme, the step of constructing the Tekton framework comprises the following steps: firstly, a configuration file is obtained, wherein the configuration file is a user-defined configuration file, and in the configuration file, a container image (Task.Spec.image) parameter, a variable (variables) parameter and a running logic (plugin) parameter are defined respectively, wherein the Task.Spec.image parameter can meet the needs of different build environments of users, the variables parameter is used for declaring a front-end UI (User Interface) display, and the plugin parameter is used for declaring a real execution logic; after the configuration file is obtained, the configuration file can be kept in a database, then a format operation is performed on the configuration file, so that a custom task related to the configuration file is obtained, then the custom task is associated with a task inline definition module, a preset task is associated with a task reference module, and finally, the Tekton framework is constructed according to the task inline definition module and the task reference module. According to the present application, the custom configuration file is associated with the task inline definition module after the format operation, so that the dynamic parameter transmission is realized, and meanwhile, the storage resource sharing between the custom task and the preset task can be realized through the workspace mechanism of the Tekton framework.

[0012] In some technical schemes, optionally, the step of generating the pipeline task running instance according to the first configuration file, the second configuration file and the Tekton framework comprises the following steps: all nodes in the first configuration file and the second configuration file are extracted; all the nodes are judged respectively; when the node is a custom node, it is judged whether the node is a task that needs to be sent; if the node is a task that does not need to be sent, it is converted into a no-task mode; if the node is a task that needs to be sent, a task inline definition is created according to the node; when the node is not a custom node, a task reference is created according to the node and a database; the task inline definition is input into the task inline definition module, and the task reference is input into the task reference module, so that the pipeline task running instance is obtained.

[0013] In the technical solution, the step of generating the pipeline task running instance according to the first configuration file, the second configuration file and the Tekton framework comprises: firstly extracting all nodes in the first configuration file and the second configuration file, then traversing all nodes, respectively judging all nodes to determine whether the node is a custom node, if the node is a custom node, judging whether the node is a task to be sent, the task to be sent refers to whether the task needs to be sent to a user, if the node is a task not to be sent, converting to a no-task mode, that is, directly ignoring the node. If the node is a task to be sent, creating a task inline definition (TaskSpec) according to the node. If the node is not a custom node, it means that the node is a preset node, then directly creating a task reference (TaskRef) according to the node and a database storing a pre-stored configuration file, then respectively inputting the task inline definition into a task inline definition module and inputting the task reference into a task reference module to obtain the pipeline task running instance. The application dynamically selects a task type (custom node or standard node) according to the node input from the front end, thereby realizing the adaptation of dynamic nodes.

[0014] In some technical solutions, optionally, the pipeline task construction method further comprises: receiving a plug-in package uploaded by a user when the pipeline task running instance is running, wherein the plug-in package comprises a project directory, an execution script and a plug-in declaration file; mapping the plug-in package to the Tekton framework by using a network attached storage technology; and updating the pipeline task running instance according to the Tekton framework to obtain a new pipeline task running instance.

[0015] In the technical solution, the pipeline task construction method further comprises: receiving a plug-in package uploaded by a user when the pipeline task running instance is running, wherein the plug-in package comprises a project directory, an execution script and a plug-in declaration file; mapping the plug-in package to the Tekton framework by using a network attached storage technology; and updating the pipeline task running instance according to the Tekton framework to obtain a new pipeline task running instance.

[0016] According to a second aspect of the present application, a pipeline task construction system is provided, comprising a first processing module configured to construct a Tekton framework, wherein the Tekton framework is composed of a task reference module and a task inline definition module; a first acquisition module configured to acquire a first configuration file and a second configuration file, wherein the first configuration file is a preset configuration file, and the second configuration file is a custom configuration file; and a second processing module configured to generate a pipeline task running instance according to the first configuration file, the second configuration file, and the Tekton framework.

[0017] The application provides a pipeline task construction system, which mainly comprises a first processing module, a first acquisition module and a second processing module. The first processing module can construct a Tekton framework, wherein the Tekton framework is composed of a task reference (TaskRef) module and a task inline definition module (TaskSpec). The Tekton framework is a cloud-native pipeline engine, and the TaskRef and the TaskSpec respectively point to a plurality of independent tasks. The TaskRef, i.e. the task reference module, can reference an existing independent task resource through a field, and the TaskSpec, i.e. the task inline definition module, can directly embed task logic without an independent task resource. In the related art, the Tekton framework is composed of a plurality of TaskRefs, while in the application, the Tekton framework is composed of a TaskSpec and a TaskRef, thereby improving the flexibility of the Tekton framework and allowing users to customize tasks. Then the first acquisition module acquires a first configuration file and a second configuration file, wherein the first configuration file is a preset configuration file, and the second configuration file is a user-defined configuration file, i.e. the second configuration file is written and added by the user. Further, the format of the second configuration file can be YAML (Yet Another Markup Language, another markup language), which includes a plurality of user-defined nodes. Then the second processing module generates a pipeline task running instance according to the first configuration file, the second configuration file and the Tekton framework. Since the first configuration file is a preset configuration file, the first configuration file can be associated with the task reference module in the Tekton framework, and since the second configuration file is a user-defined configuration file, the second configuration file can be associated with the task inline definition module, thereby constructing a pipeline task running instance (Pipeline Run) according to the associated Tekton framework. The application uses the TaskSpec and the TaskRef to jointly construct the Tekton framework, and then uses the Tekton framework, the preset configuration file and the user-defined configuration file to construct the Pipeline Run, thereby expanding the pipeline task and solving the problem of limited expandability of the pipeline task in the related art, so that the pipeline task can quickly adapt to user needs.

[0018] In some technical solutions, optionally, the first processing module comprises: a second acquisition module, configured to acquire a configuration file, wherein the configuration file comprises a container image, a variable, and a running logic; a third processing module, configured to perform a format operation on the configuration file to obtain a custom task; a first association module, configured to associate the custom task with a task inline definition module; a second association module, configured to associate a preset task with a task reference module; and a fourth processing module, configured to construct a Tekton framework according to the task inline definition module and the task reference module.

[0019] In the technical solution, the first processing module comprises the second acquisition module, the third processing module, the first association module, the second association module, and the fourth processing module. The second acquisition module can acquire a configuration file, wherein the configuration file is a user-defined configuration file, and in the configuration file, a container image (Task.Spec.image) parameter, a variable (variables) parameter, and a running logic (plugin) parameter are defined respectively. The Task.Spec.image parameter can meet the needs of different build environments of a user, the variables parameter is used to declare a front-end UI (User Interface) display, and the plugin parameter is used to declare a real execution logic. After the configuration file is obtained, the configuration file can be kept in a database, and then the third processing module performs a format operation on the configuration file to obtain a custom task related to the configuration file. Then, the first association module associates the obtained custom task with a task inline definition module, the second association module associates a preset task with a task reference module, and finally the fourth processing module constructs a Tekton framework according to the task inline definition module and the task reference module. The application associates the custom configuration file after the format operation with the task inline definition module, thereby achieving the transmission of dynamic parameters, and meanwhile, through the workspace mechanism of the Tekton framework, storage resource sharing between the custom task and the preset task can be achieved.

[0020] In some technical solutions, optionally, the second processing module includes: a first extraction module, the first extraction module is used to extract all nodes in the first configuration file and the second configuration file; a first judgment module, the first judgment module is used to judge all nodes separately; a second judgment module, the second judgment module is used to judge whether the node is a task that needs to be sent based on the fact that the node is a custom node; a fifth processing module, the fifth processing module is used to convert the node to a no-task mode if the node is a task that does not need to be sent; a sixth processing module, the sixth processing module is used to create a task inline definition based on the node if the node is a task that needs to be sent; a seventh processing module, the seventh processing module is used to create a task reference based on the node and the database when the node is not a custom node; an eighth processing module, the eighth processing module is used to input the task inline definition into the task inline definition module, and input the task reference into the task reference module to obtain a pipeline task running instance.

[0021] In this technical solution, the second processing module includes: a first extraction module, a first judgment module, a second judgment module, a fifth processing module, a sixth processing module, a seventh processing module, and an eighth processing module. The first extraction module extracts all nodes from the first and second configuration files and then traverses all nodes. The first judgment module determines whether each node is a custom node. If the node is a custom node, the second judgment module determines whether the node is a task that needs to be sent. A task that needs to be sent refers to whether the task needs to be sent to the user. If the node is a task that does not need to be sent, the fifth processing module switches to no-task mode, ignoring the node. If the node is a task that needs to be sent, the sixth processing module creates an inline task definition (TaskSpec) based on the node. If the node is not a custom node, it indicates that the node is a preset node. The seventh processing module creates a task reference (TaskRef) based on the node and a database stored as pre-stored configuration files. The eighth processing module then inputs the inline task definition into the inline task definition module and the task reference into the task reference module, resulting in a pipeline task execution instance. The present invention realizes the adaptation of dynamic nodes by dynamically selecting the task type (custom node or standard node) according to the node input from the front end.

[0022] In some technical solutions, the pipeline task construction system further comprises: a first receiving module, which is configured to receive a plug-in package uploaded by a user when the pipeline task running instance is running, wherein the plug-in package comprises a project directory, an execution script and a plug-in declaration file; a mapping module, which is configured to map the plug-in package to the Tekton framework by using a network attached storage technology; and an updating module, which is configured to update the pipeline task running instance according to the Tekton framework to obtain a new pipeline task running instance.

[0023] In this technical solution, the pipeline task construction system further comprises: a first receiving module, a mapping module and an updating module. When the pipeline task running instance is running, the first receiving module receives a plug-in package uploaded by a user, wherein the plug-in package comprises a project target, an execution script or an entry file and a plug-in declaration file, and further, the plug-in package can further comprise a second configuration file. Then the plug-in package is parsed and stored in the management platform, then the mapping module maps the plug-in package to the Tekton framework by using a network attached storage (NAS) mounting technology, and finally the updating module updates the pipeline task running instance according to the Tekton framework to obtain a new pipeline task running instance. Thus, on the one hand, the unified storage and isolation of the plug-in execution file are realized by using the NAS mounting, and on the other hand, the real-time updating of the pipeline task running instance is realized.

[0024] According to a third aspect of the present application, an electronic device is provided, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the pipeline task construction method according to any one of the above aspects when executing the computer program.

[0025] The electronic device provided by the present application can implement the technical effects of any one of the above technical solutions, and the steps of the pipeline task construction method are not described here again.

[0026] According to a fourth aspect of the present application, a storage medium is provided, which stores a computer program, wherein the computer program is executable on a processor to implement the steps of the pipeline task construction method according to any one of the above aspects.

[0027] The storage medium provided by the present application can implement the technical effects of any one of the above technical solutions, and the steps of the pipeline task construction method are not described here again.

[0028] Additional aspects and advantages of the present application will become apparent from the following description part, or can be understood by practicing the present application. BRIEF DESCRIPTION OF DRAWINGS

[0029] The above and / or additional aspects and advantages of the present application will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings in which: Figure 1 One of the flowcharts of the pipeline task construction method of one embodiment of the present application is shown; Figure 2 One of the flowcharts of the pipeline task construction method of one embodiment of the present application is shown; Figure 3 One of the flowcharts of the pipeline task construction method of one embodiment of the present application is shown; Figure 4 One of the flowcharts of the pipeline task construction method of one embodiment of the present application is shown; Figure 5 One of the schematic block diagrams of the pipeline task construction system of one embodiment of the present application is shown; Figure 6 One of the schematic block diagrams of the pipeline task construction system of one embodiment of the present application is shown; Figure 7 One of the schematic block diagrams of the pipeline task construction system of one embodiment of the present application is shown; Figure 8 One of the schematic block diagrams of the pipeline task construction system of one embodiment of the present application is shown; Figure 9 One of the schematic block diagrams of the pipeline task construction system of one embodiment of the present application is shown; Figure 10 One of the schematic block diagrams of the pipeline task construction system of one embodiment of the present application is shown; Figure 11 One of the schematic block diagrams of the pipeline task construction system of one embodiment of the present application is shown; Figure 12 One of the schematic block diagrams of the pipeline task construction system of one embodiment of the present application is shown. DETAILED DESCRIPTION

[0030] In order to enable a more clearly understanding of the above-mentioned purposes, features and advantages of the present application, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.

[0031] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present application. However, it will be recognized by one skilled in the art that the present application can be practiced without the specific details described herein. In other instances, well-known methods, procedures, components and circuits have not been described in detail as not to unnecessarily obscure aspects of the present application.

[0032] Figure 1 One of the flowcharts of the method for constructing the pipeline task of one embodiment of the present application is shown. The method comprises: Step 102: Construct a Tekton framework, wherein the Tekton framework is composed of a task reference module and a task inline definition module; Step 104: Obtain a first configuration file and a second configuration file, wherein the first configuration file is a preset configuration file, and the second configuration file is a custom configuration file; Step 106: Generate a pipeline task running instance according to the first configuration file, the second configuration file and the Tekton framework.

[0033] The application provides a pipeline task construction method, which mainly comprises the following steps: first, a Tekton framework is constructed, wherein the Tekton framework is composed of a task reference (TaskRef) module and a task inline definition module (TaskSpec). The Tekton framework is a cloud-native pipeline engine, and the TaskRef and the TaskSpec respectively point to a plurality of independent tasks. The TaskRef, i.e., the task reference module, can reference an existing independent task resource through a field, and the TaskSpec, i.e., the task inline definition module, can directly embed task logic without an independent task resource. In the related art, the Tekton framework is composed of a plurality of TaskRefs, while in the application, the Tekton framework is composed of a TaskSpec and a TaskRef, thereby improving the flexibility of the Tekton framework and allowing users to customize tasks. Then, a first configuration file and a second configuration file are obtained, wherein the first configuration file is a preset configuration file, and the second configuration file is a customized configuration file, i.e., the second configuration file is written and added by the user. Further, the format of the second configuration file can be YAML (Yet Another Markup Language, another markup language), which includes a plurality of user-defined nodes. Then, a pipeline task running instance is generated according to the first configuration file, the second configuration file and the Tekton framework. Since the first configuration file is a preset configuration file, the first configuration file can be associated with the task reference module in the Tekton framework, and since the second configuration file is a user-defined configuration file, the second configuration file can be associated with the task inline definition module, thereby constructing a pipeline task running instance (Pipeline Run) according to the associated Tekton framework. The application uses the TaskSpec and the TaskRef to jointly construct the Tekton framework, and then uses the Tekton framework, the preset configuration file and the user-defined configuration file to construct the Pipeline Run, thereby expanding the pipeline task and solving the problem of limited expandability of the pipeline task in the related art, so that the pipeline task can quickly adapt to user needs.

[0034] Figure 2 A flowchart of the step of constructing the Tekton framework in the pipeline task construction method of one embodiment of the application is shown; wherein the step of constructing the Tekton framework comprises: Step 202: obtaining a configuration file, wherein the configuration file includes a container image, a variable and a running logic; Step 204: performing a formatting operation on the configuration file to obtain a customized task; Step 206: associating the customized task with the task inline definition module; Step 208: associate the preset task with the task reference module; Step 210: construct the Tekton framework according to the task inline definition module and the task reference module.

[0035] In this embodiment, the step of constructing the Tekton framework includes: firstly, obtaining a configuration file, wherein the configuration file is a user-defined configuration file, in which a container image (Task.Spec.image) parameter, a variable (variables) parameter and a running logic (plugin) parameter are defined respectively, wherein the Task.Spec.image parameter can meet the needs of different build environments of the user, the variables parameter is used to declare a front-end UI (User Interface) display, and the plugin parameter is used to declare the real execution logic. After obtaining the configuration file, the configuration file can be kept in the database, and then the configuration file is subjected to a format operation, so as to obtain a custom task related to the configuration file, then the custom task obtained is associated with the task inline definition module, the preset task is associated with the task reference module, and finally the Tekton framework is constructed according to the task inline definition module and the task reference module. The application realizes the dynamic parameter transmission by associating the custom configuration file subjected to the format operation with the task inline definition module, and meanwhile, the storage resource sharing between the custom task and the preset task can be realized through the workspace mechanism of the Tekton framework.

[0036] Figure 3 A flowchart of a step of generating a pipeline task running instance according to a first configuration file, a second configuration file and a Tekton framework in a pipeline task construction method of an embodiment of the application is shown; wherein the step of generating the pipeline task running instance according to the first configuration file, the second configuration file and the Tekton framework includes: Step 302: extract all nodes in the first configuration file and the second configuration file; Step 304: judge all nodes respectively; Step 306: when the node is a custom node, judge whether the node is a task that needs to be sent; Step 308: if the node is a task that does not need to be sent, convert it into a no-task mode; Step 310: if the node is a task that needs to be sent, create a task inline definition according to the node; Step 312: when the node is not a custom node, create a task reference according to the node and the database; Step 314: input the task inline definition into the task inline definition module and input the task reference into the task reference module to obtain the pipeline task running instance.

[0037] In this embodiment, the step of generating the pipeline task running instance according to the first configuration file, the second configuration file and the Tekton framework comprises the following steps: firstly, all nodes in the first configuration file and the second configuration file are extracted, then all nodes are traversed, all nodes are judged respectively, it is determined whether the node is a custom node, if the node is a custom node, it is judged whether the node is a task to be sent, the task to be sent refers to whether the task needs to be sent to the user, if the node is a task not to be sent, it is converted into a no-task mode, that is, the node is directly ignored. If the node is a task to be sent, a task inline definition (TaskSpec) is created according to the node. If the node is not a custom node, it is indicated that the node is a preset node, a task reference (TaskRef) is directly created according to the node and a database storing the pre-stored configuration file, then the task inline definition is input into the task inline definition module and the task reference is input into the task reference module to obtain the pipeline task running instance. The application dynamically selects the task type (custom node or standard node) according to the node input from the front end, thereby realizing the adaptation of the dynamic node.

[0038] Figure 4 Fig. 2 shows a flowchart of a pipeline task construction method according to an embodiment of the application; wherein the pipeline task construction method comprises the following steps: Step 402: during the running of the pipeline task running instance, a plug-in package uploaded by a user is received, wherein the plug-in package comprises a project directory, an execution script and a plug-in declaration file; Step 404: the plug-in package is mapped into the Tekton framework by using a network attached storage technology; Step 406: the pipeline task running instance is updated according to the Tekton framework to obtain a new pipeline task running instance.

[0039] In this embodiment, the pipeline task construction method further includes: receiving a plug-in package uploaded by a user when the pipeline task running instance is running, wherein the plug-in package includes: a project target, an execution script or an entry file, and a plug-in declaration file, and further, the plug-in package can further include a second configuration file. Then the plug-in package is parsed and stored in the management platform, then the plug-in package is mapped to the Tekton framework through NAS (Network Attached Storage, Network Attached Storage) mounting technology, and finally the pipeline task running instance is updated according to the Tekton framework to obtain a new pipeline task running instance. Thus, on the one hand, the unified storage and isolation of the plug-in execution file are realized through NAS mounting, and on the other hand, the real-time updating of the pipeline task running instance is realized.

[0040] Exemplarily, the technical scheme of the present application mainly includes: first, as shown in Figure 9 Pipeline Run (pipeline task running instance) is constructed by using the Tekton framework, the original Task Ref (task reference) references the Task (task) in the cluster to generate the Pipeline Run, and the present application is changed to Task Ref and Task Spec (task inline definition) splicing to form the Pipeline Run, the Task Spec is a Task composed of user-defined, wherein the Task composed of user-defined can be obtained from the data in the relational database management (MySQL) through format (formatting configuration). Then the plugin.yaml (configuration file) file is specified, the user-defined pipeline node capability is developed, and the plug-in package structure is specified, and the plug-in node is specified for publication. Among them, the custom plug-in node utilizes the Task workspace (task workspace) capability to share the working directory and shared storage with other pipeline nodes TaskRef. At the same time, the Task.Spec.image (container image) parameter in the plugin.yaml file meets the needs of different build environments of users, the variables (variables) declare the front-end UI display information, and the plugin (plug-in) declares the real execution logic.

[0041] As shown in Figure 10 The pipeline task construction method further includes: Step 1002: obtaining front-end data; Step 1004: traversing all nodes; Step 1006: determining whether the node is a custom node, if the result is Y, executing step 1008; if the result is N, executing step 1014; Step 1008: judge whether it is no task, if the result is Y, execute step 1010; if the result is N, execute step 1012; Step 1010: hide no task; Step 1012: create task reference; Step 1014: generate task step; Step 1016: query database; Step 1018: pipeline task running instance.

[0042] In this embodiment, first, the front-end data is acquired, wherein the front-end data includes parameters in the first configuration file and the second configuration file, then all nodes are traversed by using Pipeline Node Enums.get Enum Type By Code (devopsTask.get Code()) code, then it is judged whether the code == custum (whether the node is a custom node), if the result is yes (true), it is judged whether it is no task (notask), if the result is yes, the no task is hidden (covert to notask), if the result is no, the task reference is created (create Task Ref). If the node is not a custom node, the database is queried by using iuap-devops-custom-node* code, and the node is assembled to generate the task step (create Task Steps). Finally, the pipeline task running instance (PipelineRun) is generated according to the task reference and the task step.

[0043] Further, as Figure 11As shown in the figure, the plug-in package includes: a project directory (such as-my-plugin-project), an execution script or an entry file (such as-my-script.xx), and a plug-in declaration file (such as-ymscloud-plugin.xml). The execution script or the entry file supports plug-in logic written in any language. The plug-in declaration file defines the plug-in name, version, and parameters (including variables front-end display fields and plugin execution logic). The plug-in package and the execution file are stored in the management platform (iuap-ums-range) after being uploaded and parsed, and then the path of the execution file is mapped to the Tekton framework through NAS mounting, wherein the path mapped to the Tekton framework can be / nas_mount / ymscloud-pipeline / tekton-cache-test / (plug-in code) / execution file. Then, a pipeline task running instance (Pipeline Run) is generated according to the mapped Tekton framework, and the plug-in can be loaded through an environment variable ($(workspace.cache.path) / (plug-in code) / execution file) when the pipeline task running instance runs. The application realizes unified storage and isolation of plug-in execution files through NAS mounting, and dynamically specifies the plug-in execution path by using an environment variable, thereby supporting hot updating.

[0044] Figure 5 A schematic block diagram of a pipeline task construction system of one embodiment of the application is shown; wherein the pipeline task construction system 50 comprises: A first processing module 502 is configured to construct a Tekton framework, wherein the Tekton framework is composed of a task reference module and a task inline definition module; A first acquisition module 504 is configured to acquire a first configuration file and a second configuration file, wherein the first configuration file is a preset configuration file, and the second configuration file is a custom configuration file; A second processing module 506 is configured to generate a pipeline task running instance according to the first configuration file, the second configuration file, and the Tekton framework.

[0045] The pipeline task construction system 50 provided by the application mainly comprises a first processing module 502, a first acquisition module 504 and a second processing module 506. The first processing module 502 can construct a Tekton framework, wherein the Tekton framework is composed of a TaskRef module and a TaskSpec module. The Tekton framework is a cloud-native pipeline engine, and the TaskRef and the TaskSpec respectively point to a plurality of independent tasks. The TaskRef can refer to an existing independent task resource through a field, and the TaskSpec can directly embed task logic without an independent task resource. In the related art, the Tekton framework is composed of a plurality of TaskRefs, and in the application, the Tekton framework is composed of a TaskSpec and a TaskRef, thereby improving the flexibility of the Tekton framework and allowing users to customize tasks. Then the first acquisition module 504 acquires a first configuration file and a second configuration file, wherein the first configuration file is a preset configuration file, and the second configuration file is a user-defined configuration file, i.e., the second configuration file is written and added by the user. Further, the format of the second configuration file can be YAML (Yet Another Markup Language), and the second configuration file includes a plurality of user-defined nodes. Then the second processing module 506 generates a pipeline task running instance according to the first configuration file, the second configuration file and the Tekton framework. Since the first configuration file is a preset configuration file, the first configuration file can be associated with the TaskRef in the Tekton framework, and since the second configuration file is a user-defined configuration file, the second configuration file can be associated with the TaskSpec, thereby constructing a pipeline task running instance (Pipeline Run) according to the associated Tekton framework. The application uses the TaskSpec and the TaskRef to construct the Tekton framework, and then uses the Tekton framework, the preset configuration file and the user-defined configuration file to construct the Pipeline Run, thereby expanding the pipeline task and solving the problem of limited expandability of the pipeline task in the related art, so that the pipeline task can quickly adapt to user needs.

[0046] Figure 6 A schematic block diagram of the first processing module in the pipeline task construction system of one embodiment of the application is shown. The first processing module 502 comprises: The second acquisition module 5022 is configured to acquire a configuration file, wherein the configuration file comprises a container image, a variable and a running logic. The third processing module 5024 is configured to perform a formatting operation on the configuration file to obtain a custom task. The first association module 5026 is configured to associate the custom task with a task inline definition module. The second association module 5028 is configured to associate a preset task with a task reference module. The fourth processing module 5030 is configured to construct a Tekton framework according to the task inline definition module and the task reference module.

[0047] In this embodiment, the first processing module 502 includes a second acquisition module 5022, a third processing module 5024, a first association module 5026, a second association module 5028, and a fourth processing module 5030. The second acquisition module 5022 can acquire a configuration file, wherein the configuration file is a configuration file defined by a user, and in the configuration file, a container image (Task.Spec.image) parameter, a variable (variables) parameter, and a running logic (plugin) parameter are defined respectively. The Task.Spec.image parameter can meet the needs of different build environments of the user, the variables parameter is used to declare a front-end UI (User Interface, user interface) display, and the plugin parameter is used to declare the real execution logic. After obtaining the configuration file, the configuration file can be kept in a database, and then the third processing module 5024 performs a formatting operation on the configuration file to obtain a custom task related to the configuration file. Then, the first association module 5026 associates the obtained custom task with a task inline definition module, the second association module 5028 associates a preset task with a task reference module, and finally the fourth processing module 5030 constructs a Tekton framework according to the task inline definition module and the task reference module. By associating the formatted custom configuration file with the task inline definition module, the present application realizes the transmission of dynamic parameters, and at the same time, through the workspace mechanism of the Tekton framework, the storage resource sharing between the custom task and the preset task can be realized.

[0048] Figure 7 A schematic block diagram of a second processing module in a pipeline task construction system of one embodiment of the present application is shown. The second processing module 506 includes: The first extraction module 5062 is configured to extract all nodes in the first configuration file and the second configuration file. The first judgment module 5064 is configured to judge all nodes respectively. The second judgment module 5066 is configured to judge whether the node is a task that needs to be sent when the node is a custom node. The fifth processing module 5068 is configured to convert into a task-free mode if the node is a task that does not need to be sent. The sixth processing module 5070 is configured to create a task inline definition according to the node if the node is a task that needs to be sent. The seventh processing module 5072 is configured to create a task reference according to the node and a database if the node is not a custom node. The eighth processing module 5074 is configured to input the task inline definition into a task inline definition module and input the task reference into a task reference module to obtain a pipeline task running instance.

[0049] In this embodiment, the second processing module 506 includes a first extraction module 5062, a first judgment module 5064, a second judgment module 5066, a fifth processing module 5068, a sixth processing module 5070, a seventh processing module 5072, and an eighth processing module 5074. The first extraction module 5062 can extract all nodes in the first configuration file and the second configuration file, and then traverse all the nodes. The first judgment module 5064 judges all the nodes respectively to determine whether the node is a custom node. If the node is a custom node, the second judgment module 5066 judges whether the node is a task that needs to be sent. The task that needs to be sent refers to whether the task needs to be sent to the user. If the node is a task that does not need to be sent, the fifth processing module 5068 converts into a task-free mode, that is, directly ignores the node. If the node is a task that needs to be sent, the sixth processing module 5070 creates a task inline definition (TaskSpec) according to the node. If the node is not a custom node, it means that the node is a preset node. The seventh processing module 5072 directly creates a task reference (TaskRef) according to the node and a database stored as a pre-stored configuration file. Then, the eighth processing module 5074 respectively inputs the task inline definition into a task inline definition module and inputs the task reference into a task reference module to obtain a pipeline task running instance. The present application dynamically selects a task type (custom node or standard node) according to the node transmitted from the front end, thereby realizing the adaptation of dynamic nodes.

[0050] Figure 8 FIG. 2 shows a schematic block diagram of a pipeline task construction system according to an embodiment of the present application; wherein the pipeline task construction system 50 further includes: The first receiving module 508 is configured to receive a plug-in package uploaded by a user when the pipeline task running instance is running, wherein the plug-in package includes a project directory, an execution script, and a plug-in declaration file. The mapping module 510 is configured to map the plug-in package into a Tekton framework by using a network attached storage technology. The updating module 512 is configured to update the pipeline task running instance according to the Tekton framework to obtain a new pipeline task running instance.

[0051] In the embodiment, the pipeline task construction system 50 further comprises a first receiving module 508, a mapping module 510, and an updating module 512. When the pipeline task running instance is running, the first receiving module 508 receives a plug-in package uploaded by a user, wherein the plug-in package comprises a project target, an execution script or an entry file, and a plug-in declaration file, and further comprises a second configuration file. The plug-in package is parsed and stored in the management platform, then the mapping module 510 maps the plug-in package to the Tekton framework through a NAS (Network Attached Storage) mounting technology, and finally the updating module 512 updates the pipeline task running instance according to the Tekton framework to obtain a new pipeline task running instance. Thus, on the one hand, the NAS mounting realizes the unified storage and isolation of the plug-in execution file, and on the other hand, the real-time updating of the pipeline task running instance is realized.

[0052] Figure 12 A schematic block diagram of an electronic device is shown, wherein the electronic device 120 comprises a memory 1202, a processor 1204, and a computer program stored in the memory 1202 and executable on the processor 1204, and the processor 1204 implements the steps of the pipeline task construction method of any one of the above when executing the computer program.

[0053] The electronic device 120 provided by the present application, when the processor 1204 executes the computer program, implements the steps of the pipeline task construction method, and can achieve the technical effects of any one of the above embodiments, which will not be repeated.

[0054] An embodiment of the present application provides a storage medium having a computer program stored thereon, and the computer program is executable by a processor to implement the steps of the pipeline task construction method of any one of the above.

[0055] The storage medium provided by the present application, when the computer program is executed by the processor, implements the steps of the pipeline task construction method, and can achieve the technical effects of any one of the above embodiments, which will not be repeated.

[0056] In this specification, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance, unless otherwise expressly specified or limited. Terms such as "connect," "install," and "fix" should be interpreted broadly. For example, "connect" can refer to a fixed connection, a detachable connection, or an integral connection; and can be directly connected or indirectly connected through an intermediary. Those skilled in the art will understand the specific meanings of these terms in the present invention based on specific circumstances.

[0057] Throughout this specification, terms such as "one embodiment," "some embodiments," and "specific embodiments" mean that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0058] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A method for constructing a pipeline task, characterized in that: include: Building a Tekton framework, wherein the Tekton framework consists of a task reference module and a task inline definition module; Obtain a first configuration file and a second configuration file, wherein the first configuration file is a preset configuration file and the second configuration file is a custom configuration file; Generate a pipeline task running instance according to the first configuration file, the second configuration file and the Tekton framework.

2. The method for constructing a pipeline task according to claim 1, characterized in that: The steps for building the Tekton framework include: Obtain a configuration file, wherein the configuration file includes: a container image, variables, and operation logic; Formatting the configuration file to obtain a custom task; Associating the custom task with the task inline definition module; Associating the preset task with the task reference module; The Tekton framework is constructed according to the task inline definition module and the task reference module.

3. The method for constructing a pipeline task according to claim 1, wherein: The step of generating a pipeline task running instance according to the first configuration file, the second configuration file, and the Tekton framework includes: Extracting all nodes in the first configuration file and the second configuration file; Making judgments on all the nodes respectively; When the node is a custom node, determining whether the node is a task that needs to be sent; If the node is a task that does not need to be sent, it is converted to a no-task mode; If the node is a task that needs to be sent, create an inline definition of the task according to the node; When the node is not the custom node, creating a task reference according to the node and a database; The task inline definition is input into the task inline definition module, and the task reference is input into the task reference module to obtain the pipeline task running instance.

4. The method for constructing a pipeline task according to any one of claims 1 to 3, characterized in that: The method for constructing the pipeline task also includes: When the pipeline task running instance is running, receiving a plug-in package uploaded by a user, wherein the plug-in package includes a project directory, an execution script, and a plug-in declaration file; Mapping the plug-in package to the Tekton framework using network attached storage technology; The pipeline task running instance is updated according to the Tekton framework to obtain a new pipeline task running instance.

5. A pipeline task construction system, characterized in that: include: A first processing module, wherein the first processing module is used to build a Tekton framework, wherein the Tekton framework is composed of a task reference module and a task inline definition module; A first acquisition module, configured to acquire a first configuration file and a second configuration file, wherein the first configuration file is a preset configuration file and the second configuration file is a custom configuration file; The second processing module is used to generate a pipeline task running instance according to the first configuration file, the second configuration file and the Tekton framework.

6. The pipeline task construction system according to claim 5, characterized in that: The first processing module includes: A second acquisition module, configured to acquire a configuration file, wherein the configuration file includes: a container image, variables, and operation logic; a third processing module, configured to format the configuration file to obtain a custom task; A first associating module, configured to associate the custom task with the task inline definition module; a second associating module, the second associating module being used to associate the preset task with the task reference module; A fourth processing module is used to build the Tekton framework according to the task inline definition module and the task reference module.

7. The pipeline task construction system according to claim 5, characterized in that: The second processing module includes: a first extraction module, configured to extract all nodes from the first configuration file and the second configuration file; A first judgment module, configured to judge all the nodes respectively; A second judgment module, the second judgment module being configured to judge whether the node is a task that needs to be sent based on the fact that the node is a custom node; a fifth processing module, configured to switch the node to a no-task mode if the node is performing a task that does not need to be sent; a sixth processing module, configured to create an inline definition of a task according to the node if the node is a task that needs to be sent; a seventh processing module, configured to create a task reference according to the node and a database when the node is not the custom node; An eighth processing module is used to input the task inline definition into the task inline definition module, input the task reference into the task reference module, and obtain the pipeline task running instance.

8. The system for constructing a pipeline task according to any one of claims 5 to 7, characterized in that: The construction system of the pipeline task also includes: a first receiving module, configured to receive a plug-in package uploaded by a user when the pipeline task running instance is running, wherein the plug-in package includes a project directory, an execution script, and a plug-in declaration file; A mapping module, configured to map the plug-in package to the Tekton framework using a network attached storage technology; An update module, wherein the update module is used to update the pipeline task running instance according to the Tekton framework to obtain a new pipeline task running instance.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the method for constructing a pipeline task according to any one of claims 1 to 4 are implemented.

10. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for constructing a pipeline task according to any one of claims 1 to 4 are implemented.

Images