A pipeline information processing method, apparatus, device, and storage medium

Abstract

The present invention discloses a pipeline information processing method, including: determining a target identifier of a target pipeline to be built and a pipeline building operation instruction according to a pipeline building command, and determining whether a target building image corresponding to the target pipeline is stored in an image repository according to the target identifier; if not, determining a pipeline compilation language according to the pipeline building command, and building the target pipeline according to the pipeline compilation language and the pipeline building operation instruction, and calling a target compilation environment image from the image repository, and running the target pipeline according to the target compilation environment image; determining a target build process directory according to the pipeline building command and first running cache information generated during the running process of the target pipeline; determining a target building image corresponding to the target pipeline according to the target identifier, the target build process directory and the target compilation environment image. It can improve the compilation and building efficiency of the pipeline and improve the resource utilization rate of pipeline building.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of computers, and in particular, to a pipeline information processing method, apparatus, device, and storage medium. Background Art

[0002] Currently, cloud native technology has been widely used. Cloud native is a distributed cloud based on distributed deployment and unified operation and management, and is a set of cloud technology product systems based on technologies such as containers, microservices, and DevOps (Development Operations, software development and operation and maintenance). Among them, DevOps is a general term for a set of processes, methods, and systems used to promote communication, collaboration, and integration between development (application / software engineering), technical operations, and quality assurance (QA) departments. When a DevOps product based on the cloud native system compiles and builds a pipeline, it often uses a container build pipeline. This build method requires the destroyed containers after use and deletes the running cache information during the pipeline operation after the pipeline is built. Therefore, every time the pipeline is built, the build completed by the pipeline needs to be built again, resulting in a long build time of the pipeline, low build efficiency, and low resource utilization rate. Therefore, how to reasonably utilize the running cache information during the pipeline operation, improve the compilation and build efficiency of the pipeline, save the build time of the pipeline, and improve the resource utilization rate of the pipeline build is a problem that needs to be solved. Summary of the Invention

[0003] The present invention provides a pipeline information processing method, apparatus, device, and storage medium, which can improve the compilation and build efficiency of the pipeline, save the build time of the pipeline, and improve the resource utilization rate of the pipeline build.

[0004] According to one aspect of the present invention, there is provided a pipeline information processing method, including:

[0005] According to the pipeline build command, determine the target identifier of the target pipeline to be built and the pipeline build operation instruction, and according to the target identifier, determine whether the target build image corresponding to the target pipeline is stored in the image repository;

[0006] If the target build image corresponding to the target pipeline is not stored in the image repository, determine the pipeline compilation language according to the pipeline build command, and build the target pipeline according to the pipeline compilation language and the pipeline build operation instruction;

[0007] Call the target compilation environment image from the image repository according to the pipeline compilation language, and run the target pipeline according to the target compilation environment image;

[0008] Determine a target build process directory according to the parsing result of the pipeline construction command and the first run cache information generated during the running of the target pipeline;

[0009] Determine a target build image corresponding to the target pipeline according to the target identifier, the target build process directory, and the target compilation environment image, and store the target build image in the image repository.

[0010] According to another aspect of the present invention, there is provided a pipeline information processing device, which includes:

[0011] A target build image query module, configured to determine a target identifier and a pipeline build operation instruction of a target pipeline to be built according to a pipeline construction command, and determine whether a target build image corresponding to the target pipeline is stored in the image repository according to the target identifier;

[0012] A target pipeline construction module, configured to, if not, determine a pipeline compilation language according to the pipeline construction command, and construct a target pipeline according to the pipeline compilation language and the pipeline build operation instruction;

[0013] A target pipeline running module, configured to call a target compilation environment image from the image repository according to the pipeline compilation language, and run the target pipeline according to the target compilation environment image;

[0014] A build process directory determination module, configured to determine a target build process directory according to the parsing result of the pipeline construction command and the first run cache information generated during the running of the target pipeline;

[0015] A target build image determination module, configured to determine a target build image corresponding to the target pipeline according to the target identifier, the target build process directory, and the target compilation environment image, and store the target build image in the image repository.

[0016] According to another aspect of the present invention, there is provided an electronic device, which includes:

[0017] At least one processor; and

[0018] A memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the pipeline information processing method according to any embodiment of the present invention.

[0020] According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing a processor to implement the pipeline information processing method according to any embodiment of the present invention when executed.

[0021] In the technical solution of the embodiment of the present invention, according to the pipeline construction command, the target identifier of the target pipeline to be constructed and the pipeline construction operation instruction are determined. According to the target identifier, it is determined whether the target construction image corresponding to the target pipeline is stored in the image repository; if the target construction image corresponding to the target pipeline is not stored in the image repository, then according to the pipeline construction command, the pipeline compilation language is determined, and according to the pipeline compilation language and the pipeline construction operation instruction, the target pipeline is constructed; according to the pipeline compilation language, the target compilation environment image is called from the image repository, and the target pipeline is run according to the target compilation environment image; according to the parsing result of the pipeline construction command and the first running cache information generated during the running process of the target pipeline, the target construction process directory is determined; according to the target identifier, the target construction process directory and the target compilation environment image, the target construction image corresponding to the target pipeline is determined, and the target construction image is stored in the image repository. The above solution solves the problem that after the construction of the target pipeline is completed, the container used for constructing the target pipeline is destroyed, resulting in the inability to query the construction process of the target pipeline in the subsequent process. When obtaining the pipeline construction command, the target pipeline is constructed according to the pipeline construction command, the target compilation environment image of the target pipeline is determined according to the pipeline compilation language, and the target construction process directory is determined according to the first running cache information generated during the running process of the target pipeline, so as to determine the target construction image corresponding to the target pipeline according to the target identifier, the target construction process directory and the target compilation environment image and store the target construction image in the image repository, so as to facilitate obtaining the target construction process directory corresponding to the first running cache information according to the target construction image in the subsequent process, and quickly reading the construction process of the target pipeline according to the target construction process directory. At the same time, when the target pipeline is constructed again in the subsequent process, the target construction image can be called from the image repository to reconstruct the target pipeline according to the target construction image, so as to improve the construction efficiency of the target pipeline when it is reconstructed again.

[0022] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] To more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for the description of the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0024] Figure 1 It is a flowchart of a pipeline information processing method provided in Embodiment 1 of the present invention;

[0025] Figure 2 It is a flowchart of a pipeline information processing method provided in Embodiment 2 of the present invention;

[0026] Figure 3 It is a flowchart of a pipeline information processing method provided in Embodiment 3 of the present invention;

[0027] Figure 4 It is a schematic structural diagram of a pipeline information processing device provided in Embodiment 4 of the present invention;

[0028] Figure 5 It is a schematic structural diagram of an electronic device provided in Embodiment 5 of the present invention. Detailed implementation manners

[0029] In order to enable those skilled in the art to better understand the solutions of the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0030] It should be noted that the terms "candidate", "target", "first", "second", etc. in the specification and claims of the present invention and the above accompanying drawings are used to distinguish similar objects, and do not necessarily need to describe a specific order or sequence. It should be understood that such used data can be interchanged under appropriate circumstances so that the embodiments of the present invention described here can be implemented in an order different from those illustrated or described here. In addition, the terms "including", "etc." and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily need to be limited to those clearly listed steps or units, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0031] Embodiment 1

[0032] Figure 1The following is a flowchart of a pipeline information processing method provided in Embodiment 1 of the present invention. This embodiment is applicable to the situation of processing pipeline information, especially applicable to the situation of processing pipeline information, constructing a target pipeline, generating a target build process directory according to the running cache information generated during the running process of the target pipeline, and constructing a target build image of the target pipeline according to the target build process directory and the target compilation environment image of the target pipeline. This method can be executed by a pipeline information processing device, which can be implemented in the form of hardware and / or software, and the pipeline information processing device can be configured in an electronic device. As Figure 1 shown, the method includes:

[0033] S110. According to the pipeline construction command, determine the target identifier of the target pipeline to be constructed and the pipeline construction operation instruction. According to the target identifier, determine whether the target build image corresponding to the target pipeline is stored in the image repository.

[0034] Among them, the pipeline construction command refers to the pipeline construction instruction issued by the user, and the pipeline is a set of automated processes. In this embodiment, the pipeline can be a continuous integration pipeline CI or a continuous delivery pipeline CD based on Kubernetes. Kubernetes is the most mainstream cloud native container orchestration system currently. The target identifier refers to the pipeline identifier of the target pipeline, and the target identifier can be a character used to represent the target pipeline. The pipeline construction operation instruction refers to the instruction code issued by the developer for constructing the pipeline. The main function of the image repository is to be responsible for the storage and distribution of image content. An image is a form of file storage and a type of redundancy. When there is an exact same copy of the data on one disk on another disk, it is called an image, and many files can be made into an image file. The target build image refers to an image containing the running cache information generated during the historical running process of the target pipeline.

[0035] Specifically, the candidate identifier of the candidate pipeline and the candidate build image of the candidate pipeline can be pre-stored in the image repository in a corresponding manner. After obtaining the pipeline construction command issued by the developer, analyze the pipeline construction command, and determine the target identifier of the target pipeline to be constructed by the pipeline construction command and the pipeline construction operation instruction according to the analysis result. Then, according to the target identifier and the corresponding relationship between the candidate identifier of the candidate pipeline stored in the image repository and the candidate build image of the candidate pipeline, determine whether the target build image corresponding to the target pipeline is stored in the image repository. Among them, the candidate pipeline can be a pipeline constructed in the historical process, and the candidate build image refers to an image containing the running cache information generated during the historical running process of the candidate pipeline.

[0036] S120. If the target build image corresponding to the target pipeline is not stored in the image repository, determine the pipeline compilation language according to the pipeline build command, and build the target pipeline according to the pipeline compilation language and the pipeline build operation instructions.

[0037] Among them, the pipeline compilation language can be java, go, c, c++, or js. The pipeline build operation instructions include image operation instructions for building the target pipeline and container startup execution instructions. The container, which is the container in the program, refers to a software package formed by bundling the code for building the target pipeline, relevant configuration files, libraries, and dependencies required to run the target pipeline.

[0038] Specifically, if the target build image corresponding to the target pipeline is not stored in the image repository, further analyze the pipeline build command. According to the further analysis result, determine the pipeline compilation language for building the target pipeline, and execute the pipeline build operation instructions using the pipeline compilation language to build the target pipeline.

[0039] S130. According to the pipeline compilation language, call the target compilation environment image from the image repository, and run the target pipeline based on the target compilation environment image.

[0040] Among them, the target compilation environment image refers to the image of the compilation environment corresponding to the pipeline compilation language that is pre-stored in the image repository.

[0041] Specifically, the target compilation environment image corresponding to the pipeline compilation language can be queried from the image repository, and the target compilation environment image can be called to run the target pipeline based on the target compilation environment image.

[0042] Exemplarily, the method of calling the target compilation environment image from the image repository according to the pipeline compilation language can be: determine the target compilation environment image from the candidate compilation environment images according to the correspondence between the pipeline compilation language and the candidate compilation environment images in the image repository, and call the target compilation environment image from the image repository.

[0043] It can be understood that determining the target compilation environment image of the target pipeline according to the correspondence between the pipeline compilation language and the candidate compilation environment images, and calling the target compilation environment image from the image repository can achieve the effect of automatically calling the target compilation environment image and improving the acquisition efficiency of the target compilation environment image.

[0044] S140. Determine the target build process directory according to the parsing result of the pipeline build command and the first run cache information generated during the running process of the target pipeline.

[0045] Among them, the first running cache information refers to the cache information generated during the running of the target pipeline directly built according to the pipeline construction command. The target build process directory refers to the directory file used to record the cache information during the running of the target pipeline.

[0046] Specifically, according to the parsing result of the pipeline construction command, determine the target compilation method of the target pipeline from the candidate compilation methods. Among them, the candidate compilation methods include but are not limited to mvn compilation, yarn compilation, and go compilation. Obtain the first running cache information generated during the running of the target pipeline, and determine the target build process directory according to the target compilation method and the first running cache information.

[0047] S150. Determine the target build image corresponding to the target pipeline according to the target identifier, the target build process directory, and the target compilation environment image, and store the target build image in the image repository.

[0048] Specifically, generate a dockerfile according to the target identifier, the target build process directory, and the target compilation environment image, and use the dockerfile as the target build image corresponding to the target pipeline, and store the target build image in the image repository. Among them, the dockerfile is a configuration file in text format, and developers can use the dockerfile to quickly create custom images. Generally speaking, the main content of the dockerfile is divided into four parts: basic image information, maintainer information, image operation instructions, and instructions executed when the container starts.

[0049] Exemplarily, the following sub-steps can be used to determine the target build image corresponding to the target pipeline according to the target identifier, the target build process directory, and the target compilation environment image:

[0050] S1501. Obtain the build timestamp of the target pipeline, and determine the version name of the target build image according to the build timestamp, the target identifier, and the target compilation environment image.

[0051] Among them, the build timestamp of the target pipeline can be the timestamp when the target pipeline is built.

[0052] S1502. Mark the target build process directory with the version name, and use the marked target build process directory as the target build image corresponding to the target pipeline.

[0053] Specifically, the version name can be placed before the target build process directory, or the version name can be placed after the target build process directory to complete the marking of the target build process directory. And use the marked target build process directory as the target build image corresponding to the target pipeline.

[0054] It can be understood that when generating the target build image corresponding to the target pipeline, the version name of the target build image is determined according to the build timestamp, the target identifier, and the target compilation environment image, which is convenient for subsequent searching or calling of the target build image and can improve the query efficiency of the target build image.

[0055] The technical solution provided in this embodiment determines the target identifier of the target pipeline to be built and the pipeline build operation instruction according to the pipeline build command. According to the target identifier, it is determined whether the target build image corresponding to the target pipeline is stored in the image repository; if the target build image corresponding to the target pipeline is not stored in the image repository, the pipeline compilation language is determined according to the pipeline build command, and the target pipeline is built according to the pipeline compilation language and the pipeline build operation instruction; the target compilation environment image is called from the image repository according to the pipeline compilation language, and the target pipeline is run according to the target compilation environment image; according to the parsing result of the pipeline build command and the first run cache information generated during the running process of the target pipeline, the target build process directory is determined; the target build image corresponding to the target pipeline is determined according to the target identifier, the target build process directory, and the target compilation environment image, and the target build image is stored in the image repository. The above solution solves the problem that after the construction of the target pipeline is completed, the container used for building the target pipeline is destroyed, resulting in the inability to query the construction process of the target pipeline in the future. When obtaining the pipeline build command, the target pipeline is built according to the pipeline build command, the target compilation environment image of the target pipeline is determined according to the pipeline compilation language, and the target build process directory is determined according to the first run cache information generated during the running process of the target pipeline, so as to determine the target build image corresponding to the target pipeline according to the target identifier, the target build process directory, and the target compilation environment image and store the target build image in the image repository, so as to facilitate obtaining the target build process directory corresponding to the first run cache information according to the target build image in the future and quickly reading the construction process of the target pipeline according to the target build process directory. At the same time, when the target pipeline is rebuilt later, the target build image can be called from the image repository to rebuild the target pipeline according to the target build image, so as to improve the construction efficiency of the target pipeline when it is rebuilt again.

[0056] Embodiment 2

[0057] Figure 2 FIG. is a flowchart of a pipeline information processing method provided in Embodiment 2 of the present invention. This embodiment is optimized on the basis of the above embodiment, and a preferred implementation scheme for building a target pipeline according to the target build image corresponding to the target pipeline stored in the image repository is given. Specifically, as Figure 2 shown, the method includes:

[0058] S210. Determine the target identifier of the target pipeline to be built and the pipeline building operation instruction according to the pipeline building command. According to the target identifier, determine whether the target building image corresponding to the target pipeline is stored in the image repository.

[0059] S220. If the target building image corresponding to the target pipeline is stored in the image repository, call the target building image from the image repository, and determine the target building process directory and the target compilation environment image of the target pipeline according to the target building image.

[0060] Specifically, if the target building image corresponding to the target pipeline is stored in the image repository, call the target building image from the image repository according to the target identifier of the target pipeline, and parse the target building image to obtain the target building process directory and the target compilation environment image of the target pipeline.

[0061] S230. Build the target pipeline according to the target compilation environment image, the pipeline building operation instruction, and the target cache information in the target building process directory.

[0062] Specifically, based on the target compilation environment image, build the target pipeline according to the pipeline building operation instruction and the target cache information in the target building process directory.

[0063] Exemplarily, on the basis of the above steps of this embodiment, during the running process of the target pipeline built according to the pipeline compilation language and the pipeline building operation instruction, the second running cache information of the target pipeline can also be monitored in real time. If the second running cache information is inconsistent with the target cache information in the target building process directory, an updated building image is built for the target pipeline. Specifically, it can be implemented through the following steps:

[0064] 1) During the running process of the target pipeline, monitor the second running cache information of the target pipeline in real time, and determine whether the second running cache information is consistent with the target cache information in the target building process directory.

[0065] The second running cache information refers to the running cache information generated during the running process of the target pipeline built according to the target building image.

[0066] Specifically, if some codes in the pipeline building operation instruction of the target pipeline change, the cache information of the target pipeline will also change accordingly. Therefore, it is also necessary to monitor the second running cache information of the target pipeline in real time during the running process of the target pipeline, and compare the second running cache information with the target cache information in the target building process directory to determine whether the second running cache information is consistent with the target cache information in the target building process directory.

[0067] 2) If not, update the target build process directory according to the second running cache information.

[0068] Specifically, if the second running cache information is inconsistent with the target cache information in the target build process directory, determine that the cache information in the second running cache information that is inconsistent with the target cache information is the unupdated cache information. Use the updated cache information to replace the target cache information corresponding to the updated cache information in the target build process directory to complete the update of the target build process directory.

[0069] 3) Determine the updated build image corresponding to the target pipeline according to the target identifier, the target compilation environment image, and the updated target build process directory, and store the updated build image in the image repository.

[0070] Exemplarily, the build timestamp of the target pipeline can be obtained, and according to the build timestamp, the target identifier, and the target compilation environment image, the version name of the target build image is determined. Use the version name to mark the updated target build process directory, and use the marked and updated target build process directory as the target build image corresponding to the target pipeline.

[0071] It can be understood that in the above solution, during the process of running the target pipeline, the second running cache information of the target pipeline and the target cache information in the target build process directory are compared for consistency in real time. If the two are inconsistent, an updated build image of the target pipeline is generated according to the second running cache information, and the updated build image is stored in the image repository. The build image of the pipeline in the image repository can be updated in real time, enriching the content in the image repository. At the same time, when the target pipeline is rebuilt later, the accuracy of the target build image retrieved from the image repository is improved.

[0072] S240. If the target build image corresponding to the target pipeline is not stored in the image repository, determine the pipeline compilation language according to the pipeline build command, and build the target pipeline according to the pipeline compilation language and the pipeline build operation instructions.

[0073] S250. Call the target compilation environment image from the image repository according to the pipeline compilation language, and run the target pipeline according to the target compilation environment image.

[0074] S260. Determine the target build process directory according to the parsing result of the pipeline build command and the first running cache information generated during the running process of the target pipeline.

[0075] S270. Determine the target build image corresponding to the target pipeline according to the target identifier, the target build process directory, and the target compilation environment image, and store the target build image in the image repository.

[0076] The technical solution of this embodiment provides a preferred real-time method for constructing a target pipeline. When it is determined that the target build image corresponding to the target pipeline is stored in the image repository, the target build process directory and the target compilation environment image of the target pipeline are determined according to the target build image corresponding to the target pipeline stored in the image repository. Based on the target compilation environment image, the pipeline build operation instructions, and the target cache information in the target build process directory, the target pipeline is constructed. This solves the problem that every time the pipeline is built, the build completed by the pipeline needs to be output, resulting in a long build time, low build efficiency, and low resource utilization rate of the pipeline. It realizes the effects of improving the compilation and build efficiency of the pipeline, saving the build time of the pipeline, and improving the resource utilization rate of the pipeline build.

[0077] Embodiment III

[0078] Figure 3 It is a flowchart of a pipeline information processing method provided by Embodiment III of the present invention. This embodiment is optimized on the basis of the above embodiment, and a preferred implementation manner for determining the target build process directory according to the parsing result of the pipeline build command and the first running cache information generated during the running of the target pipeline is given. Specifically, as Figure 3 shown, the method includes:

[0079] S310. According to the pipeline build command, determine the target identifier of the target pipeline to be built and the pipeline build operation instructions. According to the target identifier, determine whether the target build image corresponding to the target pipeline is stored in the image repository.

[0080] S320. If the target build image corresponding to the target pipeline is not stored in the image repository, determine the pipeline compilation language according to the pipeline build command, and construct the target pipeline according to the pipeline compilation language and the pipeline build operation instructions.

[0081] S330. Call the target compilation environment image from the image repository according to the pipeline compilation language, and run the target pipeline according to the target compilation environment image.

[0082] S340. Call the analysis build container to parse the pipeline build command, and determine the target compilation method of the target pipeline according to the parsing result.

[0083] Among them, the analysis and construction container can be the image of a sidecar container. The sidecar container is an auxiliary container and can be used to complete some work independent of the main container. Exemplarily, in each Pod of Kubernetes, a sidecar container can be started to complete some work independent of the main process in the Pod. A Pod is the smallest running unit in Kubernetes and consists of one or more containers, and each Pod runs on its corresponding host.

[0084] Specifically, the analysis and construction container is called to parse the pipeline construction command, so as to determine the target compilation method of the target pipeline according to the parsing result.

[0085] S350. Determine the target running cache information from the first running cache information generated during the running process of the target pipeline according to the target compilation method, and write the target running cache information into the cache information directory to determine the target construction process directory.

[0086] Among them, the target running cache information refers to the information that can be written into the cache information directory.

[0087] Specifically, the selection criteria for the target running cache information corresponding to the candidate compilation method can be preset in advance. According to the target compilation method and the selection criteria for the target running cache information corresponding to the target compilation method, the target running cache information is extracted from the first running cache information generated during the running process of the target pipeline, and the target running cache information is written into the cache information directory. The cache information directory recording the target running cache information is used as the target construction process directory.

[0088] S360. Determine the target construction image corresponding to the target pipeline according to the target identifier, the target construction process directory, and the target compilation environment image, and store the target construction image in the image repository.

[0089] In the technical solution of this embodiment, when determining the target construction process directory according to the first cache information of the target pipeline, the target compilation method of the target pipeline is determined according to the parsing result of the pipeline construction command by the analysis and construction container, and according to the target compilation method, the target running cache information used to construct the target construction process directory in the first running cache information is determined, and the target running cache information is written into the cache information directory to determine the target construction process directory. The above solution provides an optional real-time method for selectively writing the cache information generated during the running process of the target pipeline into the cache information directory, improves the acquisition efficiency of the target construction process directory, reduces the storage space for storing the target construction process directory, and is convenient for subsequent browsing of the target construction process directory.

[0090] Embodiment 4

[0091] Figure 4 This is a schematic structural diagram of a pipeline information processing device provided in the fourth embodiment of the present invention. This embodiment is applicable to the situation of processing pipeline information. As Figure 4 shown, the pipeline information processing device includes: a target build image query module 410, a target pipeline build module 420, a target pipeline operation module 430, a build process directory determination module 440, and a target build image determination module 450.

[0092] Among them, the target build image query module 410 is configured to determine the target identifier of the target pipeline to be built and the pipeline build operation instruction according to the pipeline build command, and determine whether the target build image corresponding to the target pipeline is stored in the image repository according to the target identifier;

[0093] The target pipeline build module 420 is configured to, if the target build image corresponding to the target pipeline is not stored in the image repository, determine the pipeline compilation language according to the pipeline build command, and build the target pipeline according to the pipeline compilation language and the pipeline build operation instruction;

[0094] The target pipeline operation module 430 is configured to call the target compilation environment image from the image repository according to the pipeline compilation language, and run the target pipeline according to the target compilation environment image;

[0095] The build process directory determination module 440 is configured to determine the target build process directory according to the parsing result of the pipeline build command and the first run cache information generated during the operation of the target pipeline;

[0096] The target build image determination module 450 is configured to determine the target build image corresponding to the target pipeline according to the target identifier, the target build process directory, and the target compilation environment image, and store the target build image in the image repository.

[0097] The technical solution provided in this embodiment determines the target identifier of the target pipeline to be built and the pipeline building operation instruction according to the pipeline building command. According to the target identifier, it determines whether the target building image corresponding to the target pipeline is stored in the image repository. If the target building image corresponding to the target pipeline is not stored in the image repository, it determines the pipeline compilation language according to the pipeline building command, and builds the target pipeline according to the pipeline compilation language and the pipeline building operation instruction. It calls the target compilation environment image from the image repository according to the pipeline compilation language, and runs the target pipeline according to the target compilation environment image. It determines the target building process directory according to the parsing result of the pipeline building command and the first running cache information generated during the running process of the target pipeline. It determines the target building image corresponding to the target pipeline according to the target identifier, the target building process directory and the target compilation environment image, and stores the target building image in the image repository. The above solution solves the problem that after the construction of the target pipeline is completed, the container used for building the target pipeline is destroyed, resulting in the inability to query the construction process of the target pipeline in the future. When obtaining the pipeline building command, it builds the target pipeline according to the pipeline building command, determines the target compilation environment image of the target pipeline according to the pipeline compilation language, and determines the target building process directory according to the first running cache information generated during the running process of the target pipeline, so as to determine the target building image corresponding to the target pipeline according to the target identifier, the target building process directory and the target compilation environment image and store the target building image in the image repository, so as to obtain the target building process directory corresponding to the first running cache information according to the target building image in the future, and quickly read the construction process of the target pipeline according to the target building process directory. At the same time, when the target pipeline is built again in the future, it can call the target building image from the image repository to build the target pipeline again according to the target building image, so as to improve the construction efficiency of the target pipeline when it is built again.

[0098] Exemplarily, the above pipeline information processing device further includes:

[0099] The target building image calling module is used to, if the target building image corresponding to the target pipeline is stored in the image repository, call the target building image from the image repository, and determine the target building process directory and the target compilation environment image of the target pipeline according to the target building image;

[0100] The target building image application module is used to build the target pipeline according to the target compilation environment image, the pipeline building operation instruction and the target cache information in the target building process directory.

[0101] Exemplarily, the above pipeline information processing device further includes:

[0102] The cache information monitoring module is used to monitor the second running cache information of the target pipeline in real time during the running of the target pipeline, and determine whether the second running cache information is consistent with the target cache information in the target build process directory;

[0103] The directory update module is used to, if not, update the target build process directory according to the second running cache information;

[0104] The updated build image acquisition module is used to determine the updated build image corresponding to the target pipeline according to the target identifier, the target compilation environment image, and the updated target build process directory, and store the updated build image in the image repository.

[0105] Exemplarily, the build process directory determination module 440 is specifically used for:

[0106] Call the analysis build container to parse the pipeline build command, and determine the target compilation method of the target pipeline according to the parsing result;

[0107] According to the target compilation method, determine the target running cache information from the first running cache information generated during the running of the target pipeline, and write the target running cache information into the cache information directory to determine the target build process directory.

[0108] Exemplarily, the target build image determination module 450 is specifically used for:

[0109] Obtain the build timestamp of the target pipeline, and determine the version name of the target build image according to the build timestamp, the target identifier, and the target compilation environment image;

[0110] Mark the target build process directory with the version name, and use the marked target build process directory as the target build image corresponding to the target pipeline.

[0111] Exemplarily, the target pipeline running module 430 is specifically used for:

[0112] According to the correspondence between the pipeline compilation language and the candidate compilation environment images in the image repository, determine the target compilation environment image from the candidate compilation environment images, and call the target compilation environment image from the image repository.

[0113] The pipeline information processing device provided in this embodiment can be applied to the pipeline information processing method provided in any of the above embodiments, and has corresponding functions and beneficial effects.

[0114] Embodiment Five

[0115] Figure 5FIG. 0 shows a schematic structural diagram of an electronic device 10 that can be used to implement an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as, for example, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, for example, personal digital processors, cellular telephones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely exemplary and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0116] As Figure 5 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0117] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0118] The processor 11 can be various general-purpose and / or special-purpose processing components having processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the pipeline information processing method.

[0119] In some embodiments, the pipeline information processing method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the pipeline information processing method described above may be executed. Alternatively, in other embodiments, the processor 11 may be configured to execute the pipeline information processing method by any other suitable means (e.g., by means of firmware).

[0120] The various embodiments of the systems and techniques described above in this document may be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), systems on a chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a special-purpose or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0121] The computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable pipeline information processing device, such that the computer programs, when executed by the processor, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The computer programs may be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0122] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0123] To provide for interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0124] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0125] A computing system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The client-server relationship is created by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0126] It should be understood that various forms of processes shown above can be used, steps can be reordered, added or deleted. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is made herein.

[0127] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. A pipeline information processing method, characterized in that, Including: Determine the target identifier of the target pipeline to be built and the pipeline build operation instruction according to the pipeline build command, and determine whether the target build image corresponding to the target pipeline is stored in the image repository according to the target identifier; If the target build image corresponding to the target pipeline is not stored in the image repository, determine the pipeline compilation language according to the pipeline build command, and build the target pipeline according to the pipeline compilation language and the pipeline build operation instruction; Call the target compilation environment image from the image repository according to the pipeline compilation language, and run the target pipeline according to the target compilation environment image; Determine the target build process directory according to the parsing result of the pipeline build command and the first run cache information generated during the running of the target pipeline; Determine the target build image corresponding to the target pipeline according to the target identifier, the target build process directory and the target compilation environment image, and store the target build image in the image repository.

2. The method according to claim 1, characterized in that, After determining the target identifier of the target pipeline to be built and the pipeline build operation instruction according to the pipeline build command, and determining whether the target build image corresponding to the target pipeline is stored in the image repository according to the target identifier, it further includes: If the target build image corresponding to the target pipeline is stored in the image repository, call the target build image from the image repository, and determine the target build process directory and the target compilation environment image of the target pipeline according to the target build image; Build the target pipeline according to the target compilation environment image, the pipeline build operation instruction and the target cache information in the target build process directory.

3. The method according to claim 2, wherein Including: During the running of the target pipeline, monitor the second run cache information of the target pipeline in real time, and determine whether the second run cache information is consistent with the target cache information in the target build process directory; If not, update the target build process directory according to the second run cache information; Determine the updated build image corresponding to the target pipeline according to the target identifier, the target compilation environment image and the updated target build process directory, and store the updated build image in the image repository.

4. The method according to claim 1, wherein Determine the target build process directory according to the parsing result of the pipeline build command and the first run cache information generated during the running of the target pipeline, including: Call the analysis build container to parse the pipeline build command, and determine the target compilation method of the target pipeline according to the parsing result; Determine the target run cache information from the first run cache information generated during the running of the target pipeline according to the target compilation method, and write the target run cache information into the cache information directory to determine the target build process directory.

5. The method according to claim 1, wherein Determine the target build image corresponding to the target pipeline according to the target identifier, the target build process directory and the target compilation environment image, including: Obtain the build timestamp of the target pipeline, and determine the version name of the target build image according to the build timestamp, the target identifier, and the target compilation environment image; Use the version name to mark the target build process directory, and use the marked target build process directory as the target build image corresponding to the target pipeline.

6. The method according to claim 1, wherein Call the target compilation environment image from the image repository according to the pipeline compilation language, including: Determine the target compilation environment image from the candidate compilation environment images according to the correspondence between the pipeline compilation language and the candidate compilation environment images in the image repository, and call the target compilation environment image from the image repository.

7. A pipeline information processing device, characterized in that, Include: A target build image query module, configured to determine the target identifier and the pipeline build operation instruction of the target pipeline to be built according to the pipeline build command, and determine whether the target build image corresponding to the target pipeline is stored in the image repository according to the target identifier; A target pipeline build module, configured to, if the target build image corresponding to the target pipeline is not stored in the image repository, determine the pipeline compilation language according to the pipeline build command, and build the target pipeline according to the pipeline compilation language and the pipeline build operation instruction; A target pipeline running module, configured to call the target compilation environment image from the image repository according to the pipeline compilation language, and run the target pipeline according to the target compilation environment image; A build process directory determination module, configured to determine the target build process directory according to the parsing result of the pipeline build command and the first run cache information generated during the running process of the target pipeline; A target build image determination module, configured to determine the target build image corresponding to the target pipeline according to the target identifier, the target build process directory, and the target compilation environment image, and store the target build image in the image repository.

8. The device according to claim 7, wherein Further include: A target build image call module, configured to, if the target build image corresponding to the target pipeline is stored in the image repository, call the target build image from the image repository, and determine the target build process directory and the target compilation environment image of the target pipeline according to the target build image; A target build image application module, configured to build the target pipeline according to the target compilation environment image, the pipeline build operation instruction, and the target cache information in the target build process directory.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the pipeline information processing method according to any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a processor to implement the pipeline information processing method according to any one of claims 1-6 when executed.

Images