Computer-implemented method and system for generating command pipeline for controlling technical installation

Abstract

A computer-implemented method for generating, by means of a generating device, a command pipeline as a continuous integration pipeline for controlling a technical device, the method comprising the following steps: a) providing a first command for the generating device from a local database for a first command pipeline, b) providing a second command for the generating device for the at least one second command pipeline, c) determining, by the generating device, a similarity between the first command pipeline and the at least one second command pipeline, d) identifying, by the generating device, at least one similarity between the first command pipeline and the at least one second command pipeline, and selecting at least one second command pipeline as a generated pipeline based on a predefined threshold of the at least one similarity and when the at least one similarity exceeds the threshold, e) providing the generated pipeline to the command device and controlling the technical device with the generated pipeline.

Description

[0001] The invention relates to a computer-implemented method and system for generating a command pipeline as a continuous integration pipeline for controlling a technical device by means of a generating device.

[0002] The invention also relates to a computer program, an electronically readable data carrier and a data carrier signal.

[0003] In software engineering, the combined practice of "continuous integration" (CI) and "continuous deployment and delivery" (CD) (abbreviated as "CI / CD") is applied. Figure 1 Shown in.

[0004] CI / CD is a way of developing software in which updates are foreseen to be released at any time in a sustainable manner. When changing code is routine, development cycles are more frequent, more meaningful, and faster.

[0005] Continuous integration CI is represented by the process sequences build CI_B, code CI_C, plan CI_P and test CI_T.

[0006] Continuous Deployment and Delivery CD is represented by the process sequences Deploy CD_D, Monitor CD_M, Operate CD_O and Release CD_R.

[0007] The CI / CD pipeline automates the software delivery process. The pipeline builds the code, runs the tests (CI), and safely deploys the new version of the application (CD).

[0008] The automated pipeline removes human errors, provides developers with a standardized feedback loop, and enables rapid product iteration.

[0009] Figure 2 The stages of a CI / CD pipeline according to technology are shown.

[0010] Starting from the software source SRC including the global database push GDB_P, the software build BUI phase is executed, which includes the compile COMP for the container application and the docker build DBLT phase.

[0011] Subsequently, the software testing phase TEST includes the smoke phase SMOKE, the unit phase UNIT and the integration phase INT, followed by the deployment phase DPL, where the assembly phase STA leads to the production phase PROD.

[0012] Writing continuous integration / continuous deployment and delivery (CI / CD) pipelines is a repetitive and tedious activity that is often done manually and requires a large number of project-specific scripts that are difficult to maintain and / or reuse.

[0013] In the known art, the creation of CI / CD pipelines is usually done manually and is project-specific. Some components, scripts, or pipeline configurations can be copied and applied to similar projects.

[0014] However, the identification of the items and which components can be reused is usually done manually based on the experience of experts and is therefore a manual step.

[0015] Detection of similar repositories such as GitHub has been discussed before. However, automated generation of pipelines based on similar repositories has not been addressed in any prior work.

[0016] Publication US2019 / 258976 A1 discloses a technique for generating pipeline object recommendations for a software development project. However, it is only related to software development and has nothing to do with software for controlling hardware devices. In addition, the object recommendations are related to development attributes such as data storage or software development elements or tools.

[0017] Therefore, the object of the present invention is to disclose a method for automating pipeline creation by analyzing similarities with previous projects and reusing pipeline definitions to achieve a higher level of automation, increase internal processes in software development, enhance the performance of the software and minimize functional failures.

[0018] The object of the invention is achieved by a computer-implemented method for generating a command pipeline for controlling a technical device, the method comprising the following steps:

[0019] a) providing a first command pipeline with a first command from a first database for generating a device,

[0020] b) providing at least one second command pipeline with second commands for generating the device from at least one second database,

[0021] c) determining, by the generating device, a similarity between the first command pipeline and at least one second command pipeline,

[0022] d) identifying, by the generating device, at least one similarity between the first command pipeline and at least one second command pipeline, and a predefined threshold based on the at least one similarity, and selecting the at least one second command pipeline as the generated pipeline when the at least one similarity exceeds the threshold,

[0023] e) Provide the generated pipeline to the command device and use the generated pipeline to control the technical device.

[0024] Thereby, the new first command pipeline is replaced by a known and approved similar second pipeline. Hence, further engineering efforts (such as testing) can be reduced.

[0025] The first database may be a local database which provides a first command pipeline for controlling the technical device.

[0026] The second database may be a global database which provides a second (eg approved or authenticated) command pipeline for controlling the technical device.

[0027] A command pipeline is generally a sequence of instructions containing commands.

[0028] The predefined thresholds may be defined manually or automatically using predefined quality parameters.

[0029] Furthermore, if in step c) the similarity is determined, an error metric based on the sum of squared differences is used:

[0030]

[0031] in

[0032] p i 、r i,j is the percentage of programming or scripting languages ​​in the local project and the remote project, for example by means of the number of lines of code, the number of code modules represented by the first command pipeline and the at least one second command pipeline, respectively,

[0033] w i represents the relevance to each language represented by the first command pipeline and at least one second command pipeline, which is manually assigned by the user and must be a positive number and

[0034] n is the maximum number of programming languages ​​from the local project and the remote project represented by the first command pipeline and the at least one second command pipeline,

[0035] i is an index of a programming language for the first command pipeline and the at least one second command pipeline,

[0036] j is the number of the remote item represented by the first command pipeline and at least one second command pipeline,

[0037] And the most similar item j in the first command pipeline and at least one second command pipeline corresponds to M(w, p, r j ) is the minimum value of .

[0038] Selecting parts of a common or similar programming or scripting language in the local project and the remote project within the squared difference as an error metric, like the number of lines or code or software modules, leads to the following advantage: By this a simple way is found for comparing pipelines, which can be implemented very efficiently and with low complexity.

[0039] In combination with a weighting of the programming or scripting language, the performance in similarity determination can be further improved and a beneficial impact on the overall efficiency can be achieved.

[0040] The counter i at the total in the above formula counts all different languages ​​from local and remote projects, where the maximum number of programming languages ​​is m.

[0041] In other words, the method provides the most similar pipeline in terms of the similarity of the programming or scripting language used. Thus, an improvement is achieved by inherently reducing the errors or defects generated in the selected pipeline, because fewer interface adaptations are involved, which may be affected by implicit defects and errors in the overall code or script.

[0042] The invention can be further developed if the percentage of programming or scripting languages ​​in the local project and the remote project is determined by using the number of code lines or software modules in the same language.

[0043] Thus, similarity indicates the functionality of the code itself and can limit the impact of more defects and errors within the code. Using the length of the code as the basis for similarity assessment with respect to programming or scripting languages, the result is a very reliable command pipeline.

[0044] The invention can be further developed if according to the preceding claim the similarity is determined in step c) for all executions of at least one command pipeline and a sorted list of all at least one command pipeline is generated based on the previously achieved similarities and when selecting at least one second command pipeline as the generated pipeline the list item providing the lowest error metric is selected.

[0045] In other words, the command pipeline is selected from the at least one command pipeline, which command pipeline provides the lowest error metric and is therefore most similar to the first command pipeline.

[0046] When comparing command pipelines with respect to their programming language and / or syntax and / or semantics, using the lowest error metric as defined above provides a very simple, accurate and representative measure of similarity.

[0047] The objects of the invention are also solved by a system for generating a command pipeline for controlling a technical device, the system comprising a first database and at least one second database, a generating device, and a command device, and the system is configured to perform the steps of the method according to the preceding claims.

[0048] The object of the present invention is also solved by a computer program product having machine-readable instructions stored therein which, if executed, cause a processing unit to perform the method according to the present invention.

[0049] The objects of the invention are also solved by a computer program comprising instructions which, when executed, cause a computer to perform the steps of the method according to the invention.

[0050] The objects of the invention are also solved by an electronically readable data carrier having readable control information stored thereon, the electronically readable data carrier comprising a computer program according to the preceding claims, the computer program being configured to control the processing device when the data carrier is used at the processing device and to perform the steps of the method according to the invention.

[0051] The object of the invention is also solved by a data carrier signal transmitting a computer program according to the invention.

[0052] The invention is described in the exemplary embodiments presented in detail in the following drawings. The drawings show:

[0053] Figure 1 Overview of the CI / CD pipeline,

[0054] Figure 2 The stages of the CI / CD pipeline,

[0055] Figure 3 According to a first embodiment of the system of the present invention represented by a block diagram,

[0056] Figure 4 According to an embodiment of the general process of synchronizing data between entities of a system of the present invention,

[0057] Figure 5 According to an embodiment of the synchronization process between entities of the system of the present invention,

[0058] Figure 6 An embodiment of further details of the synchronization process between entities of the system according to the invention,

[0059] Figure 7 According to an embodiment of a flow chart of the method of the present invention,

[0060] Figure 8 Embodiment of a block diagram of a system according to the invention.

[0061] Figure 3 A first embodiment of a system according to the invention represented by a block diagram is shown.

[0062] The present invention proposes a method for automating pipeline creation by means of analyzing (ie determining) similarities with previous projects and reusing pipeline definitions.

[0063] The system implementing the method according to the invention uses four main components, namely an indexer INDEX, a pipeline generator PG, a similarity analyzer SA and a local database LDB, which are included in the process along with external components as local projects LP and a global database GDB (such as a GitLab instance), and the interaction between them is described in Figure 3 In the presentation, Figure 3 A method MAI according to the invention is shown as an exemplary embodiment.

[0064] The pipeline generator PG generates configuration and code for the instantiation of the pipeline and the connection to CI / CD tools.

[0065] The Similarity Analyzer SA analyzes and determines the project properties and compares them with existing projects in the repository.

[0066] The local database LDB stores the properties of the project in the repository.

[0067] The indexer INDEX monitors the repository for changes and updates the local database.

[0068] The global database GDB (eg Gitlab instance) stores repositories.

[0069] When a local project LP requires a new CI / CD pipeline definition, on the new pipeline requirement NP, the pipeline generator PG supported by the similarity analyzer SA returns the most relevant configuration.

[0070] The local database LDB allows synchronous and asynchronous updates of content retrieved from the GitLab instance on synchronous requests RS.

[0071] This update is performed by the indexer INDEX, which queries the GitLab API for the most recent changes in the repository.

[0072] Figure 4 A general process of synchronizing data between entities of a system according to the present invention is shown.

[0073] The local item requests synchronization RS, the relevant entity performs the corresponding action through the synchronization step SYNC, and then sends a signal through the synchronization completion SD.

[0074] like Figure 5As depicted in , when synchronization is called, the indexer INDEX will request a list of projects from the supporting Git instance (Gitlab), and for each project, update the corresponding entry in the local database LDB with the new value of the programming language code percentage. The new value can now be used to provide a more accurate reference for similar projects, according to which the pipeline structure is matched and crawled.

[0075] Optionally, one or more synchronization steps SYNC may be repeated, indicated by a common reference LOOP.

[0076] Within a sequence diagram, the "par" symbol is used to indicate a "parallel" process.

[0077] Figure 4 It is shown that the synchronization process can occur synchronously or asynchronously, and is therefore either explicitly triggered by a local project and / or handled on an external trigger, such as a cron job (job scheduling utility).

[0078] In a sequence diagram, the "ref" symbol is used for reference. It is used to show another sequence, e.g. Figure 5 It is called in, which affects three roles on top: the indexer INDEX, the local database LDB, and the Gitlab instance / global database GDB.

[0079] To generate a new pipeline configuration for a local project LP, perform the following steps, which are also in Figures 5 to 7 Described in:

[0080] a) The indexer INDEX is triggered, for example, by a scheduler or a developer.

[0081] b) The indexer INDEX queries the GitLab instance GDB to update the local database LDB with the remote projects and updates their corresponding programming with the command QAPI_P and / or updates the script language with percentages with the command QAPI_P QAPI_L. The GitLab instance GDB provides the corresponding return list for the language RL_L and / or the return list for the project RL_P. If necessary, the query is repeated in the loop LOOP. Subsequently, the indexer INDEX updates the local database LDB accordingly via the signal UDB and optionally receives a confirmation signal UDB_A.

[0082] c) The Similarity Analyzer provides an algorithm that uses an error metric based on the sum of squared differences:

[0083]

[0084] in

[0085] p i 、ri,j are the percentages of programming or scripting languages ​​used in local and remote projects, respectively.

[0086] w i represents the relevance that the user assigns to each language, it must be a positive number and

[0087] n is the maximum number of programming languages ​​from local projects and remote projects,

[0088] i is the index of the programming language,

[0089] j is the remote project number.

[0090] The most similar item j corresponds to M(w, p, r j ) is the minimum value of .

[0091] First, the local project LP executes a request pipeline RP call to the pipeline generator PG.

[0092] The pipeline generator PG calls the similarity analyzer SA with the call analyzer CA, which executes the algorithm explained before using the signals for the query item and the pipeline QPPL and delivers the return result RR.

[0093] The similarity analyzer SA runs the similarity determination (ie analyzes RSA) and returns the return item ID to the pipeline generator PG using step RPID.

[0094] The Pipeline Generator PG calls the step GP to get the Pipeline YAML from the Git repository REPO.

[0095] GitLab uses a simplified markup language, Yet Another Markup Language (YAML), for data serialization to define the structure of the CI / CD pipeline. YAML is used to reference the pipeline configuration of a specific project.

[0096] The similarity analyzer SA returns at most one item below the error threshold by returning the YAML RY, in particular by returning the YAML structure RYS.

[0097] For GitLab Pipeline configuration files, any details about the original project will be removed, and the configuration returned by the GitLab instance will go through an internal process in the Pipeline generator, which will sanitize it: removing comments, references to customer environment variables, etc. Only the skeleton detailing the overall flow of the Pipeline should be left.

[0098] d) accepting or not accepting the remote item selected in step c) based on a predefined threshold.

[0099] e) The pipeline generator PG receives the accepted project and gets its CI / CD pipeline configuration from the remote GitLab instance GDB.

[0100] f) The pipeline generator PG then parses the configuration and returns the template to the local project LP by optionally involving the indexer INDEX.

[0101] Based on a predefined threshold of at least one similarity, when the at least one similarity exceeds the threshold, the remote or at least one second command pipeline is selected as the generated pipeline.

[0102] Obviously, for all embodiments, the predefined value range is equal to the previous threshold value to which the value can be compared, and the predefined value range includes two threshold values ​​that can be exceeded in both directions, i.e. the value falls below a predefined minimum value or exceeds a predefined maximum value.

[0103] In practice, if more than one pipeline exceeds a predefined threshold, the pipeline with the highest degree of similarity may be selected.

[0104] The indexer INDEX updates the local database LDB accordingly via the signal UDB. This way, the pipeline configuration is not stored in the database. To determine the best project from which to crawl the pipeline framework, the algorithm uses only the language information, so the local database can be considered as a cache for those. When a response is received from GitLab GDB with the language information for a specific project, the percentage of each language used for that project is updated in the database and can be used immediately by the similarity analyzer.

[0105] As mentioned before, the pipeline configuration is not stored in the local database LDB. After the best project has been identified, the configuration needs to be obtained from the repository that supports this specific GitLab project.

[0106] In step f), the pipeline generator PG returns the template to the LP local project. However, the indexer INDEX is only involved in crawling the Gitlab project and the synchronization process.

[0107] Figure 7 An embodiment with a simplified flow chart of the method according to the invention is shown.

[0108] Step f) as described above is optional.

[0109] Figure 8 An embodiment with a simplified block diagram of a system S according to the invention is shown.

[0110] The system S for generating a command pipeline for controlling a technical device D comprises a first database and at least one second database LDB, GDB, a generating device GD, a command device CD for controlling the technical device D.

[0111] The system is configured to perform the steps of the method according to the invention.

[0112] List of reference numerals:

[0113] BUI Build

[0114] CA Call Analyzer

[0115] CD_D Deployment

[0116] CD_M Monitoring

[0117] CD_O Operation

[0118] CD_R Release

[0119] CI_B build

[0120] CI_C Code

[0121] CI_P Project

[0122] CI_T Test

[0123] COMP Compilation

[0124] D Device

[0125] DBLT docker build

[0126] DPL Deployment

[0127] GDB Global Database

[0128] GDB_P Global database push

[0129] GP gets the pipeline yami from the repo

[0130] INDEX Indexer

[0131] INT Integration

[0132] LDB Local Database

[0133] LOOP, for example every x hours or every x days

[0134] LP Local Project

[0135] MAI Embodiments of the method according to the invention

[0136] NP New Pipeline

[0137] PG Pipeline Generator

[0138] PROD

[0139] QAPI_L Query API for Languages

[0140] QAPI_P Query API for projects

[0141] QPPL Query Project and Pipeline

[0142] RL_L Return list for languages

[0143] RL_P is used for the return list of items

[0144] RP Request Pipeline

[0145] RPID returns the project ID

[0146] RR Return results

[0147] RS Request Synchronization

[0148] RSA Run Similarity Analysis

[0149] RY returns to yami

[0150] RYS returns to yami structure

[0151] S system

[0152] SA Similarity Analyzer

[0153] SD Sync Complete

[0154] SMOKE

[0155] SRC Source

[0156] STA Assembly

[0157] TEST

[0158] UDB Update Database

[0159] UDB_A database has been updated

[0160] UNIT

[0161] YAML is yet another markup language"

[0162] Ref

[0163] Par Parameter

Claims

1. A computer-implemented method for generating, by means of a generating device, a command pipeline as a continuous integration pipeline for controlling a technical device (D), the method comprising the following steps: a) providing a first command pipeline with a first command from a first database for generating a device, b) providing at least one second command pipeline with second commands for generating the device from at least one second database, c) determining, by the generating device, a similarity between the first command pipeline and at least one second command pipeline, d) identifying, by the generating device, at least one similarity between the first command pipeline and at least one second command pipeline, and a predefined threshold value based on the at least one similarity, and selecting the at least one second command pipeline as the generated pipeline when the at least one similarity exceeds the threshold value, e) providing the generated pipeline to the command device and controlling the technical device (D) with the generated pipeline, In step c), when determining the similarity, an error metric based on the sum of squared differences is used: in p i 、r i,j is the percentage of programming or scripting languages ​​in the local project and the remote project represented by the first command pipeline and the at least one second command pipeline, respectively, w i represents the relevance to each language represented by the first command pipeline and at least one second command pipeline, which is manually assigned by the user and must be a positive number and n is the maximum number of programming languages ​​from the local project and the remote project represented by the first command pipeline and the at least one second command pipeline, i is an index of a programming language for the first command pipeline and the at least one second command pipeline, j is the number of the remote item represented by the first command pipeline and the at least one second command pipeline, And the most similar item j in the first command pipeline and at least one second command pipeline corresponds to M(w, p, r j ) is the minimum value of .

2. The method according to the preceding claim, wherein the percentage of programming or scripting languages ​​in the local project and the remote project is determined by using the number of code lines or software modules in the same language.

3. A method according to one of the preceding claims, wherein in step c) the determination of the similarity is performed for all of at least one command pipeline, and a sorted list of all of at least one command pipeline is generated based on the previously achieved similarities, and when selecting at least one second command pipeline as the generated pipeline, the list item that provides the lowest error metric is selected.

4. A system (S) for generating a command pipeline for controlling a technical device (D), comprising a first database and at least one second database (LDB, GDB), a generating device (GD), and a command device (CD) for controlling the technical device (D), and the system is configured to perform the steps of the method according to one of the preceding claims.

5. A computer program product having machine-readable instructions stored therein which, if executed, cause a processing unit to perform the method according to one of the preceding method claims.

Images