A deployment method and system for vault elements

Abstract

The present application discloses a method and system for deploying vault elements, which involves computer and database technologies, and includes the following steps: obtaining a version list of vault elements to be deployed; performing configuration verification based on the obtained version list, wherein the configuration verification includes YAML syntax verification, field existence verification, field type verification, and field value verification based on the version list; obtaining service components corresponding to the vault elements to be deployed from the target address according to a preset configuration file, and arranging and archiving them; performing calculations and allocations according to a specific deployment template based on the arranged and archived data to complete the deployment. The present application embodiment proposes a one-click deployment system to simplify and automate the software deployment process, ensure the closed-loop delivery of productized services, and improve deployment efficiency and the accuracy of product delivery.

Description

Technical Field

[0001] The present application relates to the fields of computers and databases, and in particular to a method and system for deploying vault elements. Background Art

[0002] SealOS and KubeSphere provide an open source distributed operating system for cloud-native applications built on Kubernetes, including Hadoop automation control, code library-image library-version library construction.

[0003] Traditional deployment processes are often complex and error-prone, leading to inconsistent versions, configuration errors, and other issues during product delivery. Although there are some solutions on the market that focus on private deployment, most of them cannot fully meet the above requirements. Summary of the invention

[0004] The embodiments of the present application provide a method and system for deploying vault elements to ensure consistency and efficiency of product delivery during software development and deployment.

[0005] The present application embodiment provides a method for deploying treasury elements, including the following steps:

[0006] Get the version list of vault elements to be deployed;

[0007] Perform configuration verification based on the obtained version list, wherein the configuration verification includes YAML syntax verification, field existence verification, field type verification, and field value verification based on the version list;

[0008] According to the preset configuration file, obtain the service components corresponding to the vault elements to be deployed from the target address, and organize and archive them;

[0009] Based on the organized and archived data, computing allocation is performed according to a specific deployment template to complete the deployment.

[0010] Optionally, the obtained version parameters include: version information, multi-service definition, common component information, basic component information, product information, and service YAML storage repository;

[0011] According to the obtained version list, configuration verification includes:

[0012] Check for syntax errors in the build manifest;

[0013] Verify that all necessary fields are complete;

[0014] For any field in the version list, determine whether any field is compliant and perform validity check on the value of any field.

[0015] Optionally, based on the obtained version list, configuration verification is performed, including:

[0016] When any language code in the version list is merged, configure the webhook to perform rule checking before merging, and verify whether all SQL files are compliant based on DDL / DML / syntax tree; and,

[0017] Load the grammar rules of the specified database for interception checking;

[0018] Check whether the submitted YAML file meets the type compliance of the element based on the YAML syntax library;

[0019] Determine whether the naming of data elements in the version list is compliant.

[0020] Optionally, send a notification to a specified address if the validation is not satisfied.

[0021] Optionally, the preset configuration file is a pre-configured manifest file, which obtains the service components corresponding to the vault elements to be deployed from the target address and organizes and archives them, including:

[0022] According to the pre-configured manifest file, the corresponding service components are obtained from the target address in sequence, and the service components include the configuration files, images, and services of the vault elements to be deployed;

[0023] The obtained service components are cleaned by element rules, converted by labels, compressed by signatures, and integrated by watermarks. The labels are converted into exclusive labels of product version, service, batch, and timestamp according to the extracted image labels.

[0024] The generated data files are classified and stored according to the hierarchy, and the different services, components, AI models, and materials of the elements are classified and stored according to the type and stage of the data elements.

[0025] Optionally, obtaining the service components corresponding to the vault elements to be deployed from the target address and arranging and archiving them also includes:

[0026] Select the configuration required by the current version from the service's YAML repository, archive the scenes used, and organize them into dependency packages;

[0027] The generated dependency packages are converted and signed, and then transferred to the directory assigned by the archive server, where the directory assigned by the server is assigned based on time and operator based on set rules;

[0028] If the download image cannot be accurately obtained from the target address, or the configured md5 verification rule is incorrect, an exception is returned.

[0029] Optionally, based on the organized and archived data, computing allocation is performed according to a specific deployment template, including:

[0030] Convert the archived files to the corresponding directory according to the corresponding archive file template mirror template.

[0031] The embodiment of the present application further proposes a vault element deployment system, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps of the vault element deployment method as described above are implemented.

[0032] The embodiment of the present application proposes a one-click deployment system to simplify and automate the software deployment process, ensure a closed-loop delivery of productized services, and improve deployment efficiency and the accuracy of product delivery.

[0033] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Various other advantages and benefits will become apparent to those of ordinary skill in the art by reading the detailed description of the preferred embodiments below. The accompanying drawings are only for the purpose of illustrating the preferred embodiments and are not to be considered as limiting the present application. Also, the same reference symbols are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0035] Figure 1 This is a basic flow diagram of the deployment method of the treasury elements for this application;

[0036] Figure 2 This is a flow chart of the deployment method of the treasury elements for this application;

[0037] Figure 3 The following is a technical architecture diagram of the deployment method of the treasury elements of this application. DETAILED DESCRIPTION

[0038] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0039] Requirements for automated software deployment:

[0040] Fast deployment: For any product, fast, accurate, and error-free deployment is extremely important. Automated deployment tools should provide one-click deployment, deployment templates, deployment guides, and other functions to achieve fast deployment and reduce deployment difficulty and error rate.

[0041] Configuration verification: In the process of private deployment, the correctness of configuration is an important factor affecting the success of deployment. Automated deployment tools should provide a configuration verification mechanism to detect and correct incorrect configurations in a timely manner to avoid deployment failures.

[0042] Quick archiving: During product iteration, how to effectively save and manage historical versions is the key to improving efficiency. Automated deployment tools need to provide quick archiving functions to help users easily manage historical versions.

[0043] Version control: In the process of product iteration, the management and control of different versions becomes particularly important. This includes not only the version control of software, but also the version control of configuration files, dependent libraries, etc. A good private automated deployment tool should provide powerful version control functions to ensure that users can clearly manage and track different versions of products.

[0044] The present application embodiment provides a method for deploying treasury elements, such as Figure 1 , Figure 2 As shown, the one-click deployment system of the present application includes: version list, configuration verification, one-click archiving, deliverable generation, and one-click deployment. The method of the present application completes the delivery closed loop of productized services, ensuring that after all services complete version testing, the fingerprints of all deliverables remain consistent.

[0045] The method of the present application automatically sets access control and webhook confirmation for all code branches during the merging process of various code branches of the product. After each code is submitted, a scanning event is automatically triggered. In addition to conventional security code scanning and element specification scanning, archive configuration verification is additionally supplemented. If the verification fails in any link, the merge is automatically blocked.

[0046] When the product is tested and ready for release, the production process of the element product version is completed based on the automatic DBPM workflow. It is configured based on the pipeline. Each service obtains the code from the corresponding specified branch of git, builds an image, aggregates the data element processing and trading center, and archives the service modules of each system. It builds dependent AI models, third-party integration libraries, etc., and has built-in component kits that each version depends on. The kits are self-developed, integrated, and outsourced based on the goals of each version of the elements. After integration, they are output as component libraries to facilitate pipeline integration. The specific method of the embodiment of the present application includes the following steps:

[0047] In step S101, a version list of vault elements to be deployed is obtained;

[0048] In step S102, configuration verification is performed according to the obtained version list, wherein the configuration verification includes YAML syntax verification, field existence verification, field type verification and field value verification based on the version list.

[0049] In some embodiments, the obtained version list includes: version information (release), multi-service definition (service), public component information (public), basic component information (component), product information (product) and service YAML storage repository (serviceYAMLRepo).

[0050] In the process of making the version library, if there is no verification stage, the deliverables cannot be used and various configuration errors will occur, similar to downloading an installation package but failing to install it successfully. The technical difficulty of verification is how to find errors in various code files and images in advance. There are various languages ​​​​such as Java, Python, Go, Vue, etc., and it is also necessary to support multiple databases such as MySQL, DAMO, Renmin University of China Jincang, as well as R&D specifications. In some embodiments of this application, according to the obtained version list, configuration verification includes:

[0051] Check for syntax errors in the build manifest;

[0052] Verify that all necessary fields are complete;

[0053] For any field in the version list, determine whether any field is compliant and perform validity check on the value of any field.

[0054] In a specific example, the configuration verification of this application includes ensuring that the syntax of the configuration file is correct, and then further checking whether the required fields are complete. Then, for each field, verify whether its type is compliant, and finally perform validity verification on the value of each field. The entire process is carefully and rigorously verified through four steps to ensure the correctness and usability of the configuration file.

[0055] In step S103, according to the preset configuration file, the service components corresponding to the vault elements to be deployed are obtained from the target address, and organized and archived. The difficulty of archiving is that it is necessary to integrate all the dependent services, components, and configurations of the product for processing and management to form a unified file. Common archiving only does the movement action, but the element vault product involves many element configuration conventions and process dependencies, which need to be handled uniformly throughout the steps. The organization and archiving of this application works precisely according to the definition in the version list to ensure the consistency and traceability of all deployed resources.

[0056] In step S104, calculation allocation is performed according to the sorted and archived data and a specific deployment template to complete the deployment.

[0057] The embodiment of the present application proposes a one-click deployment system to simplify and automate the software deployment process, ensure a closed-loop delivery of productized services, and improve deployment efficiency and the accuracy of product delivery.

[0058] In some embodiments, performing configuration verification according to the obtained version list specifically includes:

[0059] When any language code in the version list is merged, configure the webhook to perform rule checking before merging, and verify whether all SQL files are compliant based on DDL / DML / syntax tree; and,

[0060] Load the grammar rules of the specified database for interception checking, such as loading the grammar rules of mainstream databases such as MySQL and DAMO for interception checking.

[0061] Check whether the submitted YAML file meets the type compliance of the elements based on the YAML syntax library. For example, check whether the submitted YAML file meets the type compliance of the elements based on the YAML syntax library. The header file, body, various configurations at the bottom, and keys must meet the type compliance rationality of the elements.

[0062] Determine whether the naming of the data elements in the version list is compliant, for example, whether the various names of the data element system development are compliant, whether the length, type, and space matching are accurate, and whether the configuration of the microservice matches the element architecture. In some embodiments, if the verification is not met, a notification is sent to a specified address. That is, if the verification is not met, an IM notification or email is automatically sent to the specified code submitter.

[0063] The element vault product targeted by this application involves many element configuration conventions and process dependencies, which need to be handled uniformly throughout the entire step. In some embodiments, the preset configuration file is a pre-configured manifest file, which obtains the service components corresponding to the vault elements to be deployed from the target address, and organizes and archives them, including:

[0064] According to the pre-configured manifest file, the corresponding service components are obtained from the target address in sequence, and the service components include the configuration files, images, and services of the vault elements to be deployed.

[0065] The acquired service components are cleaned by element rules, converted by labels, compressed by signatures, and integrated by watermarks. Label conversion is performed based on the extracted image labels and uniformly converted into exclusive labels for product version, service, batch, and timestamp. In a specific example, during the archiving phase, element rule cleaning, label conversion, signature compression, and watermark integration are performed for the downloaded service component packages. Label conversion is uniformly converted into exclusive labels to ensure that each archive output has an independent and traceable label.

[0066] The generated data files are classified and stored according to the hierarchy, and the different services, components, AI models, and materials of the elements are classified and stored according to the type and stage of the data elements. For the generated files, the files are classified and stored according to the hierarchy to reduce the storage space. The different services, components, AI models, and materials of the elements are classified and stored according to the type and stage of the data elements. And the files are compressed and signed for the entire element suite to ensure the security of the output. The compressed signature uses an encryption algorithm and configures different public and private key processing.

[0067] In some embodiments, obtaining service components corresponding to the vault elements to be deployed from the target address and arranging and archiving them further includes:

[0068] Select the configuration required by the current version from the service's YAML repository, archive the scenes used, and organize them into dependency packages;

[0069] The generated dependency packages are converted, signed, and transferred to the directory assigned by the archive server, where the directory assigned by the server is assigned based on time and operator based on set rules. In a specific example, the configuration required for the current version can be selected from the YAML repository of the service, the images used by public components can be archived, and the big data deployment packages and AI models used by the service can also be archived. Finally, all dependent packages are converted, signed, and processed, and then uniformly moved to the directory automatically assigned by the archive server. The assigned directory is determined based on time and operator based on preset rules.

[0070] If the download image cannot be accurately obtained from the target address, or the configured md5 verification rule is incorrect, an exception is returned. After the exception is returned, the corresponding service provider developer is notified. There is no standard mature solution for archiving in the prior art. Basically, it is directly extracted and stored. During the archiving process, the accuracy of each configuration item, specification, and address of the end-to-end configuration file will not be checked, and no specific signature compression will be used.

[0071] In some embodiments, computing and allocating according to a specific deployment template based on the organized and archived data includes: converting the organized and archived files into corresponding directories according to the corresponding archive file template image template. Figure 2 The main task of the deliverable generation phase shown in the figure is to distribute the archived products of the version in an orderly and logical manner according to a specific deployment template. For example, all image files (whether public components or application services) are converted to the specified version library according to the preset image template and assembled and integrated.

[0072] Similarly, the deployment files of the common components will be placed in the specified directory of the common component deployment template and packaged into a common component installation package.

[0073] The process of delivering the product is to process, organize, assemble and label the information (release), multi-service definition (service), public component information (public), basic component information (component), product information (product) and service YAML storage warehouse (serviceYAMLRepo) from the archived semi-finished products to the version library according to the storage rules of the element design. The version library is securely encrypted and users need to apply for permission to download and use it.

[0074] In the entire process of generating deliverables, we complete configuration files, storage directories, formats, parsing, and encryption based on self-developed technologies to ensure that deliverables can be used out of the box. We perform complete pre-verification, processing parsing, and automatic processing on the built-in configuration files, configuration formats, specifications, file locations, and encryption forms of the element versions. The basic processing logic on the market will generate an ordinary compressed package, which is installed based on the standard installation format, has security risks, and many personalized functions are not supported.

[0075] Finally, the method of this application performs one-click deployment. According to the generated element-specific deliverables, after the fingerprint is verified to be consistent, this link mainly realizes private one-click deployment, including operating system initialization, infrastructure, public components, application service deployment, as well as public component initialization, mTLS configuration and other operations.

[0076] like Figure 3 As shown, this application is based on the internally developed workflow deployment and orchestration capabilities, and the custom assembly deployment phase includes image deployment, common component deployment, application service deployment, infrastructure deployment, and service initialization phases. Each phase can be divided into multiple execution tasks, and each execution task corresponds to a process in the background. The system can automatically and intelligently assign different resource priorities to different task types. Different types such as images, common basic components, element services, and vault services will be assigned synchronized weights to run the workflow.

[0077] This application proposes a one-click deployment system that aims to simplify and automate the software deployment process and ensure a closed-loop delivery of productized services, thereby improving deployment efficiency and the accuracy of product delivery.

[0078] This application provides a complete automation solution for code libraries, image libraries, version libraries, and product libraries from version management to multi-library separation. The method of this application can quickly build a suite-type version library, extract the parts that change in the project, and can meet the rapid privatization standard automated installation and deployment based on the suite, and can adapt to various personalized scenarios, such as different numbers of machines, different models, open source components, trusted components, etc. During the deployment process, the method of this application can more conveniently analyze and view logs, solve problems, and support short-point processing. After the deployment is completed, a complete inspection and verification can be performed on the completed deployment to check for problems, security vulnerabilities, etc.

[0079] The embodiment of the present application further proposes a vault element deployment system, including a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the steps of the vault element deployment method as described above are implemented.

[0080] It should be noted that in the various embodiments of the present application, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0081] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0082] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0083] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for deploying treasury elements, characterized in that: The steps include: Get the version list of vault elements to be deployed; Perform configuration verification based on the obtained version list, wherein the configuration verification includes YAML syntax verification, field existence verification, field type verification, and field value verification based on the version list; According to the preset configuration file, obtain the service components corresponding to the vault elements to be deployed from the target address, and organize and archive them; Based on the organized and archived data, computing allocation is performed according to a specific deployment template to complete the deployment.

2. The method for deploying treasury elements according to claim 1, characterized in that: The obtained version parameters include: version information, multi-service definition, common component information, basic component information, product information and service YAML storage repository; According to the obtained version list, configuration verification includes: Check for syntax errors in the build manifest; Verify that all necessary fields are complete; For any field in the version list, determine whether any field is compliant and perform validity check on the value of any field.

3. The method for deploying treasury elements according to claim 2, characterized in that: According to the obtained version list, configuration verification specifically includes: When any language code in the version list is merged, configure the webhook to perform rule checking before merging, and verify whether all SQL files are compliant based on DDL / DML / syntax tree; and, Load the grammar rules of the specified database for interception checking; Check whether the submitted YAML file meets the type compliance of the element based on the YAML syntax library; Determine whether the naming of data elements in the version list is compliant.

4. The method for deploying treasury elements according to claim 3, characterized in that: If the validation fails, a notification is sent to the specified address.

5. The method for deploying treasury elements according to claim 2, characterized in that: The preset configuration file is a pre-configured manifest file, which obtains the service components corresponding to the vault elements to be deployed from the target address and organizes and archives them, including: According to the pre-configured manifest file, the corresponding service components are obtained from the target address in sequence, and the service components include the configuration files, images, and services of the vault elements to be deployed; The obtained service components are cleaned by element rules, converted by labels, compressed by signatures, and integrated by watermarks. The labels are converted into exclusive labels of product version, service, batch, and timestamp according to the extracted image labels. The generated data files are classified and stored according to the hierarchy, and the different services, components, AI models, and materials of the elements are classified and stored according to the type and stage of the data elements.

6. The method for deploying treasury elements as claimed in claim 5, characterized in that: Obtain the service components corresponding to the vault elements to be deployed from the target address, and organize and archive them, including: Select the configuration required by the current version from the service's YAML repository, archive the scenes used, and organize them into dependency packages; The generated dependency packages are converted and signed, and then transferred to the directory assigned by the archive server, where the directory assigned by the server is assigned based on time and operator based on set rules; If the download image cannot be accurately obtained from the target address, or the configured md5 verification rule is incorrect, an exception is returned.

7. The method for deploying treasury elements according to claim 5, characterized in that: Based on the organized and archived data, calculation allocation is performed according to a specific deployment template, including: Convert the archived files to the corresponding directory according to the corresponding archive file template mirror template.

8. A deployment system for vault elements, characterized in that: It comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the method for deploying vault elements as described in any one of claims 1 to 7 are implemented.

Images