Method, device and equipment for generating training pipeline model and storage medium

Abstract

The present disclosure provides a generation method and device of a training pipeline model, equipment and a storage medium, relates to the technical field of artificial intelligence such as deep learning and computer vision, and can be used in a target detection scene. The specific implementation scheme is as follows: a pre-set configuration file and a data processing function are acquired, the configuration file includes training parameters for generating a training pipeline model; a plurality of training parameters and a training processing function are embedded into a training code template to generate a training code file; an inference code corresponding to the configuration file and a service architecture framework are acquired, and the inference code, the service architecture framework and a service processing function are embedded into a service code template to generate a service code file; and the training pipeline model is generated based on the training code file and the service code file. The present disclosure solves the technical problem that the self-generated algorithm development capability is limited in the prior art, needs to rely on a third party to provide a training pipeline, and there is a data leakage risk.

Description

Technical Field

[0001] This disclosure relates to the fields of artificial intelligence technology such as deep learning and computer vision, and specifically to a method, apparatus, device, and storage medium for generating training pipeline models. Background Technology

[0002] With the continuous advancement of technology, more and more companies are using their product data to train models in order to accelerate product development.

[0003] However, most companies have limited in-house algorithm development capabilities and need to rely on training pipelines provided by third parties. Therefore, there is a risk of leakage of their own product data. Moreover, the currently common model training methods involve a lot of repetitive work, which requires a lot of manpower and time.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This disclosure provides a method, apparatus, device, and storage medium for generating training pipeline models.

[0006] According to one aspect of this disclosure, a method for generating a training pipeline model is provided, comprising: obtaining a pre-set configuration file and data processing functions, wherein the configuration file includes at least one training parameter for generating the training pipeline model, and the data processing functions include a training processing function and a service processing function; embedding multiple training parameters and the training processing functions into a training code template to generate the training code file, wherein the training code template is obtained based on the configuration file; obtaining inference code and a service architecture framework corresponding to the configuration file, embedding the inference code, the service architecture framework, and the service processing functions into a service code template to generate the service code file; and generating the training pipeline model based on the training code file and the service code file.

[0007] Optionally, obtaining the data processing function mentioned above includes: obtaining the training processing function from a pre-set training processing module, and obtaining the service processing function from a pre-set training processing module.

[0008] Optionally, before embedding the above training parameters and the above training processing functions into the training code template to generate the above training code file, the above method further includes: using a training module to provide multiple training code templates for generating the above training pipeline model, wherein the above training code includes: deep learning task training code; selecting the above training code template corresponding to the above configuration file from the multiple training code templates.

[0009] Optionally, obtaining the inference code and service architecture framework corresponding to the configuration file includes: obtaining the inference code and service architecture framework corresponding to the configuration file from the service deployment module, wherein the service deployment module is used to provide the corresponding inference code and service architecture framework for generating the training pipeline model, the inference code is used to provide model inference services for the training pipeline model, and the service architecture framework is used to build training services for the training pipeline model.

[0010] Optionally, before generating the training code file, the method further includes: obtaining a pre-configured model encryption / decryption module and authentication module; and embedding the model encryption / decryption module and authentication module into the training code file.

[0011] Optionally, before generating the service code file, the method further includes: obtaining a pre-configured model encryption / decryption module and authentication module; and embedding the model encryption / decryption module and authentication module into the service code file.

[0012] Optionally, the above-mentioned generation of the training pipeline model based on the above-mentioned training code file and the above-mentioned service code file includes: performing hardening processing on the above-mentioned training code file and the above-mentioned service code file to generate a hardened code file; and using the above-mentioned hardened code file to generate the above-mentioned training pipeline model.

[0013] According to another aspect of this disclosure, an apparatus for generating a training pipeline model is provided, comprising: an acquisition module for acquiring a pre-set configuration file and data processing functions, wherein the configuration file includes at least one training parameter for generating the training pipeline model, and the data processing functions include a training processing function and a service processing function; a first generation module for embedding multiple training parameters and the training processing functions into a training code template to generate the training code file, wherein the training code template is obtained based on the configuration file; a second generation module for acquiring inference code and a service architecture framework corresponding to the configuration file, embedding the inference code, the service architecture framework, and the service processing functions into a service code template to generate the service code file; and a processing module for generating the training pipeline model based on the training code file and the service code file.

[0014] Optionally, the above-mentioned acquisition module includes: a first acquisition unit, configured to acquire the above-mentioned training processing function from a pre-set training processing module, and to acquire the above-mentioned service processing function from a pre-set training processing module.

[0015] Optionally, the first generation module includes: a supply unit, used to provide multiple training code templates for generating the training pipeline model using the training module, wherein the training code includes deep learning task training code; and a selection unit, used to select the training code template corresponding to the configuration file from the multiple training code templates.

[0016] Optionally, the second generation module includes: a second acquisition unit, used to acquire the inference code and the service architecture framework corresponding to the configuration file from the service deployment module, wherein the service deployment module is used to provide the corresponding inference code and the service architecture framework for generating the training pipeline model, the inference code is used to provide model inference services for the training pipeline model, and the service architecture framework is used to build training services for the training pipeline model.

[0017] Optionally, the first generation module further includes: a third acquisition unit, used to acquire a pre-configured model encryption / decryption module and authentication module; and a first embedding unit, used to embed the model encryption / decryption module and authentication module into the training code file.

[0018] Optionally, the second generation module further includes: a fourth acquisition unit, used to acquire a pre-configured model encryption / decryption module and authentication module; and a second embedding unit, used to embed the model encryption / decryption module and authentication module into the service code file.

[0019] Optionally, the above processing module includes: a hardening unit, used to harden the above training code file and the above service code file to generate a hardened code file; and a generation unit, used to generate the above training pipeline model using the above hardened code file.

[0020] According to another aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the above-described methods for generating a training pipeline model.

[0021] According to another aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform any of the above-described methods for generating a training pipeline model.

[0022] According to another aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements any of the above-described methods for generating training pipeline models.

[0023] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0024] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0025] Figure 1 This is a flowchart illustrating the steps of a method for generating a training pipeline model according to an embodiment of the present disclosure.

[0026] Figure 2 This is a schematic diagram of the training pipeline model generation process according to an embodiment of the present disclosure;

[0027] Figure 3 This is a schematic diagram of the structure of a training pipeline model generation apparatus according to an embodiment of the present disclosure;

[0028] Figure 4 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. Detailed Implementation

[0029] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] According to an embodiment of this disclosure, an embodiment of a method for generating a training pipeline model is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0032] Figure 1 This is a flowchart illustrating the steps of the method for generating a training pipeline model according to the first embodiment of this disclosure, as shown below. Figure 1 As shown, the method includes the following steps:

[0033] Step S102: Obtain a pre-set configuration file and data processing functions, wherein the configuration file includes at least one training parameter for generating a training pipeline model, and the data processing functions include a training processing function and a service processing function.

[0034] Step S104: Embed the above-mentioned training parameters and training processing functions into the training code template to generate the above-mentioned training code file, wherein the above-mentioned training code template is obtained based on the above-mentioned configuration file;

[0035] Step S106: Obtain the inference code and service architecture framework corresponding to the above configuration file, embed the above inference code, the above service architecture framework and the above service processing function into the service code template, and generate the above service code file;

[0036] Step S108: Generate the training pipeline model based on the training code file and the service code file.

[0037] In this embodiment of the disclosure, a pre-set configuration file, training processing function, and service processing function are obtained, and corresponding training parameters and training code templates are generated according to the configuration file; multiple training parameters are embedded into the training code template to generate the training code file; inference code and service architecture framework corresponding to the configuration file are obtained, and the inference code, service architecture framework, and service processing function are embedded into the service code template to generate the service code file; finally, the training pipeline model is generated based on the training code file and the service code file.

[0038] It should be noted that the above configuration file can be a default configuration file pre-generated by the system, or it can be pre-set by a user. For example, the above configuration file is used to determine which template to use for the subsequent pipeline model and how to set some hyperparameters. The above configuration file includes some required fields, such as: what training template to use, data processing functions, etc. These required fields can be filled in through the visualization page. In the visualization page, each required field has fixed options. After selecting the corresponding option, the selected options will automatically generate the final configuration file; or, when filling in the above required fields, you can also manually write them according to the format, and the final configuration file will be generated based on the writing result.

[0039] Optionally, the training parameters mentioned above include hyperparameters such as learning rate and number of training epochs. The training template contains fields for these parameters, and the integration module will directly fill in these hyperparameters in the corresponding positions. Some hyperparameters may not be specified in the configuration file. When these hyperparameters are not explicitly specified, the integration module can directly fill in the pre-written training parameters in the corresponding positions.

[0040] In the embodiments disclosed herein, such as Figure 2 The diagram showing the training pipeline model generation process breaks down the original complete training pipeline model into several major modules, namely the training processing module, training module, service deployment module, data processing module, and integration module. In addition to these modules, a configuration template is also included, which users can use to fill in the above configuration file.

[0041] It should be noted that the data processing module and training processing module are mainly used for data preprocessing and postprocessing during algorithm training and prediction. They include various commonly used preprocessing and postprocessing functions, and also allow for the creation of custom output processing functions.

[0042] In an optional embodiment, the above-described data processing function includes:

[0043] Step S202: Obtain the training processing function and the service processing function from the pre-set training processing module.

[0044] In this embodiment of the disclosure, both the training processing function and the service processing function are obtained from a pre-set training processing module.

[0045] It should be noted that the training processing module described above is used to store the training processing function and the service processing function generated by the integration module. As an optional embodiment, when selecting a corresponding data processing function, the configuration file will provide data processing function usage information, and the integration module will embed it into the reserved location based on the data processing function usage information provided in the configuration file.

[0046] In an optional embodiment, before embedding the above-mentioned training parameters and training processing functions into the training code template to generate the above-mentioned training code file, the method further includes:

[0047] Step S302: The training module provides multiple training code templates for generating the above training pipeline model, wherein the above training code includes: deep learning task training code;

[0048] Step S304: Select the training code template corresponding to the above configuration file from multiple training code templates.

[0049] In this embodiment of the disclosure, multiple training code templates in the training module are obtained, and the integration module reads the configuration file. Based on the configuration file, the training code template corresponding to the configuration file is selected from the multiple training code templates.

[0050] It's important to note that the training module contains model training code, including training code for common deep learning tasks such as image processing, image recognition, object detection, and object classification. This code is used as a template to generate the final training code when creating the training pipeline model. A complete training code file includes the data processing section, and the training code template includes the interfaces for this section. For most training tasks, the data processing part consists of several common processing functions.

[0051] In an optional embodiment, obtaining the inference code and service architecture framework corresponding to the configuration file includes:

[0052] Step S402: Obtain the inference code and service architecture framework corresponding to the above configuration file from the service deployment module. The service deployment module is used to provide the corresponding inference code and service architecture framework for generating the above training pipeline model. The inference code is used to provide model inference services for the above training pipeline model, and the service architecture framework is used to build training services for the above training pipeline model.

[0053] In this embodiment of the disclosure, the integration module obtains the inference code and service architecture framework corresponding to the configuration file from the service deployment module based on the configuration file, and then selects post-processing functions from the data processing module to form a service code file.

[0054] It should be noted that the above service deployment module is used to provide the corresponding inference code and service architecture framework for generating the above training pipeline model. The inference code is used to provide model inference services for the above training pipeline model, and the service architecture framework is used to build training services for the above training pipeline model. The above service deployment module includes inference code and service architecture framework for some common tasks, which are used to build the final service after the model is trained.

[0055] In an optional embodiment, before generating the training code file described above, the method further includes:

[0056] Step S502: Obtain the pre-configured model encryption / decryption module and authentication module;

[0057] Step S504: Embed the above-mentioned model encryption / decryption module and the above-mentioned authentication module into the above-mentioned training code file.

[0058] In this embodiment of the disclosure, a pre-configured model encryption / decryption module and authentication module are obtained; the model encryption / decryption module and authentication module are embedded into the training code file, and the model encryption / decryption and authentication modules are also embedded into the training model to generate the final training code file.

[0059] In an optional embodiment, before generating the aforementioned service code file, the method further includes:

[0060] Step S602: Obtain the pre-configured model encryption / decryption module and authentication module;

[0061] Step S604: Embed the above-mentioned model encryption / decryption module and the above-mentioned authentication module into the above-mentioned service code file.

[0062] In this embodiment of the disclosure, a pre-configured model encryption / decryption module and an authentication module are obtained; the model encryption / decryption module and the authentication module are embedded into the service code file to generate the final service code file.

[0063] It should be noted that the above-mentioned model encryption / decryption module and authentication module are both provided by the above-mentioned integration module; the above-mentioned integration module can also be used to generate encryption / decryption code, authentication code, inference code, etc.

[0064] In an optional embodiment, generating the training pipeline model based on the training code file and the service code file includes:

[0065] Step S702: Strengthen the above training code file and the above service code file to generate a strengthened code file;

[0066] Step S704: Use the reinforced code file to generate the training pipeline model.

[0067] In this embodiment, after generating the training code file and the service code file, the integration module calls the code hardening module to harden the code, generating unreadable encrypted code to make the file unreadable. The training pipeline model is then generated using the hardened code file. When using the model, the trained model needs to be placed in the designated location of the service code.

[0068] It should be noted that the above-mentioned training model refers to the trained model weights and other files. The training code will store the trained model in a fixed folder, and the service code will specify a folder to read the model weights and other files. You can simply copy the trained model weights and other files to the folder specified by the service code.

[0069] This disclosure discloses a technical solution that involves obtaining a pre-set configuration file and data processing functions. The configuration file includes at least one training parameter for generating a training pipeline model, and the data processing functions include training processing functions and service processing functions. Multiple training parameters and training processing functions are embedded into a training code template to generate a training code file, where the training code template is based on the configuration file. Inference code and a service architecture framework corresponding to the configuration file are obtained, and the inference code, service architecture framework, and service processing functions are embedded into a service code template to generate a service code file. The training pipeline model is then generated based on the training code file and the service code file. This technical solution templates and modularizes the code, enabling the automatic generation of the entire training pipeline code from a single configuration file. This reduces development difficulty, minimizes repetitive work, improves delivery efficiency, integrates training and service code directly generated from modules, facilitates management, eliminates redundant functions, and is more targeted. When existing templates cannot meet the requirements, the functions and templates of each module can be expanded through customization, minimizing development effort and allowing for easy reuse later.

[0070] According to embodiments of this disclosure, an apparatus embodiment for implementing the above-described method for generating training pipeline models is also provided. Figure 3 This is a schematic diagram of the structure of the apparatus for generating a training pipeline model according to the second embodiment of this disclosure, as shown below. Figure 3As shown, the above-mentioned training pipeline model generation device includes: an acquisition module 40, a first generation module 42, a second generation module 44, and a processing module 46, wherein:

[0071] The acquisition module 40 is used to acquire a pre-set configuration file and data processing functions, wherein the configuration file includes at least one training parameter for generating a training pipeline model, and the data processing functions include a training processing function and a service processing function.

[0072] The first generation module 42 is used to embed multiple training parameters and training processing functions into a training code template to generate the training code file, wherein the training code template is obtained based on the configuration file.

[0073] The second generation module 44 is used to obtain the inference code and service architecture framework corresponding to the above configuration file, embed the above inference code, the above service architecture framework and the above service processing function into the service code template, and generate the above service code file.

[0074] Processing module 46 is used to generate the training pipeline model based on the training code file and the service code file.

[0075] In this embodiment of the disclosure, the above-mentioned acquisition module includes: a first acquisition unit, configured to acquire the above-mentioned training processing function from a pre-set training processing module, and to acquire the above-mentioned service processing function from a pre-set training processing module.

[0076] In this embodiment of the disclosure, the first generation module includes: a supply unit, configured to provide multiple training code templates for generating the training pipeline model using a training module, wherein the training code includes deep learning task training code; and a selection unit, configured to select the training code template corresponding to the configuration file from the multiple training code templates.

[0077] In this embodiment of the disclosure, the second generation module includes: a second acquisition unit, configured to acquire from the service deployment module the inference code and the service architecture framework corresponding to the configuration file, wherein the service deployment module is configured to provide the corresponding inference code and the service architecture framework for generating the training pipeline model, the inference code is configured to provide model inference services for the training pipeline model, and the service architecture framework is configured to build training services for the training pipeline model.

[0078] In this embodiment of the disclosure, the first generation module further includes: a third acquisition unit, used to acquire a pre-configured model encryption / decryption module and an authentication module; and a first embedding unit, used to embed the model encryption / decryption module and the authentication module into the training code file.

[0079] In this embodiment of the disclosure, the second generation module further includes: a fourth acquisition unit, used to acquire a pre-configured model encryption / decryption module and authentication module; and a second embedding unit, used to embed the model encryption / decryption module and authentication module into the service code file.

[0080] In this embodiment of the disclosure, the above-mentioned processing module includes: a hardening unit, used to harden the above-mentioned training code file and the above-mentioned service code file to generate a hardened code file; and a generation unit, used to generate the above-mentioned training pipeline model using the above-mentioned hardened code file.

[0081] It should be noted that the above modules can be implemented by software or hardware. For example, for the latter, it can be implemented in the following ways: the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0082] It should be noted that the acquisition module 40, the first generation module 42, the second generation module 44, and the processing module 46 mentioned above correspond to steps S102 to S108 in Embodiment 1. The instances and application scenarios implemented by the above modules and their corresponding steps are the same, but they are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules, as part of the device, can run on a computer terminal.

[0083] It should be noted that the optional or preferred implementation methods of this embodiment can be found in the relevant description in Embodiment 1, and will not be repeated here.

[0084] The above-mentioned training pipeline model generation device may also include a processor and a memory. The above-mentioned acquisition module 40, first generation module 42, second generation module 44 and processing module 46 are all stored in the memory as program units, and the processor executes the above-mentioned program units stored in the memory to realize the corresponding functions.

[0085] The processor contains a core that retrieves corresponding program units from memory. One or more cores may be configured. Memory may include non-persistent memory in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory includes at least one memory chip.

[0086] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0087] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0088] Figure 4 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0089] like Figure 4 As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0090] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0091] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as methods for obtaining pre-set configuration files and data processing functions. For example, in some embodiments, methods for obtaining pre-set configuration files and data processing functions may be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the methods described above for obtaining pre-set configuration files and data processing functions may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured by any other suitable means (e.g., by means of firmware) to execute a method to obtain a pre-set configuration file and data processing functions.

[0092] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0093] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0094] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0095] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0096] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0097] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0098] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0099] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A generating method of training a pipeline model, wherein, The method comprises the following steps: obtaining a preset configuration file and a data processing function, wherein the configuration file comprises at least one training parameter for generating a training pipeline model and usage information of the data processing function, the configuration file is generated based on mandatory fields of a visualization page, and the data processing function comprises a training processing function and a service processing function; using a training module to provide a plurality of training code templates for generating the training pipeline model, wherein the training code comprises: deep learning task training code applied to image processing scenarios, image recognition scenarios, target detection scenarios, and object classification scenarios; selecting a training code template corresponding to the configuration file from the plurality of training code templates; embedding a plurality of the training parameters and the training processing function into the training code template corresponding to the configuration file to generate a training code file; obtaining inference code and a service architecture framework corresponding to the configuration file, embedding the inference code, the service architecture framework, and the service processing function into a service code template to generate a service code file; generating the training pipeline model based on the training code file and the service code file; wherein the data processing function is obtained by: obtaining the training processing function from a preset training processing module, and obtaining the service processing function from a preset training processing module.

2. The method of claim 1, wherein, The method further comprises the following steps before generating the training code file: obtaining a pre-configured model encryption and decryption module and an authentication module; 3. The method of claim 1, wherein, embedding the model encryption and decryption module and the authentication module into the training code file. The method further comprises the following steps before generating the service code file: obtaining a pre-configured model encryption and decryption module and an authentication module; 4. The method of claim 1, wherein, embedding the model encryption and decryption module and the authentication module into the service code file. The method further comprises the following steps before generating the training pipeline model based on the training code file and the service code file: performing reinforcement processing on the training code file and the service code file to generate a reinforced code file; 5. The method of claim 1, wherein, generating the training pipeline model using the reinforced code file. The method comprises the following steps: obtaining a preset configuration file and a data processing function, wherein the configuration file comprises at least one training parameter for generating a training pipeline model and usage information of the data processing function, the configuration file is generated based on mandatory fields of a visualization page, and the data processing function comprises a training processing function and a service processing function; 6. A generating apparatus of training a pipeline model, wherein, ​ ​ The first generation module comprises: a supply unit configured to provide a plurality of training code templates for generating the training pipeline model by using the training module, wherein the training code comprises: deep learning task training code applied to an image processing scene, an image recognition scene, a target detection scene, and an object classification scene; and a selection unit configured to select a training code template corresponding to the configuration file from the plurality of training code templates; The first generation module is further configured to embed the plurality of training parameters and the training processing function into the training code template corresponding to the configuration file to generate a training code file; The second generation module is configured to obtain inference code and a service architecture framework corresponding to the configuration file, embed the inference code, the service architecture framework, and the service processing function into a service code template to generate a service code file; The processing module is configured to generate the training pipeline model based on the training code file and the service code file. The obtaining module comprises: a first obtaining unit configured to obtain the training processing function from a pre-set training processing module and obtain the service processing function from a pre-set training processing module.

7. The apparatus of claim 6, wherein, The second generation module comprises: A second obtaining unit configured to obtain the inference code and the service architecture framework corresponding to the configuration file from a service deployment module, wherein the service deployment module is configured to provide the inference code and the service architecture framework corresponding to the training pipeline model, the inference code is configured to provide model inference services for the training pipeline model, and the service architecture framework is configured to build training services for the training pipeline model.

8. The apparatus of claim 6, wherein, The first generation module further comprises: A third obtaining unit configured to obtain a pre-configured model encryption and decryption module and an authentication module; A first embedding unit configured to embed the model encryption and decryption module and the authentication module into the training code file.

9. The apparatus of claim 6, wherein, The second generation module further comprises: A fourth obtaining unit configured to obtain a pre-configured model encryption and decryption module and an authentication module; A second embedding unit configured to embed the model encryption and decryption module and the authentication module into the service code file.

10. The apparatus of claim 6, wherein, The processing module comprises: A reinforcement unit configured to perform reinforcement processing on the training code file and the service code file to generate a reinforced code file; A generation unit configured to generate the training pipeline model by using the reinforced code file.

11. An electronic device comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method for generating a training pipeline model according to any one of claims 1-5.

12. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method for generating a training pipeline model according to any one of claims 1-6.

13. A computer program product comprising a computer program which, when executed by a processor, implements the method of claim 1-5 for generating a training pipeline model.

Images