Pre-training service system and service providing method based on the pre-training service system

Abstract

One or more embodiments of the present application provide a pre-training service system and a service providing method based on the pre-training service system, in which a model maintained in the pre-training service system includes an obfuscated model structure code and an obfuscated model parameter, and the obfuscated model structure code and the obfuscated model parameter are generated after previously performing an associated obfuscation process on an initial model structure code and an initial model parameter, and the system includes: a producer service module for providing a model producer to generate a pre-trained model; an optimizer service module for optimizing the pre-trained model based on a fine-tuning dataset provided by a model optimizer to obtain an optimized model; and a consumer service module for providing a service interface for the pre-trained model and / or the optimized model to a model consumer, in which the pre-trained model or the optimized model infers data provided by the model consumer and outputs a model prediction result.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims priority to a Chinese patent application filed with the China Patent Office on April 6, 2022, bearing application number 202210353465.X and entitled "Pre-training service system and service provision method based on pre-training service system", the entire contents of which are incorporated herein by reference.

[0002] One or more embodiments of the present application relate to the field of deep learning, and in particular to a pre-training service system and a service providing method based on the pre-training service system. [Background technology]

[0003] In the background of the rapid development of artificial intelligence technology, various large-scale pre-trained models have appeared according to the times, and the comprehension, generation and search capabilities of pre-trained models have been applied to various industries with remarkable success. However, the conditions for using pre-trained models are high, so how to reduce the difficulty of use and provide a platform that allows more users to operate pre-trained models with simple operations is an issue that needs to be resolved immediately. In addition, because pre-trained models have extremely strong comprehension, generation and search capabilities, how to avoid the misuse of pre-trained models is also a possible issue. Summary of the Invention [Problem to be solved by the invention]

[0004] In light of this, one or more embodiments of the present application provide a pre-training services system. [Means for solving the problem]

[0005] To achieve the above objectives, one or more embodiments of the present application provide the following technical solutions:

[0006] According to a first aspect of one or more embodiments of the present application, there is provided a pre-training service system, wherein a model maintained by the pre-training service system includes an obfuscated model structure code and an obfuscated model parameter, and the obfuscated model structure code and the obfuscated model parameter are generated after previously performing a related obfuscation process on an initial model structure code and an initial model parameter, and the pre-training service system: a producer service module for providing a model producer with pre-training capabilities for a model on a pre-training dataset and generating a corresponding pre-trained model; an optimizer service module for optimizing the pre-trained model based on a fine-tuning dataset provided by a model optimizer to obtain an optimized model; and a consumer service module for providing a service interface for the pre-trained model and / or the optimized model to a model consumer, wherein the pre-trained model or the optimized model is configured to infer data provided by the model consumer and output a model prediction result.

[0007] According to a second aspect of one or more embodiments of the present application, there is provided a service providing method based on a pre-training service system, the pre-training service system including a producer service module, an optimizer service module and a consumer service module, a model maintained by the pre-training service system includes an obfuscated model structure code and an obfuscated model parameter, the obfuscated model structure code and the obfuscated model parameter are generated after previously performing a related obfuscation process on an initial model structure code and an initial model parameter, the method including: Obtaining identity information of a target user who accesses the pre-training service system, and determining a pre-defined user type to which the target user belongs; Opening a service module that matches the predefined user type to which the target user belongs, wherein: If the target user is a model producer, open the producer service module to the target user, the producer service module being used to provide the model producer with a pre-training function for a model on a pre-training dataset and generate a corresponding pre-trained model; If the target user is a model optimizer, open the optimizer service module to the target user, and the optimizer service module is used to optimize the pre-trained model based on a fine-tuning dataset provided by the model optimizer to obtain an optimized model; If the target user is a model consumer, opening the consumer service module to the target user, the consumer service module is used to provide the model consumer with a service interface for the pre-trained model and / or the optimized model, and the pre-trained model or the optimized model is used to infer data provided by the model consumer and output model prediction results.

[0008] According to a third aspect of one or more embodiments of the present application, there is provided a service providing apparatus based on a pre-training service system, the pre-training service system including a producer service module, an optimizer service module and a consumer service module, a model maintained by the pre-training service system includes an obfuscated model structure code and an obfuscated model parameter, the obfuscated model structure code and the obfuscated model parameter are generated after previously performing a related obfuscation process on an initial model structure code and an initial model parameter, and the apparatus: an acquisition unit for acquiring identity information of a target user who accesses the pre-training service system and determining a pre-defined user type to which the target user belongs; an opening unit for opening a service module matching a preset user type to which the target user belongs, comprising: If the target user is a model producer, open the producer service module to the target user, the producer service module being used to provide the model producer with a pre-training function for a model on a pre-training dataset and generate a corresponding pre-trained model; If the target user is a model optimizer, open the optimizer service module to the target user, and the optimizer service module is used to optimize the pre-trained model based on a fine-tuning dataset provided by the model optimizer to obtain an optimized model; and an opening unit for opening the consumer service module to the target user when the target user is a model consumer, the consumer service module being used to provide the model consumer with a service interface for the pre-trained model and / or the optimized model, and the pre-trained model or the optimized model being used to infer data provided by the model consumer and output model prediction results.

[0009] According to a fourth aspect of one or more embodiments of the present application, there is provided an electronic device, comprising: a processor; a memory; and a non-volatile memory; The processor is adapted to read a corresponding computer program from the non-volatile memory into the memory, and to execute the method according to the second aspect based on the computer program.

[0010] According to a fifth aspect of one or more embodiments of the present application, there is provided a computer-readable storage medium storing a computer program, the computer program being characterized in that, when executed by a processor, the computer program implements the method according to the second aspect above.

[0011] According to a sixth aspect of one or more embodiments of the present application, there is provided a computer program product comprising a computer program which, when executed, causes a computer to perform the method according to the second aspect above. Effect of the Invention

[0012] In the technical solution provided in the present application, a pre-training service system including a producer service module, an optimizer service module and a consumer service module provides different types of services to users with different statuses, reduces the difficulty of using a pre-trained model, improves the versatility of the pre-trained model, provides a model optimizer who does not have the ability to generate a pre-trained model with the ability to quickly and easily obtain an optimized model based on the pre-trained model, and further provides a service interface for a model consumer, through which the model consumer can quickly use and reason on the pre-trained model or the optimized model to obtain a model prediction result. And the model structure code and model parameters provided by the pre-training service system are obfuscated model structure code and obfuscated model parameters, which are functionally similar to the initial model structure code and initial model parameters, but the obfuscated model structure code makes it difficult to reproduce the pre-trained model, and the obfuscated structure code and obfuscated model parameters can be used to avoid the abuse of the pre-trained model, thereby achieving the effect of protecting the pre-trained model. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a structural schematic diagram of a pre-training service system provided by an exemplary embodiment of the present application; [Diagram 2] FIG. 2 is a schematic diagram of a flowchart of a service providing method based on a pre-training service system provided by an exemplary embodiment of the present application. [Diagram 3] 1 is a structural schematic diagram of an electronic device provided by one exemplary embodiment of the present application. [Figure 4] FIG. 1 is a schematic diagram of a service providing device based on a pre-training service system provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, exemplary embodiments will be described in detail based on the drawings. When the following description refers to the drawings, the same reference numbers in different drawings indicate the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to include all embodiments consistent with one or more embodiments of the present application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of one or more embodiments of the present application, as detailed in the appended claims.

[0015] It should be noted that other embodiments may not necessarily perform corresponding method steps in the order shown and described herein. In some other embodiments, the method may include more or less steps than those described herein. Also, a single step described herein may be decomposed and described as multiple steps in other embodiments, but multiple steps described herein may be integrated and described as a single step in other embodiments.

[0016] Due to the complexity of the application scenarios of artificial intelligence (AI) technology, there are differences in the aspects of task goals, training data, problem setting, etc. in different application scenarios. Therefore, by providing users with common pre-trained models, users can fine-tune based on the pre-trained models to obtain application modes of optimized models that match specific application scenarios, thereby reducing the demand for training data, improving training efficiency, and providing more small and medium-sized enterprises with the ability to obtain optimized models that match specific application scenarios, thereby promoting the application and popularization of AI technology. Moreover, users can select different pre-trained models or optimized models for specific service scenarios according to application scenarios, and use their own data to make inferences, thereby obtaining model output results based on the selected pre-trained models or optimized models.

[0017] In order to facilitate providing services based on different types of pre-trained models to users with different types of identities, an exemplary embodiment of the present application provides a pre-training service system as shown in Fig. 1, which includes a producer service module 11, an optimizer service module 12 and a consumer service module 13. Among them, the producer service module 11 is used to provide a model producer with a model pre-training function for a pre-training dataset 111 to generate a corresponding pre-trained model 112, the optimizer service module 12 is used to optimize the pre-trained model 112 based on a fine-tuning dataset 121 provided by a model optimizer to obtain an optimized model 122, and the consumer service module 13 is used to provide a service interface for the pre-trained model 112 and / or the optimized model 122 to a model consumer, and the pre-trained model 112 or the optimized model 122 is used to reason on data 131 provided by the model consumer to output a model prediction result 132.

[0018] Among them, the above-mentioned model producer is an algorithm developer on the pre-training service system side and a producer of the pre-trained model 112. In one exemplary embodiment of the present application, the model producer launches distributed pre-training using the model pre-training function provided by the pre-training service system based on the pre-training dataset 111 to generate the pre-trained model 112, which is the basis for the pre-training service system to provide artificial intelligence related services, and other pre-trained models are all obtained after being modified by the pre-trained model 112.

[0019] In one exemplary embodiment of the present application, in the pre-training process of the pre-trained model 112, the pre-training function of the model realizes an activation checkpointing technique. Several checkpoints are set in the network, and some temporarily useless intermediate results are discarded in the downstream process to reduce memory occupation, and when these intermediate results are required in the subsequent process, the corresponding intermediate results are restored from the most recent checkpoint through additional downstream calculation, thus not only saving graphic memory, but also avoiding the cumbersome process of calculating from scratch, thereby reducing the graphic memory occupation caused by intermediate activation in the training process of the pre-trained model.

[0020] In addition, the pre-training process further adopts mixed precision training technology, and uses 16-bit floating point (FP16) to train the deep learning of the pre-trained model, thereby reducing the graphic memory required for training and improving the calculation speed. Moreover, the pre-training system further adopts technologies such as sparse expert parallelism and data parallelism to improve the calculation efficiency of the pre-training process, ensuring the low carbon and environmental protection of the pre-training process. Among them, the sparse expert parallelism technology is a sparse gate mixture of experts network (Mixture of Experts, MoE), which divides a large model into multiple small models, and each small model reserves independent parameters. In the model pre-training process, it is not necessary to calculate all the small models for one sample, but selects an appropriate small model according to a preset policy and selects and calculates the corresponding parameters. In this solution, the total number of pre-trained parameters is very high, but only a small part is activated, which saves the use of graphic memory in the calculation process.

[0021] In an exemplary embodiment of the present application, in order to make the capabilities of the generated pre-trained models 112 more diverse and more common, a multi-modal pre-training format is adopted in the pre-training process of the pre-trained models 112. Modality refers to the format in which data exists, such as file formats such as text, audio, image, video, etc.

[0022] In an exemplary embodiment of the present application, in the pre-training process of the pre-trained model 112, the pre-trained model 112 can use multiple tasks, including language modeling, text denoising, image captioning, image-based text denoising, etc. In addition, in the pre-training process, the scale and breadth of the field of the pre-training dataset 111 affect the ability and commonality of the finally obtained pre-trained model 112. Therefore, in an exemplary embodiment of the present application, in the pre-training process of the pre-trained model 112, a super-large unimodal and multimodal dataset covering multiple fields is used as the pre-training dataset 111 of the pre-trained model 112. By starting the multimodal pre-training and expanding the scale and field coverage of the pre-training dataset 111, the pre-trained model 112 has the abilities of unimodal and multimodal understanding and generation at the same time, and has good adaptability to tasks in different fields, thereby improving the commonality of the pre-trained model 112.

[0023] Corresponding to a model developer, a model optimizer is a user in another field who wants to obtain an optimized model 122 suitable for the field he or she belongs to based on the pre-trained model 112. In one exemplary embodiment of the present application, the pre-training service system as shown in FIG. 1 is an optimizer service module 12 provided by a service optimizer, which optimizes the pre-trained model 112 generated by the producer service module 11 based on a fine-tuning dataset 121 provided by a model optimizer to obtain an optimized model 122.

[0024] In one exemplary embodiment of the present application, the pre-training function of the model provided by the pre-training service system is used to train based on the pre-training dataset 111 to obtain a common pre-trained model. Then, the producer service module generates a dedicated pre-trained model corresponding to the downstream task instructed by the model producer based on the common pre-trained model. That is, the model producer instructs the pre-training service system based on the atomic capabilities of the common pre-trained model to generate a dedicated pre-trained model that has a dedicated purpose and is applicable to a certain type of downstream task based on the common pre-trained model. Among them, the atomic capabilities of the common pre-trained model include cross-modal generation capabilities, cross-modal understanding capabilities, and cross-modal retrieval capabilities, etc. Accordingly, the dedicated pre-trained model can include a cross-modal generation model, a cross-modal understanding model, and a cross-modal retrieval model, etc. The pre-trained model 112 includes a common pre-trained model and a dedicated pre-trained model.

[0025] In one exemplary embodiment of the present application, the producer service module 11 in the pre-training service system is further used to generate a code development template corresponding to the dedicated pre-trained model, which is provided to the model optimizer by the optimizer service module 12 in the pre-training service system.

[0026] Any pre-trained model mainly includes two parts: model structure code and model parameters, where the model structure code is the code for constructing the pre-trained model, the model can be regarded as an assumption, the model parameters are specific adjustments to the assumption based on a specific data set, and the model parameter values ​​can define the model function.

[0027] The model optimizer refers to the code development template and issues a model modification command to the pre-training service system according to its own optimization requirements. Specifically, the optimizer service module 12 in the pre-training service system is a model modification interface provided by the model optimizer, the model optimizer refers to the code development template and uses the model modification interface to issue a model modification command for the model structure code and / or model parameters of the dedicated pre-trained model, the optimizer service module modifies the corresponding model structure code and / or model parameters according to the model modification command, and then launches training for the modified pre-trained model according to the fine-tuning dataset 121 provided by the model optimizer to obtain an optimized model.

[0028] In one exemplary embodiment of the present application, for a model optimizer who wants to quickly try out a pre-trained model, the data (fine-tuned data set) in the field can be utilized by simply modifying the exemplary parameters provided by the code development template, and the pre-training service system launches a distributed data parallel distributed training task to generate optimized model parameters belonging to the model optimizer. The optimized model parameters can be combined with the model structure code of the pre-trained model to generate an optimized model.

[0029] In another exemplary embodiment of the present application, if the model optimizer wants to further optimize the pre-trained model, the pre-training service system can further provide an exemplary example of optimizing the pre-trained model for use in other fields, and the optimizer can issue modification instructions to the pre-trained model according to his / her requirements by referring to the above exemplary example and input the fine-tuning dataset of the present field, and the pre-training service system will launch a distributed training task and generate corresponding optimized model parameters, thereby obtaining the optimized model.

[0030] The optimized model is trained based on the fine-tuning data set provided by the model optimizer, and the model optimizer can optimize and fine-tune the model according to his / her own requirements, so that the optimized model is applicable to the field to which the model optimizer belongs. In addition, in the process of using the pre-training service system to perform model optimization, the model optimizer does not need to pay attention to specific implementation details such as data IO, data parallelism, etc., such as the development process of the pre-trained model, the distributed training of the pre-trained model, and the evaluation flow of the pre-trained model, but can obtain an optimized model applicable to the industry to which the model optimizer belongs by simply modifying the corresponding model structure code or model parameters based on the code development template provided by the pre-training service system and launching model training. The pre-training service system reduces the difficulty for users to optimize the pre-trained model, and model optimizers who do not have the ability to develop pre-trained models can also optimize the model through the pre-training service system.

[0031] In one exemplary embodiment of the present application, mature industry solutions (such as copywriting generation, search recall, clothing design, smart dialogue, financial question answering, etc.) can be provided as test examples for other users (including other developers) to refer to through the pre-training service system. In addition to referring to the code development template to send modification instructions to the pre-trained model, the model optimizer can also refer to the industry solutions to modify the model.

[0032] In an exemplary embodiment of the present application, the pre-training service system further provides a sharing interface, and the model optimizer can upload the optimization process for the pre-trained model to the pre-training service system through the sharing interface and share it as a new test case, which can be provided to other model optimizers for optimization reference. The case sharing function can expand the application scope of the pre-trained model, and further reduce the difficulty of using the pre-training service system.

[0033] In this application, the user of the pre-training service system further includes a model consumer, and the model consumer is a user who uses the pre-trained model or the optimized model to make inferences using data belonging to the user and obtain model prediction results. The pre-training service system provides the model consumer with a service interface for the pre-trained model and / or the optimized model through a consumer service module, and the pre-trained model or the optimized model is used to infer data provided by the model consumer and output model prediction results. As shown in FIG. 1, the model consumer can call the pre-trained model 112 or the optimized model 122 to make inferences. The above inference process may be offline, and the pre-training service system is used to place the pre-trained model 112 or the optimized model 122 selected by the model consumer on a device corresponding to the model consumer through the service interface, and perform offline inferences on the data provided by the model consumer. Of course, the above-mentioned pre-trained model 112 or optimized model 122 can be deployed as a low-latency online service, in which consumers directly input their own data 131 into a web page provided by the pre-training service system, which corresponds to a service interface provided by the pre-training service system. After obtaining the model consumer's data 131, the corresponding pre-trained model 112 or optimized model 122 is called to perform online reasoning, and model prediction results are obtained, and the model prediction results are provided to the model consumer in the form of a web page.

[0034] In one exemplary embodiment of the present application, the pre-training service system can provide users with multiple different unimodal or cross-modal artificial intelligence services based on the cross-modal generation, cross-modal search, and cross-modal understanding capabilities of the pre-trained model.

[0035] For example, a news event can be displayed in multiple formats, such as text, image, or video, so when performing information search, if only the data of a single modality is searched, the acquired information content will be greatly limited, and extensive recognition of the same object or event is required from the data of other modalities, and at this time, it is necessary to realize the search between data of different modalities through cross-modal search. In one exemplary embodiment of the present application, the pre-training service system can obtain the video of the news event based on the cross-modal search ability of the pre-training model according to the news information in the form of a picture input by the user.

[0036] In one exemplary embodiment of the present application, the pre-training service system can provide a model consumer with a clothing design service based on the cross-modal generation ability of the pre-training model. For example, the model consumer can input "men's striped shirt" through a web page provided by the pre-training service system, and generate some clothing design pictures that match the text description of "men's striped shirt" by calculating the target pre-trained model in the field of clothing design provided by the pre-training service system. The above example utilizes the cross-modal generation ability of the pre-training model. Of course, the pre-training service system provides a cross-modal artificial intelligence service to the user, but the pre-training service system also has the ability to provide an artificial intelligence service based on unimodality. For example, when a consumer performs the following clothing design operation through the offline reasoning function of the target pre-trained model in the field of clothing design provided by the pre-training service system, the model consumer can input two clothing pictures with different styles, and after performing offline reasoning, the model consumer can obtain the result of offline reasoning, that is, the clothing picture obtained after fusing two clothing with different styles.

[0037] In addition to the function of clothing design, the pre-training service system can also use the cross-modal search, generation and understanding capabilities of the pre-trained model to perform training reasoning related to character processing, such as copywriting generation. For example, in the reasoning process for a target pre-trained model in the field of character processing, a picture of a latex pillow is input, and when the target pre-trained model in the field of character processing provided by the pre-training service system is used for reasoning, the cross-modal understanding capability of the pre-trained model can be used to obtain the advertising copywriting for the latex pillow. In addition, the pre-training service system can also use the above capabilities of the pre-trained model to realize text-to-text conversion, and perform text noise removal and language modeling in the conversion process.

[0038] As can be seen from the above introduction to the pre-training service system, in the service process of the pre-training service system, the pre-training model plays a very important role. Users can call up a target pre-trained model through the pre-training service system. Since the pre-trained model has very strong text and image generation capabilities, in order to avoid the misuse of the pre-trained model, the pre-training service system further sets up several defense mechanisms to prevent the misuse of the pre-trained model, and the defense mechanisms are also applied to avoid the misuse of the optimized model.

[0039] As can be seen from the above introduction, the pre-trained model or the optimized model includes two parts: model structure code and model parameters. For the model structure code, based on the common framework and other materials in the industry, the user may reproduce the complete model structure code backwards, and combine the reproduced model structure code and the model parameters obtained by other means to obtain the pre-trained model or the optimized model. This is disadvantageous to the management control applied to the pre-trained model or the optimized model. Because the model parameters define the function of the model, the protection of the model parameters and the protection of the model structure code are equally important to the pre-training service system. When protecting the entire pre-trained model, not only the protection of the model structure code needs to be considered, but also the protection of the model parameters needs to be considered.

[0040] In one exemplary embodiment of the present application, the pre-trained model and / or the optimized model includes an obfuscated model structure code and an obfuscated model parameter, in which the obfuscated model structure code and the obfuscated model parameter are generated after performing a related obfuscation process on the initial model structure code and the initial model parameter in advance. By associating and obfuscating the model structure code and the model parameter, the obfuscated model structure code and the obfuscated model parameter have equivalent functions to the initial model structure code and the initial model parameter, but are not reproduced formally, thereby achieving the effect of avoiding the pre-trained model from being exploited. The model parameter value can define the model function.

[0041] Specifically, the related obfuscation process can be realized in the following form: In performing a normal obfuscation operation (usually by removing annotations, injecting irrelevant code, renaming functions, classes, variables, etc., to reduce the readability of the code and improve the cracking conditions, but ensure that the execution result is consistent with that before obfuscation) on at least a part of the code in the initial model structure code of the pre-trained model, modify at least one calculation process in the model structure code, and at the same time modify some parameters corresponding to the calculation process in the model parameters, so that the execution result of the obfuscated pre-trained model constructed by combining the modified obfuscated model structure code and the corresponding modified obfuscated model parameters is consistent with the initial pre-trained model before obfuscation. Since the obfuscation process is combined with the modification adjustment to the model parameters, it is difficult for others to locate the specific position where obfuscation occurs, and the above method further improves the difficulty of reproducing the model structure code, thereby realizing protection for the model structure code.

[0042] Modifying at least one calculation process in the model structure code includes performing a forward obfuscation process on at least a portion of the unmodified initial model structure code, and performing a reverse obfuscation process on model parameters related to the modified portion of the initial model parameters, thereby increasing the difficulty of reproducing the pre-trained model without changing the execution result of the pre-trained model.

[0043] For example, the forward obfuscation process may include subtracting a vector during the calculation of the model structure code in performing a normal obfuscation operation on at least a portion of the code in the initial model structure code of the pre-trained model, and the corresponding reverse obfuscation process may include adding the vector to a corresponding portion of the model parameters to compensate for the difference, so that the execution result of the obfuscated pre-trained model matches the initial pre-trained model before obfuscation.

[0044] In order to further improve the security of the pre-training model, in one exemplary embodiment of the present application, the model parameters are stored in a storage space managed by a pre-training service system, and when a target user calls for a target pre-trained model or a target optimized model, the corresponding model parameters are obtained from the storage space according to a reading path indicated in the call request of the target user. Wherein, the storage space may be a target storage (cloud storage) managed by the pre-training service system, or the storage space may be directly provided by the pre-training service system. The model parameters may be obfuscated model parameters. After obtaining the obfuscated model parameters, the pre-training service system combines them with the corresponding obfuscated model structure code to obtain the target pre-trained model or the target optimized model.

[0045] The users of the pre-training service system have different identities, including model developers, model optimizers and model consumers. Different types of users have different needs, which use the pre-training service system to complete different tasks, so if the pre-training service system opens all usage rights to all types of users, it is disadvantageous to the protection of the pre-trained model.

[0046] In order to provide corresponding model usage rights based on user identity, the pre-training service system in this application further provides the following services:

[0047] an optimizer service module is further used to verify whether the model optimizer has the authority to call the obfuscated model parameters corresponding to the pre-trained model, and if the model optimizer has the authority to call the obfuscated model parameters, combine the obfuscated model parameters with the obfuscated model structure code to obtain the pre-trained model and provide it to the model optimizer; and / or The consumer service module is further used to verify whether the model consumer has authority to call obfuscated model parameters corresponding to the pre-trained model or the optimized model, and if the model consumer has authority to call the obfuscated model parameters, combine the obfuscated model parameters with the obfuscated model structure code to obtain the pre-trained model or the optimized model and provide it to the model consumer.

[0048] For example, the optimizer service module in the pre-training service system verifies that a model optimizer has the authority to call the model parameters of the selected target pre-training model, where the model parameters may be obfuscated model parameters. If the model optimizer has the authority to call the obfuscated model parameters, the system obtains the obfuscated parameter model and combines it with the corresponding obfuscated model structure code to obtain a target pre-training model. The target pre-training model is provided to the model optimizer for optimizing the target pre-training model. For the model consumer, the verification process is the same as that of the model optimizer.

[0049] Of course, in the process of running the pre-training service system, in addition to the model structure code and model parameters of the pre-trained model and the optimized model, the pre-training service system also runs a model access code and a model training reasoning code for any pre-trained model or optimized model to call and use the pre-trained model or the optimized model, and this part of the code is decoupled from the model structure code to obtain an independent model access code and a model training reasoning code, and the model access code and the model structure code are encrypted to avoid exposing the storage location of the pre-trained model or the optimized model or leaking the model structure code, thereby improving the security of the pre-training service system and avoiding the misuse of the pre-trained model or the optimized model. However, the model training reasoning code can be maintained in plain text by the pre-training service system.

[0050] In one exemplary embodiment of the present application, a service providing method based on a pre-training service system as shown in Figure 2 is provided. The method can be applied to a pre-training service system as shown in Figure 1. The pre-training service system can include a producer service module 11, an optimizer service module 12 and a consumer service module 13. A model maintained in the pre-training service system includes an obfuscated model structure code and an obfuscated model parameter, and the obfuscated model structure code and the obfuscated model parameter are generated after previously performing a related obfuscation process on an initial model structure code and an initial model parameter, and the method can include the following steps:

[0051] In S201, the identity information of a target user who accesses the pre-training service system is obtained, and a preset user type to which the target user belongs is determined.

[0052] For example, the pre-training service system can determine the identity information of the target user based on the login account of the target user, and determine the pre-defined user type to which the target user belongs. Or, the pre-training service system can determine the identity information of the target user based on a call request of the target user.

[0053] In S202, a service module that matches the target user with the preset user type that it belongs to is opened.

[0054] Wherein, if the target user is a model producer, the producer service module is opened to the target user, and the producer service module is used to provide the model producer with a model pre-training function for a pre-training dataset and generate a corresponding pre-trained model.

[0055] If the target user is a model optimizer, the optimizer service module is opened to the target user, and the optimizer service module is used to optimize the pre-trained model based on a fine-tuning dataset provided by the model optimizer to obtain an optimized model.

[0056] If the target user is a model consumer, the consumer service module is opened to the target user, and the consumer service module is used to provide the model consumer with a service interface for the pre-trained model and / or the optimized model, and the pre-trained model or the optimized model is used to infer data provided by the model consumer and output model prediction results.

[0057] Wherein, the model pre-training function is used to train based on a pre-training dataset to obtain a common pre-trained model, where the pre-trained model includes at least one of a common pre-trained model and a dedicated pre-trained model corresponding to a downstream task instructed by the model producer, which is generated by a producer service module based on the common pre-trained model.

[0058] In an exemplary embodiment of the present application, the service providing method based on the above pre-training service system further includes: generating a code development template corresponding to the dedicated pre-trained model; providing the code development template to the model optimizer; and modifying a model structure code and / or model parameters of the dedicated pre-trained model according to a model modification instruction issued by the model optimizer.

[0059] For example, in one exemplary embodiment of the present application, assume that a model optimizer (target user A) in the field of clothing design needs to modify based on a target pre-trained model, and obtain an optimized model for the target field, i.e., the field of clothing design, and assume that the target pre-trained model is a pre-trained model 112 as shown in FIG. 1. At this time, after the target user A in the field of clothing design logs into the pre-training service system, the target pre-training system obtains the identity information of the target user and verifies that the target user is a model optimizer. The pre-training service system opens the optimization service module 12 to the target user A. The target user A can refer to the code development template provided by the pre-training service system to modify the parameters of the pre-trained model 112, and use the data in the fine-tuning dataset 121 of the target user A belonging to the field of clothing design to train the modified pre-trained model, and obtain an optimized model 122. Or the target user A can choose to use the optimization authority granted by the pre-training service system, and issue a modification command through a modification interface provided by the pre-training service system to instruct the pre-training service system to modify the corresponding model structure code, and similarly use the fine-tuning data set to launch model training, and finally obtain an optimized model 122. The optimized model 122 is applied in the field of clothing design.

[0060] In another exemplary embodiment of the present application, when the pre-training service system determines that the user type of the target user B is a model consumer, the pre-training service system opens the consumer service module 13 to the target user B, and the consumer service module 13 is used to provide the model consumer with a service interface for the pre-trained model and / or the optimized model, in which the pre-trained model or the optimized model is used to infer data provided by the model consumer and output model prediction results. For example, when the target user B wants to obtain a clothing design means of a specific theme through the artificial intelligence service provided by the pre-training service system, the target user B requests to call a target optimized model in the field of clothing design, that is, the optimized model 122 obtained by optimizing the pre-trained model 112 using the developer target user A in the previous embodiment. The consumer service module 13 of the pre-training service system provides the target user B with a model usage right to infer and obtain model prediction results using the optimized model 122 through the service interface. Specifically, the target user B can use the reasoning service arranged online in the pre-training service system, input "men's striped shirt", call the optimized model 122 to perform online reasoning calculation, and obtain the model prediction result, for example, some clothing design pictures that match the text description of "men's striped shirt". Of course, the reasoning function can also be set as an offline function, for example, the optimized model 122 can be arranged on a device corresponding to the model consumer, i.e., user B, through the service interface provided by the consumer service module of the pre-training service system, to perform offline reasoning on the data provided by the model consumer B.

[0061] Similarly, model consumers can also call up optimized models in different fields, such as those obtained by model optimizers in the fields of character processing and finance using the same optimization methods as in the above embodiments, and use inference to obtain model prediction results in different fields.

[0062] After the model optimizer optimizes the target pre-training model through the pre-training service system, the model optimizer needs to provide the optimized model to other users for use. Therefore, in one exemplary embodiment of the present application, the pre-training service system further provides the function of storing the parameters of the optimized model generated by the pre-training service system in a storage space managed by the pre-training service system. The uploaded optimized model parameters can be called up according to the above-mentioned manner based on the reading path provided by the target user. For example, in the above-mentioned embodiment, the target user A obtains the optimized model 122 and then stores the model parameters in the storage space managed by the pre-training service system. When the target user B wants to call up the optimized model 122, the pre-training service system can call up the model parameters of the optimized model 122 according to the reading path provided by the user B. Of course, the above-mentioned model parameters may be obfuscated model parameters, and the optimized model 122 can be obtained by combining the obfuscated model parameters with the corresponding obfuscated model structure code. Wherein, the obfuscated model structure code and the obfuscated model parameters are generated after performing a related obfuscation process on the initial model structure code and the initial model parameters of the pre-optimized model 122. For specific methods of the above-mentioned related obfuscation process, please refer to the related introduction of the above-mentioned pre-training service system.

[0063] In one exemplary embodiment of the present application, the pre-training service system can verify whether a target user has the authority to call obfuscated model parameters, and if the target user has the authority to call the obfuscated model parameters, combine the obfuscated model parameters with the corresponding obfuscated model structure code to obtain a target pre-trained model or a target optimized model. The model parameters are usually stored in the form of a ckpt file, and in order to ensure that the ckpt file of the model parameters cannot be used independently apart from the pre-training service system, a license server is introduced to verify whether the target user has the authority to call the model parameters (ckpt file) provided by the pre-training service system, and if the target user has the authority, combine the obtained model parameters with the model structure code to obtain a target pre-trained model.

[0064] FIG. 3 is a structural schematic diagram of an electronic device according to an exemplary embodiment of the present application. Referring to FIG. 3, in terms of hardware, the device includes a processor 302, an internal bus 304, a network interface 306, a memory 308 and a non-volatile memory 310. Of course, other hardware required for other functions may be included. The processor 302 reads the corresponding computer program from the non-volatile memory 310 into the memory 308 and then executes it, forming a service providing device based on a pre-training service system in terms of logic. Of course, in addition to the software implementation, one or more embodiments of the present application do not exclude other implementation methods, such as a logic device or a combination of software and hardware, that is, the execution subject of the following process flow is not limited to each logic unit, but may be hardware or a logic device.

[0065] Corresponding to an embodiment of the above method, as shown in FIG. 4, the present application further provides a service providing apparatus based on a pre-training service system, the pre-training service system including a producer service module, an optimizer service module and a consumer service module, a model maintained by the pre-training service system includes an obfuscated model structure code and an obfuscated model parameter, the obfuscated model structure code and the obfuscated model parameter are generated after previously performing a related obfuscation process on an initial model structure code and an initial model parameter, and the apparatus: an acquiring unit 410 for acquiring identity information of a target user who accesses the pre-training service system and determining a pre-defined user type to which the target user belongs; an opening unit 420 for opening a service module matching a preset user type to which the target user belongs, comprising: If the target user is a model producer, open the producer service module to the target user, the producer service module being used to provide the model producer with a pre-training function for a model on a pre-training dataset and generate a corresponding pre-trained model; If the target user is a model optimizer, open the optimizer service module to the target user, and the optimizer service module is used to optimize the pre-trained model based on a fine-tuning dataset provided by the model optimizer to obtain an optimized model; and an opening unit 420 for opening the consumer service module to the target user when the target user is a model consumer, the consumer service module being used to provide the model consumer with a service interface for the pre-trained model and / or the optimized model, and the pre-trained model or the optimized model being used to infer data provided by the model consumer and output model prediction results.

[0066] Optionally, the model pre-training function is used to train based on the pre-training dataset to obtain a common pre-trained model, in which the pre-trained model includes at least one of the common pre-trained model and a dedicated pre-trained model corresponding to a downstream task specified by the model producer that is generated by the producer service module using the common pre-trained model.

[0067] Optionally, the service providing device based on the pre-training service system further comprises: a generation unit 430 for generating a code development template corresponding to said dedicated pre-trained model; a providing and modifying unit 440 for providing the code development template to the model optimizer, and modifying model structure code and / or model parameters of the dedicated pre-trained model based on model modification instructions issued by the model optimizer.

[0068] Optionally, the service providing device based on the pre-training service system further comprises: an offline deployment unit 450 for deploying the pre-trained model or the optimized model to a device corresponding to the model consumer via the service interface and performing offline inference on data provided by the model consumer; or and an online reasoning unit 460 for obtaining data provided by the model consumer via the service interface and calling the pre-trained model or the optimized model to perform online reasoning.

[0069] Optionally, the associated obfuscation process includes performing a forward obfuscation process on at least some of the initial model structure code, and performing a reverse obfuscation process on model parameters associated with the at least some of the initial model parameters.

[0070] Optionally, the obfuscated model parameters are stored in a storage space managed by the pre-training service system.

[0071] Optionally, the service providing device based on the pre-training service system further comprises: a first verification unit 470 for verifying whether the model optimizer has the authority to call the obfuscated model parameters corresponding to the pre-trained model, and if the model optimizer has the authority to call the obfuscated model parameters, combining the obfuscated model parameters with the obfuscated model structure code to obtain the pre-trained model and providing it to the model optimizer; and / or The method may further include a second verification unit 480 for verifying whether the model consumer has an authority to call obfuscated model parameters corresponding to the pre-trained model or the optimized model, and if the model consumer has an authority to call the obfuscated model parameters, combining the obfuscated model parameters with the obfuscated model structure code to obtain the pre-trained model or the optimized model, and providing the obfuscated model to the model consumer.

[0072] Optionally, the pre-training service system maintains an independent model access code and a model training reasoning code for the target pre-training model, in which the model structure code and the model access code are maintained by the pre-training service system after being encrypted, and the model training reasoning code is maintained by the pre-training service system in plain text.

[0073] The process of implementing the functions and actions of each unit in the above device is specifically referred to as the process of implementing the corresponding steps in the above method, and the description is omitted here.

[0074] The present application further provides a computer program product, comprising a computer program, which, when executed, causes a computer to perform the method according to the above embodiments.

[0075] The present application further provides a computer readable storage medium, characterized in that a computer program is stored in the computer readable storage medium, and the computer program is adapted to implement the method according to the above embodiment when executed by a processor.

[0076] The system, device, module or unit described in the above embodiments can be specifically realized by a computer chip or entity, or can be realized by a product having a certain function. A typical implementation device is a computer, and the specific form of the computer may be a personal computer, a laptop computer, a mobile phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an e-mail sending and receiving device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0077] In one typical configuration, a computer includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0078] The memory may include any form of computer readable medium, such as non-volatile memory, random access memory (RAM) and / or non-volatile memory, such as read only memory (ROM) or flash memory (flashRAM). The memory is one example of a computer readable medium.

[0079] Computer-readable media includes permanent and non-permanent, and portable and non-portable media can realize information storage in any manner or technology. Information may be computer-readable instructions, data structures, modules of programs, or other data. Computer storage media may include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic tape cartridge, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices or other non-transmission media, and the like, which may be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transitory storage computer-readable media, such as modulated data signals and carrier waves.

[0080] In one or more embodiments of the present application, the terms "comprise", "including", or any other variation thereof, are intended to include a non-exclusive inclusion, whereby a process, method, article, or device that includes a set of elements not only includes those elements, but also includes other elements not expressly listed or that are inherent to such process, method, article, or device. In the absence of more limitations, an element qualified by the phrase "comprises a..." does not exclude the presence of additional identical elements in the process, method, article, or device that includes said element.

[0081] The terms used in one or more embodiments of the present application are for the purpose of describing particular embodiments only and are not intended to limit one or more embodiments of the present application. As used in one or more embodiments of the present application and in the appended claims, the singular forms "a," "said," and "the" are intended to include the plural forms unless the context clearly indicates otherwise. In addition, the term "and / or" as used herein should be understood to mean including any and all combinations of one or more of the associated listed items.

[0082] It should be understood that, although terms such as first, second, and third may be used to describe various pieces of information in one or more embodiments of the present application, the information is not limited to these terms. These terms are merely used to distinguish between the same types of information. For example, the first information may be referred to as the second information, and similarly, the second information may be referred to as the first information, without departing from the scope of one or more embodiments of the present application. Also, depending on the context, "if" as used herein may be interpreted as "when" or "in the event of" or "responsive to a determination."

[0083] What has been described above is merely a preferred embodiment of one or more embodiments of the present application, and is not intended to limit the one or more embodiments of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of one or more embodiments of the present application should be included within the protection scope of one or more embodiments of the present application.

Claims

1. A pre-training service system, wherein a model maintained by the pre-training service system includes an obfuscated model structure code and an obfuscated model parameter, and the obfuscated model structure code and the obfuscated model parameter are generated after previously performing a related obfuscation process on an initial model structure code and an initial model parameter, and the pre-training service system: a producer service module for providing a model producer with pre-training capabilities for a model on a pre-training dataset and generating a corresponding pre-trained model; an optimizer service module for optimizing the pre-trained model based on a fine-tuning dataset provided by a model optimizer to obtain an optimized model; A consumer service module for providing a service interface for the pre-trained model and / or the optimized model to a model consumer, wherein the pre-trained model or the optimized model is configured to infer data provided by the model consumer and output a model prediction result.

2. 2. The system of claim 1, wherein the model pre-training function is used to train based on the pre-training dataset to obtain a common pre-trained model, in which the pre-trained model includes at least one of the common pre-trained model and a dedicated pre-trained model corresponding to a downstream task specified by the model producer, which is generated by the producer service module based on the common pre-trained model.

3. the producer services module is further adapted to generate a code development template corresponding to the specialized pre-trained model; 3. The system of claim 2, wherein the optimizer service module is further used to provide the code development template to the model optimizer and to modify model structure code and / or model parameters of the dedicated pre-trained model based on model modification instructions issued by the model optimizer.

4. The consumer service module includes: The pre-trained model or the optimized model is deployed to a device corresponding to the model consumer via the service interface, and is used to perform offline inference on data provided by the model consumer; or The system according to any one of claims 1 to 3, characterized in that it is used to obtain data provided by the model consumer via the service interface, and to call the pre-trained model or the optimized model to perform online inference.

5. The system of any one of claims 1 to 4, characterized in that the associated obfuscation process includes performing a forward obfuscation process on at least a portion of the initial model structure code, and performing a reverse obfuscation process on model parameters associated with the at least a portion of the initial model parameters.

6. The system according to any one of claims 1 to 5, characterized in that the obfuscated model parameters are stored in a storage space managed by the pre-training service system.

7. the optimizer service module is further used to verify whether the model optimizer has the authority to call the obfuscated model parameters corresponding to the pre-trained model, and if the model optimizer has the authority to call the obfuscated model parameters, combine the obfuscated model parameters with the obfuscated model structure code to obtain the pre-trained model and provide it to the model optimizer; and / or the consumer service module is further used to verify whether the model consumer has an authority to call the obfuscated model parameters corresponding to the pre-trained model or the optimized model, and if the model consumer has an authority to call the obfuscated model parameters, combine the obfuscated model parameters with the obfuscated model structure code to obtain the pre-trained model or the optimized model and provide it to the model consumer; A system according to any one of claims 1 to 6.

8. The system of any one of claims 1 to 7, characterized in that the pre-training service system maintains an independent model access code and a model training reasoning code for the pre-training model or the optimized model, in which the model structure code and the model access code are maintained by the pre-training service system after being encrypted, and the model training reasoning code is maintained by the pre-training service system in plain text.

9. A service providing method based on a pre-training service system, the pre-training service system including a producer service module, an optimizer service module and a consumer service module, a model maintained by the pre-training service system includes an obfuscated model structure code and an obfuscated model parameter, the obfuscated model structure code and the obfuscated model parameter are generated after previously performing a related obfuscation process on an initial model structure code and an initial model parameter, the method comprising: Obtaining identity information of a target user who accesses the pre-training service system, and determining a pre-defined user type to which the target user belongs; Opening a service module that matches a predefined user type to which the target user belongs, wherein: If the target user is a model producer, open the producer service module to the target user, the producer service module being used to provide the model producer with a pre-training function for a model on a pre-training dataset and generate a corresponding pre-trained model; If the target user is a model optimizer, open the optimizer service module to the target user, and the optimizer service module is used to optimize the pre-trained model based on a fine-tuning dataset provided by the model optimizer to obtain an optimized model; If the target user is a model consumer, opening the consumer service module to the target user, the consumer service module is used to provide the model consumer with a service interface for the pre-trained model and / or the optimized model, and the pre-trained model or the optimized model is used to infer data provided by the model consumer and output model prediction results.

10. 10. The method of claim 9, wherein the model pre-training function is used to train based on the pre-training dataset to obtain a common pre-trained model, in which the pre-trained model includes at least one of the common pre-trained model and a dedicated pre-trained model corresponding to a downstream task specified by the model producer, which is generated by the producer service module based on the common pre-trained model.

11. generating a code development template corresponding to the dedicated pre-trained model; providing the code development template to the model optimizer, and modifying model structure code and / or model parameters of the dedicated pre-trained model based on model modification instructions issued by the model optimizer.

11. The method of claim 10.

12. Deploying the pre-trained model or the optimized model to a device corresponding to the model consumer via the service interface, which is used to perform offline inference on data provided by the model consumer; or and acquiring data provided by the model consumer through the service interface, and calling the pre-trained model or the optimized model to perform online inference. The method according to any one of claims 9 to 11.

13. 13. The method according to claim 9, wherein the associated obfuscation process comprises performing a forward obfuscation process on at least a portion of the initial model structure code, and performing a reverse obfuscation process on model parameters associated with the at least a portion of the initial model parameters.

14. The method according to any one of claims 9 to 13, characterized in that the obfuscated model parameters are stored in a storage space managed by the pre-training service system.

15. Verifying whether the model optimizer has the authority to call the obfuscated model parameters corresponding to the pre-trained model, and if the model optimizer has the authority to call the obfuscated model parameters, combining the obfuscated model parameters with the obfuscated model structure code to obtain the pre-trained model, and providing the pre-trained model to the model optimizer; and / or verifying whether the model consumer has an authority to call the obfuscated model parameters corresponding to the pre-trained model or the optimized model, and if the model consumer has an authority to call the obfuscated model parameters, combining the obfuscated model parameters with the obfuscated model structure code to obtain the pre-trained model or the optimized model, and providing the obfuscated model parameters to the model consumer. The method according to any one of claims 9 to 14.

16. The method according to any one of claims 9 to 15, characterized in that the pre-training service system maintains an independent model access code and a model training reasoning code for the target pre-training model, in which the model structure code and the model access code are maintained by the pre-training service system after being encrypted, and the model training reasoning code is maintained by the pre-training service system in plain text.

17. A service providing apparatus based on a pre-training service system, the pre-training service system including a producer service module, an optimizer service module and a consumer service module, a model maintained by the pre-training service system includes an obfuscated model structure code and an obfuscated model parameter, the obfuscated model structure code and the obfuscated model parameter are generated after previously performing a related obfuscation process on an initial model structure code and an initial model parameter, the apparatus comprising: an acquisition unit for acquiring identity information of a target user who accesses the pre-training service system and determining a pre-defined user type to which the target user belongs; an opening unit for opening a service module matching a preset user type to which the target user belongs, comprising: If the target user is a model producer, open the producer service module to the target user, the producer service module being used to provide the model producer with a pre-training function for a model on a pre-training dataset and generate a corresponding pre-trained model; If the target user is a model optimizer, open the optimizer service module to the target user, and the optimizer service module is used to optimize the pre-trained model based on a fine-tuning dataset provided by the model optimizer to obtain an optimized model; and an opening unit for opening the consumer service module to the target user when the target user is a model consumer, the consumer service module being used to provide the model consumer with a service interface for the pre-trained model and / or the optimized model, and the pre-trained model or the optimized model being used to infer data provided by the model consumer and output model prediction results.

18. 18. The apparatus of claim 17, wherein the model pre-training function is used to train based on the pre-training dataset to obtain a common pre-trained model, in which the pre-trained model includes at least one of the common pre-trained model and a dedicated pre-trained model corresponding to a downstream task specified by the model producer, which is generated by the producer service module based on the common pre-trained model.

19. a generation unit for generating a code development template corresponding to the dedicated pre-trained model; a providing and modifying unit for providing the code development template to the model optimizer and modifying a model structure code and / or a model parameter of the dedicated pre-trained model according to a model modification instruction issued by the model optimizer.

20. The apparatus of claim 18.

20. an offline deployment unit for deploying the pre-trained model or the optimized model to a device corresponding to the model consumer via the service interface and performing offline inference on data provided by the model consumer; or An online reasoning unit for obtaining data provided by the model consumer through the service interface, and calling the pre-trained model or the optimized model to perform online reasoning. Apparatus according to any one of claims 17 to 19.

21. 21. The apparatus of claim 17, wherein the associated obfuscation process comprises performing a forward obfuscation process on at least a portion of the initial model structure code, and performing a reverse obfuscation process on model parameters associated with the at least a portion of the initial model parameters.

22. The apparatus according to any one of claims 17 to 21, characterized in that the obfuscated model parameters are stored in a storage space managed by the pre-training service system.

23. a first verification unit for verifying whether the model optimizer has the authority to call obfuscated model parameters corresponding to the pre-trained model, and if the model optimizer has the authority to call the obfuscated model parameters, combining the obfuscated model parameters with the obfuscated model structure code to obtain the pre-trained model and providing it to the model optimizer; and / or a second verification unit for verifying whether the model consumer has an authority to call the obfuscated model parameters corresponding to the pre-trained model or the optimized model, and if the model consumer has an authority to call the obfuscated model parameters, combining the obfuscated model parameters with the obfuscated model structure code to obtain the pre-trained model or the optimized model for providing to the model consumer; Apparatus according to any one of claims 17 to 22.

24. The apparatus of any one of claims 17 to 23, characterized in that the pre-training service system maintains an independent model access code and a model training reasoning code for the target pre-training model, in which the model structure code and the model access code are maintained by the pre-training service system after being encrypted, and the model training reasoning code is maintained by the pre-training service system in plain text.

25. A processor, a memory and a non-volatile memory, The electronic device is characterized in that the processor is used to read a corresponding computer program from the non-volatile memory into the memory and execute the method according to any one of claims 9 to 16 based on the computer program.

26. A computer-readable storage medium storing a computer program which, when executed by a processor, implements the method according to any one of claims 9 to 16.

27. A computer program product comprising a computer program, the computer program causing a computer to carry out the method according to any one of claims 9 to 16 when the computer program is executed.

Images