Model adaptation method and device and storage medium

Abstract

The invention provides a model adaptation method and device and a storage medium, relates to the technical field of artificial intelligence, and is used for solving the problem that a deep learning model is adaptive to a high-performance computing carrier during migration. The method comprises the following steps: acquiring model information of a to-be-migrated model, initial carrier information of an initial carrier unit and target carrier information of a target carrier unit; a carrier difference is determined based on the initial carrier information and the target carrier information. And adjusting the to-be-migrated model based on the model information and the carrier difference to obtain an adjusted model.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a model adaptation method, device and storage medium. Background Art

[0002] Currently, the training and reasoning of deep learning models require a lot of computing resources. Since graphics processing units (GPUs) cannot efficiently and in real time meet the requirements for training and reasoning deep learning models, it is necessary to migrate deep learning models to high-performance computing carriers to improve computing efficiency.

[0003] However, the process of migrating deep learning models to high-performance computing carriers may lead to a decrease in the performance of deep learning models. Therefore, how to ensure that deep learning models are compatible with high-performance computing carriers during migration has become an urgent problem to be solved. Summary of the invention

[0004] The present application provides a model adaptation method, device and storage medium for solving the problem of adapting a deep learning model to a high-performance computing carrier during migration.

[0005] In order to achieve the above objectives, this application adopts the following technical solutions:

[0006] In a first aspect, the present application provides a model adaptation method. In the method, model information of a model to be migrated, initial carrier information of an initial carrier unit, and target carrier information of a target carrier unit are obtained. Based on the initial carrier information and the target carrier information, a carrier difference is determined. Based on the model information and the carrier difference, the model to be migrated is adjusted to obtain an adjusted model.

[0007] Based on the above technical solution, the model information of the model to be migrated, the initial carrier information of the initial carrier unit, and the target carrier information of the target carrier unit are obtained. Afterwards, based on the initial carrier information and the target carrier information, the carrier difference can be determined. In this way, by determining the difference between the two carriers, a suitable migration strategy can be formulated to ensure that the model can run correctly on the target carrier after migration. Afterwards, the model to be migrated is adjusted based on the model information and the carrier difference to obtain the adjusted model. In this way, by adjusting the model according to the difference between the two carriers, the performance of the model on the carrier can be optimized, so that the adjusted model can be more suitable for the target carrier.

[0008] In a possible design, the model information includes: the size of the model to be migrated and the multiple initial modules included in the model to be migrated. Based on the model information and the carrier difference information, the size of the model to be migrated and the multiple initial modules included in the model to be migrated are adjusted to obtain an adjusted model.

[0009] In a possible design, based on the model information and the carrier difference information, the size of the model to be migrated and the multiple initial modules included in the model to be migrated are adjusted to obtain a first model including multiple first modules, the size of the first model is smaller than the size of the model to be migrated, and the number of the multiple first modules is smaller than the number of the multiple initial modules. The first model is parsed, and the order of the multiple first modules is adjusted based on the parsing result and the target carrier information to obtain a second model, which is the adjusted model.

[0010] In a possible design, based on the initial operator of the model to be migrated, a target operator adapted to the target carrier unit is determined from the operator library, and the function of the initial operator is the same as that of the target operator. The multiple target operators are used as operators in the second model.

[0011] In one possible design, a candidate operator with the same function as the initial operator is determined from the operator library. The target environment of the target carrier unit is used as the operator in the adjusted model in the target environment, and the adjusted model is tested to obtain a test result, which is used to reflect the matching degree between the candidate operator and the target environment. If the test result is greater than the preset test condition, the candidate operator is used as the target operator.

[0012] In one possible design, adversarial sample data is obtained, and adversarial training is performed on the adjusted model based on the adversarial sample data when the adjusted model is not transferred and / or transferred.

[0013] In a second aspect, the present application provides a model adaptation device, which includes an acquisition module and a processing module.

[0014] The acquisition module is used to acquire model information of the model to be migrated, initial carrier information of the initial carrier unit, and target carrier information of the target carrier unit. The processing module is used to determine carrier differences based on the initial carrier information and the target carrier information. The processing module is also used to adjust the model to be migrated based on the model information and the carrier differences to obtain an adjusted model.

[0015] In one possible design, the model information includes: the size of the model to be migrated and the multiple initial modules included in the model to be migrated. The processing module is also used for the processing module to adjust the size of the model to be migrated and the multiple initial modules included in the model to be migrated based on the model information and the carrier difference information to obtain an adjusted model.

[0016] In a possible design, the processing module is further used to adjust the size of the model to be migrated and the multiple initial modules included in the model to be migrated based on the model information and the carrier difference information, so as to obtain a first model including multiple first modules, wherein the size of the first model is smaller than the size of the model to be migrated, and the number of the multiple first modules is smaller than the number of the multiple initial modules. The processing module is further used to parse the first model, and adjust the order of the multiple first modules based on the parsing result and the target carrier information to obtain a second model, wherein the second model is the adjusted model.

[0017] In one possible design, the processing module is further used to determine a target operator adapted to the target carrier unit from the operator library based on the initial operator of the model to be migrated, and the function of the initial operator is the same as that of the target operator. The processing module is further used to use the multiple target operators as operators in the second model.

[0018] In a possible design, the processing module is further used to determine a candidate operator with the same function as the initial operator from the operator library. The processing module is further used to initialize the target environment of the target carrier unit, use the candidate operator as the operator in the adjusted model in the target environment, and test the adjusted model to obtain a test result, which is used to reflect the degree of matching between the candidate operator and the target environment. The processing module is further used to use the candidate operator as the target operator if the test result is greater than a preset test condition.

[0019] In a possible design, the acquisition module is further used to acquire adversarial sample data. The processing module is further used to perform adversarial training on the adjusted model based on the adversarial sample data when the adjusted model is not transferred and / or transferred.

[0020] In the third aspect, the present application provides a model adaptation device, which includes: a processor and a memory; the processor and the memory are coupled; the memory is used to store one or more programs, and the one or more programs include computer execution instructions. When the model adaptation device is running, the processor executes the computer execution instructions stored in the memory to implement the method described in the first aspect and any possible implementation of the first aspect.

[0021] In a fourth aspect, the present application provides a computer-readable storage medium, which stores instructions. When the instructions are executed on a computer, the computer executes the method described in the above-mentioned first aspect and any possible implementation of the first aspect.

[0022] In a fifth aspect, the present application provides a chip, comprising a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run a computer program or instructions to implement the method described in the first aspect and any possible implementation of the first aspect.

[0023] In a sixth aspect, the present application provides a computer program product comprising instructions, which, when executed by a computer, enables the computer to execute the method described in the above-mentioned first aspect and any possible implementation manner of the first aspect.

[0024] In the above scheme, the technical problems that can be solved and the technical effects achieved by the model adaptation device, computer equipment, computer storage medium, chip or computer program product can refer to the technical problems and technical effects solved by the above-mentioned first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 A system architecture diagram of a model adaptation system provided in an embodiment of the present application;

[0026] Figure 2 A flow chart of a model adaptation method provided in an embodiment of the present application;

[0027] Figure 3 A flowchart of another model adaptation method provided in an embodiment of the present application;

[0028] Figure 4 A flowchart of another model adaptation method provided in an embodiment of the present application;

[0029] Figure 5 A flowchart of another model adaptation method provided in an embodiment of the present application;

[0030] Figure 6 A schematic diagram of the structure of a model adaptation device provided in an embodiment of the present application;

[0031] Figure 7 A schematic diagram of the structure of another model adaptation device provided in an embodiment of the present application;

[0032] Figure 8 A conceptual partial view of a computer program product provided for an embodiment of the present application. DETAILED DESCRIPTION

[0033] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0034] The terms “first” and “second” in the description and claims of the present application are used to distinguish different objects rather than to describe a specific order of the objects.

[0035] In addition, the terms "including" and "having" and any variations thereof mentioned in the description of this application are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not limited to the listed steps or modules, but may optionally include other steps or modules that are not listed, or may optionally include other steps or modules that are inherent to these processes, methods, products or devices.

[0036] In addition, in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present concepts in a specific way.

[0037] In recent years, with the development of artificial intelligence technology, the role of artificial intelligence technology in the development of human society has become increasingly prominent, and the high requirements of artificial intelligence technology for computing power and storage have also become more prominent in restricting the popularization of projects. When the end-side device trains and infers the deep learning model, it is not possible to meet the requirements of high efficiency and real-time. Therefore, it is necessary to migrate the deep learning model from the end-side device to the cloud device to improve computing efficiency. However, the storage and computing power of the end-side device and the cloud device are different, so the deep learning model needs to be quantized and pruned in a targeted manner. In the face of different computing architectures of different devices, it is necessary to determine the corresponding computing method. If a completely consistent computing method is used for the deep learning model, it will greatly affect the computing efficiency and actual application effect of the deep learning model on the platform.

[0038] In addition, during the application of deep learning models in cloud devices, it is necessary to iterate the design and deploy the deep learning models at a higher frequency according to the requirements of different scenarios and tasks. At present, the mainstream neural network models are developed and trained based on GPUs, and it is difficult to adapt them well to cloud devices. Therefore, it is necessary to transplant and adapt the deep learning models on the GPU to the cloud devices. During the transplantation and adaptation process, there are problems such as the decrease in accuracy of the deep learning model after transplantation, the lack of support for some operators, and the unsatisfactory performance of the deep learning model after transplantation and adaptation. In addition, after the deep learning model is transplanted and adapted, there may be a certain gap between the performance and running on the GPU. It is necessary to optimize the performance in various ways to ensure that the performance of each acceleration chip for the algorithm and the deep learning model is maximized.

[0039] Therefore, how to ensure that deep learning models are compatible with high-performance computing carriers during migration has become a problem that needs to be solved urgently.

[0040] In order to solve the above technical problems, an embodiment of the present application provides a model adaptation method. In this method, the model information of the model to be migrated, the initial carrier information of the initial carrier unit, and the target carrier information of the target carrier unit are obtained. Afterwards, based on the initial carrier information and the target carrier information, the carrier difference can be determined. In this way, by determining the difference between the two carriers, a suitable migration strategy can be formulated to ensure that the model can run correctly on the target carrier after migration. Afterwards, the model to be migrated is adjusted based on the model information and the carrier difference to obtain the adjusted model. In this way, by adjusting the model according to the difference between the two carriers, the performance of the model on the carrier can be optimized, so that the adjusted model can be more adapted to the target carrier.

[0041] The implementation environment of the embodiments of the present application is introduced below.

[0042] like Figure 1 As shown, a model adaptation system provided by an embodiment of the present application includes: a model conversion module 101, a hardware adaptation module 102, a performance optimization module 103 and a monitoring feedback module 104. The model conversion module 101, the hardware adaptation module 102, the performance optimization module 103 and the monitoring feedback module 104 are integrated in the model adaptation device.

[0043] The model conversion module 101 is used to adjust the structure of the model. For example, the model conversion module 101 can adjust the size of the model. For another example, the model conversion module 101 can adjust the modules included in the model, such as adding modules, deleting modules, adjusting the positions between modules, etc. The model conversion module 101 can also be used to adjust the operators used by the model.

[0044] The hardware adaptation module 102 is used to test the model. Furthermore, the hardware adaptation module 102 can also be used to send the tested model to the model conversion module 101 for adjustment.

[0045] It should be noted that the hardware adaptation module 102 can use a compiler to generate code with an instruction set supported by the target carrier unit, and use a debugging tool to check the execution of the generated code. Afterwards, the model conversion module 101 sorts the modules of the model, replaces the new operator, and completes the conversion of the entire model in the target carrier unit.

[0046] The performance optimization module 103 is used to optimize the adjusted model. For example, the performance optimization module 103 can train the adjusted model. The performance optimization module 103 is also used to compile the operators of the model and add the compiled operators to the operator library.

[0047] Exemplarily, the performance optimization module 103 may write new operators through a deep learning framework or a compiler.

[0048] It should be noted that the target carrier can identify the type of the compiled operator, and the compiled operator has the same function as the operator before compilation.

[0049] It should be noted that the performance optimization module 103 can also optimize the execution efficiency of the model on the target carrier, and adjust the computing unit and memory access mode of the model to increase the running speed.

[0050] The monitoring feedback module 104 is used to monitor the operation of the migrated model on the target carrier unit and to update the migrated model.

[0051] The embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0052] like Figure 2 As shown, a model adaptation method provided in an embodiment of the present application includes:

[0053] S201, obtaining model information of a model to be migrated, initial carrier information of an initial carrier unit, and target carrier information of a target carrier unit.

[0054] The model information includes at least one of the following: model type, model memory information, model architecture information, data format of model data, etc.

[0055] In a possible design, the model memory information includes at least one of the following: the size of the model to be migrated, memory capacity, memory bandwidth, memory access time, and model data.

[0056] Exemplarily, the memory allocation method includes static memory allocation and dynamic memory allocation.

[0057] It should be noted that there is a corresponding relationship between the memory allocation method of the model and the carrier corresponding to the model.

[0058] In a possible design, the model architecture information includes at least one of the following: a model hierarchy, a model data path, model module information, and a model operator. The model data path is a path for data to flow through the model. The model module information includes the number of modules and the order in which the modules are arranged in the model.

[0059] Exemplarily, the model hierarchy may be an input layer, a convolution layer, a pooling layer, a fully connected layer, and an output layer. The model operator may be a convolution type operator, a pooling type operator, an activation function type operator, a fully connected type operator, and a normalization type operator.

[0060] Exemplarily, the convolution type operator may be a convolution algorithm in a convolution layer, and the fully connected type operator may be a weight summation process in a fully connected layer.

[0061] It should be noted that a module is composed of multiple computing units, which are the basic computing elements that perform computing tasks in the model.

[0062] Exemplarily, the computing unit may be a neuron, a convolution kernel, or a weight matrix in a fully connected layer.

[0063] In a possible design, the data format of the model data includes at least one of the following: FP32 floating point type, FP16 floating point type, INT8 integer type, INT16 integer type, BF16 floating point type.

[0064] It should be noted that the embodiments of the present application do not limit the model to be migrated. For example, the model to be migrated can be a neural network model, a language processing model, an image processing model, a speech processing model, a code processing model, and an attention mechanism model.

[0065] In a possible implementation, the model may be parsed to obtain model information.

[0066] In another possible implementation, the model adaptation device pre-stores preset model information of a plurality of models. Based on the model type, the model information corresponding to the model type can be determined from the plurality of preset model information.

[0067] In an embodiment of the present application, the initial carrier information includes at least one of the following: the initial carrier type, the architecture information of the initial carrier, the memory information of the initial carrier, the performance indicator information of the initial carrier, and the operator type supported by the initial carrier. The performance indicator information is used to indicate the computing capability of the carrier. The target carrier information includes at least one of the following: the target carrier type, the architecture information of the target carrier, the memory information of the target carrier, the performance indicator information of the target carrier, and the operator type supported by the target carrier.

[0068] Exemplarily, the architecture information includes: the number of computing units, the distribution of computing units, and the data path of the carrier. The data path of the carrier is the path for data to flow on the model. The memory information includes memory capacity, memory type, memory bandwidth, and storage hierarchy. The storage hierarchy includes global memory, shared memory, and cache.

[0069] It should be noted that the embodiments of the present application do not limit the carrier unit. For example, the carrier unit may be a central processing unit (CPU), a GPU, a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or a neural processing unit (NPU). The following takes the initial carrier unit as a GPU and the target carrier unit as an ASIC as an example to introduce the embodiments of the present application.

[0070] In a possible implementation, the initial carrier unit is parsed to obtain initial carrier information, and the target carrier unit is parsed to obtain target carrier information.

[0071] In another possible implementation, a plurality of preset carrier information is pre-stored in the model adaptation device. Based on the carrier type, the model information corresponding to the carrier type can be determined from the plurality of preset carrier information.

[0072] S202: Determine carrier differences based on the initial carrier information and the target carrier information.

[0073] In a possible implementation manner, the initial carrier information and the target carrier information are compared to determine the carrier difference between the initial carrier and the target carrier.

[0074] In one possible design, carrier differences include differences in carrier architecture, carrier memory, carrier performance indicators, and differences in the types of operators supported by the carriers.

[0075] In one possible design, based on the architecture information of the initial carrier and the architecture information of the target carrier, the carrier architecture difference is determined. Based on the memory information of the initial carrier and the memory information of the target carrier, the carrier memory difference is determined. Based on the performance indicator information of the initial carrier and the performance indicator information of the target carrier, the carrier performance indicator difference is determined. Based on the operator type supported by the initial carrier and the operator type supported by the target carrier, the difference in the operator type supported by the carrier is determined.

[0076] It should be noted that by determining the differences in carrier architecture, we can obtain the differences in computing power and power consumption of the carriers. By determining the differences in carrier memory, we can obtain the differences in data storage and access speed of the carriers.

[0077] It can be understood that in this way, the model can be adjusted by determining the carrier differences and choosing the appropriate strategy.

[0078] S203: Adjust the model to be migrated based on the model information and carrier differences to obtain an adjusted model.

[0079] In a possible implementation, based on the model information and the carrier difference information, the size of the model to be migrated and the multiple initial modules included in the model to be migrated are adjusted to obtain an adjusted model.

[0080] In one possible design, based on the carrier memory difference, it is determined whether the target carrier meets the memory requirements of the model. If the carrier memory difference indicates that the memory of the target carrier unit is greater than or equal to the memory of the initial carrier, it is determined that the target carrier meets the memory requirements of the model. If the carrier memory difference indicates that the memory of the target carrier unit is less than or equal to the memory of the initial carrier, it is determined that the target carrier cannot meet the memory requirements of the model. If the target carrier meets the memory requirements of the model, the modules of the model can be increased. If the target carrier cannot meet the memory requirements of the model, the modules of the model can be reduced.

[0081] In one possible design, based on the model structure information and carrier architecture differences, the model modules are compiled, and the module distribution positions of the model adapted to the target carrier are determined, and the distribution positions of the model modules to be migrated are adjusted.

[0082] In another possible design, based on the data format of the model, differences in carrier performance indicators and carrier memory, the data format of the model adapted to the target carrier is determined, and the data format of the model is converted into a data format supported by the target carrier.

[0083] Exemplarily, the target carrier performance index is higher than the initial carrier performance index, the target carrier memory is larger than the initial carrier memory, and the weight of the model can be converted from FP16 floating point type to FP32 floating point type.

[0084] Based on the above technical solution, the model information of the model to be migrated, the initial carrier information of the initial carrier unit, and the target carrier information of the target carrier unit are obtained. Afterwards, based on the initial carrier information and the target carrier information, the carrier difference can be determined. In this way, by determining the difference between the two carriers, a suitable migration strategy can be formulated to ensure that the model can run correctly on the target carrier after migration. Afterwards, the model to be migrated is adjusted based on the model information and the carrier difference to obtain the adjusted model. In this way, by adjusting the model according to the difference between the two carriers, the performance of the model on the carrier can be optimized, so that the adjusted model can be more suitable for the target carrier.

[0085] like Figure 3 As shown, another model adaptation method provided in an embodiment of the present application, in which S203 may include:

[0086] S301. Based on the model information and the carrier difference information, adjust the size of the model to be migrated and the multiple initial modules included in the model to be migrated to obtain a first model including multiple first modules.

[0087] The size of the first model is smaller than the size of the model to be migrated, and the number of the first modules is smaller than the number of the initial modules.

[0088] In a possible design, based on the model memory information and the carrier memory difference, a model adapted to the target carrier is determined to perform data quantization and pruning on the initial model to obtain a first model.

[0089] Exemplarily, the weights of the model are converted from FP32 floating point type to FP16 floating point type.

[0090] It should be noted that by quantizing the data of the initial model and converting the model's weights from high precision to low precision, the size and computational complexity of the model can be adjusted; by pruning the initial model and removing weights, the redundant calculation and storage requirements of the model can be reduced, and the converted model architecture, model weights, and data types of the model data can be obtained.

[0091] S302: Analyze the first model, and adjust the order of the multiple first modules based on the analysis result and the target carrier information to obtain the second model.

[0092] The second model is the adjusted model. The analysis results include the model's performance indicators, module compatibility information, and operator compatibility information.

[0093] In a possible implementation, based on the performance indicators of the model and the architecture information of the target carrier, the order of the plurality of first modules is adjusted to obtain the second model.

[0094] In one possible design, based on the performance indicators of the model and the architecture information of the target carrier, the memory information of the target carrier, and the performance indicator information of the target carrier, the number and level of modules are increased, and adjustments are made by determining the connection method between modules to obtain a second model.

[0095] Exemplarily, the order of the convolution module and the pooling module of the model is convolution first, then pooling. The order of the convolution module and the pooling module is swapped, and the model is first pooled, and then convolution.

[0096] In some embodiments, based on the analysis results and the target carrier information, the initial operator of the model to be migrated is determined. Based on the initial operator of the model to be migrated, a target operator adapted to the target carrier unit is determined from the operator library, and the function of the initial operator is the same as that of the target operator. The multiple target operators are used as operators in the second model.

[0097] In a possible implementation, a candidate operator having the same function as the initial operator is determined from the operator library. The target environment of the target carrier unit is initialized, the candidate operator is used as the operator in the adjusted model in the target environment, and the adjusted model is tested to obtain a test result, which is used to reflect the matching degree between the candidate operator and the target environment. If the test result is greater than a preset test condition, the candidate operator is used as the target operator.

[0098] In one possible design, the test result is the accuracy of the model trained on the target carrier unit. The preset test condition is that the test result is greater than the accuracy of the model trained on the initial carrier unit.

[0099] In another possible design, the test result is the speed at which the model processes the task on the target carrier unit. The preset test condition is that the test result is greater than the speed at which the model processes the task on the initial carrier unit.

[0100] In some embodiments, if a compatibility problem occurs in the adjusted model during the testing process, the adjusted model is repaired.

[0101] In a possible implementation, a suitable operator is selected from an operator library to replace an incompatible operator.

[0102] It should be noted that after obtaining the first model, the execution performance of the modules and operators of the first model can be tested by a performance analysis tool, and the order of the modules of the multiple first models can be adjusted. After that, a new target operator adapted to the target carrier unit is determined from the operator library, so that the adjusted model has a higher degree of adaptability to the target carrier unit.

[0103] like Figure 4 As shown, another model adaptation method provided in an embodiment of the present application includes:

[0104] S401. Obtain adversarial sample data.

[0105] In one possible design, adversarial example data is obtained for the adjusted model under transfer conditions.

[0106] In another possible design, adversarial sample data is obtained for the adjusted model without transfer.

[0107] In a possible implementation, original sample data is obtained based on target carrier information, and adversarial sample data is obtained through an adversarial sample generation method based on the original sample data.

[0108] In one possible design, adversarial sample generation methods include fast gradient sign method (FGSM), fast gradient method (FGM), projected gradient descent (PGD), Gaussian noise, and salt and pepper noise.

[0109] It should be noted that when generating adversarial sample data, hardware perturbations can be simulated through perturbations, thereby simulating errors and noise that may occur in the actual operation of the hardware.

[0110] Exemplarily, the perturbation method may be to quantize the generated adversarial samples to simulate the errors caused by low-precision calculations. The perturbation method may also be to simulate the impact of memory access delay on data. The perturbation method may also be to add Gaussian noise or salt and pepper noise to simulate random noise in hardware.

[0111] In another possible implementation,

[0112] S402. When the adjusted model is not migrated and / or migrated, perform adversarial training on the adjusted model based on adversarial sample data.

[0113] In a possible implementation, training sample data is obtained. The training sample data includes adversarial sample data. Based on the adversarial sample data, adversarial training is performed on the adjusted model.

[0114] In a possible implementation, a generative adversarial network model is constructed. The generative adversarial network model includes a generator and a discriminator. The model is adversarially trained by using the generative adversarial network model and adversarial sample data.

[0115] It should be noted that the generator can generate fake samples, and the discriminator can distinguish between fake and real samples. Through adaptive adversarial training, the ability of the generator to generate realistic samples and the ability of the discriminator to identify fake samples can be enhanced.

[0116] It should be noted that adaptive adversarial training adopts a dynamic difficulty adjustment strategy, which gradually increases the difficulty of adversarial training as training progresses, which can enhance the generator's ability to generate realistic samples. Afterwards, a regularization term is added to the discriminator loss, so that the discriminator outputs a more confident prediction, thereby improving the discriminator's robustness to false samples.

[0117] Based on the above technical solution, adversarial sample data is obtained. When the adjusted model is not transferred and / or migrated, adversarial training is performed on the adjusted model based on the adversarial sample data. In this way, the model can learn how to correctly identify and resist attacks through adversarial training, improve the robustness of the model performance, and enable the model to maintain a high degree of stability and accuracy in the face of various attacks and noise. Furthermore, adversarial training can improve the accuracy of the model when the adjusted model is not transferred and migrated.

[0118] In some embodiments, after the model is migrated to the target carrier unit, the migrated model is monitored and updated.

[0119] In a possible implementation, user feedback information and training sample data are obtained, and the migrated model is trained based on the user feedback information and the training sample data to update the migrated model.

[0120] In a possible design, training sample data is obtained based on an importance-weighted active learning method. Based on user feedback information and the training sample data, the training sample data is classified using a nearest neighbor algorithm to obtain target training data. Afterwards, the migrated model is trained using the target training data to obtain a training result. Based on the training result, the migrated model is updated.

[0121] It should be noted that when the training sample data is small, the samples can be increased through sample enhancement technology.

[0122] For example, at time t, after model h receives a sample xt and its feedback yt, it triggers a divergence threshold calculation subroutine, which combines the sample record S t-1 , update the hypothesis space H according to machine learning theory t , calculate the rejection probability p according to the loss function t , the inverse of which reflects the importance of the current instance. After this, an important instance will be added to the instance set S t and record its importance weight w = 1 / p t , and then combine S according to the loss function t , based on empirical risk minimization from the hypothesis space H t The new model h is obtained t (Implement training or updating models).

[0123] Model h on set S t The formula for calculating the empirical risk loss value is as follows:

[0124]

[0125] In the formula, each random variable Q is independent of each other and obeys the parameter p t Bernoulli distribution, l(h(x t ),y t ) is the loss function calculation term;

[0126] In the divergence threshold calculation subroutine, the hypothesis space is first generated based on prior knowledge (samples and feedback). t-1 After finding the minimum empirical loss value in , according to the active learning theory, the size of the hypothesis space can be constrained in advance:

[0127]

[0128] In the formula, For model h on set S t-1 The true loss value, δ is a random number, and the relaxation Δ t-1 Derived from sample complexity analysis, this process can ensure that the optimal model is always in H t and try to shrink the hypothesis space to a smaller space.

[0129] Based on the shrunk hypothesis space H t , p t The calculation formula is as follows:

[0130]

[0131] In the formula, p t The value is H t Regarding the instance (x t ,y t ), and it can also be regarded as the error probability of the current instance in the new hypothesis space at time t. The lower the error probability, the higher the representativeness of the current sample. According to the active learning theory, the importance weight w is set to p t The inverse of is used to correct the sample bias.

[0132] It can be understood that obtaining target training data based on feedback information and introducing an active learning method based on importance weighting can continuously update the optimization strategy and the model on the target carrier unit to adapt to the new environment and data distribution.

[0133] The following is an introduction to the embodiments of the present application with reference to specific examples. Figure 5As shown, it is a flow chart of another model adaptation method provided in an embodiment of the present application. The model information of the model to be migrated, the initial carrier information of the initial carrier unit, and the target carrier information of the target carrier unit are obtained. Based on the initial carrier information and the target carrier information, the carrier difference is determined. Based on the model structure information and the carrier architecture difference, the modules of the model are compiled, and the module distribution positions of the model adapted to the target carrier are determined, and the distribution positions of the modules of the model to be migrated are adjusted. Afterwards, adversarial sample data is obtained. In the case where the adjusted model is not migrated and / or migrated, adversarial training is performed on the adjusted model based on the adversarial sample data. Afterwards, the model after migration is monitored. User feedback information and training sample data are obtained, and the migrated model is trained based on the user feedback information and the training sample data, and the migrated model is updated.

[0134] The above mainly introduces the solution provided by the embodiment of the present application from the perspective of the method. It is understandable that the adaptation device of the model includes a hardware structure and / or software module corresponding to each function in order to realize the above functions. Those skilled in the art should easily realize that, in combination with the adaptation method steps of the model of each example described in the embodiment disclosed in this application, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0135] The embodiment of the present application also provides a model adaptation device. The model adaptation device can be a computer, a CPU in the above computer, a module in the above computer for adapting the model, or a client in the above computer for adapting the model.

[0136] The embodiment of the present application can divide the adaptation device of the model into functional modules or functional units according to the above method example. For example, each functional module or functional unit can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above integrated module can be implemented in the form of hardware or in the form of software functional modules or functional units. Among them, the division of modules or units in the embodiment of the present application is schematic, which is only a logical function division, and there may be other division methods in actual implementation.

[0137] The present application embodiment provides a model adaptation device. Figure 6 As shown, the model adaptation device may include: an acquisition module 601 and a processing module 602.

[0138] The acquisition module 601 is used to acquire the model information of the model to be migrated, the initial carrier information of the initial carrier unit and the target carrier information of the target carrier unit. The processing module 602 is used to determine the carrier difference based on the initial carrier information and the target carrier information. The processing module 602 is also used to adjust the model to be migrated based on the model information and the carrier difference to obtain the adjusted model.

[0139] Figure 7 7 is a schematic diagram of a structure of another model adaptation device according to an exemplary embodiment. The model adaptation device may include a processor 702, and the processor 702 is used to execute application code to implement the model adaptation method in the present application.

[0140] The processor 702 may be a central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the present application.

[0141] like Figure 7 As shown, the model adaptation device may further include a memory 703. The memory 703 is used to store application program codes for executing the solution of the present application, and the execution is controlled by the processor 702.

[0142] The memory 703 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 703 may exist independently and be connected to the processor 702 via a bus 704. The memory 703 may also be integrated with the processor 702.

[0143] like Figure 7As shown, the model adaptation device may further include a communication interface 701, wherein the communication interface 701, the processor 702, and the memory 703 may be coupled to each other, for example, via a bus 704. The communication interface 701 is used to perform information interaction with other devices, for example, to support information interaction between the model adaptation device and other devices.

[0144] It should be pointed out that Figure 7 The equipment structure shown in the figure does not constitute a limitation on the adaption device of the model, except Figure 7 In addition to the components shown, the adaptation device of the model may include more or fewer components than shown, or combine certain components, or have a different arrangement of components.

[0145] In actual implementation, the functions implemented by the processing unit can be Figure 7 The processor 702 shown calls the program code in the memory 703 to implement.

[0146] The present application also provides a computer-readable storage medium, on which instructions are stored, and when the instructions in the computer-readable storage medium are executed by a processor of a computer device, the computer is enabled to execute the model adaptation method provided in the above-mentioned embodiment. For example, the computer-readable storage medium may be a memory 703 including instructions, and the above instructions may be executed by a processor 702 of a computer device to complete the above method. Optionally, the computer-readable storage medium may be a non-temporary computer-readable storage medium, for example, the non-temporary computer-readable storage medium may be a ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc.

[0147] Figure 8 A conceptual partial view of a computer program product provided by an embodiment of the present application is schematically shown, where the computer program product includes a computer program for executing a computer process on a computing device.

[0148] In one embodiment, the computer program product is provided using a signal bearing medium 800. The signal bearing medium 800 may include one or more program instructions that, when executed by one or more processors, may provide the above-described Figure 2 , Figure 3 , Figure 4 Thus, for example, reference to Figure 2 In the embodiment shown in , one or more features of S201 to S203 may be undertaken by one or more instructions associated with the signal bearing medium 800. In addition, Figure 8 The program instructions in also describe example instructions.

[0149] In some examples, the signal bearing medium 800 may include a computer readable medium 801, such as, but not limited to, a hard drive, a compact disk (CD), a digital video disk (DVD), a digital tape, a memory, a read-only memory (ROM) or a random access memory (RAM), and the like.

[0150] In some implementations, the signal bearing medium 800 may include a computer recordable medium 802 such as, but not limited to, a memory, a read / write (R / W) CD, a R / W DVD, or the like.

[0151] In some implementations, signal bearing medium 800 may include communication medium 803 such as, but not limited to, digital and / or analog communication media (eg, fiber optic cables, waveguides, wired communication links, wireless communication links, etc.).

[0152] The signal bearing medium 800 may be communicated by a wireless form of communication medium 803. The one or more program instructions may be, for example, computer executable instructions or logic implemented instructions.

[0153] In some examples, the model adaptation device can be configured to provide various operations, functions, or actions in response to one or more program instructions in computer-readable medium 801, computer-recordable medium 802, and / or communication medium 803.

[0154] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0155] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of modules or units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0156] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0157] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0158] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or the full classification part or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute the full classification part or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes various media that can store program codes, such as-U disk, mobile hard disk, ROM, RAM, disk or CD.

[0159] The above are only specific implementations of the present application, but the protection scope of the present application is not limited thereto, and any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A model adaptation method, characterized in that: The method comprises: Acquire model information of the model to be migrated, initial carrier information of the initial carrier unit, and target carrier information of the target carrier unit; Determining a carrier difference based on the initial carrier information and the target carrier information; The model to be migrated is adjusted based on the model information and the carrier difference to obtain an adjusted model.

2. The method according to claim 1, characterized in that The model information includes: the size of the model to be migrated and a plurality of initial modules included in the model to be migrated; and adjusting the model to be migrated based on the model information and the carrier difference to obtain an adjusted model includes: Based on the model information and the carrier difference information, the size of the model to be migrated and the multiple initial modules included in the model to be migrated are adjusted to obtain the adjusted model.

3. The method according to claim 2, characterized in that The adjusting, based on the model information and the carrier difference information, the size of the model to be migrated and the multiple initial modules included in the model to be migrated to obtain the adjusted model includes: Based on the model information and the carrier difference information, adjust the size of the model to be migrated and the multiple initial modules included in the model to be migrated to obtain a first model including multiple first modules, the size of the first model is smaller than the size of the model to be migrated, and the number of the multiple first modules is smaller than the number of the multiple initial modules; The first model is parsed, and the order of the plurality of first modules is adjusted based on the parsing result and the target carrier information to obtain a second model, where the second model is the adjusted model.

4. The method according to claim 3, characterized in that The method further comprises: Based on the initial operator of the model to be migrated, determine a target operator adapted to the target carrier unit from an operator library, wherein the function of the initial operator is the same as that of the target operator; A plurality of target operators are used as operators in the second model.

5. The method according to claim 4, characterized in that The determining, based on the initial operator of the model to be migrated, a target operator adapted to the target carrier unit from an operator library comprises: Determine from the operator library a candidate operator having the same function as the initial operator; Initializing a target environment of the target carrier unit, using the operator to be selected as an operator in the adjusted model in the target environment, and testing the adjusted model to obtain a test result, wherein the test result is used to reflect the matching degree between the operator to be selected and the target environment; If the test result is greater than a preset test condition, the operator to be selected is used as the target operator.

6. The method according to any one of claims 1 to 5, characterized in that The method further comprises: Obtain adversarial sample data; In a case where the adjusted model is not migrated and / or migrated, adversarial training is performed on the adjusted model based on the adversarial sample data.

7. A model adaptation device, characterized in that: The device comprises an acquisition module and a processing module: The acquisition module is used to acquire model information of the model to be migrated, initial carrier information of the initial carrier unit and target carrier information of the target carrier unit; The processing module is used to determine a carrier difference based on the initial carrier information and the target carrier information; The processing module is further used to adjust the model to be migrated based on the model information and the carrier difference to obtain an adjusted model.

8. A model adaptation device, characterized in that: include: Processor and memory; The processor is coupled to the memory; The memory is used to store one or more programs, and one or more of the programs include computer-executable instructions. When the adaptation device of the model is running, the processor executes the computer-executable instructions stored in the memory to enable the adaptation device of the model to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, characterized in that: When a computer executes the instruction, the computer performs the method according to any one of claims 1 to 6.

10. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device, the computing device is caused to perform the method according to any one of claims 1 to 6.

Images