Model alignment method, platform and apparatus

Abstract

The present disclosure provides a model alignment method, platform and device, relates to the field of artificial intelligence, in particular to the field of deep learning. The specific implementation scheme is: obtaining a first model and a second model to be aligned uploaded by a user through a model alignment platform; aligning a data pipeline of the second model with a data pipeline of the first model; aligning the accuracy of the second model with the accuracy of the first model; aligning the optimization parameters of the second model with the optimization parameters of the first model; storing the aligned second model to a specified path for the user to download through the model alignment platform. This embodiment realizes the alignment of models under different framework mechanisms, reduces the labor cost, and improves the positioning efficiency.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence, in particular to the field of deep learning, and specifically to a model alignment method, platform and device. BACKGROUND

[0002] In the initial stage of deep learning, each deep learning researcher needs to write a large amount of repetitive code. In order to improve work efficiency, these researchers write these codes into a framework and put it on the Internet for all researchers to use. Then, different frameworks appear on the Internet. With the passage of time, a few of the most useful frameworks are used by a large number of people and thus become popular. The most popular deep learning frameworks in the world are Paddle, Tensorflow, Caffe, Theano, MXNet, Torch and PyTorch.

[0003] Model teams often need to implement different models and align the forward, backward and training with other models.

[0004] Due to the differences in implementation of framework mechanisms, operators (Operator, referred to as Op), initialization, optimizers, APIs, etc., the model alignment work is difficult, repetitive or unable to converge, etc., which wastes a lot of manpower and effort. SUMMARY

[0005] The present disclosure provides a model alignment method, device, equipment, storage medium and computer program product.

[0006] According to a first aspect of the present disclosure, a model alignment method is provided, comprising: obtaining a first model and a second model to be aligned uploaded by a user through a model alignment platform; aligning a data pipeline of the second model with a data pipeline of the first model; aligning the accuracy of the second model with the accuracy of the first model; aligning the optimization parameters of the second model with the optimization parameters of the first model; storing the aligned second model to a specified path for the user to download through the model alignment platform.

[0007] According to a second aspect of the present disclosure, a model alignment platform is provided, comprising: an input module configured to receive a first model and a second model to be aligned uploaded by a user; an execution module configured to execute the method of any one of the first aspect; an output module configured to output the output results and gradients of each layer of the first model and the second model, and compare whether the difference between the output results of each layer of the first model and the second model meets the accuracy requirement, and store the aligned second model to a specified path for the user to download.

[0008] According to a third aspect of the present disclosure, a model alignment apparatus is provided, comprising: an acquisition unit configured to acquire a first model and a second model to be aligned uploaded by a user through a model alignment platform; a data alignment unit configured to align a data pipeline of the second model with a data pipeline of the first model; a precision alignment unit configured to align a precision of the second model with a precision of the first model; an optimization alignment unit configured to align an optimization parameter of the second model with an optimization parameter of the first model; and an output unit configured to store the aligned second model to a specified path for the user to download through the model alignment platform.

[0009] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of the first aspect.

[0010] According to a fifth aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the method of any one of the first aspect.

[0011] According to a sixth aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method of any one of the first aspect.

[0012] Embodiments of the present disclosure provide a model alignment method, platform and apparatus, which abstract the model alignment work into three stages, and propose an alignment strategy of control variable factors to improve the efficiency of positioning problems. At the same time, through the platform, the alignment time is saved, and the waste of manpower and energy is avoided.

[0013] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS

[0014] The accompanying drawings are used to better understand the present scheme, and do not limit the present disclosure. Among them:

[0015] Figure 1 is an exemplary system architecture diagram to which an embodiment of the present disclosure can be applied;

[0016] Figure 2 is a flowchart of one embodiment of the model alignment method according to the present disclosure;

[0017] Figure 3 is a schematic diagram of one application scenario of the model alignment method according to the present disclosure;

[0018] Figure 4 is a schematic diagram of one embodiment of the model alignment platform according to the present disclosure;

[0019] Figure 5 is a structural schematic diagram of one embodiment of the model alignment device according to the present disclosure;

[0020] Figure 6 is a structural schematic diagram of a computer system of an electronic device suitable for implementing embodiments of the present disclosure. DETAILED DESCRIPTION

[0021] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to assist in understanding, which should be considered in a descriptive sense only. Thus, it will be apparent to those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, for the sake of brevity and clarity, descriptions of well-known functions and constructions are omitted from the following description.

[0022] Figure 1 An exemplary system architecture 100 to which the model alignment method of the embodiments of the present disclosure can be applied is shown.

[0023] As shown in Figure 1 , the system architecture 100 can include terminals 101, 102, a network 103, a database server 104, and a server 105. The network 103 serves as a medium to provide communication links between the terminals 101, 102, the database server 104, and the server 105. The network 103 can include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0024] A user 110 can use the terminals 101, 102 to interact with the server 105 through the network 103 to receive or send messages, etc. Various client applications can be installed on the terminals 101, 102, such as model training applications, target detection applications, shopping applications, payment applications, web browsers, and instant messaging tools, etc.

[0025] The terminals 101 and 102 herein can be hardware or software. When the terminals 101 and 102 are hardware, they can be various electronic devices with display screens, including but not limited to smartphones, tablet computers, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), laptop computers, desktop computers, and the like. When the terminals 101 and 102 are software, they can be installed in the above-listed electronic devices. They can be implemented as multiple software or software modules (for example, to provide distributed services) or as a single software or software module. No specific limitation is made herein.

[0026] When the terminals 101 and 102 are hardware, they can also have image acquisition devices installed thereon. The image acquisition devices can be various devices capable of acquiring images, such as cameras, sensors, and the like. The user 110 can use the image acquisition devices on the terminals 101 and 102.

[0027] The database server 104 can be a database server providing various services. For example, the database server can store a sample set. The sample set contains a small number of samples. For example, the samples can include sample images and new class labels of target objects. In this way, the user 110 can also select samples from the sample set stored by the database server 104 through the terminals 101 and 102.

[0028] The server 105 can also be a server providing various services, such as a background server supporting various applications displayed on the terminals 101 and 102. The background server can use the samples in the sample set sent by the terminals 101 and 102 to align the model of one framework with the model of another framework, then train the aligned models, and can send the trained models to the terminals 101 and 102. In this way, the user can apply the generated models to implement image recognition, speech recognition, text classification, and the like.

[0029] The database server 104 and the server 105 herein can also be hardware or software. When they are hardware, they can be implemented as a distributed server cluster composed of multiple servers or as a single server. When they are software, they can be implemented as multiple software or software modules (for example, to provide distributed services) or as a single software or software module. No specific limitation is made herein. The database server 104 and the server 105 can also be servers of a distributed system or servers combined with blockchains. The database server 104 and the server 105 can also be cloud servers or intelligent cloud computing servers or intelligent cloud hosts with artificial intelligence technology.

[0030] It should be noted that the model alignment method provided in this embodiment is generally executed by server 105. Accordingly, the model alignment device is also generally provided in server 105.

[0031] It should be noted that if server 105 can perform the relevant functions of database server 104, database server 104 may not be set up in system architecture 100.

[0032] It should be understood that Figure 1 The number of terminals, networks, database servers, and servers shown is merely illustrative. Depending on implementation needs, any number of terminals, networks, database servers, and servers can be included.

[0033] See also Figure 2 The diagram illustrates a flow 200 of an embodiment of a model alignment method according to the present disclosure. The model alignment method may include the following steps:

[0034] Step 201: Obtain the first and second models to be aligned uploaded by the user through the model alignment platform.

[0035] In this embodiment, the execution body of the model alignment method (e.g.) Figure 1 The server shown can obtain the first and second models uploaded by the user for alignment from a server with the model alignment platform installed. The first and second models can be neural network models from various fields such as image processing, speech recognition, and natural language processing. The first and second models are models developed under different frameworks but have the same function. For example, both the first and second models are object detection models, with the first model being a model from the torch framework and the second model being a model from the paddle framework.

[0036] Step 202: Align the data pipeline of the second model with the data pipeline of the first model.

[0037] In this embodiment, the data pipeline mainly consists of three parts: data preprocessing, dataset, and data loader. It is crucial to align the computations in the preprocessing part and the batch sampler strategy in the data loader. The shuffle strategy of the data pipeline will affect the final convergence effect and also needs attention. Finally, it is necessary to ensure that the number of examples processed by the pipeline is the same. Inconsistencies may lead to misaligned loss values ​​during training or validation, resulting in poor convergence.

[0038] The data preprocessing, data set, and data loading process related code of the data pipeline of the first model can be covered to align the data preprocessing, data set, and data loading process code of the data pipeline of the second model.

[0039] In step 203, the accuracy of the second model is aligned with the accuracy of the first model.

[0040] In this embodiment, the accuracy of the forward, backward, and loss processes needs to be aligned. Generally, the forward and loss are aligned to 1e-5 (i.e., the difference between the output of each layer in the forward and loss processes of the second model and the output of each layer in the forward and loss processes of the first model is less than 1e-5, because floating-point data cannot be completely equal), and the backward is aligned to 1e-6 (i.e., the difference between the output of each layer in the backward process of the second model and the output of each layer in the backward process of the first model is less than 1e-6). However, in actual operation, there may be framework factors or dependent library versions that cause the op implementation to not meet the above accuracy requirements, such as Layer Norm. At this time, the proportion of the accuracy that is not aligned and the maximum absolute error are needed to be considered to make a comprehensive decision. For example: diff 0.0003%, maxabsoluate diff: 0.00002. Although this case does not meet the alignment requirements, it has little or negligible impact on model convergence.

[0041] In step 204, the optimization parameters of the second model are aligned with the optimization parameters of the first model.

[0042] In this embodiment, whether the optimization parameters of the second model are aligned with the optimization parameters of the first model is checked, which can include: whether the training process is different, whether the gradclip is calculated correctly, whether the init strategy is consistent, whether the step number is consistent, whether the checkpoint resume is correct, whether the lr decrease is as expected, whether the loss decrease trend is consistent, whether the optimizer implementation is problematic, whether the optimization parameters are the same, whether the multi-optimizer behavior is aligned with the single-optimizer effect, whether the valid loss is consistent (which can affect the learning rate decrease), whether the accum_grad strategy is correct, whether the freeze model is not effective, and the like.

[0043] The optimization parameters inconsistent with the first model in the second model are modified to the optimization parameters of the first model.

[0044] In step 205, the aligned second model is stored in a specified path for the user to download through the model alignment platform.

[0045] In this embodiment, the second model is delivered to the user after alignment, and the aligned second model is stored in a specified path that can be accessed externally, a download link is generated, and the user can access the download link through the model alignment platform to download the aligned second model.

[0046] The method provided by the above embodiments of the present disclosure abstracts the model alignment work into three stages, improves the positioning efficiency, saves the alignment time, and avoids wasting manpower and energy.

[0047] In the model alignment process, since some parts have been aligned, alignment can be performed only on the unaligned parts. The implementation manners of data pipeline alignment, precision alignment, and optimization parameter alignment can be arbitrarily combined.

[0048] In some optional implementation manners of this embodiment, the aligning the data pipeline of the second model with the data pipeline of the first model comprises aligning the data preprocessing strategy of the second model with the data preprocessing strategy of the first model. The function called by the data preprocessing strategy of the second model can be directly replaced by the function called by the data preprocessing strategy of the first model. Thus, the second model has the same data preprocessing strategy as the first model, so that the training data input by the second model is consistent with the first model in the case that the batch sampling strategy and the data shuffling strategy are the same.

[0049] In some optional implementation manners of this embodiment, the aligning the data pipeline of the second model with the data pipeline of the first model comprises aligning the batch sampling strategy of the second model with the batch sampling strategy of the first model. The function called by the batch sampling strategy of the second model can be directly replaced by the function called by the batch sampling strategy of the first model. Thus, the second model has the same batch sampling strategy as the first model, so that the training data input by the second model is consistent with the first model in the case that the data preprocessing strategy and the data shuffling strategy are the same.

[0050] In some optional implementations of the embodiment, the aligning the data pipeline of the second model with the data pipeline of the first model comprises aligning a data shuffling strategy of the second model with a data shuffling strategy of the first model. The function called by the data shuffling strategy of the second model can be directly replaced by the function called by the data shuffling strategy of the first model. Thus, the second model has the same data shuffling strategy as the first model, so as to ensure that the training data input by the second model is consistent with the first model in the case that the data preprocessing strategy and the batch sampling strategy are the same.

[0051] In some optional implementations of the embodiment, the aligning the data pipeline of the second model with the data pipeline of the first model comprises aligning the number of samples processed by the data pipeline of the second model with the number of samples processed by the data pipeline of the first model. The ultimate purpose of the data pipeline alignment is to make the number of samples output by the two models consistent. In the case that the data preprocessing strategy, the batch sampling strategy and the data shuffling strategy are not completely aligned, the number of samples is aligned, so as to ensure that the training data input by the second model is consistent with the first model.

[0052] In some optional implementations of the embodiment, the aligning the data pipeline of the second model with the data pipeline of the first model comprises obtaining a batch of training data from the data pipeline of the first model as training data for subsequent training of the second model. This method can not align the data preprocessing strategy, the batch sampling strategy and the data shuffling strategy, and directly construct the same training data as the first model, so as to simplify the data pipeline alignment process and ensure subsequent precision alignment and optimization parameter alignment.

[0053] In some optional implementations of the embodiment, the aligning the precision of the second model with the precision of the first model comprises aligning the precision of the forward training stage of the second model to 1e-5. The output results of the forward training stage of the second model and the output results of the forward training stage of the first model are required to be less than 1e-5, so as to consider that the output results of the two models are the same. If the precision does not reach 1e-5, the alignment can be achieved by adjusting the implementation of the operator. In this way, it can be avoided that the output results of the two models cannot be the same due to the framework factor or the dependent library version, and it is mistakenly considered that the forward stage cannot be aligned.

[0054] In some optional implementations of the embodiment, the aligning the precision of the second model with the precision of the first model comprises: aligning the precision of the loss value of the second model to 1e-5. The loss values of the two models are considered to be the same only if the difference between the loss value of the second model and the loss value of the first model is less than 1e-5. If the precision does not reach 1e-5, the alignment can be achieved by adjusting the implementation of the operator. In this way, it can be avoided that the loss values of the two models cannot be the same on the op implementation due to framework factors or dependent library versions, and it is mistakenly considered that the loss values cannot be aligned.

[0055] In some optional implementations of the embodiment, the aligning the precision of the second model with the precision of the first model comprises: aligning the precision of the backpropagation stage of the second model to 1e-6. The output results of the two models are considered to be the same only if the difference between the output result of the backpropagation stage of the second model and the output result of the backpropagation stage of the first model is less than 1e-6. If the precision does not reach 1e-6, the alignment can be achieved by adjusting the implementation of the operator. In this way, it can be avoided that the output results of the two models cannot be the same on the op implementation due to framework factors or dependent library versions, and it is mistakenly considered that the backpropagation stage cannot be aligned.

[0056] In some optional implementations of the embodiment, the aligning the precision of the second model with the precision of the first model comprises: if the proportion of the misaligned precision is less than a first predetermined value and the maximum absolute error is less than a second predetermined value, stopping the aligning the precision of the second model with the precision of the first model. The output results and the loss values of each layer of the forward stage and the backpropagation stage of the two models are compared. If the difference between the output results or the loss values is greater than the predetermined precision, it is considered that there is no alignment. The proportion of the misaligned precision is calculated as the ratio of the frequency of the output results and the loss values that are not aligned to the total frequency of the output results and the loss values. For example, if the difference between the output results and the loss values of the forward stage and the backpropagation stage of the two models is 100, the number of times that the difference between the output results of the forward stage of the two models is greater than 1e-5 is 1, and the number of times that the difference between the output results of the backpropagation stage of the two models is greater than 1e-6 is 2, the proportion of the misaligned precision is 3%. The maximum absolute error refers to the maximum value of the difference between the output results and the loss values of all layers of the forward stage and the backpropagation stage of the two models.

[0057] In some optional implementations of the embodiment, the aligning of the accuracy of the second model with the accuracy of the first model comprises: setting the second model to a prediction mode to remove the influence of random factors. Aligning forward / loss / backword accuracy. Removing random factors in the model: such as setting the model to eval() mode, removing dropout (dropout), bn (batch normalization) and other random factors. This can ensure that after aligning the accuracy, the output results of the two models will not be different due to random factors.

[0058] In some optional implementations of the embodiment, the aligning of the optimization parameters of the second model with the optimization parameters of the first model comprises: the initialization model of the second model uses the initialization model of the first model. Using the same ckpt initialization model, first remove the interference of init. This avoids the case that even if the data pipeline and the accuracy are aligned and other optimization parameters are the same, the output results and loss value of the two models are greatly different due to inconsistent initialization.

[0059] In some optional implementations of the embodiment, the aligning of the optimization parameters of the second model with the optimization parameters of the first model comprises: fixing the optimizer and learning rate of the second model. The sgd optimizer and learning rate can be fixed to avoid problems such as lr scheduler (learning rate scheduler) and optimzier (optimizer) implementation.

[0060] In some optional implementations of the embodiment, the aligning of the optimization parameters of the second model with the optimization parameters of the first model comprises: deleting the gradient clipping strategy in the second model. Deleting gradclip (gradient clipping) and other factors that affect grad (gradient). This can prevent the problem that the loss value curve trend cannot be consistent due to inconsistent gradient clipping strategies of the two models.

[0061] Optionally, the initialization model constraint, the fixed optimizer and learning rate, and the deleted gradient clipping strategy can be added in sequence, and then the training loss value trends of the two models are aligned to solve the implementation problem of the "optimization" related code.

[0062] After alignment, gradually release the constraints. For example, add gradclip, then release lr, opt, and init. Finally, use the aligned datapipline to train the model.

[0063] At this point, the model alignment and problem positioning are achieved.

[0064] In some optional implementations of the embodiment, the method further comprises: after the alignment of the optimization parameters of the second model with the optimization parameters of the first model is completed, adding a gradient clipping strategy in the second model; training the first model and the second model respectively using the same training data to obtain a first training loss trend graph and a second training loss trend graph; and locating, according to the difference between the first training loss trend graph and the second training loss trend graph, a problem of inconsistency between the gradient clipping strategy of the second model and the gradient clipping strategy of the first model. In the case of other fixed optimization parameters, the addition of gradclip can locate the inconsistency of the output results of each layer and the loss value caused by the difference between the gradclip of the second model and the gradclip of the first model, and prevent the interference of other optimization parameters.

[0065] In some optional implementations of the embodiment, the method further comprises: after the alignment of the optimization parameters of the second model with the optimization parameters of the first model is completed, aligning the optimizer and the learning rate respectively by any one of the following ways: setting the optimizer of the second model to be adjustable while fixing the learning rate, then performing optimizer alignment, and then setting the learning rate to be adjustable to perform learning rate alignment; or preferentially performing learning rate alignment, and then performing optimizer alignment; training the first model and the second model respectively using the same training data to obtain a first training loss trend graph and a second training loss trend graph; and locating, according to the difference between the first training loss trend graph and the second training loss trend graph, a problem of non-equivalence of the implementation of the optimizer of the second model and the optimizer of the first model. There are two ways to align the optimizer and the learning rate, which can be selected according to the complexity of the model. In a simple case, the learning rate can be aligned offline, and the optimizer can be aligned first. In a complex case, the learning rate adjustment depends on the value of the val_loss (validation set loss), and at this time, the optimizer needs to be aligned together. Because the two need to be adjusted during the training process. For the complex case, the learning rate can be fixed first, the optimizer can be aligned, and then the learning rate can be aligned.

[0066] In the case of the same initialization model and the aligned gradient clipping strategy, only the adjustment strategies of the optimizer and the learning rate of the second model are different from those of the first model, the inconsistency of the output results of each layer and the loss value caused by the difference between the adjustment strategies of the optimizer and the learning rate of the second model and the adjustment strategies of the optimizer and the learning rate of the first model can be located, and the interference of other optimization parameters can be prevented.

[0067] In some optional implementations of the embodiment, the method further comprises: after the alignment of the optimized parameters of the second model and the optimized parameters of the first model is completed, re-generating an initialized model using the initialization strategy of the second model; training the first model and the second model using the same training data to obtain a first training loss trend graph and a second training loss trend graph; and locating the initialization strategy problem of the second model according to the difference between the first training loss trend graph and the second training loss trend graph. In the case where the gradient clipping strategy is aligned and the adjustment strategies of the optimizer and the learning rate are also aligned, the difference between the optimized parameters of the second model and the optimized parameters of the first model is only that the initialized models are different, and the inconsistency of the output results of each layer and the loss values caused by the difference between the initialization strategy of the second model and the initialization strategy of the first model can be located, thereby preventing the interference of other optimized parameters.

[0068] In practice, through this strategy, the init strategy problem, the inequality between the multi-opt and the single-opt implementation, the inconsistency between the gradclip strategy and torch, and other problems are located. These problems are difficult to think of and locate in actual alignment work.

[0069] In some optional implementations of the embodiment, the first model is a model of a torch framework, and the second model is a model of a paddle framework. The scheme of the application can align any combination of models of different frameworks, but in actual application, the scenario of aligning the model of the paddle framework with the model of the torch framework is more common and more complex. Through the method of the application, the alignment time and the labor cost can be reduced, and the model development efficiency can be improved.

[0070] In some optional implementations of the embodiment, the first model is a first image processing model, and the second model is a second image processing model. The model alignment platform can be applied to the image processing field, and receives the first image processing model and the second image processing model in the image processing field uploaded by a user. The aligned second image processing model can be used to implement image semantic segmentation, target detection, target tracking, image recognition and the like. The application scenarios can include but are not limited to video monitoring, face recognition, three-dimensional animation production, automatic driving and the like.

[0071] Continuing to refer to Figure 3 , Figure 3 is a schematic diagram of an application scenario of the model alignment method according to the embodiment. In Figure 3 the application scenario, it is necessary to align the model of the paddle framework with the model of the torch framework. In order to simplify the alignment process, we need to start from a "naked" starting point, gradually add variables, and finally achieve overall alignment.

[0072] 1, First align data: dump a batch of data from torch's pipeline. Subsequent training only loads this batch of data.

[0073] 2, Second align model: align the forward / loss / backward precision of the second model with the first model according to the requirements in the previous section. Remove random factors in the second model: such as setting the model to eval() mode, removing dropout, bn, etc.

[0074] 3, Finally align optimization parameters:

[0075] 1) The second model uses the first model's ckpt to initialize the model, and first removes the interference of init.

[0076] 2) Fix the sgd optimizer and lr of the second model to avoid lr scheduler and optimizer implementation issues.

[0077] 3) Remove gradclip and other factors that affect grad in the second model.

[0078] After the above steps, the training loss trend of paddle / troch can be aligned, and the implementation problem of "optimization" related code can be solved.

[0079] After alignment, gradually release the constraints. Such as adding gradclip, then releasing lr, opt, init. Finally, use the aligned datapipline to train the model.

[0080] Thus, the model alignment and problem positioning are realized.

[0081] In practice, through this strategy, the init strategy problem, the implementation inequality of multiple opt and single opt, the gradclip strategy and torch inconsistency, etc. These are difficult to think of and locate in the actual alignment work.

[0082] Further reference Figure 4 , which shows a schematic diagram of a model alignment platform. The model alignment platform includes: an input module, an execution module and an output module. Figure 4 The front-end interface of the model alignment platform is shown in the figure. The user inputs the model to be aligned through the front-end interface, the input module of the back-end model alignment platform receives the data, and the execution module is executed, and the output module outputs the data of each layer in the training process on the front-end interface.

[0083] In the embodiment, the input module is configured to receive a first model and a second model uploaded by a user to be aligned. The user can select two models to be aligned through a front-end interface. Alternatively, the input module can also receive training data input by the user.

[0084] In the embodiment, the execution module is configured to execute the method described in the process 200. After the user confirms to start training through the front-end interface, the execution module trains the two models and collects the activation and grad of each layer. The user can specify the precision to check whether the diff meets the expectation and whether there is an op implementation problem.

[0085] In the embodiment, the output module is configured to output the output results and gradients of each layer of the first model and the second model, compare whether the difference between the output results of each layer of the first model and the second model meets the precision requirement, and store the aligned second model to a specified path for the user to download. The output module can visualize the intermediate data (e.g., the output results and loss values of each layer in the forward stage and the reverse stage) in the training process. These data can be output in the form of a coordinate graph, Figure 4 only for illustration, the coordinate axes and key point values are not drawn. After the second model is aligned, the aligned second model is stored to a specified path, a download link is generated, and the user can download the aligned second model by clicking the link.

[0086] In some optional implementation manners of the embodiment, the output module is further configured to visualize the initialization strategy of the first model and the second model. The distribution of the initialized model init is visualized to check whether the initialization strategy is consistent.

[0087] In some optional implementation manners of the embodiment, the output module is further configured to visualize the loss curves of the first model and the second model under different variables. The experimental configuration and the corresponding loss curves can be tracked through scripts and platforms. The loss curves under different variables are analyzed by visualization to obtain intermediate results. Meanwhile, when a problem is found, the experimental environment can be provided for research and development, the communication and environment construction costs can be reduced, and the problem positioning can be accelerated.

[0088] Further reference Figure 5 , as an implementation of the method shown in the above figures, the present disclosure provides an embodiment of a model alignment device, which corresponds to the method embodiment shown in Figure 2 , and the device can be applied to various electronic devices.

[0089] As Figure 5As shown, the model alignment device 500 of the embodiment includes an acquisition unit 501, a data alignment unit 502, a precision alignment unit 503, and an optimization alignment unit 504. The acquisition unit 501 is configured to acquire a first model and a second model to be aligned uploaded by a user through a model alignment platform. The data alignment unit 502 is configured to align a data pipeline of the second model with a data pipeline of the first model. The precision alignment unit 503 is configured to align precision of the second model with precision of the first model. The optimization alignment unit 504 is configured to align optimization parameters of the second model with optimization parameters of the first model. An output unit 505 is configured to store the aligned second model to a specified path for the user to download through the model alignment platform.

[0090] In some optional implementations of the embodiment, the data alignment unit 502 is further configured to align a data preprocessing strategy of the second model with a data preprocessing strategy of the first model.

[0091] In some optional implementations of the embodiment, the data alignment unit 502 is further configured to align a batch sampling strategy of the second model with a batch sampling strategy of the first model.

[0092] In some optional implementations of the embodiment, the data alignment unit 502 is further configured to align a data shuffling strategy of the second model with a data shuffling strategy of the first model.

[0093] In some optional implementations of the embodiment, the data alignment unit 502 is further configured to align a number of samples of data pipeline processing of the second model with a number of samples of data pipeline processing of the first model.

[0094] In some optional implementations of the embodiment, the data alignment unit 502 is further configured to acquire a batch of training data from the data pipeline of the first model as training data for subsequent training of the second model.

[0095] In some optional implementations of the embodiment, the precision alignment unit 503 is further configured to align precision of a forward training stage of the second model to 1e-5.

[0096] In some optional implementations of the embodiment, the precision alignment unit 503 is further configured to align precision of a loss value of the second model to 1e-5.

[0097] In some optional implementations of the present embodiment, the precision alignment unit 503 is further configured to align the precision of the backward training stage of the second model to 1e-6.

[0098] In some optional implementations of the present embodiment, the precision alignment unit 503 is further configured to stop aligning the precision of the second model with the precision of the first model if the proportion of the precision of the second model that is not aligned is less than a first predetermined value and the maximum absolute error is less than a second predetermined value.

[0099] In some optional implementations of the present embodiment, the precision alignment unit 503 is further configured to set the second model to a prediction mode to remove the influence of random factors.

[0100] In some optional implementations of the present embodiment, the precision alignment unit 503 is further configured to use the initialization model of the first model as the initialization model of the second model.

[0101] In some optional implementations of the present embodiment, the precision alignment unit 503 is further configured to fix the optimizer and learning rate of the second model.

[0102] In some optional implementations of the present embodiment, the precision alignment unit 503 is further configured to delete the gradient clipping strategy in the second model.

[0103] In some optional implementations of the present embodiment, the optimization alignment unit 504 is further configured to add a gradient clipping strategy to the second model after completing the alignment of the optimization parameters of the second model with the optimization parameters of the first model; train the first model and the second model using the same training data to obtain a first training loss trend graph and a second training loss trend graph; and locate the inconsistency between the gradient clipping strategy of the second model and the gradient clipping strategy of the first model according to the difference between the first training loss trend graph and the second training loss trend graph.

[0104] In some optional implementations of the embodiment, the optimization alignment unit 504 is further configured to, after completing the alignment of the optimization parameters of the second model with the optimization parameters of the first model, align the optimizer and the learning rate respectively by any one of the following: setting the optimizer of the second model to be adjustable and fixing the learning rate to perform optimizer alignment, and then setting the learning rate to be adjustable to perform learning rate alignment; or preferentially performing learning rate alignment and then performing optimizer alignment; training the first model and the second model respectively using the same training data to obtain a first training loss trend graph and a second training loss trend graph; and locating the problem of non-equivalence of the implementation of the optimizer of the second model and the optimizer of the first model according to the difference between the first training loss trend graph and the second training loss trend graph.

[0105] In some optional implementations of the embodiment, the optimization alignment unit 504 is further configured to, after completing the alignment of the optimization parameters of the second model with the optimization parameters of the first model, regenerate an initialized model using the initialization strategy of the second model; train the first model and the second model respectively using the same training data to obtain a first training loss trend graph and a second training loss trend graph; and locate the problem of the initialization strategy of the second model according to the difference between the first training loss trend graph and the second training loss trend graph.

[0106] In some optional implementations of the embodiment, the optimization alignment unit 504 is further configured to: the first model is a model of a torch framework, and the second model is a model of a paddle framework.

[0107] In the technical solution of the disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the technical solution comply with relevant laws and regulations and do not violate public order and good customs.

[0108] According to the embodiments of the disclosure, the disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0109] An electronic device includes at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of flow 200.

[0110] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable the computer to perform the method of flow 200.

[0111] A computer program product comprising a computer program which, when executed by a processor, implements the method of flow 200.

[0112] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present disclosure described and / or claimed in this document.

[0113] As shown in Figure 6 The device 600 includes a computing unit 601 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the device 600 can also be stored in the RAM 603. The computing unit 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0114] Various components in the device 600 are connected to the I / O interface 605, including an input unit 606, such as a keyboard, a mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; the storage unit 608, such as magnetic disks, optical disks, etc.; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 609 allows the device 600 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0115] The computing unit 601 can be various general and / or special purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 601 performs various methods and processes described above, such as the model alignment method. For example, in some embodiments, the model alignment method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 600 via the ROM 602 and / or the communication unit 609. When the computer program is loaded onto the RAM 603 and executed by the computing unit 601, one or more steps of the model alignment method described above can be performed. Alternatively, in other embodiments, the computing unit 601 can be configured to perform the model alignment method by any other appropriate means, such as by means of firmware.

[0116] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0117] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine or entirely on a remote machine or server.

[0118] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0119] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0120] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0121] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0122] It should be understood that the various forms of flow shown above can be used to reorder, add, or remove steps. For example, the steps described in the present disclosure can be performed in parallel, in series, or in a different order, without limitation herein, so long as the desired results of the technology described in the present disclosure are achieved.

[0123] The specific implementation described above does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A model alignment method, comprising: obtaining a first model and a second model to be aligned uploaded by a user through a model alignment platform; aligning a data pipeline of the second model with a data pipeline of the first model; aligning precision of the second model with precision of the first model; aligning optimization parameters of the second model with optimization parameters of the first model; storing the aligned second model to a specified path for the user to download through the model alignment platform.

2. The method of claim 1, wherein, The aligning the data pipeline of the second model with the data pipeline of the first model comprises: aligning a data preprocessing strategy of the second model with a data preprocessing strategy of the first model.

3. The method of claim 1, wherein, The aligning the data pipeline of the second model with the data pipeline of the first model comprises: aligning a batch sampling strategy of the second model with a batch sampling strategy of the first model.

4. The method of claim 1, wherein, The aligning the data pipeline of the second model with the data pipeline of the first model comprises: aligning a data shuffling strategy of the second model with a data shuffling strategy of the first model.

5. The method of claim 1, wherein, The aligning the data pipeline of the second model with the data pipeline of the first model comprises: aligning a number of samples processed by the data pipeline of the second model with a number of samples processed by the data pipeline of the first model.

6. The method of claim 1, wherein, The aligning the data pipeline of the second model with the data pipeline of the first model comprises: obtaining a batch of training data from the data pipeline of the first model as training data for subsequent training of the second model.

7. The method of claim 1, wherein, The aligning the precision of the second model with the precision of the first model comprises: aligning the precision of a forward training stage of the second model to 1e-5.

8. The method of claim 1, wherein, The aligning the precision of the second model with the precision of the first model comprises: aligning the precision of a loss value of the second model to 1e-5.

9. The method of claim 1, wherein, The aligning the precision of the second model with the precision of the first model comprises: aligning the precision of a backward training stage of the second model to 1e-6.

10. The method of claim 1, wherein, The aligning the precision of the second model with the precision of the first model comprises: if a proportion of the precision of the second model that is not aligned is less than a first predetermined value and a maximum absolute error is less than a second predetermined value, then stopping aligning the precision of the second model with the precision of the first model.

11. The method of claim 1, wherein, The aligning the precision of the second model with the precision of the first model comprises: setting the second model to a prediction mode to remove the influence of random factors.

12. The method of claim 1, wherein, The aligning the optimization parameters of the second model with the optimization parameters of the first model comprises: an initialization model of the second model adopts an initialization model of the first model.

13. The method of claim 1, wherein, The aligning the optimization parameters of the second model with the optimization parameters of the first model comprises: fixing an optimizer and a learning rate of the second model.

14. The method of claim 1, wherein, The aligning the optimization parameters of the second model with the optimization parameters of the first model comprises: deleting a gradient clipping strategy in the second model.

15. The method of claim 14, wherein, The method further comprises: After the alignment of the optimization parameters of the second model with the optimization parameters of the first model is completed, a gradient clipping strategy is added to the second model; The first model and the second model are respectively trained using the same training data to obtain a first training loss trend graph and a second training loss trend graph; According to the difference between the first training loss trend graph and the second training loss trend graph, the inconsistency problem of the gradient clipping strategy of the second model and the gradient clipping strategy of the first model is located.

16. The method of claim 13, wherein, The method further comprises: After the alignment of the optimization parameters of the second model with the optimization parameters of the first model is completed, the optimizer and the learning rate are respectively aligned in any one of the following ways: setting the optimizer of the second model to be adjustable and fixing the learning rate, then performing optimizer alignment, and then setting the learning rate to be adjustable and performing learning rate alignment; or preferentially performing learning rate alignment, and then performing optimizer alignment; The first model and the second model are respectively trained using the same training data to obtain a first training loss trend graph and a second training loss trend graph; According to the difference between the first training loss trend graph and the second training loss trend graph, the non-equivalence problem of the implementation of the optimizer of the second model and the optimizer of the first model is located.

17. The method of claim 12, wherein, The method further comprises: After the alignment of the optimization parameters of the second model with the optimization parameters of the first model is completed, an initialization model is regenerated using the initialization strategy of the second model; The first model and the second model are respectively trained using the same training data to obtain a first training loss trend graph and a second training loss trend graph; According to the difference between the first training loss trend graph and the second training loss trend graph, the initialization strategy problem of the second model is located.

18. The method of claim 1, wherein, The first model is a model of a torch framework, and the second model is a model of a paddle framework.

19. The method of claim 1, wherein, The first model is a first image processing model, and the second model is a second image processing model.

20. A model alignment platform, comprising: an input module configured to receive a first model and a second model uploaded by a user and to be aligned; an execution module configured to execute the method of any one of claims 1-19; an output module configured to output the output results and gradients of each layer of the first model and the second model, and to compare whether the difference between the output results of each layer of the first model and the second model meets the accuracy requirement, and to store the aligned second model to a specified path for the user to download.

21. The platform of claim 20, wherein, The output module is further configured to: visually output the initialization strategies of the first model and the second model.

22. The platform of claim 20, wherein, The output module is further configured to: visually output the loss curves of the first model and the second model under different variables.

23. A model alignment device, comprising: an acquisition unit configured to acquire a first model and a second model uploaded by a user through a model alignment platform and to be aligned; a data alignment unit configured to align a data pipeline of the second model with a data pipeline of the first model; an acquisition unit configured to acquire a first model and a second model uploaded by a user through a model alignment platform and to be aligned; a data alignment unit configured to align a data pipeline of the second model with a data pipeline of the first model; a precision alignment unit configured to align a precision of the second model with a precision of the first model; an optimization alignment unit configured to align an optimization parameter of the second model with an optimization parameter of the first model; an output unit configured to store the aligned second model to a designated path for the user to download through the model alignment platform.

24. The apparatus of claim 23, wherein, The data alignment unit is further configured to: align a data preprocessing strategy of the second model with a data preprocessing strategy of the first model.

25. The apparatus of claim 23, wherein, The data alignment unit is further configured to: align a batch sampling strategy of the second model with a batch sampling strategy of the first model.

26. The apparatus of claim 23, wherein, The data alignment unit is further configured to: align a data shuffling strategy of the second model with a data shuffling strategy of the first model.

27. The apparatus of claim 23, wherein, The data alignment unit is further configured to: align a number of samples of data pipeline processing of the second model with a number of samples of data pipeline processing of the first model.

28. The apparatus of claim 23, wherein, The data alignment unit is further configured to: obtain a batch of training data from a data pipeline of the first model as training data for subsequent training of the second model.

29. The apparatus of claim 23, wherein, The precision alignment unit is further configured to: align a precision of a forward training stage of the second model to 1e-5.

30. The apparatus of claim 23, wherein, The precision alignment unit is further configured to: align a precision of a loss value of the second model to 1e-5.

31. The apparatus of claim 23, wherein, The precision alignment unit is further configured to: align a precision of a backward training stage of the second model to 1e-6.

32. The apparatus of claim 23, wherein, The precision alignment unit is further configured to: stop aligning the precision of the second model with the precision of the first model if a proportion of misaligned precision of the second model is less than a first predetermined value and a maximum absolute error is less than a second predetermined value.

33. The apparatus of claim 23, wherein, The precision alignment unit is further configured to: set the second model to a prediction mode to remove the influence of random factors.

34. The apparatus of claim 23, wherein, The precision alignment unit is further configured to: an initialization model of the second model adopts an initialization model of the first model.

35. The apparatus of claim 23, wherein, The precision alignment unit is further configured to: fix an optimizer and a learning rate of the second model.

36. The apparatus of claim 23, wherein, The precision alignment unit is further configured to: delete a gradient clipping strategy in the second model.

37. The apparatus of claim 36, wherein, The optimization alignment unit is further configured to: add a gradient clipping strategy in the second model after completing alignment of the optimization parameter of the second model with the optimization parameter of the first model; train the first model and the second model respectively using the same training data to obtain a first training loss trend chart and a second training loss trend chart; according to a difference between the first training loss trend chart and the second training loss trend chart, locate a problem of inconsistency between a gradient clipping strategy of the second model and a gradient clipping strategy of the first model.

38. The apparatus of claim 35, wherein, The optimization alignment unit is further configured to: After the alignment of the optimization parameters of the second model with the optimization parameters of the first model, the optimizer and the learning rate are aligned in the following ways: the optimizer of the second model is set to be adjustable and the learning rate is fixed, and then the optimizer is aligned, and then the learning rate is set to be adjustable and the learning rate is aligned; Or, the learning rate alignment is performed first, and then the optimizer alignment is performed; The first model and the second model are respectively trained using the same training data to obtain a first training loss trend graph and a second training loss trend graph; According to the difference between the first training loss trend graph and the second training loss trend graph, the problem of non-equivalence of the implementation of the optimizer of the second model and the optimizer of the first model is located.

39. The apparatus of claim 34, wherein, The optimization alignment unit is further configured to: After the alignment of the optimization parameters of the second model with the optimization parameters of the first model, the initialization model is regenerated using the initialization strategy of the second model; The first model and the second model are respectively trained using the same training data to obtain a first training loss trend graph and a second training loss trend graph; According to the difference between the first training loss trend graph and the second training loss trend graph, the problem of the initialization strategy of the second model is located.

40. The apparatus of claim 23, wherein, The first model is a model of a torch framework, and the second model is a model of a paddle framework.

41. The apparatus of claim 23, wherein, The first model is a first image processing model, and the second model is a second image processing model.

42. An electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-19.

43. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method according to any one of claims 1-19.

44. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1-19.

Images