Replicability of model

Abstract

According to implementations of the present disclosure, a solution for generating and verifying a replicable model is provided. In this solution, the target addition process associated with a target model to be generated can be determined, wherein the target addition process includes accumulation of a set of data. Further, an expression of the target addition process can be adjusted, so that the target addition process is executed in a predetermined addition order during execution of the target model. Accordingly, the target model can be generated based on the adjusted target addition process. Therefore, embodiments of the present disclosure can enable an accumulation process of the model to be always executed according to the predetermined addition order, thereby avoiding non-replicability of the model due to randomness of the accumulation process.

Description

REPLICABILITY OF MODELBACKGROUND ART

[0001] In recent years, machine learning techniques are experiencing rapid development and have been applied to many aspects of people's lives. For example, people can use the machine learning techniques to perform scientific research and publish their research results.

[0002] However, a model used in the conventional machine learning techniques is nonreplicable. That is, it is difficult to reproduce the same model across different computing platforms, which makes it difficult to effectively verily relevant statements made by people using the machine learning model.SUMMARY OF THE INVENTION

[0003] According to implementations of the present disclosure, a solution for generating a model is provided. In this solution, a target addition process associated with a target model to be generated can be determined, the target addition process comprising accumulation of a set of data. Further, the expression of the target addition process can be adjusted, so that the target addition process is executed in a predetermined addition order during execution of the target model. Accordingly, the target model can be generated based on the adjusted target addition process. Therefore, embodiments of the present disclosure can enable an accumulation process of the model to be always executed in a predetermined order, thereby avoiding non-replicability of the model due to randomness of the accumulation process.

[0004] According to the implementations of the present disclosure, a solution for verifying the model is also provided. In this solution, at least one statement associated with a target model can be obtained, wherein the at least one statement is related to running of the target model on a first platform. Further, a replication model of the target model can be executed on a second platform, wherein the replication model is constructed based on model information of the target model. Accordingly, the at least one statement can be verified based on an execution result of the replication model on the second platform. Therefore, the embodiments of the present disclosure can verify the statement associated with the target model by executing the replication model of the target model on another platform.

[0005] The Summary section is provided to introduce the identification of concepts in a simplified form, which will be further described in the Detailed Description below. The Summary section is intended to neither identify key features or essential features of the claimed subject matter, nor limit the scope of the claimed subject matter.BRIEF DESCRIPTION OF THE DRAWINGS

[0006] Fig. 1 shows a schematic diagram of an exemplary environment according to someimplementations of the present disclosure;

[0007] Figs. 2A-2C show schematic diagrams for processing an addition process according to some implementations of the present disclosure;

[0008] Fig. 3 shows a schematic diagram of a comparison computational graph according to some implementations of the present disclosure;

[0009] Fig. 4 shows a flowchart of an exemplary method for generating a model according to some implementations of the present disclosure;

[0010] Fig. 5 shows a flowchart of an exemplary method for verifying a model according to some implementations of the present disclosure; and

[0011] Fig. 6 shows a block diagram of an exemplary computing device according to some implementations of the present disclosure.

[0012] In these drawings, the same or similar reference signs are used to indicate the same or similar elements.DETAILED DESCRIPTION

[0013] The present disclosure will now be discussed with reference to several exemplary implementations. It should be understood that these implementations are provided only to enable those ordinary skilled in the art to better understand and thus realize the present disclosure, without implying any limitation on the scope of the subject matter.

[0014] As used herein, the term "includes" and its variants are to be interpreted as an open- ended term meaning "including but not limited to". The term "based on" is to be interpreted as "at least partially based on". The terms "an implementation" and "one implementation" are to be interpreted as "at least one implementation". The term "another implementation" is to be interpreted as "at least one another implementation". The terms "first", "second" and so on may refer to different or the same object. Other explicit and implicit definitions may be included below.

[0015] As discussed above, in the field of machine learning, the replicability of a model is people's focus of attention. Due to the complexity of a machine learning model, the results of the model on different computing platforms exhibit greater randomness, which makes it difficult to replicate the model across different computing platforms. Therefore, it is also difficult for people to verify the model or its related statements by reproducing the model, which makes it difficult to effectively guarantee the credibility of the model and its related statements.

[0016] According to the implementations of the present disclosure, a solution for generating a model is provided. In this solution, a target addition process associated with a target model to be generated can be determined, the target addition process comprising accumulation of a set of data. Further, the expression of the target addition process can be adjusted, so that the targetaddition process is executed in a predetermined addition order during execution of the target model. Accordingly, the target model can be generated based on the adjusted target addition process. Therefore, the embodiments of the present disclosure can enable an accumulation process of the model to be always executed in a predetermined order, thereby avoiding nonreplicability of the model due to randomness of the accumulation process. In this way. the embodiments of the present disclosure can improve the credibility of the model and its related statements.

[0017] According to the implementations of the present disclosure, a solution for verifying the model is also provided. In this solution, at least one statement associated with a target model can be obtained, wherein the at least one statement is related to running of the target model on a first platform. Further, a replication model of the target model can be executed on a second platform, wherein the replication model is constructed based on model information of the target model. Accordingly, the at least one statement can be verified based on an execution result of the replication model on the second platform. Therefore, the embodiments of the present disclosure can verify the statement associated with the target model by executing the replication model of the target model on another platform.

[0018] Basic principles and several exemplary' implementations of the present disclosure w ill be explained below with reference to the attached drawings.Exemplary environment

[0019] Fig. 1 shows an exemplary environment 100 according to some implementations of the present disclosure. As shown in Fig. 1, the environment 100 may include a generation device 110. As shown in Fig. 1, the generation device 110 may be configured to generate a target model 120. Such a target model 120 may include any suitable machine learning model, examples of which may include, but not limited to, a neural network model, a decision tree model, and the like.

[0020] As will be described in detail below, the generation device 110 may generate a replicable target model 120 and may. for example, provide model information 130 of the target model 120. Such model information 130 may, for example, be used to reproduce the target model on the same or different devices. Exemplarily, such model information 130 may, for example, include code expression, model parameters, and the like of the target model 120.

[0021] Further, the generation device 110 may also provide at least one statement 140 associated with the target model 120. Such a statement 140 may, for example, include an identify relation statement associated with the target model 120. In some implementations, such a statement 140 may be associated w ith a training process of the target model 120. For example, the statement 140 may be configured to indicate training data used in training the target model120.

[0022] In some implementations, such a statement 140 may be associated with an inference process of the target model 120. For example, the statement 140 may include an inference result obtained by inference through the target model 120.

[0023] Additionally, as shown in Fig. 1, the environment 100 may, for example, further include a verification device 160. The verification device 160 may, for example, generate a replication model 150 of the target model 120 based on the model information 130 of the target model 120.

[0024] As will be described in detail below, the generation device 110 may make the generated target model 120 replicable. Thus, the verification device 160 can construct the replication model 150 completely corresponding to the target model 120.

[0025] Further, the verification device 160 may also verify the at least one statement 140 associated with the target model 120 by using the constructed replication model 150.

[0026] The generation process of the target model 120 and the verification process of the statement 140 will be further described below.Randomness related to accumulation process

[0027] In an operation process of the machine learning model, the accumulation process is an important reason leading to randomness of the result. For example, Fig. 2A shows an exemplary accumulation process. As shown in Fig. 2A. during the operation of the machine learning model, for example, the model needs to execute an addition operation on multiple data (for example, data "Al" to data "A8").

[0028] In some cases, when the model is deployed to different computing devices, different computing devices may execute different data sub-blocks for the accumulation process, resulting in different addition orders.

[0029] For example, some computing devices may divide the illustrated eight data into two data blocks and respectively execute the addition of data "Al" to data "A4" and the addition of data "A5" to data "A8" by using two accumulators.

[0030] On the contrary, other computing devices may divide the illustrated eight data into four data blocks, and respectively execute the addition of data "Al" to data "A2", the addition of data "A3" to data "A4", the addition of data "A5" to data "A6", and the addition of data "A7" to data "A8" by using four accumulators.

[0031] In these two scenarios, the addition order of data "Al" to data "A8" has changed, so that the result of the accumulation calculated by the two computing platforms will be different.

[0032] In some implementations, in order to solve the randomness caused by the accumulation process, the generation device 110 may determine a target addition process associated with the target model 120 to be generated, and such a target addition process may comprise theaccumulation of a set of data.

[0033] Further, the generation device 110 may adjust the expression (e.g., codes) of the target model 120, so that the set of data are always accumulated in a predetermined addition order. Therefore, the accumulation result is not affected by the computing platform.

[0034] In some implementations, the generation device 110 may execute the accumulation of the set of data serially using a single thread. Taking Fig. 2A as an example, the generation device 110 may sequentially accumulate data "Al" to data "A8" by using, for example, the single thread, so that different computing platforms can execute a target addition operation in the same addition order. In some implementations, the generation device 110 may execute the accumulation of the set of data by using, for example, a single accumulator.

[0035] In some implementations, the generation device 110 may, for example, support the division of the set of data into a plurality of data sub-blocks. As shown in Fig. 2B, the set of data may, for example, be divided into data sub-block 205-1 to data sub-block 205-2.

[0036] Further, the generation device 1 10 may execute the addition of the two data sub-blocks 205-1 and 205-2 using a single thread. For example, the generation device 110 may execute the addition of the two data sub-blocks by using a single accumulator, and when the calculation of the addition of data sub-block 205-2 is started, the addition result of data sub-block 205-1 is retained as a basis for the addition operation of data sub-block 205-2.

[0037] In this way, the implementations of the present disclosure can continue to retain flexibility of the data sub-blocks in the accumulation operation, which is thus favorable to improve the access and storage efficiency in a model operation process.

[0038] In some implementations, the generation device 110 may. for example, also determine the addition order based on a binary tree corresponding to the set of data. As shown in Fig. 2C, the generation device 110 may, for example, construct a binary tree 200C corresponding to the set of data to be added.

[0039] As shown in Fig. 2C, in the binary tree 200C, multiple data may be constructed as leaf nodes of the binary tree 200C. Further, in the process of accumulating the set of data, the generation device 110 may always add a pair of data corresponding to a pair of associated nodes to determine the data of their parent node(s). Such a pair of associated nodes may correspond to the same parent node.

[0040] Taking Fig. 2C as an example, the generation device 110 may, for example, always execute the addition operation on data "Al" and data "A2" to determine the data corresponding to parent node 210-1. Similarly, data "A3" and data "A4" are always subjected to the addition operation to determine the data corresponding to parent node 210-2. Data "A5" and data "A6" are always subjected to the addition operation to determine the data corresponding to parentnode 210-3. Data A7 and data A8 are always subjected to the addition operation to determine the data corresponding to parent node 210-4.

[0041] Further, the data corresponding to node 210-1 and the data corresponding to node 210-2 may be subjected to the addition to determine the data of parent node 210-5. The data corresponding to node 210-3 and the data corresponding to node 210-4 may be subjected to the addition to determine the data of parent node 210-6.

[0042] Finally, the data corresponding to node 210-5 and the data corresponding to node 210-6 may be subjected to the addition to determine the data of root node 210-7, that is, the accumulation result of data "Al" to data "A8".

[0043] By determining the addition order for the set of data through the binary tree, the implementations of the present disclosure can ensure that the addition order for the set of data by different computing platforms is always fixed. In addition, in such an addition process, the model may also, for example, support the execution of different addition operations in the addition process by using different threads (or different accumulators).

[0044] Taking Fig. 2C as an example, the target model 120 may, for example, support the calculation of the addition of data "Al" and data "A2", the addition of data "A3" and data "A4", the addition of data "A5" and data "A6", and the addition of data "A7" and data "A8" by using four threads (or four accumulators) respectively. Therefore, the embodiments of the present disclosure can support different computing devices to adopt different parallel mechanisms while ensuring uniformity of the addition order, thereby improving computation efficiency of the addition operation.Randomness related to rounding operation

[0045] In some scenarios, the calculation of the machine learning model will also cause some basic mathematical operations to produce different computation results on different computing platforms due to an inaccurate rounding operation.

[0046] In some implementations, the generation device 110 may identify the operation process associated with the rounding operation in the target model 120. For example, the rounding operation may be involved in a square root operation, an exponential operation, a logarithmic operation and the like.

[0047] In some implementations, the generation device 110 may, for example, implement such an operation process according to predetermined rounding rules, so that the rounding operation accuracy corresponding to the operation process is independent of the platform on which the model runs. Exemplarily, the generation device 110 may implement such an operation process according to the rounding rules defined by IEEE-754, so as to ensure that the result of the rounding operation is independent of the platform on which the model runs. Therefore, thereplicability of the generated target model 120 can be ensured.Randomness related to compilation process

[0048] In some scenarios, the computation result may have randomness due to a compilation process.

[0049] Exemplarily, a batch normalization process may be defined as:BatchNorm(

[0050] During the operation, the batch normalization process may, for example, be expressed in the form of a first line. However, after compilation, it may be realized as the expression of a second line or a third line, which leads to the randomness of the batch normalization result and makes the model nonreplicable.

[0051] In some implementations, in order to ensure the replicability of the model, the generation device 110 may determine a first computational graph associated with the target model 120 based on code representation of the target model 120.

[0052] Taking the fused multiply-add (FMA) operation as an example, its code representation may, for example, include "a*b+c". , and therefore, its corresponding first computational graph may be expressed as "a->Mul, b->Mul, Mul->Add, c->Add, Add->res".

[0053] In some implementations, the generation device 110 may also determine a second computational graph associated with the target model 120 by compiling the target model 120. In some scenarios, the second computational graph is also called a circuit diagram. Continuously taking the FMA operation as an example, it may be compiled as "FMA(a,b,c)", and its corresponding computational graph may be expressed as "a->FMA. b->FMA, c->FMA, FMA- >res".

[0054] Further, the generation device 110 may adjust the target model 120 based on comparison between the first computational graph and the second computational graph, so that the difference between the first computational graph and the second computational graph is less than a threshold. Continuously taking the FMA operation as an example, since the first computational graph is different from the second computational graph, the results of the FMA operation on different computing platforms may be different. Accordingly, the generation device 110 may, for example, adjust the codes or compilation process of the target model 120, so that its corresponding first and second computational graphs are kept consistent.

[0055] In this way, the implementations of the present disclosure can also overcome the randomness caused by the compilation process, thereby ensuring the replicability of the model.Verification based on replication model

[0056] In some implementations, as discussed with reference to Fig. 1, the generation device 110 may generate the target model 120 based on the process discussed above, so that the target model 120 is replicable.

[0057] Accordingly, the generation device 110 may provide the model information 130 of the target model 120, for example. Such model information 130 may, for example, be used to reproduce the target model 120 on the same or different device(s). Exemplarily, such model information 130 may, for example, include code expression, model parameters, and the like of the target model 120.

[0058] Further, as shown in Fig. 1, the generation device 110 may also provide at least one statement 140 associated with the target model 120. Such a statement 140 may, for example, be related to running of the target model 120 on a first platform (e.g.. the generation device 110 or other computing platforms).

[0059] In some implementations, such a statement 140 may be associated with a training process of the target model 120. For example, the statement 140 may be used to indicate training data for training the target model 120.

[0060] In some implementations, such a statement 140 may be associated with an inference process of the target model 120. For example, the statement 140 may include an inference result obtained by inference through the target model 120.

[0061] In some implementations, the verification device 160 may generate a replication model 150 of the target model 120 based on the model information 130 of the target model 120. The generation device 110 can ensure the replicability of the target model 120 based on the discussed process. Therefore, the replication model 150 constructed by the verification device 160 can completely correspond to the target model 120.

[0062] Further, the verification device 160 may also execute the constructed replication model 150 on a second platform (for example, the verification device 160 or other computing platforms), thereby verifying the at least one statement 140 associated with the target model 120.

[0063] Exemplarily, the statement 140 may be associated with the training process of the target model 120. The verification device 160 may train the replication model 150 by using the training data corresponding to the statement 140, and verify whether the statement 140 is accurate based on a training result. As another example, the statement 140 may be associated with the inference process of the target model 120. For example, the statement 140 may include a specific inference result acquired by using the target model 120. Accordingly, the verificationdevice 160 may execute the same inference process by using the replication model 150, and verify whether the statement 140 is accurate according to the inference result of the replication model 150.

[0064] Based on the above way, the implementations of the present disclosure can verify the related statement of the model by using the replicability of the model, thereby improving reliability of the statement.

[0065] In some other implementations, the replication model 150 may, for example, also be used for other appropriate purposes, examples of which may include, but not limited to: cooperating with the target model 120 to complete a common processing task, etc.Exemplary process

[0066] Fig. 4 shows a flowchart of an exemplary process 400 for generating a model according to some implementations of the present disclosure. The process 400 may, for example, be implemented by the generation device 110 in Fig. 1 or other suitable devices (such as the device 600 to be discussed in combination with Fig. 6).

[0067] At block 410, the generation device 110 determines a target addition process associated with a target model to be generated, and the target addition process comprises accumulation of a set of data.

[0068] At block 420, the generation device 110 adjusts an expression of the target addition process, so that the target addition process is executed in a predetermined addition order during execution of the target model.

[0069] At block 430, the generation device 110 generates the target model based on the adjusted target addition process.

[0070] In some implementations, adjusting the expression of the target addition process includes: executing the accumulation of the set of data serially using a single thread.

[0071] In some implementations, executing the accumulation of the set of data serially using the single thread includes: dividing the set of data into a plurality of data sub-blocks; and executing accumulation of the plurality of data sub-blocks serially using the single thread.

[0072] In some implementations, executing the accumulation of the set of data serially using the single thread includes: executing the accumulation of the set of data serially by using a single accumulator.

[0073] In some implementations, adjusting the expression of the target addition process includes: determining the predetermined addition order for the set of data based on a binary tree corresponding to the set of data, so that a pair of data corresponding to a pair of associated nodes in the binary' tree are added to determine data of a parent node, wherein the pair of associated nodes correspond to the same parent node; and adjusting the expression of the target additionprocess according to the predetermined addition order.

[0074] In some implementations, the process further includes: determining a target operation process associated with the target model; and adjusting an expression of the target operation process, so that a rounding operation accuracy corresponding to the target operation process is independent of a platform on which the target model runs.

[0075] In some implementations, the process further includes: determining a first computational graph associated with the target model based on code representation of the target model; determining a second computational graph associated with the target model by compiling the target model; and adjusting the target model based on comparison between the first computational graph and the second computational graph, so that a difference between the first computational graph and the second computational graph is less than a threshold.

[0076] In some implementations, the method further includes: determining at least one statement based on running of the target model on a first platform; and providing model information of the target model to verify the at least one statement based on an execution result of a replication model of the target model on a second platform, wherein the replication model is constructed based on the model information.

[0077] Fig. 5 shows a flowchart of an exemplary process 500 for verifying a model according to some implementations of the present disclosure. The process 500 may, for example, be implemented by the verification device 160 in Fig. 1 or other suitable devices (such as the device 600 to be discussed in combination with Fig. 6).

[0078] At block 510, the verification device 160 obtains at least one statement associated with a target model, and the at least one statement is related to running of the target model on a first platform.

[0079] At block 520, the verification device 160 executes a replication model of the target model on a second platform, and the replication model is constructed based on model information of the target model.

[0080] At block 530, the verification device 160 verifies the at least one statement based on an execution result of the replication model on the second platform.

[0081] In some implementations, the at least one statement includes: a first statement associated with a training process of the target model; and / or a second statement associated with an inference process of the target model.

[0082] In some implementations, a target addition process of a set of data in the replication model is configured to be executed in a predetermined addition order, and the predetermined addition order is independent of the second platform for executing the replication model.

[0083] In some implementations, during the execution of the replication model, the targetaddition process is executed by a single thread.

[0084] In some implementations, during the execution of the replication model, the target addition process is executed by a single accumulator.

[0085] In some implementations, the predetermined addition order is determined based on a binary tree corresponding to the set of data, so that a pair of data corresponding to a pair of associated nodes in the binary tree are added to determine data of a parent node, wherein the pair of associated nodes correspond to the same parent node.

[0086] In some implementations, the accuracy of a target operation process in the replication model is independent of the second platform for executing the replication model.

[0087] In some implementations, the difference between a first computational graph associated with the target model and a second computational graph associated with the target model is less than a threshold, the first computational graph is determined based on code representation of the target model, and the second computational graph is determined based on compilation of the target model.Exemplary device

[0088] Fig. 6 shows a schematic block diagram of an exemplary' device 600 that can be used to implement the embodiments of the present disclosure. It should be understood that the device 600 shown in Fig. 6 is merely exemplary and should not constitute any limitation on the functions and scope of the implementations described in the present disclosure. As shown in Fig. 6, components of the device 600 may include, but not limited to, one or more processors or processing units 610, a memory' 620, a storage device 630, one or more communication units 640, one or more input devices 650 and one or more output devices 660.

[0089] In some implementations, the device 600 may be implemented as vanous user terminals or service terminals. The service terminal may be a server, a large computing device, etc., provided by various sendee providers. The user terminal is, for example, any ty pe of mobile terminal, fixed terminal or portable terminal, including a mobile phone, a multimedia computer, a multimedia tablet, an internet node, a communicator, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an electronic book device, a gaming device or any combination thereof, including accessories and peripherals of such devices or any combination thereof. It is also foreseeable that the device 600 can support any type of user-oriented interface (such as a "wearable" circuit).

[0090] The processing unit 610 may be an actual or virtual processor and can execute various processes according to programs stored in the memory 620. In a multiprocessor system, multipleprocessing units execute computer-executable instructions in parallel to improve a parallel processing capability of the device 600. The processing unit 610 may also be called a central processing unit (CPU), a microprocessor, a controller, and a microcontroller.

[0091] The device 600 generally includes a plurality of computer storage mediums. Such mediums may be any available mediums accessible to the device 600. including but not limited to volatile and nonvolatile mediums and removable and non-removable mediums. The memory 620 may be a volatile memory (e.g., a register, a cache, a random access memory’ (RAM)), a nonvolatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM) and a flash memory) or some combination thereof. The memory’ 620 may include one or more generation modules and / or a generation module 625, and these program modules are configured to execute layout generation and / or page generation functions of various implementations described herein. The generation module 625 may be accessed and run by the processing unit 610 to realize corresponding functions. The storage device 630 may be a removable or non-removable medium and may include a machine-readable medium, which can be used to store information and / or data and can be accessed within the device 600.

[0092] Functions of the components of the device 600 may be realized in a single computing cluster or a plurality of computing machines, and these computing machines can communicate through communication connections. Therefore, the device 600 can operate in a networked environment by using a logical connection with one or more other servers, a personal computer (PC) or another general network node. The device 600 may also communicate with one or more external devices (not shown) through the communication unit 640 as needed, and the external device, such as a database, other storage devices, a server and a display device, communicates with one or more devices that enable users to interact with the device 600. or with any device (e g., a network card, a modem, etc.) that enables the device 600 to communicate with one or more other computing devices. Such communication may be carried out via an input / output (I / O) interface (not shown).

[0093] The input device 650 may be one or more of various input devices, such as a mouse, a keyboard, a trackball, a voice input device and a camera. The output device 660 may be one or more output devices, such as a display, a speaker and a printer.Exemplary implementations

[0094] Some exemplary implementations of the present disclosure are listed below.

[0095] In a first aspect of the present disclosure, a computer implemented method is provided. The method includes: determining a target addition process associated with a target model to be generated, the target addition process comprising accumulation of a set of data; adjusting an expression of the target addition process such that the target addition process is executedaccording to a predetermined addition order during execution of the target model; and generating the target model based on the adjusted target addition process.

[0096] In some implementations, adjusting the expression of the target addition process includes: executing the accumulation of the set of data serially using a single thread.

[0097] In some implementations, executing the accumulation of the set of data serially using the single thread includes: dividing the set of data into a plurality of data sub-blocks; and executing accumulation of the plurality' of data sub-blocks serially using the single thread.

[0098] In some implementations, executing the accumulation of the set of data serially using the single thread includes: executing the accumulation of the set of data serially by using a single accumulator.

[0099] In some implementations, adjusting the expression of the target addition process includes: determining the predetermined addition order for the set of data based on a binary' tree corresponding to the set of data, such that a pair of data corresponding to a pair of associated nodes in the binary tree are added to determine data of a parent node, wherein the pair of associated nodes correspond to the same parent node; and adjusting the expression of the target addition process according to the predetermined addition order.

[0100] In some implementations, the method further includes: determining a target operation process associated with the target model; and adjusting an expression of the target operation process such that a rounding operation accuracy corresponding to the target operation process is independent of a platform on which the target model runs.

[0101] In some implementations, the method further includes: determining a first computational graph associated with the target model based on code representation of the target model; determining a second computational graph associated with the target model by compiling the target model; and adjusting the target model based on comparison between the first computational graph and the second computational graph, such that a difference between the first computational graph and the second computational graph is less than a threshold.

[0102] In some implementations, the method further includes: determining at least one statement based on running of the target model on a first platform; and providing model information of the target model to verify the at least one statement based on an execution result of a replication model of the target model on a second platform, wherein the replication model is constructed based on the model information.

[0103] In a second aspect of the present disclosure, an electronic device is provided. The device includes: a processing unit; and a memory' coupled to the processing unit and containing instructions stored thereon. The instructions, when executed by the processing unit, cause the device to perform the following actions: determining a target addition process associated with atarget model to be generated, the target addition process comprising accumulation of a set of data; adjusting an expression of the target addition process such that the target addition process is executed according to a predetermined addition order during execution of the target model; and generating the target model based on the adjusted target addition process.

[0104] In some implementations, adjusting the expression of the target addition process includes: executing the accumulation of the set of data serially using a single thread.

[0105] In some implementations, executing the accumulation of the set of data serially using the single thread includes: dividing the set of data into a plurality of data sub-blocks; and executing accumulation of the plurality of data sub-blocks serially using the single thread.

[0106] In some implementations, executing the accumulation of the set of data serially using the single thread includes: executing the accumulation of the set of data serially by using a single accumulator.

[0107] In some implementations, adjusting the expression of the target addition process includes: determining the predetermined addition order for the set of data based on a binary tree corresponding to the set of data, such that a pair of data corresponding to a pair of associated nodes in the binary tree are added to determine data of a parent node, wherein the pair of associated nodes correspond to the same parent node; and adjusting the expression of the target addition process according to the predetermined addition order.

[0108] In some implementations, the actions further include: determining a target operation process associated with the target model; and adjusting an expression of the target operation process such that a rounding operation accuracy corresponding to the target operation process is independent of the platform on which the target model runs.

[0109] In some implementations, the actions further include: determining a first computational graph associated with the target model based on code representation of the target model; determining a second computational graph associated with the target model by compiling the target model; and adjusting the expression of the target model based on comparison between the first computational graph and the second computational graph, such that a difference between the first computational graph and the second computational graph is less than a threshold.

[0110] In some implementations, the actions further include: determining at least one statement based on running of the target model on a first platform; and providing model information of the target model to verily the at least one statement based on an execution result of a replication model of the target model on a second platform, wherein the replication model is constructed based on the model information.

[0111] In a third aspect of the present disclosure, a computer program product is provided. The computer program product is tangibly stored in a non-transitory computer storage mediumand includes machine-executable instructions. The machine-executable instructions, when executed by a device, cause the device to perform the following actions: determining a target addition process associated with a target model to be generated, the target addition process comprising accumulation of a set of data; adjusting an expression of the target addition process such that the target addition process is executed according to a predetermined addition order during execution of the target model; and generating the target model based on the adjusted target addition process.

[0112] In some implementations, adjusting the expression of the target addition process includes: executing the accumulation of the set of data serially using a single thread.

[0113] In some implementations, executing the accumulation of the set of data serially using the single thread includes: dividing the set of data into a plurality of data sub-blocks; and executing accumulation of the plurality of data sub-blocks serially using the single thread.

[0114] In some implementations, executing the accumulation of the set of data serially using the single thread includes: executing the accumulation of the set of data serially by using a single accumulator.

[0115] In some implementations, adjusting the expression of the target addition process includes: determining the predetermined addition order for the set of data based on a binary tree corresponding to the set of data, such that a pair of data corresponding to a pair of associated nodes in the binary tree are added to determine data of a parent node, wherein the pair of associated nodes correspond to the same parent node; and adjusting the expression of the target addition process according to the predetermined addition order.

[0116] In some implementations, the actions further include: determining a target operation process associated with the target model; and adjusting an expression of the target operation process such that a rounding operation accuracy corresponding to the target operation process is independent of the platform on which the target model runs.

[0117] In some implementations, the actions further include: determining a first computational graph associated with the target model based on code representation of the target model; determining a second computational graph associated with the target model by compiling the target model; and adjusting the expression of the target model based on comparison between the first computational graph and the second computational graph, such that a difference between the first computational graph and the second computational graph is less than a threshold.

[0118] In some implementations, the actions further include: determining at least one statement based on running of the target model on a first platform; and providing model information of the target model to verify the at least one statement based on an execution result of a replication model of the target model on a second platform, wherein the replication model isconstructed based on the model information.

[0119] In a fourth aspect of the present disclosure, a computer implemented method is provided. The method includes: obtaining at least one statement associated with a target model, the at least one statement being related to running of the target model on a first platform; executing a replication model of the target model on a second platform, the replication model being constructed based on model information of the target model; and verifying the at least one statement based on an execution result of the replication model on the second platform.

[0120] In some implementations, the at least one statement includes: a first statement associated with a training process of the target model; and / or a second statement associated with an inference process of the target model.

[0121] In some implementations, a target addition process of a set of data in the replication model is configured to be executed in a predetermined addition order, and the predetermined addition order is independent of the second platform for executing the replication model.

[0122] In some implementations, during the execution of the replication model, the target addition process is executed by a single thread.

[0123] In some implementations, during the execution of the replication model, the target addition process is executed by a single accumulator.

[0124] In some implementations, the predetermined addition order is determined based on a binary tree corresponding to the set of data, so that a pair of data corresponding to a pair of associated nodes in the binary tree are added to determine data of a parent node, wherein the pair of associated nodes correspond to the same parent node.

[0125] In some implementations, the accuracy of a target operation process in the replication model is independent of the second platform for executing the replication model.

[0126] In some implementations, the difference between a first computational graph associated with the target model and a second computational graph associated with the target model is less than a threshold, the first computational graph is determined based on code representation of the target model, and the second computational graph is determined based on compilation of the target model.

[0127] In a fifth aspect of the present disclosure, an electronic device is provided. The device includes: a processing unit; and a memory coupled to the processing unit and containing instructions stored thereon. The instructions, when executed by the processing unit, cause the device to perform the following actions: obtaining at least one statement associated with a target model, the at least one statement being related to running of the target model on a first platform; executing a replication model of the target model on a second platform, the replication model being constructed based on model information of the target model; and verifying the at least onestatement based on an execution result of the replication model on the second platform.

[0128] In some implementations, the at least one statement includes: a first statement associated with a training process of the target model; and / or a second statement associated with an inference process of the target model.

[0129] In some implementations, a target addition process of a set of data in the replication model is configured to be executed in a predetermined addition order, and the predetermined addition order is independent of the second platform for executing the replication model.

[0130] In some implementations, during the execution of the replication model, the target addition process is executed by a single thread.

[0131] In some implementations, during the execution of the replication model, the target addition process is executed by a single accumulator.

[0132] In some implementations, the predetermined addition order is determined based on a binary tree corresponding to the set of data, so that a pair of data corresponding to a pair of associated nodes in the binary tree are added to determine data of a parent node, wherein the pair of associated nodes correspond to the same parent node.

[0133] In some implementations, the accuracy of a target operation process in the replication model is independent of the second platform for executing the replication model.

[0134] In some implementations, the difference between a first computational graph associated with the target model and a second computational graph associated with the target model is less than a threshold, the first computational graph is determined based on code representation of the target model, and the second computational graph is determined based on compilation of the target model.

[0135] In a sixth aspect of the present disclosure, a computer program product is provided. The computer program product is tangibly stored in a non-transitory computer storage medium and includes machine-executable instructions. The machine-executable instructions, when executed by a device, cause the device to perform the following actions: obtaining at least one statement associated with a target model, the at least one statement being related to running of the target model on a first platform; executing a replication model of the target model on a second platform, the replication model being constructed based on model information of the target model; and verifying the at least one statement based on an execution result of the replication model on the second platform.

[0136] In some implementations, the at least one statement includes: a first statement associated with a training process of the target model; and / or a second statement associated with an inference process of the target model.

[0137] In some implementations, a target addition process of a set of data in the replicationmodel is configured to be executed in a predetermined addition order, and the predetermined addition order is independent of the second platform for executing the replication model.

[0138] In some implementations, during the execution of the replication model, the target addition process is executed by a single thread.

[0139] In some implementations, during the execution of the replication model, the target addition process is executed by a single accumulator.

[0140] In some implementations, the predetermined addition order is determined based on a binary' tree corresponding to the set of data, so that a pair of data corresponding to a pair of associated nodes in the binary tree are added to determine data of a parent node, wherein the pair of associated nodes correspond to the same parent node.

[0141] In some implementations, the accuracy of a target operation process in the replication model is independent of the second platform for executing the replication model.

[0142] In some implementations, the difference between a first computational graph associated with the target model and a second computational graph associated with the target model is less than a threshold, the first computational graph is determined based on code representation of the target model, and the second computational graph is determined based on compilation of the target model.

[0143] The functions described above herein may be at least partially performed by one or more hardware logic components. For example, without limitation, exemplary types of available hardware logic components include: a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), an application specific standard product (ASSP), a system-on- chip (SOC), a complex programmable logic device (CPLD) and so on.

[0144] Program codes for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer or other programmable data processing apparatuses, so that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program codes may be completely executed on a machine, partially executed on a machine, partially executed on a machine as a separate software package and partially executed on a remote machine or completely executed on a remote machine or server.

[0145] In the context of the present disclosure, the machine-readable medium may be a tangible medium, which may contain or store a program for use by or in combination with an instruction execution system, apparatus or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readablemedium may include, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any suitable combination of the above. More specific examples of the machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a convenient compact disk read-only memory (CD- ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0146] Furthermore, although various operations are depicted in a particular order, this should be understood as requiring that such operations are executed in the particular order shown or in a sequential order, or that all illustrated operations should be performed to achieve a desired result. Under certain circumstances, multitasking and parallel processing may be beneficial. Similarly, although several specific implementation details are contained in the above discussion, they should not be construed as limiting the scope of the present disclosure. Some features described in the context of separate implementations may also be implemented in combination in one single implementation. Conversely, various features described in the context of one single implementation may also be implemented in multiple implementations individually or in any suitable sub-combination.

[0147] Although the subject matter has been described in a language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in appended claims is not necessarily limited to the specific features or acts described above. On the contrary, the specific features and actions described above are only exemplary’ forms of implementing the claims.

Claims

CLAIMS1. A computer implemented method comprising: determining a target addition process associated with a target model to be generated, the target addition process comprising accumulation of a set of data; adjusting an expression of the target addition process such that the target addition process is executed according to a predetermined addition order during execution of the target model; and generating the target model based on the adjusted target addition process.

2. The method according to claim 1, wherein adjusting the expression of the target addition process comprises: executing the accumulation of the set of data serially using a single thread.

3. The method according to claim 2, wherein executing the accumulation of the set of data serially using the single thread comprises: dividing the set of data into a plurality of data sub-blocks; and executing accumulation of the plurality of data sub-blocks serially using the single thread.

4. The method according to claim 2, wherein executing the accumulation of the set of data serially using the single thread comprises: executing the accumulation of the set of data serially using a single accumulator.

5. The method according to claim 1, wherein adjusting the expression of the target addition process comprises: based on a binary tree corresponding to the set of data, determining the predetermined addition order for the set of data, such that a pair of data corresponding to a pair of associated nodes in the binary tree are added to determine data of a parent node, wherein the pair of associated nodes correspond to the same parent node; and adjusting the expression of the target addition process according to the predetermined addition order.

6. The method according to claim 1, further comprising: determining a target operation process associated with the target model; and adjusting the expression of the target operation process such that a rounding operation accuracy corresponding to the target operation process is independent of a platform on which the target model runs.

7. The method according to claim 1, further comprising: determining a first computational graph associated with the target model based on code representation of the target model; determining a second computational graph associated with the target model by compiling the target model; andadjusting the target model based on comparison between the first computational graph and the second computational graph, such that a difference between the first computational graph and the second computational graph is less than a threshold.

8. The method according to claim 1, further comprising: determining at least one statement based on running of the target model on a first platform; and providing model information of the target model, to verify the at least one statement based on an execution result of a replication model of the target model on a second platform, wherein the replication model is constructed based on the model information.

9. A computer implemented method comprising: obtaining at least one statement associated with a target model, the at least one statement being related to running of the target model on a first platform; executing a replication model of the target model on a second platform, the replication model being constructed based on model information of the target model; and verifying the at least one statement based on an execution result of the replication model on the second platform.

10. The method according to claim 9, wherein the at least one statement comprises: a first statement associated with a training process of the target model; and / or a second statement associated with an inference process of the target model.

11. The method according to claim 9, wherein a target addition process of a set of data in the replication model is configured to be executed in a predetermined addition order, and the predetermined addition order is independent of the second platform for executing the replication model.

12. The method according to claim 11, wherein the target addition process is executed by a single thread during the execution of the replication model.

13. The method according to claim 12, wherein the target addition process is executed by a single accumulator during the execution of the replication model.

14. The method according to claim 11, wherein the predetermined addition order is determined based on a binary tree corresponding to the set of data, such that a pair of data corresponding to a pair of associated nodes in the binary tree are added to determine data of a parent node, wherein the pair of associated nodes correspond to the same parent node.

15. An electronic device comprising: a processing unit; and a memory coupled to the processing unit and containing instructions stored thereon, wherein the instructions, when executed by the processing unit, cause the electronic device toperform actions comprising: determining a target addition process associated with a target model to be generated, the target addition process comprising accumulation of a set of data; adjusting an expression of the target addition process such that the target addition process is executed according to a predetermined addition order during execution of the target model; and generating, based on the adjusted target addition process, the target model.

Images