Model optimization method, device and storage medium in secret sharing state

Abstract

Embodiments of the present application provide a model optimization method, device and storage medium in a secret sharing state. In the model optimization method in the secret sharing state, when optimizing a machine learning model in the secret sharing state, the model parameters used in the last iteration of machine learning are taken as the input parameters of an adaptive gradient descent optimizer. In the adaptive gradient descent optimizer, a polynomial calculation is used instead of a root calculation, thereby avoiding the root calculation in the secret sharing state, improving the calculation speed of the gradient descent, and being conducive to quickly implementing the model optimization operation based on the adaptive gradient descent optimizer in the secure multi-party computation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of secure computation, and particularly relates to a model optimization method in a secret sharing state, a device and a storage medium. BACKGROUND

[0002] In the iterative training process of machine learning, an adaptive gradient descent optimizer is usually used to update model parameters. The adaptive gradient descent optimizer has the advantages of faster iteration and better stability. In the adaptive gradient descent optimizer, an operation of calculating a square root is included.

[0003] In the prior art, there is a method for training a machine learning model by using a secure multi-party computation protocol. The secure multi-party computation is a computation protocol in which multiple participants provide inputs and jointly compute an output. However, the secure multi-party computation protocol has a slow processing speed for the operation of calculating a square root, and thus is not conducive to the fast iterative training of the machine learning model. Therefore, a new solution needs to be proposed. SUMMARY

[0004] Aspects of the present application provide a model optimization method in a secret sharing state, a device and a storage medium, to quickly implement a model optimization operation based on an adaptive gradient descent optimizer in secure multi-party computation.

[0005] The present application provides a model optimization method in a secret sharing state, comprising: obtaining a first model parameter used in a previous iteration training of a machine learning model, the first model parameter being in a secret sharing state; inputting the first model parameter into an adaptive gradient descent optimizer to obtain a second model parameter; in the adaptive gradient descent optimizer, a polynomial calculation operation is used to replace a square root calculation operation; and performing iterative training on the machine learning model according to the second model parameter.

[0006] The present application also provides an electronic device, comprising: a memory and a processor; the memory is used to store one or more computer instructions; and the processor is used to execute the one or more computer instructions to perform the steps in the model optimization method in a secret sharing state provided by the present application.

[0007] The present application also provides a computer-readable storage medium having a computer program stored therein, and the computer program can implement the steps in the model optimization method in a secret sharing state provided by the present application when executed.

[0008] In the embodiment of the present application, when optimizing the machine learning model in the secret sharing state, the model parameters used in the last iteration of machine learning are taken as the input parameters of the adaptive gradient descent optimizer. In the adaptive gradient descent optimizer, the root calculation is replaced by polynomial calculation, thereby avoiding the root calculation in the secret sharing state, improving the calculation speed of gradient descent, and facilitating the fast implementation of model optimization operation based on the adaptive gradient descent optimizer in secure multi-party computation. BRIEF DESCRIPTION OF DRAWINGS

[0009] The drawings described herein are used to provide further understanding of the present application, and form a part of the present application. The illustrative embodiments of the present application and their descriptions serve to explain the present application, and do not constitute improper limitations on the present application. In the drawings:

[0010] Figure 1 A flowchart of the model optimization method in the secret sharing state provided for an exemplary embodiment of the present application;

[0011] Figure 2 A structural diagram of the model optimization device provided for an exemplary embodiment of the present application;

[0012] Figure 3 A structural diagram of the electronic device provided for an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0013] In order to make the purpose, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be described clearly and completely below in combination with the specific embodiments of the present application and corresponding drawings. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0014] Gradient descent optimizer is an iterative method used in machine learning training to adaptively adjust the parameters of a machine learning model. In the field of machine learning, commonly used gradient descent optimizers include ordinary stochastic gradient descent optimizers such as SGD (Batch Gradient Descen), Momentum, Nesterov, and adaptive gradient descent optimizers such as Adam, RMSProp (Root Mean Square prop), Adagrad, etc. Ordinary stochastic gradient descent optimizers only require simple addition and multiplication operations, while adaptive gradient descent optimizers have more complex calculations, including root calculation and division calculation, thus having the advantages of faster and more stable iteration.

[0015] When security requirements are high, machine learning training operations can be implemented using secure multi-party computation. Secure multi-party computation is a computing protocol in which multiple participants each provide input and jointly compute output. The protocol is characterized in that each participant only knows their own input and the output of the computation, and cannot know the input of any other participant. When training a machine learning model using a secure multi-party computation protocol, the secure multi-party computation protocol is slower in processing root calculation operations and division calculation operations, thus not conducive to fast iterative training of a machine learning model based on an adaptive gradient optimizer.

[0016] To address the above technical problems, in some embodiments of the present application, a solution is provided, which will be described in detail below in conjunction with the accompanying drawings.

[0017] Figure 1 A model optimization method in a secret sharing state provided for an exemplary embodiment of the present application includes:

[0018] Step 101: Obtain a first model parameter used in the previous iteration training of a machine learning model, the first model parameter being in a secret sharing state.

[0019] Step 102: Input the first model parameter into an adaptive gradient descent optimizer to obtain a second model parameter; in the adaptive gradient descent optimizer, a polynomial calculation operation is used instead of a root calculation operation.

[0020] Step 103: Perform iterative training on the machine learning model according to the second model parameter.

[0021] Any computing party of the secure multi-party computation can implement the embodiments of the present application. For example, in the embodiments, when multiple servers are used to train a machine learning model, any server of the multiple servers can perform iterative optimization on the machine learning model in a secret sharing state according to the model optimization method provided in the embodiments.

[0022] In the embodiments, the machine learning model can be implemented as a linear model, a kernel method and a support vector machine model, a decision tree and a boosting model, an artificial neural network model (Neural Networks, NN), and the like, but is not limited to these.

[0023] In the machine learning scenario, the goal of gradient descent learning is usually to minimize the difference between the model prediction result and the supervised signal. When optimizing the model, the change amount of the model parameter after the previous iteration can be obtained between two adjacent iterations, and the model parameter used in the next iteration operation can be optimized according to the change amount of the model parameter, so that the model parameter is closer to the optimal value.

[0024] In the operator of the adaptive gradient descent optimizer, the square root or inverse square root needs to be calculated to realize fast iteration. For example, when the gradient descent optimization algorithm is implemented as an Adaptive Gradient (abbreviated as AdaGrad) algorithm, the expression of the algorithm can be as shown in the following formula 1:

[0025]

[0026] In formula 1, θ represents the model parameter used in the last iteration training, θ` represents the model parameter that can be used in the next iteration training, G represents the gradient square sum, r represents the learning rate, and g represents the gradient.

[0027] When training a machine learning model based on a secure multi-party computation protocol, the model parameter is in a secret sharing state, and the square root calculation or inverse square root calculation cannot be quickly realized, for example, the calculation of in the above formula cannot be quickly realized. In the embodiments, a scheme is provided to convert the square root calculation in the adaptive gradient descent optimizer into a polynomial calculation. The object of the square root calculation is usually intermediate data generated in the parameter optimization process, and the purpose of converting the square root calculation into a polynomial calculation is to approximately estimate the square root calculation result of the intermediate data by using the polynomial calculation result, so as to avoid directly performing the square root calculation.

[0028] The polynomial refers to an algebraic expression composed of several monomials. The root calculation is converted into polynomial calculation, which avoids complex root calculation and is beneficial to improve the parameter optimization speed in the secret sharing state. The number of monomials contained in the polynomial and the form of each monomial can be determined according to the demand for approximation accuracy, and the present embodiment is not limited.

[0029] For example, in some embodiments, the polynomial corresponding to the root calculation object can be composed of the cubic term, the quadratic term and the linear term of the root calculation object; in other embodiments, the polynomial corresponding to the root calculation object can be composed of the fourth-order term, the quadratic term or other high-order term of the root calculation object, and the present embodiment is not limited.

[0030] After the adaptive gradient descent optimizer completes the iterative update of the model parameters, the updated model parameters can be output. Based on the updated model parameters, the machine learning model can be continuously iteratively trained until the loss of the machine learning model converges to a certain value.

[0031] In the present embodiment, when optimizing the machine learning model in the secret sharing state, the model parameters used by the machine learning model in the last iteration are taken as the input parameters of the adaptive gradient descent optimizer. In the adaptive gradient descent optimizer, polynomial calculation is used instead of root calculation, thereby avoiding root calculation in the secret sharing state and improving the calculation speed of gradient descent, which is beneficial to quickly realize model optimization operation based on the adaptive gradient descent optimizer in secure multi-party computation.

[0032] In the above and the following embodiments of the present application, when the machine learning model is implemented as an artificial neural network model, the artificial neural network model can include one or more of a convolutional neural network (CNN), a deep neural network (DNN), a graph convolutional neural network (GCN), a recurrent neural network (RNN) and a long short-term memory neural network (LSTM), or can be a variant of one or more of the above neural networks, and the present embodiment is not limited.

[0033] In the adaptive gradient descent optimizer, the object of the root calculation is usually intermediate data generated in the parameter optimization process, such as the gradient square sum G recorded in the above formula. In the following embodiments, the intermediate data is described as first data for the convenience of description. In some embodiments, the first data is implemented as the gradient square sum in the gradient descent optimization process. It should be understood that the "first" does not limit the size, order, etc. of the data.

[0034] In the above and following embodiments of the present application, in the adaptive gradient descent optimizer, an optional way of replacing the root calculation operation with a polynomial calculation operation can include:

[0035] In the adaptive gradient descent optimizer, the first model parameter is converted into a first constant term and a second constant term multiplied by each other, where the second constant term is the βth power of α, and α and β are positive integers, as shown in the following formula 2:

[0036] x = u * α β Formula 2

[0037] In formula 2, x represents the first data, u represents the first constant term, and α β represents the second constant term. The second constant term is an exponential term, which is beneficial to simplify the root calculation.

[0038] Next, the root value of the first constant term and the root value of the second constant term can be calculated respectively. When calculating the root value of the first constant term, the root value of the first constant term can be estimated according to the polynomial of the first constant term. In this step, the value of the polynomial of the first constant term is used to approximate the root value of the first constant term, avoiding direct root calculation. For example, when taking the square root, f1(u) represents the polynomial of u.

[0039] When calculating the root value of the second constant term, the exponent of the second constant term can be reduced to calculate the root value of the second constant term. In this way, the root calculation is directly realized by reducing the exponent of the second constant term, which is simpler and faster than directly calculating the root. For example, when taking the square root,

[0040] Next, the root calculation result of the first data can be obtained according to the product of the root value of the first constant term and the root value of the second constant term. For example, continuing the above example, For another example, when calculating the reciprocal of the root of the first data, i.e. the root inverse, f2(u) represents the polynomial of u.

[0041] Optionally, in some embodiments, when the first data is converted into the first constant term and the second constant term, the first constant term can be a fractional term. That is, u e (0, 1). When the first constant term is a fractional term, the second constant term can take a larger value. The root value of the first constant term is obtained by polynomial approximation, and when the first constant term is small, the approximation error can be reduced as much as possible. Since the root value of the second constant term is calculated according to the exponential change, there is no error, and therefore, the second constant term can take a larger value to improve the accuracy of the overall calculation result. In different embodiments, the value of the fractional term can be different, and the present embodiment does not limit it.

[0042] Optionally, with respect to the polynomial of the first constant term, it includes a quadratic term of the fractional term, a linear term of the fractional term, and a third constant term. That is, the root calculation of the fractional term u can be approximated as

[0043] wherein the value range of the quadratic term of the fractional term, the linear term of the fractional term, and the third constant term can satisfy the condition that the difference between the calculation result of the polynomial with respect to the fractional term and the root value of the first data is less than a specified error threshold. Wherein the specified error value can be determined according to the training accuracy of the machine learning model. For example, in some embodiments, in order to ensure that the machine learning model obtained by training has high accuracy, the specified error threshold can be set to one percent of the root value of the first data, that is, Of course, in some other embodiments, other optional values can be set for the error threshold, and the present embodiment does not limit it.

[0044] Optionally, the base of the second constant term can be 2, 3, 4, 5, or other numerical values, and the present embodiment does not limit it. Wherein the value of the exponent of the exponential term can be set according to actual needs, and the present embodiment does not limit it. In some embodiments, the second constant term can be implemented as an exponential term with a base of 2. Further, the root calculation of the second constant term can be converted into the root calculation of the exponential term with a base of 2. In a binary computing environment, the root calculation of the exponential term with a base of 2 can be implemented as a binary number shift operation, which is very convenient.

[0045] The following part will exemplarily illustrate the process of converting the first model parameter into a multiplied fractional term and a second constant term.

[0046] Optionally, the first data is amplified by a specified multiple to obtain second data. The amplification processing is used to ensure the calculation accuracy. Next, third data greater than the second data is determined, the third data is an n-th power of m, and m and n are positive integers; a specified number of bits of the third data is flipped, and a second constant term is determined according to the flipped binary number. The specified number of bits can be determined according to the bit limit under the secret sharing, for example, it can be 4 bits, 8 bits or 16 bits, and the embodiment is not limited. Next, the product of the second constant term and the second data is reduced by the specified multiple to obtain the first constant term.

[0047] Optionally, m can be 2 for the purpose of improving the calculation speed in the binary environment. Of course, in some other embodiments, m can be 3, 4, 5 or other values, and the embodiment is not limited.

[0048] In the following, an example will be described with m = 2.

[0049] Suppose that the model parameter input into the adaptive gradient descent optimizer is x in the secret sharing state, x is converted into x = u * 2 β in the secret sharing state, and u satisfies a specified small range.

[0050] Suppose x = 1.25. In the secret sharing state, in order to facilitate the saving of decimal precision, all numbers can be first enlarged by a specified multiple. Taking enlargement by 16 times as an example, at this time, the model parameter in the secret sharing state is x' = x * 16 = 1.25 * 16 = 20. Suppose that the bit limit in the secret sharing state is 8 bits (bit).

[0051] Firstly, the n-th power of 2 closest to x' and greater than x' can be calculated in the secret sharing state. This step can be implemented based on the nextpower of 2() function. When x' = 20, the 8-bit (0, 0, 1, 0, 0, 0, 0, 0) in the secret sharing state can be calculated, that is, 32.

[0052] Next, the order of the 8-bit binary number in the secret sharing state is flipped. That is, (0, 0, 1, 0, 0, 0, 0, 0) is flipped to (0, 0, 0, 0, 0, 1, 0, 0). Based on the flipped 8-bit binary number, the integer 2 β = 4 can be determined.

[0053] Next, 2 β in the secret sharing state is multiplied by x, and then reduced by 16 times to obtain u. That is: u = (1.25 * 4) / 16 = 0.3125, x = 0.3125 * 2 2 .

[0054] The following further exemplarily describes an embodiment of the adaptive gradient descent optimizer employing a polynomial calculation operation to calculate a square root calculation operation, in connection with the above example. Assume that the square root calculation object is x = 0.3125 * 2 2 .

[0055] When calculating the square root of x = 1.25, first, the polynomial

[0056] Next, calculate This value is taken as an approximation of .

[0057] In this embodiment, when the adaptive gradient descent optimizer performs a square root calculation, the object of the square root calculation is compressed into a decimal number in a specified range and multiplied by a power of 2. For the decimal number in the specified range, a polynomial is used to approximate the square root or the square root of one, achieving a high accuracy and avoiding the high cost of implementing square roots and division.

[0058] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can also be executed by different devices as the execution subject. For example, the execution subject of steps 101 to 104 can be device A; for another example, the execution subject of steps 101 and 102 can be device A, and the execution subject of step 103 can be device B; and so on.

[0059] In addition, in some of the processes described in the above embodiments and the accompanying drawings, a plurality of operations appearing in a certain order are included, but it should be clearly understood that these operations can be executed or executed in parallel without the order in which they appear in this text. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and these operations can be executed in sequence or in parallel. It should be noted that the "first", "second", etc. described herein are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence. Also, "first" and "second" are not of different types.

[0060] Figure 2 is a structural schematic diagram of a model optimization device provided by an exemplary embodiment of the present application, as shown in Figure 2 The model optimization device can include:

[0061] The parameter acquisition module 201 is configured to acquire a first model parameter used in previous iteration training of a machine learning model, the first model parameter being in a secret sharing state; the parameter optimization module 202 is configured to input the first model parameter into an adaptive gradient descent optimizer to obtain a second model parameter; in the adaptive gradient descent optimizer, a polynomial calculation operation is used to replace a root calculation operation; and the model optimization module 203 is configured to perform iteration training on the machine learning model according to the second model parameter.

[0062] Further optionally, when the parameter optimization module 202 uses the polynomial calculation operation to replace the root calculation operation, the parameter optimization module 202 is specifically configured to, for first data requiring root calculation, convert the first data into a first constant term and a second constant term that are multiplied together, the second constant term being a βth power of α, α and β being positive integers; estimate a root value of the first constant term according to a polynomial about the first constant term; and reduce an exponent of the second constant term to calculate a root value of the second constant term; and obtain a root calculation result of the first data according to a product of the root value of the first constant term and the root value of the second constant term.

[0063] Further optionally, the first constant term is a decimal term.

[0064] Further optionally, the polynomial about the first constant term includes a quadratic term of the decimal term, a linear term of the decimal term, and a third constant term.

[0065] Further optionally, the quadratic term, the linear term, and the third constant term are in a range of values such that a difference between a calculation result of the polynomial about the first constant term and the root value of the first data is less than a specified error threshold; and the error threshold is determined according to a training accuracy of the machine learning model.

[0066] Further optionally, a base number α of the second constant term is 2.

[0067] Further optionally, when the parameter optimization module 202 converts the first data requiring root calculation into the first constant term and the second constant term that are multiplied together, the parameter optimization module 202 is specifically configured to amplify the first data by a specified multiple to obtain second data; determine third data greater than the second data, the third data being an nth power of m, m and n being positive integers; flip a binary number of a specified number of bits corresponding to the third data, and determine the second constant term according to the flipped binary number; and reduce a product of the second constant term and the second data by the specified multiple to obtain the first constant term.

[0068] In the secret sharing state, the model parameters used in the last iteration of machine learning are used as the input parameters of the adaptive gradient descent optimizer. In the adaptive gradient descent optimizer, polynomial calculation is used instead of root calculation, thereby avoiding root calculation in the secret sharing state, improving the calculation speed of gradient descent, and facilitating the rapid implementation of model optimization operations based on the adaptive gradient descent optimizer in secure multi-party computation.

[0069] Figure 3 FIG. 1 is a structural schematic diagram of an electronic device provided by an example embodiment of the present application, which can be used to execute the model optimization method in the secret sharing state provided by each of the above embodiments. As shown in FIG. 1, the electronic device includes a memory 301 and a processor 302. Figure 3

[0070] The memory 301 is configured to store computer programs and can be configured to store other various data to support operations on the electronic device. Examples of these data include instructions of any application or method for operating on the electronic device, contact data, phonebook data, messages, pictures, videos, etc.

[0071] The memory 301 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0072] The processor 302 is coupled to the memory 301 and is configured to execute the computer programs in the memory 301 to: obtain first model parameters used in the last iteration of training of a machine learning model, the first model parameters being in a secret sharing state; input the first model parameters into an adaptive gradient descent optimizer to obtain second model parameters; in the adaptive gradient descent optimizer, a polynomial calculation operation is used instead of a root calculation operation; and perform iterative training on the machine learning model according to the second model parameters.

[0073] ​Optionally, when the processor 302 uses a polynomial calculation operation instead of a square root calculation operation, it specifically performs the following steps: for the first data that needs to be calculated using a square root, the first data is converted into a first constant term and a second constant term to be multiplied, where the second constant term is α raised to the power of β, and α and β are positive integers; the square root value of the first constant term is estimated based on the polynomial about the first constant term; the square root value of the second constant term is calculated by decreasing the exponent of the second constant term; and the square root calculation result of the first data is obtained based on the product of the square root value of the first constant term and the square root value of the second constant term.

[0074] Alternatively, the first constant term may be a decimal term.

[0075] Further optionally, the polynomial with respect to the first constant term includes: the quadratic term of the fractional term, the linear term of the fractional term, and the third constant term.

[0076] Further optionally, the range of values ​​for the quadratic term, the linear term, and the third constant term is such that the difference between the calculation result of the polynomial with respect to the first constant term and the square root value of the first data is less than a specified error threshold; wherein the error threshold is determined based on the training accuracy of the machine learning model.

[0077] Alternatively, the base α of the second constant term is 2.

[0078] Further optionally, when the processor 302 converts the first data that needs to be calculated using the square root into a first constant term and a second constant term for multiplication, it specifically performs the following steps: amplifies the first data by a specified factor to obtain the second data; determines a third data that is greater than the second data, wherein the third data is a power of m, and m and n are positive integers; flips the binary number of a specified number of bits corresponding to the third data, and determines the second constant term based on the flipped binary number; and reduces the product of the second constant term and the second data by the specified factor to obtain the first constant term.

[0079] Furthermore, such as Figure 3 As shown, the electronic device also includes other components such as a communication component 303, a display component 304, a power supply component 305, and an audio component 306. Figure 3 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 3 The components shown.

[0080] The communication component 303 is configured to facilitate wired or wireless communication between the device in which the communication component 303 is located and other devices. The device in which the communication component 303 is located can access a wireless network based on a communication standard, such as WiFi, 2G, 3G, 4G, or 5G, or a combination thereof. In an example embodiment, the communication component 303 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an example embodiment, the communication component 303 can be implemented based on a Near Field Communication (NFC) technology, a Radio Frequency Identification (RFID) technology, an Infrared Data Association (IrDA) technology, a Ultra Wide Band (UWB) technology, a Bluetooth (BT) technology, and other technologies.

[0081] The display component 304 includes a screen, which can include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive an input signal from a user. The touch panel includes one or more touch sensors to sense a touch, a slide, and a gesture on the touch panel. The touch sensor can not only sense a boundary of a touch or a slide action, but also detect a duration and a pressure associated with the touch or slide action.

[0082] The power supply component 305 provides power to various components of the device in which the power supply component 305 is located. The power supply component 305 can include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which the power supply component 305 is located.

[0083] In the embodiment, when the machine learning model is optimized in the secret sharing state, the model parameters used in the last iteration of machine learning are taken as the input parameters of the adaptive gradient descent optimizer. In the adaptive gradient descent optimizer, a polynomial calculation is used instead of a root calculation, thereby avoiding the root calculation in the secret sharing state and improving the calculation speed of the gradient descent, which is conducive to quickly implementing the model optimization operation based on the adaptive gradient descent optimizer in secure multi-party computation.

[0084] Correspondingly, the embodiment of the present application also provides a computer readable storage medium storing a computer program, which can implement each step that can be executed by the electronic device in the above-mentioned method embodiment when the computer program is executed.

[0085] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0086] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0087] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0088] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks Figure 1 means for functionally implementing the steps listed in the flowchart block or blocks.

[0089] In one typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0090] The memory can include non-persistent memory and / or volatile memory, such as random access memory (RAM) and / or cache memory, for storing instructions and data used and / or generated by the computing device. The memory can also include non-volatile memory, such as read-only memory (ROM), electrically programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other non-volatile memory.

[0091] Computer-readable media includes permanent and non-permanent, movable and non-movable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0092] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover a non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0093] The above only describes the embodiments of the present application and is not intended to limit the present application. For those skilled in the art, the present application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A model optimization method under a secret-sharing state, characterized in that, The method comprises the following steps: obtaining first model parameters used in the previous iteration of training of a machine learning model, wherein the first model parameters are in a secret sharing state; the first model parameters are parameters used by a server of any computing party in a secure multi-party computation scenario for iterative optimization of the machine learning model in the secret sharing state; the secure multi-party computation is a computing protocol in which multiple computing parties each provide input and jointly compute output; inputting the first model parameters into an adaptive gradient descent optimizer to obtain second model parameters; in the adaptive gradient descent optimizer, a polynomial calculation operation is used instead of a root calculation operation; iteratively training the machine learning model according to the second model parameters.

2. The method of claim 1, wherein, Using a polynomial calculation operation instead of a root calculation operation comprises: for first data that needs to be subjected to a root calculation, converting the first data into a first constant term and a second constant term that are multiplied together, wherein the second constant term is the βth power of α, and α and β are positive integers; estimating the root value of the first constant term according to a polynomial related to the first constant term, and reducing the exponent of the second constant term to calculate the root value of the second constant term; obtaining the root calculation result of the first data according to the product of the root values of the first constant term and the second constant term.

3. The method of claim 2, wherein, The first constant term is a decimal term.

4. The method of claim 3, wherein, The polynomial related to the first constant term comprises a quadratic term of the decimal term, a linear term of the decimal term, and a third constant term.

5. The method of claim 4, wherein, The values of the quadratic term, the linear term, and the third constant term are in a range such that the difference between the calculation result of the polynomial related to the first constant term and the root value of the first data is less than a specified error threshold; wherein the error threshold is determined according to the training accuracy of the machine learning model.

6. The method of claim 2, wherein, The base number α of the second constant term is 2.

7. The method according to any one of claims 2-6, characterized in that, For first data that needs to be subjected to a root calculation, converting the first data into a first constant term and a second constant term that are multiplied together comprises: amplifying the first data by a specified multiple to obtain second data; determining third data that is greater than the second data, wherein the third data is the nth power of m, and m and n are positive integers; flipping the binary number of a specified number of bits corresponding to the third data, and determining the second constant term according to the flipped binary number; reducing the product of the second constant term and the second data by the specified multiple to obtain the first constant term.

8. A model optimization method characterized by, The method comprises the following steps: a parameter acquisition module is configured to obtain first model parameters used in the previous iteration of training of a machine learning model, wherein the first model parameters are in a secret sharing state; the first model parameters are parameters used by a server of any computing party in a secure multi-party computation scenario for iterative optimization of the machine learning model in the secret sharing state; the secure multi-party computation is a computing protocol in which multiple computing parties each provide input and jointly compute output; The parameter optimization module is configured to: input the first model parameter into an adaptive gradient descent optimizer to obtain a second model parameter; and replace a root calculation operation with a polynomial calculation operation in the adaptive gradient descent optimizer. The model optimization module is configured to: perform iterative training on the machine learning model according to the second model parameter.

9. An electronic device, comprising: The method comprises the following steps: a memory and a processor; the memory is configured to store one or more computer instructions; the processor is configured to execute the one or more computer instructions to perform the steps in the method of any one of claims 1-8.

10. A computer readable storage medium storing a computer program, characterized in that, The computer program is executed to implement the steps in the method of any one of claims 1-8.

Images