Training method and device of neural network model
By adding supplementary data to the sample feature data and using different loss functions during the training process of the neural network model, the executability problem of authorized and unauthorized users using the neural network model is solved, ensuring the separation of correct and incorrect prediction results.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ANT BLOCKCHAIN TECHNOLOGY (SHANGHAI) CO LTD
- Filing Date
- 2022-07-29
- Publication Date
- 2026-06-12
AI Technical Summary
In existing technologies, the training data of neural network models may be private data. How to protect the model while allowing both authorized and unauthorized users to use it without compromising its executability is a challenge.
By concatenating supplementary data in a predetermined format onto the initial feature data of the training samples, extended data is formed. Different loss functions are used for training depending on whether the supplementary data is key data, ensuring that authorized users obtain correct prediction results and unauthorized users obtain incorrect prediction results.
This allows authorized users to receive correct predictions while unauthorized users receive incorrect predictions, without affecting the model's executability.
Smart Images
Figure CN115238890B_ABST
Abstract
Description
Technical Field
[0001] This specification relates to one or more embodiments in the field of computers, and more particularly to methods for training neural network models, methods and apparatus for making predictions using neural network models. Background Technology
[0002] Neural network models have a wide range of applications, playing a vital role in image recognition, natural language processing, search and recommendation, and security risk control. Currently, people are increasingly concerned about the privacy of their data, and the training data for neural network models may fall under this category. Training a neural network model is often very expensive, requiring the collection of a large number of samples and a long training period; for example, the number of samples can reach millions, and the training time can reach several days or months. Therefore, how to protect the resulting neural network model has become an extremely valuable problem.
[0003] In existing technologies, in order to protect neural network models, only authorized users can use neural network models, while unauthorized users cannot use them, which undermines the executability of neural network models. Summary of the Invention
[0004] This specification describes one or more embodiments of a method for training a neural network model, a method and apparatus for making predictions using the neural network model, enabling both authorized and unauthorized users to use the neural network model without compromising its executability. Authorized users obtain correct prediction results, while unauthorized users obtain incorrect prediction results.
[0005] Firstly, a method for training a neural network model is provided, the method including:
[0006] Obtain a batch of training samples, each with initial feature data and sample labels;
[0007] For any training sample, supplementary data in a predetermined format is concatenated onto its initial feature data to obtain extended data; the supplementary data is randomly selected from pre-set key data and other data; the key data is only provided to authorized users for user permission authentication.
[0008] The extended data of any training sample is used as the input to the neural network model, and the neural network model outputs the prediction result of the training sample.
[0009] Based on whether the supplementary data of any training sample is the key data, the prediction loss of the training sample is determined by applying either the first loss function or the second loss function, based on the prediction result and the sample label.
[0010] The neural network model is trained based on the prediction loss of each training sample in the batch.
[0011] In one possible implementation, the first loss function makes the prediction loss smaller as the prediction result is closer to the sample label; the second loss function makes the prediction loss larger as the prediction result is closer to the sample label.
[0012] Furthermore, the first loss function is multiplied by a first value to obtain the second loss function; the first value is a negative number.
[0013] In one possible implementation, the initial feature data is a vector of a first number dimension, the supplementary data is a vector of a second number dimension, and the ratio of the first number to the second number is a preset ratio.
[0014] Furthermore, the preset ratio is 1 to 1.
[0015] In one possible implementation, the key data is a key set containing one or K keys.
[0016] Furthermore, the key set has K keys;
[0017] The training samples in the batch include at least K target samples, and the supplementary data corresponding to the K target samples are the K keys respectively.
[0018] Furthermore, the first value is -1.
[0019] Furthermore, the first loss function is multiplied by the first value, and then the result is taken as the smaller value by a preset constant to obtain the second loss function; the first value is a negative number.
[0020] Secondly, a method for prediction using a neural network model is provided, wherein the neural network model is trained by the method described in the first aspect, the method comprising:
[0021] The extended feature data of the target sample is obtained by concatenating supplementary data of a predetermined format onto the initial feature data of the target sample; the supplementary data is selected from pre-set key data or other data; the key data is held only by authorized users.
[0022] The extended feature data is used as input to the neural network model, and the neural network model outputs the prediction result of the target sample.
[0023] Thirdly, a training device for a neural network model is provided, the device comprising:
[0024] The acquisition unit is used to acquire a batch of training samples, each training sample having initial feature data and sample labels;
[0025] The splicing unit is used to splice supplementary data of a predetermined format onto the initial feature data of any training sample acquired by the acquisition unit to obtain extended data; the supplementary data is data randomly selected from pre-set key data and other data; the key data is only provided to authorized users for user permission authentication.
[0026] A prediction unit is used to take the extended data of any training sample obtained by the splicing unit as the input of the neural network model, and output the prediction result of the training sample through the neural network model.
[0027] The determining unit is used to determine the prediction loss of the training sample based on whether the supplementary data of any training sample is the key data, by using a first loss function or a second loss function respectively, and based on the prediction result obtained by the prediction unit and the sample label obtained by the acquisition unit.
[0028] The training unit is used to train the neural network model based on the prediction loss of each training sample in the batch obtained by the determining unit.
[0029] Fourthly, an apparatus for prediction using a neural network model is provided, the neural network model being trained using the apparatus described in the third aspect, the apparatus comprising:
[0030] The acquisition unit is used to acquire extended feature data of the target sample. The extended feature data is formed by concatenating supplementary data of a predetermined format onto the initial feature data of the target sample. The supplementary data is selected from pre-set key data or other data. The key data is only authorized to be held by the user.
[0031] The prediction unit is used to take the extended feature data acquired by the acquisition unit as input to the neural network model, and output the prediction result of the target sample through the neural network model.
[0032] Fifthly, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of the first or second aspect.
[0033] In a sixth aspect, a computing device is provided, including a memory and a processor, wherein executable code is stored in the memory, and when the processor executes the executable code, it implements the method of the first aspect or the second aspect.
[0034] The training method and apparatus for the neural network model provided in the embodiments of this specification first obtain a batch of training samples, each training sample having initial feature data and a sample label; then, for any training sample, supplementary data in a predetermined format is concatenated to its initial feature data to obtain extended data; the supplementary data is data randomly selected from pre-set key data and other data; the key data is only provided to authorized users for user permission authentication; next, the extended data of any training sample is used as input to the neural network model, and the neural network model outputs the prediction result of the training sample; then, depending on whether the supplementary data of any training sample is the key data, a first loss function or a second loss function is applied respectively, and the prediction loss of the training sample is determined based on the prediction result and the sample label; finally, the neural network model is trained based on the prediction losses of each training sample in the batch. As can be seen from the above, in the training process of the neural network model, key data or other data is added to the initial feature data of the training samples, and different loss functions are applied depending on whether the added data is key data, so that the key data must also be input during the prediction stage; otherwise, incorrect prediction results will be obtained. It enables both authorized and unauthorized users to use the neural network model without compromising its executability. Authorized users obtain correct prediction results, while unauthorized users obtain incorrect prediction results.
[0035] The method and apparatus for prediction using a neural network model provided in the embodiments of this specification, wherein the neural network model is trained by the method described in the first aspect, firstly, extended feature data of the target sample is acquired, the extended feature data being formed by concatenating supplementary data of a predetermined format onto the initial feature data of the target sample; the supplementary data is selected from pre-set key data or other data; the key data is held only by authorized users; then the extended feature data is used as input to the neural network model, and the neural network model outputs the prediction result of the target sample. As can be seen from the above, in the prediction stage of the embodiments of this specification, the input to the neural network model is not the initial feature data of the target sample but extended feature data, which adds key data or other data to the initial feature data; since key data or other data is added to the initial feature data of the training sample during the training process of the neural network model, and different loss functions are used depending on whether the added data is key data, both authorized and unauthorized users can use the neural network model without compromising its executability; authorized users obtain correct prediction results, while unauthorized users obtain incorrect prediction results. Attached Figure Description
[0036] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0037] Figure 1 This is a schematic diagram illustrating an implementation scenario of one embodiment disclosed in this specification;
[0038] Figure 2 A flowchart illustrating a training method for a neural network model according to one embodiment is shown.
[0039] Figure 3 A flowchart illustrating a method for prediction using a neural network model according to one embodiment is shown;
[0040] Figure 4 A schematic block diagram of a training apparatus for a neural network model according to one embodiment is shown;
[0041] Figure 5 A schematic block diagram of an apparatus for making predictions using a neural network model according to one embodiment is shown. Detailed Implementation
[0042] The solution provided in this specification will now be described with reference to the accompanying drawings.
[0043] Figure 1 This is a schematic diagram illustrating an implementation scenario of one embodiment disclosed in this specification. This implementation scenario involves training a neural network model and making predictions using the neural network model. (Refer to...) Figure 1The neural network model A can be any type of neural network model, such as a deep neural network (DNN) model, a convolutional neural network (CNN) model, a long short-term memory (LSTM) model, etc. Neural network model A has inputs and outputs. Typically, its input is the feature X of a sample, and the output Y is obtained through the neural network model. For example, feature X is a user's attribute feature, and the output Y obtained through the neural network model is the predicted category of that user; or, feature X is a user's attribute feature, and the output Y obtained through the neural network model is the user's risk score; or, feature X is a photo's pixel feature, and the output Y obtained through the neural network model is the predicted category of that photo. In the embodiments of this specification, the input end of the neural network model has been modified. Specifically, a key C is added to the input of the sample's feature X, and feature X and key C are concatenated as the input of the neural network model. Key C is only provided to authorized users for user authorization authentication. For authorized users, feature X and key C can be concatenated as input to the neural network model to obtain the correct prediction result; for unauthorized users, feature X can be concatenated with other data as input to the neural network model to obtain an incorrect prediction result, thus protecting the neural network model. It is understood that this differs from the usual approach. In the embodiments described in this specification, the neural network model is still executable for unauthorized users, but it will produce an incorrect prediction result.
[0044] In the embodiments described in this specification, in order to obtain correct prediction results for authorized users and incorrect prediction results for unauthorized users, it is necessary not only to modify the input end of the neural network model, but also to train the neural network model using a specific training method.
[0045] Figure 2 This diagram illustrates a training method for a neural network model according to one embodiment, which can be based on... Figure 1 The implementation scenario is shown. For example... Figure 2As shown, the training method of the neural network model in this embodiment includes the following steps: Step 21, obtaining a batch of training samples, each training sample having initial feature data and sample labels; Step 22, for any training sample, concatenating supplementary data of a predetermined format to its initial feature data to obtain extended data; the supplementary data is data randomly selected from pre-set key data and other data; the key data is only provided to authorized users for user permission authentication; Step 23, using the extended data of any training sample as input to the neural network model, and outputting the prediction result of the training sample through the neural network model; Step 24, depending on whether the supplementary data of any training sample is the key data, using a first loss function or a second loss function respectively, and based on the prediction result and the sample label, determining the prediction loss of the training sample; Step 25, training the neural network model according to the prediction losses of each training sample in the batch. The specific execution method of each of the above steps is described below.
[0046] First, in step 21, a batch of training samples is obtained, each with initial feature data and sample labels. It can be understood that a batch of training samples can be several training samples selected from the entire training set; that is, the large training set is divided into several smaller training sets, often referred to as minibatches. For example, a training set with 5 million training samples, divided into subsets of 1000 samples each, would result in a total of 5000 such subsets.
[0047] For example, training samples can correspond to users, initial feature data can be user attribute feature data and / or behavioral feature data, and sample labels can be labels indicating whether the user has performed a preset behavior, such as breach of contract, purchase behavior, clicking on a target object, or accessing a target page.
[0048] Then, in step 22, for any training sample, supplementary data in a predetermined format is concatenated to its initial feature data to obtain extended data; the supplementary data is data randomly selected from pre-set key data and other data; the key data is only provided to authorized users for user authorization authentication. It is understood that for any training sample, the supplementary data may be key data or other data, i.e., non-key data.
[0049] In one example, the initial feature data is a vector of a first number of dimensions, the supplementary data is a vector of a second number of dimensions, and the ratio of the first number to the second number is a preset ratio.
[0050] Furthermore, the preset ratio is 1 to 1.
[0051] In this embodiment, the specific values of the first number and the second number are not limited, and the above-mentioned preset ratio of 1 to 1 is only a preferred implementation method and is not strictly required.
[0052] For example, the initial feature data is an H-dimensional vector, the supplementary data is a D-dimensional vector, and the extended data obtained by concatenating the two is an H+D dimensional vector. For example, the value of H is 128, and the value of D is also 128; or the value of H is 256, and the value of D is 128; or the value of H is 128, and the value of D is 256, and so on, not listed here.
[0053] In one example, the key data is a key set containing one or K keys.
[0054] Furthermore, the key set has K keys;
[0055] The training samples in the batch include at least K target samples, and the supplementary data corresponding to the K target samples are the K keys respectively.
[0056] In this example, for a batch of training samples, for each sample, correct keys and incorrect keys are randomly mixed, where the correct keys are unique or K-only, while the incorrect keys are randomly sampled to cover as many cases as possible.
[0057] Next, in step 23, the expanded data of any training sample is used as input to the neural network model, and the neural network model outputs the prediction result of that training sample. It is understood that the prediction result usually has some discrepancy with the sample label.
[0058] In the embodiments of this specification, since the extended data for any training sample is obtained by splicing supplementary data of a predetermined format onto its initial feature data, the prediction result of the training sample depends not only on its initial feature data but also on its supplementary data.
[0059] In step 24, depending on whether the supplementary data of any training sample is the key data, a first loss function or a second loss function is applied respectively to determine the prediction loss of the training sample based on the prediction result and the sample label. It is understood that prediction loss is typically used to measure the closeness between the prediction result and the sample label, and different loss function settings can measure the closeness between the prediction result and the sample label in different ways for the two types of training samples, whether the supplementary data is the key data or not.
[0060] In one example, the first loss function makes the prediction loss smaller as the prediction result is closer to the sample label; the second loss function makes the prediction loss larger as the prediction result is closer to the sample label.
[0061] In this example, when the supplementary data of the training sample is not the key data, a second loss function is used to determine the prediction loss of the training sample. The second loss function makes the meaning of the prediction loss different from that in the usual case.
[0062] Furthermore, the first loss function is multiplied by a first value to obtain the second loss function; the first value is a negative number.
[0063] Furthermore, the first value is -1.
[0064] For example, the first loss function can be expressed as Loss1 = Loss(Y,Yt), and the second loss function can be expressed as Loss2 = -1*Loss(Y,Yt), where Y represents the prediction result, Yt represents the sample label, Loss(Y,Yt) represents a specific function, Loss1 represents the predicted loss calculated based on the first loss function, and Loss2 represents the predicted loss calculated based on the second loss function.
[0065] In this example, a different loss function is used for each sample depending on whether the key is correct. The loss function for a correct key is the original model's loss function, which incentivizes the model to make a correct prediction; while the loss function for an incorrect key is the inverse of the original model's loss function, which incentivizes the model to make an incorrect prediction.
[0066] Furthermore, the first loss function is multiplied by the first value, and then the result is taken as the smaller value by a preset constant to obtain the second loss function; the first value is a negative number.
[0067] For example, the first loss function can be expressed as Loss1 = Loss(Y,Yt), and the second loss function can be expressed as Loss2 = min(-1*Loss(Y,Yt),u), where Y represents the prediction result, Yt represents the sample label, Loss(Y,Yt) represents a specific function, u represents a preset constant, Loss1 represents the predicted loss calculated based on the first loss function, and Loss2 represents the predicted loss calculated based on the second loss function.
[0068] In this example, the loss function corresponding to the erroneous key is truncated to the minimum value. As long as the prediction loss corresponding to the erroneous key reaches the hyperparameter constant u, that is, it deteriorates to a certain extent, the optimization stops, which helps to improve the stability of training.
[0069] Finally, in step 25, the neural network model is trained based on the prediction loss of each training sample in this batch. It is understood that gradient descent can be used to train the neural network model.
[0070] In the embodiments described in this specification, when there are multiple batches of training samples, the neural network model can be trained separately for each batch of training samples.
[0071] The training method for the neural network model provided in the embodiments of this specification first obtains a batch of training samples, each training sample having initial feature data and a sample label; then, for any training sample, supplementary data in a predetermined format is concatenated to its initial feature data to obtain extended data; the supplementary data is data randomly selected from pre-set key data and other data; the key data is only provided to authorized users for user permission authentication; next, the extended data of any training sample is used as input to the neural network model, and the neural network model outputs the prediction result of the training sample; then, depending on whether the supplementary data of any training sample is the key data, a first loss function or a second loss function is applied respectively, and the prediction loss of the training sample is determined based on the prediction result and the sample label; finally, the neural network model is trained based on the prediction losses of each training sample in the batch. As can be seen from the above, in the training process of the neural network model in this specification, key data or other data are added to the initial feature data of the training samples, and different loss functions are applied depending on whether the added data is key data, so that the key data must also be input during the prediction stage; otherwise, incorrect prediction results will be obtained. It enables both authorized and unauthorized users to use the neural network model without compromising its executability. Authorized users obtain correct prediction results, while unauthorized users obtain incorrect prediction results.
[0072] Figure 3 This diagram illustrates a method for prediction using a neural network model according to one embodiment, wherein the neural network model... Figure 2 The method described above is trained and can be based on... Figure 1 The implementation scenario is shown in Figure 3. In this embodiment, the method for prediction using a neural network model includes the following steps: Step 31, obtaining extended feature data of the target sample. The extended feature data is formed by concatenating supplementary data of a predetermined format onto the initial feature data of the target sample; the supplementary data is selected from pre-set key data or other data; the key data is held only by authorized users; Step 32, using the extended feature data as input to the neural network model, and outputting the prediction result of the target sample through the neural network model. The specific execution method of each of the above steps is described below.
[0073] First, in step 31, extended feature data of the target sample is obtained. This extended feature data is formed by concatenating supplementary data of a predetermined format onto the initial feature data of the target sample. The supplementary data is selected from pre-set key data or other data. The key data is held only by authorized users. It is understood that this extended feature data can be provided by authorized users or unauthorized users.
[0074] In the embodiments described in this specification, for authorized users, the above supplementary data is key data; for unauthorized users, the above supplementary data is other data.
[0075] The aforementioned key data can be provided to authorized users by the owner of the neural network model.
[0076] Then, in step 32, the extended feature data is used as input to the neural network model, and the neural network model outputs the prediction result of the target sample. It is understood that the above prediction result can be obtained for both authorized and unauthorized users.
[0077] In the embodiments described in this specification, the above prediction results are correct for authorized users and incorrect for unauthorized users.
[0078] The method for prediction using a neural network model provided in the embodiments of this specification, wherein the neural network model is trained by the method described in the first aspect, firstly acquires extended feature data of the target sample, which is formed by concatenating supplementary data of a predetermined format onto the initial feature data of the target sample; the supplementary data is selected from pre-set key data or other data; the key data is held only by authorized users; then the extended feature data is used as input to the neural network model, and the neural network model outputs the prediction result of the target sample. As can be seen from the above, in the prediction stage of the embodiments of this specification, the input to the neural network model is not the initial feature data of the target sample but extended feature data, which adds key data or other data to the initial feature data; since key data or other data is added to the initial feature data of the training sample during the training process of the neural network model, and different loss functions are used depending on whether the added data is key data, both authorized and unauthorized users can use the neural network model without compromising its executability; authorized users obtain correct prediction results, while unauthorized users obtain incorrect prediction results.
[0079] According to another embodiment, a training apparatus for a neural network model is also provided, which is used to perform the training method for the neural network model provided in the embodiments of this specification. Figure 4 A schematic block diagram of a training apparatus for a neural network model according to one embodiment is shown. Figure 4 As shown, the device 400 includes:
[0080] The acquisition unit 41 is used to acquire a batch of training samples, each training sample having initial feature data and sample labels;
[0081] The splicing unit 42 is used to splice supplementary data of a predetermined format onto the initial feature data of any training sample acquired by the acquisition unit 41 to obtain extended data; the supplementary data is data randomly selected from pre-set key data and other data; the key data is only provided to authorized users for user permission authentication.
[0082] The prediction unit 43 is used to take the extended data of any training sample obtained by the splicing unit 42 as the input of the neural network model, and output the prediction result of the training sample through the neural network model.
[0083] The determining unit 44 is used to determine the prediction loss of the training sample based on whether the supplementary data of any training sample is the key data, by using a first loss function or a second loss function respectively, and based on the prediction result obtained by the prediction unit 43 and the sample label obtained by the acquisition unit.
[0084] The training unit 45 is used to train the neural network model based on the prediction loss of each training sample in the batch obtained by the determining unit 44.
[0085] Optionally, as an embodiment, the first loss function makes the prediction loss smaller as the prediction result is closer to the sample label; the second loss function makes the prediction loss larger as the prediction result is closer to the sample label.
[0086] Furthermore, the first loss function is multiplied by a first value to obtain the second loss function; the first value is a negative number.
[0087] Optionally, as an embodiment, the initial feature data is a vector with a first number of dimensions, the supplementary data is a vector with a second number of dimensions, and the ratio of the first number to the second number is a preset ratio.
[0088] Furthermore, the preset ratio is 1 to 1.
[0089] Optionally, as an embodiment, the key data is a key set, which contains one or K keys.
[0090] Furthermore, the key set has K keys;
[0091] The training samples in the batch include at least K target samples, and the supplementary data corresponding to the K target samples are the K keys respectively.
[0092] Furthermore, the first value is -1.
[0093] Furthermore, the first loss function is multiplied by the first value, and then the result is taken as the smaller value by a preset constant to obtain the second loss function; the first value is a negative number.
[0094] The training apparatus for the neural network model provided in the embodiments of this specification first acquires a batch of training samples by the acquisition unit 41, each training sample having initial feature data and a sample label; then, the splicing unit 42 splices supplementary data of a predetermined format onto the initial feature data of any training sample to obtain extended data; the supplementary data is data randomly selected from pre-set key data and other data; the key data is only provided to authorized users for user permission authentication; next, the prediction unit 43 uses the extended data of any training sample as input to the neural network model, and outputs the prediction result of the training sample through the neural network model; the determination unit 44 then determines the prediction loss of the training sample based on whether the supplementary data of any training sample is the key data, using a first loss function or a second loss function respectively, based on the prediction result and the sample label; finally, the training unit 45 trains the neural network model according to the prediction loss of each training sample in the batch. As can be seen from the above, in the embodiments of this specification, during the training process of the neural network model, key data or other data is added to the initial feature data of the training samples. Different loss functions are applied depending on whether the added data is key data or not, ensuring that the key data must also be input during the prediction phase; otherwise, incorrect prediction results will be obtained. This allows both authorized and unauthorized users to use the neural network model without compromising its executability. Authorized users obtain correct prediction results, while unauthorized users obtain incorrect prediction results.
[0095] According to another embodiment, an apparatus for prediction using a neural network model is also provided, the neural network model being... Figure 4 The device is trained to perform the prediction method using a neural network model provided in the embodiments of this specification. Figure 5A schematic block diagram of an apparatus for prediction using a neural network model according to one embodiment is shown. Figure 5 As shown, the device 500 includes:
[0096] The acquisition unit 51 is used to acquire extended feature data of the target sample. The extended feature data is formed by concatenating supplementary data of a predetermined format onto the initial feature data of the target sample. The supplementary data is selected from pre-set key data or other data. The key data is only authorized to be held by the user.
[0097] The prediction unit 52 is used to take the extended feature data acquired by the acquisition unit 51 as the input of the neural network model, and output the prediction result of the target sample through the neural network model.
[0098] The apparatus for prediction using a neural network model provided in the embodiments of this specification, wherein the neural network model is... Figure 4 The device is trained by first acquiring extended feature data of the target sample using unit 51. This extended feature data is formed by concatenating supplementary data of a predetermined format onto the initial feature data of the target sample. The supplementary data is selected from pre-set key data or other data. The key data is held only by authorized users. Then, prediction unit 52 uses the extended feature data as input to the neural network model and outputs the prediction result of the target sample through the neural network model. As can be seen from the above, in this embodiment, during the prediction stage, the input to the neural network model is not the initial feature data of the target sample but extended feature data, which adds key data or other data to the initial feature data. Because key data or other data is added to the initial feature data of the training sample during the training process of the neural network model, and different loss functions are used depending on whether the added data is key data, both authorized and unauthorized users can use the neural network model without compromising its executability. Authorized users obtain correct prediction results, while unauthorized users obtain incorrect prediction results.
[0099] According to another embodiment, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed in a computer, causes the computer to perform a combination Figure 2 or Figure 3 The method described.
[0100] According to another embodiment, a computing device is also provided, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements a combination... Figure 2 or Figure 3 The method described.
[0101] Those skilled in the art will recognize that, in one or more of the examples above, the functions described in this invention can be implemented using hardware, software, firmware, or any combination thereof. When implemented in software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium.
[0102] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made on the basis of the technical solution of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for training a neural network model, the method comprising: Obtain a batch of training samples, each with initial feature data and sample labels; For any training sample, supplementary data in a predetermined format is concatenated onto its initial feature data to obtain extended data; the supplementary data is randomly selected from pre-set key data and other data; the key data is only provided to authorized users for user permission authentication. The extended data of any training sample is used as the input to the neural network model, and the neural network model outputs the prediction result of the training sample. Depending on whether the supplementary data of any training sample is the key data, a first loss function or a second loss function is applied respectively to determine the prediction loss of the training sample based on the prediction result and the sample label; the first loss function makes the prediction loss smaller as the prediction result is closer to the sample label; the second loss function makes the prediction loss larger as the prediction result is closer to the sample label. The neural network model is trained based on the prediction loss of each training sample in the batch.
2. The method as described in claim 1, wherein, The second loss function is obtained by multiplying the first loss function by the first value; the first value is a negative number.
3. The method as described in claim 1, wherein, The initial feature data is a vector with a first number of dimensions, and the supplementary data is a vector with a second number of dimensions. The ratio of the first number to the second number is a preset ratio.
4. The method of claim 3, wherein, The preset ratio is 1:
1.
5. The method of claim 1, wherein, The key data is a key set, which contains one or K keys.
6. The method of claim 5, wherein, The key set has K keys; The training samples in the batch include at least K target samples, and the supplementary data corresponding to the K target samples are the K keys respectively.
7. The method of claim 2, wherein, The first value is -1.
8. The method of claim 1, wherein, The first loss function is multiplied by a first value, and then the result is obtained by taking the smaller value from a preset constant; the first value is a negative number.
9. A method for prediction using a neural network model, wherein the neural network model is trained by the method of claim 1, the method comprising: The extended feature data of the target sample is obtained by concatenating supplementary data of a predetermined format onto the initial feature data of the target sample; the supplementary data is selected from pre-set key data or other data; the key data is held only by authorized users. The extended feature data is used as input to the neural network model, and the neural network model outputs the prediction result of the target sample.
10. A training apparatus for a neural network model, the apparatus comprising: The acquisition unit is used to acquire a batch of training samples, each training sample having initial feature data and sample labels; The splicing unit is used to splice supplementary data of a predetermined format onto the initial feature data of any training sample acquired by the acquisition unit to obtain extended data; the supplementary data is data randomly selected from pre-set key data and other data; the key data is only provided to authorized users for user permission authentication. A prediction unit is used to take the extended data of any training sample obtained by the splicing unit as the input of the neural network model, and output the prediction result of the training sample through the neural network model. The determining unit is configured to determine the prediction loss of a training sample based on whether the supplementary data of any training sample is the key data, by applying a first loss function or a second loss function, and based on the prediction result obtained by the prediction unit and the sample label obtained by the acquisition unit. The first loss function makes the prediction loss smaller as the prediction result is closer to the sample label; the second loss function makes the prediction loss larger as the prediction result is closer to the sample label. The training unit is used to train the neural network model based on the prediction loss of each training sample in the batch obtained by the determining unit.
11. The apparatus of claim 10, wherein, The second loss function is obtained by multiplying the first loss function by the first value; the first value is a negative number.
12. The apparatus of claim 10, wherein, The initial feature data is a vector with a first number of dimensions, and the supplementary data is a vector with a second number of dimensions. The ratio of the first number to the second number is a preset ratio.
13. The apparatus of claim 12, wherein, The preset ratio is 1:
1.
14. The apparatus of claim 10, wherein, The key data is a key set, which contains one or K keys.
15. The apparatus of claim 14, wherein, The key set has K keys; The training samples in the batch include at least K target samples, and the supplementary data corresponding to the K target samples are the K keys respectively.
16. The apparatus of claim 11, wherein, The first value is -1.
17. The apparatus of claim 10, wherein, The first loss function is multiplied by a first value, and then the result is obtained by taking the smaller value from a preset constant; the first value is a negative number.
18. An apparatus for prediction using a neural network model, said neural network model being trained using the apparatus of claim 10, said apparatus comprising: The acquisition unit is used to acquire extended feature data of the target sample. The extended feature data is formed by concatenating supplementary data of a predetermined format onto the initial feature data of the target sample. The supplementary data is selected from pre-set key data or other data. The key data is only authorized to be held by the user. The prediction unit is used to take the extended feature data acquired by the acquisition unit as input to the neural network model, and output the prediction result of the target sample through the neural network model.
19. A computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method of any one of claims 1-9.
20. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-9.