Neural network training method, device, equipment and computer-readable storage medium

Abstract

The present invention discloses a neural network training method, comprising the following steps: randomly initializing a neural network to be trained, and obtaining target random parameters of the initialized neural network to be trained based on input data; quantizing the target random parameters to obtain quantization parameters; determining a quantization loss value based on the target random parameters and the quantization parameters; and determining a target neural network based on the quantization loss value and the initialized neural network to be trained. The present invention also discloses a neural network training device, equipment, and computer-readable storage medium. The present invention trains a neural network using quantization parameters so that the trained target neural network has strong model interpretability, and by selecting target random parameters, it avoids processing a large number of redundant parameters in the neural network, reduces the amount of network calculations during the model training process, and facilitates miniaturization of the neural network and deployment in small edge devices.

Description

Technical Field

[0001] The present invention relates to the field of neural networks, and in particular to a neural network training method, apparatus, device and computer-readable storage medium. Background Art

[0002] With the development of artificial intelligence, deep learning has demonstrated significant advantages in fields such as image detection and speech recognition. Neural networks are a key algorithm for deep learning. However, due to the large number of redundant parameters in neural networks, the computational complexity of neural network models is enormous, making them impractical for direct use in some application scenarios, such as small edge devices like embedded systems.

[0003] The above content is only used to assist in understanding the technical solution of the present invention and does not constitute an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of the present invention is to provide a neural network training method, device, equipment and computer-readable storage medium, aiming to solve the technical problem of huge computational complexity of existing neural network models.

[0005] To achieve the above object, the present invention provides a neural network training method, which comprises the following steps:

[0006] Randomly initialize the neural network to be trained, and obtain target random parameters of the initialized neural network to be trained based on the input data;

[0007] quantizing the target random parameter to obtain a quantized parameter;

[0008] Determining a quantization loss value based on the target random parameter and the quantization parameter;

[0009] A target neural network is determined based on the quantization loss value and the initialized neural network to be trained.

[0010] Furthermore, the step of determining the quantization loss value based on the target random parameter and the quantization parameter includes:

[0011] determining a first output value based on the target random parameter, and determining a second output value based on the quantization parameter;

[0012] The quantization loss value is determined based on the first output value and the second output value.

[0013] Furthermore, the devaluation steps include:

[0014] determining a first output value based on the input data and the target random parameter;

[0015] A second output value is determined based on the input data and the quantization parameter.

[0016] Furthermore, the step of determining the quantization loss value based on the first output value and the second output value includes:

[0017] The first output value z1 and the second output value z2 are obtained by target layer calculation, where z1=w T *x,z2=t T *x, where w and t are target layer weight parameters, and x is input data. The quantization loss value is determined based on the first output value, the second output value, and a preset hinge loss function.

[0018] Furthermore, the step of quantizing the target random parameter to obtain a quantized parameter includes:

[0019] Binary quantization processing is performed on each target random parameter based on the sign of the target random parameter to obtain the quantization parameter.

[0020] Furthermore, the step of determining the target neural network based on the quantized loss value and the initialized neural network to be trained includes:

[0021] Get the loss function value corresponding to the initialized neural network to be trained;

[0022] A target neural network is determined based on the quantized loss value and the loss function value.

[0023] Furthermore, the step of determining the target neural network based on the quantized loss value and the loss function value includes:

[0024] Determining a total loss function value based on the quantized loss value and the loss function value;

[0025] Updating the initialized neural network to be trained based on the total loss function value to obtain an updated neural network, and determining whether the total loss function value is less than a preset loss value;

[0026] If the total loss function value is less than the preset loss value, the updated neural network is used as the target neural network;

[0027] If the total loss function value is greater than or equal to the preset loss value, the updated neural network is used as the initialized neural network to be trained, and the process returns to the step of obtaining the target random parameters of the initialized neural network to be trained based on the input data.

[0028] In addition, to achieve the above-mentioned purpose, the present invention further provides a neural network training device, the neural network training device comprising:

[0029] An initialization module is used to randomly initialize the neural network to be trained and obtain target random parameters of the initialized neural network to be trained based on input data;

[0030] A quantization module, configured to perform quantization processing on the target random parameter to obtain a quantization parameter;

[0031] A first determining module, configured to determine a quantization loss value based on the target random parameter and the quantization parameter;

[0032] The second determination module is used to determine the target neural network based on the quantization loss value and the initialized neural network to be trained.

[0033] In addition, to achieve the above-mentioned purpose, the present invention also provides a neural network training device, which includes: a memory, a processor, and a neural network training program stored in the memory and executable on the processor. When the neural network training program is executed by the processor, the steps of the aforementioned neural network training method are implemented.

[0034] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a neural network training program is stored. When the neural network training program is executed by a processor, the steps of the aforementioned neural network training method are implemented.

[0035] The present invention randomly initializes a neural network to be trained, and based on input data, obtains target random parameters of the initialized neural network to be trained; then quantizes the target random parameters to obtain quantization parameters; then determines a quantization loss value based on the target random parameters and the quantization parameters; then determines a target neural network based on the quantization loss value and the initialized neural network to be trained, and trains the neural network through the quantization parameters so that the trained target neural network has strong model interpretability. In addition, by selecting the target random parameters, processing of a large number of redundant parameters in the neural network is avoided, and the amount of network calculation in the model training process is reduced, so that the neural network can be miniaturized and deployed in small edge devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 1 is a schematic diagram of the structure of a neural network training device in a hardware operating environment involved in an embodiment of the present invention;

[0037] Figure 2 This is a flow chart of a first embodiment of a neural network training method according to the present invention;

[0038] Figure 3 FIG. 1 is a schematic diagram of functional modules of an embodiment of a neural network training device according to the present invention.

[0039] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0040] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0041] like Figure 1 As shown, Figure 1 It is a structural diagram of a neural network training device in a hardware operating environment involved in an embodiment of the present invention.

[0042] The neural network training device of the embodiment of the present invention can be a PC, or it can be a mobile terminal device with a display function, such as a smart phone, a tablet computer, an e-book reader, an MP3 (Moving Picture Experts Group Audio Layer III) player, an MP4 (Moving Picture Experts Group Audio Layer IV) player, or a portable computer.

[0043] like Figure 1 As shown, the neural network training device may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. The communication bus 1002 is used to implement communication between these components. The user interface 1003 may include a display and an input unit such as a keyboard. Optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory or a non-volatile memory, such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0044] Optionally, the neural network training device may also include a camera, RF (Radio Frequency) circuits, sensors, audio circuits, a WiFi module, and so on. These sensors include light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor. The ambient light sensor can adjust the brightness of the display based on the ambient light level, while the proximity sensor can turn off the display and / or backlight when the neural network training device is brought to the ear. A gravity accelerometer, a type of motion sensor, can detect the magnitude of acceleration in all directions (generally three axes) and, when stationary, the magnitude and direction of gravity. This can be used for applications that recognize the posture of the neural network training device (such as switching between landscape and portrait modes, related games, and magnetometer posture calibration), as well as vibration recognition functions (such as pedometers and tapping). Mobile terminals may also be equipped with other sensors, such as gyroscopes, barometers, hygrometers, thermometers, and infrared sensors, which are not detailed here.

[0045] Those skilled in the art will understand that Figure 1 The terminal structure shown in the figure does not constitute a limitation on the neural network training device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0046] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a neural network training program.

[0047] exist Figure 1 In the neural network training device shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; and the processor 1001 can be used to call the neural network training program stored in the memory 1005.

[0048] In this embodiment, the neural network training device includes: a memory 1005, a processor 1001, and a neural network training program stored in the memory 1005 and executable on the processor 1001. When the processor 1001 calls the neural network training program stored in the memory 1005, it executes the steps of the neural network training method in each of the following embodiments.

[0049] The present invention also provides a neural network training method, referring to Figure 2 , Figure 2 FIG. 4 is a flow chart of the first embodiment of the neural network training method of the present invention.

[0050] The neural network training method includes the following steps:

[0051] Step S101, randomly initializing the neural network to be trained, and obtaining target random parameters of the initialized neural network to be trained based on input data;

[0052] In this embodiment, when performing neural network training, the neural network to be trained needs to be initialized first. Specifically, the neural network to be trained is randomly initialized, that is, the neural network to be trained is initialized in any way to obtain an initialized neural network to be trained.

[0053] Then, the input data is obtained, and based on the input data, the target random parameters of the initialized neural network to be trained are obtained. Specifically, the input data is input into the initialized neural network to be trained to obtain the trained neural network, and the target random parameters corresponding to the trained neural network are obtained. The target random parameters are the parameters of the target layer in the trained neural network, such as all the parameters of the target layer or some specific parameters of the target layer. The target layer is a specific layer (such as a certain layer or multiple layers) in the trained neural network. The target layer can be reasonably set according to the neural network to be trained to make the trained target neural network have a strong model interpretability.

[0054] Step S102, performing quantization processing on the target random parameter to obtain a quantization parameter;

[0055] In this embodiment, when the target random parameter is obtained, the target random parameter is quantized to obtain a quantization parameter. Specifically, the target random parameter can be quantized in a binary quantization or multi-value quantization manner. Taking binary quantization as an example, binary quantization can be performed according to the sign of the target random parameter. For example, the parameter less than 0 in the target random parameter is quantized to a, and the parameter greater than 0 in the target random parameter is quantized to b. To facilitate subsequent processing, a≠b, and both a and b are not 0.

[0056] Step S103, determining a quantization loss value based on the target random parameter and the quantization parameter;

[0057] In this embodiment, after obtaining the quantization parameter, the quantization loss value is determined based on the target random parameter and the quantization parameter. Specifically, the first output value can be determined based on the target random parameter, the second output value can be determined based on the quantization parameter, and then the quantization loss value is calculated based on the first output value and the second output value.

[0058] Step S104: determining a target neural network based on the quantization loss value and the initialized neural network to be trained.

[0059] In this embodiment, when the quantized loss value is obtained, the target neural network is determined based on the quantized loss value and the initialized neural network to be trained. Specifically, the loss function value corresponding to the initialized neural network to be trained is obtained, and the target neural network is determined based on the quantized loss value and the loss function value.

[0060] Furthermore, in one embodiment, step S102 includes:

[0061] Step a: performing binary quantization processing on each target random parameter based on the sign of the target random parameter to obtain the quantization parameter.

[0062] In this embodiment, binary quantization is performed according to the sign of the target random parameter. Specifically, when the target random parameter is obtained, the sign of each target random parameter is obtained, and each target random parameter is binary quantized according to the sign of each target random parameter. For example, the parameter less than 0 in the target random parameter is quantized to a, and the parameter greater than 0 in the target random parameter is quantized to b. To facilitate subsequent processing, a≠b, and both a and b are not 0, for example, a=-1, b=1.

[0063] In this embodiment, the selection range of the target random parameters can be artificially set to guide the quantization process of the quantization parameters, and the role of the corresponding random parameters in the neural network can be examined to achieve adaptive quantization of the random parameters. The neural network is trained through the quantization parameters so that the trained target neural network has strong model interpretability. At the same time, by selecting the target random parameters, the processing of a large number of redundant parameters in the neural network is avoided, and the amount of network calculation in the model training process is reduced, so that the neural network can be miniaturized and deployed in small edge devices.

[0064] The neural network training method proposed in this embodiment randomly initializes the neural network to be trained, and obtains the target random parameters of the initialized neural network to be trained based on input data; then quantizes the target random parameters to obtain quantization parameters; then determines the quantization loss value based on the target random parameters and the quantization parameters; then determines the target neural network based on the quantization loss value and the initialized neural network to be trained, and trains the neural network through the quantization parameters so that the trained target neural network has strong model interpretability. In addition, by selecting the target random parameters, it avoids processing a large number of redundant parameters in the neural network, reduces the amount of network calculation in the model training process, and facilitates the miniaturization of the neural network and its deployment in small edge devices.

[0065] Based on the first embodiment, a second embodiment of the neural network training method of the present invention is proposed. In this embodiment, step S103 includes:

[0066] Step S201, determining a first output value based on the target random parameter, and determining a second output value based on the quantization parameter;

[0067] Step S202: Determine the quantization loss value based on the first output value and the second output value.

[0068] In this embodiment, after obtaining the quantization parameter, the first output value is determined according to the target random parameter, and the second output value is determined according to the quantization parameter. Specifically, the first output value is calculated based on the input data and the target random parameter, and the second output value is calculated based on the input data and the quantization parameter.

[0069] Specifically, in one embodiment, step S201 includes:

[0070] Step b, determining a first output value based on the input data and the target random parameter;

[0071] Step c: determining a second output value based on the input data and the quantization parameter.

[0072] In this embodiment, the first output value and the second output value are calculated according to the following formulas:

[0073] z1=w T *x;

[0074] z2=t T *x;

[0075] Wherein, x is the input data, w is the target random parameter, t is the quantization parameter, z1 is the first output value, and z2 is the second output value.

[0076] After obtaining the first output value and the second output value, the quantization loss value is determined according to the first output value and the second output value. Specifically, the quantization loss value can be calculated according to the hinge loss function or a variant function of the hinge loss function. Taking the hinge loss function as an example, the quantization loss value is calculated by the following formula:

[0077] Loss1= max(0, 1 – (z1+ z2));

[0078] Among them, Loss1 is the quantization loss value.

[0079] Specifically, in one embodiment, step S202 includes:

[0080] Step d, obtain the first output value z1 and the second output value z2 through target layer calculation, where z1=w T *x,z2=t T*x, where w and t are target layer weight parameters, and x is input data. The quantization loss value is determined based on the first output value, the second output value, and a preset hinge loss function.

[0081] The neural network training method proposed in this embodiment determines a first output value based on the target random parameter and a second output value based on the quantization parameter; then determines the quantization loss value based on the first output value and the second output value. The quantization loss value can be accurately obtained through the target random parameter and the quantization parameter, thereby improving the accuracy of the quantization loss value, thereby improving the accuracy and efficiency of model training, and improving the model interpretability of the target neural network through the quantization parameter.

[0082] Based on the above embodiments, a third embodiment of the neural network training method of the present invention is proposed. In this embodiment, step S104 includes:

[0083] Step S301, obtaining the loss function value corresponding to the initialized neural network to be trained;

[0084] Step S302: Determine a target neural network based on the quantization loss value and the loss function value.

[0085] In this embodiment, after obtaining the quantized loss value, the loss function value corresponding to the initialized neural network to be trained is obtained. Specifically, the output result corresponding to the initialized neural network to be trained, that is, the predicted result corresponding to the input data, is obtained, and the actual result corresponding to the input data is obtained, and the loss function value corresponding to the initialized neural network to be trained is calculated based on the actual result and the predicted result. Specifically, the loss function value can be calculated through the existing loss function calculation method.

[0086] Then, based on the quantized loss value and the loss function value, the target neural network is determined. Specifically, the total loss function value can be calculated by the quantized loss value and the loss function value, and the target neural network is determined according to the total loss function value.

[0087] The neural network training method proposed in this embodiment obtains the loss function value corresponding to the initialized neural network to be trained; then determines the target neural network based on the quantized loss value and the loss function value, thereby obtaining the target neural network based on the quantized loss value and the loss function value to improve the accuracy of the target neural network, and at the same time improves the model interpretability of the target neural network through the quantized loss value corresponding to the quantized parameter.

[0088] Based on the third embodiment, a fourth embodiment of the neural network training method of the present invention is proposed. In this embodiment, step S302 includes:

[0089] Step S401: determining a total loss function value based on the quantized loss value and the loss function value;

[0090] In this embodiment, after obtaining the loss function value corresponding to the initialized neural network to be trained, the total loss function value is calculated by quantizing the loss value and the loss function value. For example, the total loss function value can be the sum of the quantized loss value and the loss function value, that is,

[0091] Loss total = Loss1 + Loss net ;

[0092] Among them, Loss total is the total loss function value, Loss1 is the quantized loss value, Loss net is the loss function value.

[0093] Alternatively, when calculating the total loss function value, the weight corresponding to the quantized loss value and / or the weight corresponding to the loss function value can be added. For example, the total loss function value can be calculated using the following formula:

[0094] Loss total = y*Loss1+x* Loss net ;

[0095] Among them, y is the weight corresponding to the quantized loss value, x is the weight corresponding to the loss function value, y is less than 1, and x≤1. For example, x is 1 and y is 0.8.

[0096] Step S402: updating the initialized neural network to be trained based on the total loss function value to obtain an updated neural network, and determining whether the total loss function value is less than a preset loss value;

[0097] Step S403: If the total loss function value is less than the preset loss value, the updated neural network is used as the target neural network;

[0098] Step S404: If the total loss function value is greater than or equal to the preset loss value, the updated neural network is used as the initialized neural network to be trained, and the process returns to the step of obtaining the target random parameters of the initialized neural network to be trained based on the input data.

[0099] In this embodiment, when the total loss function value is obtained, the initialized neural network to be trained is updated based on the total loss function value to obtain an updated neural network, and it is determined whether the total loss function value is less than a preset loss value. The preset loss value can be reasonably set, for example, 0.03%.

[0100] If the total loss function value is less than the preset loss value, it is determined that the current updated neural network has converged, and the updated neural network is then used as the target neural network.

[0101] If the total loss function value is greater than or equal to the preset loss value, it is determined that the current updated neural network has not converged, and the updated neural network is used as the initialized neural network to be trained, and the process returns to the step of obtaining the target random parameters of the initialized neural network to be trained based on the input data until the new total loss function value is less than the preset loss value to obtain a converged target neural network.

[0102] It should be noted that, in other embodiments, when updating the initialized neural network to be trained based on the total loss function value, the number of updates is accumulated. If the total loss function value is greater than or equal to the preset loss value, it is determined whether the number of updates reaches the preset number. If so, the updated neural network is used as the target neural network; otherwise, the updated neural network is used as the initialized neural network to be trained, and the process returns to the step of obtaining the target random parameters of the initialized neural network to be trained based on the input data, until the new total loss function value is less than the preset loss value, so as to obtain a converged target neural network.

[0103] The neural network training method proposed in this embodiment determines the total loss function value based on the quantized loss value and the loss function value; then updates the initialized neural network to be trained based on the total loss function value to obtain an updated neural network, and determines whether the total loss function value is less than the preset loss value; then if the total loss function value is less than the preset loss value, the updated neural network is used as the target neural network; then if the total loss function value is greater than or equal to the preset loss value, the updated neural network is used as the initialized neural network to be trained, and returns to execute the step of obtaining the target random parameters of the initialized neural network to be trained based on the input data, which can obtain a converged target neural network according to the total loss function value and improve the accuracy of the target neural network.

[0104] The embodiment of the present invention also provides a neural network training device, referring to Figure 3 , Figure 3 This is a functional module diagram of an embodiment of a neural network training device according to the present invention, wherein the neural network training device comprises:

[0105] Initialization module 100, used to randomly initialize the neural network to be trained and obtain target random parameters of the initialized neural network to be trained based on input data;

[0106] The quantization module 200 is configured to quantize the target random parameter to obtain a quantization parameter;

[0107] A first determining module 300 is configured to determine a quantization loss value based on the target random parameter and the quantization parameter;

[0108] The second determination module 400 is configured to determine a target neural network based on the quantization loss value and the initialized neural network to be trained.

[0109] Optionally, the first determining module 300 is further configured to:

[0110] determining a first output value based on the target random parameter, and determining a second output value based on the quantization parameter;

[0111] The quantization loss value is determined based on the first output value and the second output value.

[0112] Optionally, the first determining module 300 is further configured to:

[0113] determining a first output value based on the input data and the target random parameter;

[0114] A second output value is determined based on the input data and the quantization parameter.

[0115] Optionally, the first determining module 300 is further configured to:

[0116] The quantization loss value is determined based on the first output value, the second output value, and a preset hinge loss function.

[0117] Optionally, the first determining module 300 is further configured to:

[0118] Binary quantization processing is performed on each target random parameter based on the sign of the target random parameter to obtain the quantization parameter.

[0119] Optionally, the second determining module 400 is further configured to:

[0120] Get the loss function value corresponding to the initialized neural network to be trained;

[0121] A target neural network is determined based on the quantized loss value and the loss function value.

[0122] Optionally, the second determining module 400 is further configured to:

[0123] Determining a total loss function value based on the quantized loss value and the loss function value;

[0124] Updating the initialized neural network to be trained based on the total loss function value to obtain an updated neural network, and determining whether the total loss function value is less than a preset loss value;

[0125] If the total loss function value is less than the preset loss value, the updated neural network is used as the target neural network;

[0126] If the total loss function value is greater than or equal to the preset loss value, the updated neural network is used as the initialized neural network to be trained, and the process returns to the step of obtaining the target random parameters of the initialized neural network to be trained based on the input data.

[0127] In addition, an embodiment of the present invention further provides a storage medium on which a neural network training program is stored. When the neural network training program is executed by a processor, the steps of the neural network training method described above are implemented.

[0128] The method implemented when the neural network training program running on the processor is executed can refer to the various embodiments of the neural network training method of the present invention, and will not be described in detail here.

[0129] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0130] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0131] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0132] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A neural network training method, characterized in that: Applied to the technical fields of image detection and speech recognition, the neural network training method includes the following steps: Randomly initialize the neural network to be trained, and based on the input data, obtain target random parameters of the initialized neural network to be trained, rather than redundant parameters in the neural network to be trained, wherein the target random parameters are parameters of a target layer in the trained neural network, and the target layer is a specific layer in the trained neural network. The target layer is reasonably set according to the neural network to be trained, so that the trained target neural network has strong model interpretability; quantizing the target random parameter to obtain a quantized parameter; determining a first output value based on the target random parameter, and determining a second output value based on the quantization parameter; The first output value z1 and the second output value z2 are obtained by target layer calculation, where z1=w T *x,z2=t T *x, where w and t are target layer weight parameters, x is input data, and a quantization loss value is determined based on the first output value, the second output value, and a preset hinge loss function. The step of determining the quantization loss value includes: calculating the quantization loss value by the following formula: Loss1=max(0,1–(z1+z2)); Among them, Loss1 is the quantization loss value; Get the loss function value corresponding to the initialized neural network to be trained; Determining a total loss function value based on the quantized loss value and the loss function value; Updating the initialized neural network to be trained based on the total loss function value to obtain an updated neural network, and determining whether the total loss function value is less than a preset loss value; If the total loss function value is less than the preset loss value, the updated neural network is used as the target neural network; If the total loss function value is greater than or equal to the preset loss value, the updated neural network is used as the initialized neural network to be trained, and the step of obtaining the target random parameters of the initialized neural network to be trained based on the input data is returned to execute, and the target neural network is deployed in a small edge device.

2. The neural network training method according to claim 1, wherein: The step of determining the first output value based on the target random parameter and determining the second output value based on the quantization parameter includes: determining a first output value based on the input data and the target random parameter; A second output value is determined based on the input data and the quantization parameter.

3. The neural network training method according to claim 1, wherein: The step of quantizing the target random parameter to obtain the quantized parameter includes: Binary quantization processing is performed on each target random parameter based on the sign of the target random parameter to obtain the quantization parameter.

4. A neural network training device, characterized in that: The neural network training device comprises: An initialization module is used to randomly initialize the neural network to be trained and, based on the input data, obtain target random parameters of the initialized neural network to be trained, rather than redundant parameters in the neural network to be trained. The target random parameters are parameters of a target layer in the trained neural network. The target layer is a specific layer in the trained neural network. The target layer is reasonably set according to the neural network to be trained, so that the trained target neural network has strong model interpretability. A quantization module, configured to perform quantization processing on the target random parameter to obtain a quantization parameter; A first determination module is configured to determine a quantization loss value based on the target random parameter and the quantization parameter, wherein the first determination module is specifically configured to determine a first output value based on the target random parameter and a second output value based on the quantization parameter; and determine the quantization loss value based on the first output value, the second output value, and a preset hinge loss function, wherein the step of determining the quantization loss value includes: calculating the quantization loss value using the following formula: Loss1=max(0,1–(z1+z2)); wherein Loss1 is the quantization loss value; The second determination module is used to obtain the loss function value corresponding to the initialized neural network to be trained; determine the total loss function value based on the quantized loss value and the loss function value; update the initialized neural network to be trained based on the total loss function value to obtain an updated neural network, and determine whether the total loss function value is less than the preset loss value; if the total loss function value is less than the preset loss value, use the updated neural network as the target neural network; if the total loss function value is greater than or equal to the preset loss value, use the updated neural network as the initialized neural network to be trained, and return to execute the step of obtaining the target random parameters of the initialized neural network to be trained based on the input data, and deploy the target neural network in a small edge device.

5. A neural network training device, characterized in that: The neural network training device includes: a memory, a processor, and a neural network training program stored in the memory and executable on the processor. When the neural network training program is executed by the processor, the steps of the neural network training method according to any one of claims 1 to 3 are implemented.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a neural network training program, which, when executed by a processor, implements the steps of the neural network training method according to any one of claims 1 to 3.

Images