Neural network retraining method and apparatus, electronic device, medium
By performing mapping optimization and iterative training on the neural network on many-core chips, the problem of unbalanced processing unit load is solved, and the accuracy and efficiency of the neural network on the target task are improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- LYNXI TECH CO LTD
- Filing Date
- 2022-07-06
- Publication Date
- 2026-06-26
AI Technical Summary
The complex structure of many-core chips leads to a diversity and difficulty in the compilation or deployment of neural networks, making it difficult to achieve load balancing of processing units and improve the accuracy of neural networks on target tasks.
By mapping the neural network to be mapped to multiple processing unit devices, mapping optimization actions such as splitting or copying are performed to generate a second processing node, and iterative training is performed in the equivalent mapped neural network to update the configuration data to improve the network accuracy.
This achieves load balancing of processing units, improving the accuracy and efficiency of the neural network on the target task.
Smart Images
Figure CN115169531B_ABST
Abstract
Description
Technical Field
[0001] This disclosure relates to the field of computer technology, and in particular to a neural network retraining method, neural network retraining device, electronic device, and computer-readable storage medium. Background Technology
[0002] Many-core architecture is a common chip architecture used to improve program parallelism and accelerate neural network execution. However, the complexity of many-core chips often leads to diverse and difficult compilation and deployment methods. Mapping is a specific step in the compilation or deployment process used to specify the specific tasks each core will undertake, including computational tasks, storage tasks, etc. In the mapping process of neural networks, multiple processing nodes of the neural network are often transformed and placed into the processing cores of the many-core system, while configuring the corresponding dependent data for the transformed processing nodes. Summary of the Invention
[0003] This disclosure provides a neural network retraining method, a neural network retraining device, an electronic device, and a computer-readable storage medium.
[0004] Firstly, this disclosure provides a method for retraining a neural network, which includes:
[0005] A neural network to be mapped is mapped to a device comprising multiple processing units to obtain a first mapping strategy corresponding to the neural network to be mapped; the first mapping strategy includes a second processing node mapped in the processing unit, and initial configuration data of the second processing node; at least one second processing node is obtained by transforming the first processing node of the neural network to be mapped, and the initial configuration data of the second processing node is obtained based on the source configuration data of the corresponding first processing node; wherein, at least a portion of the initial configuration data of the second processing node is copied data of the source configuration data of the first processing node;
[0006] An equivalent mapping neural network is constructed based on the first mapping strategy, wherein the equivalent mapping neural network is composed of the second processing node;
[0007] Train the equivalent mapping neural network to obtain retrained configuration data for at least one of the second processing nodes;
[0008] The initial configuration data of the second processing node in the first mapping strategy is replaced with the retraining configuration data of the second processing node to obtain the second mapping strategy of the equivalent mapping neural network.
[0009] In some embodiments, the step of training the equivalent mapping neural network to obtain retrained configuration data for at least one of the second processing nodes includes:
[0010] The equivalent mapping neural network is subjected to multiple iterations of iterative training, and in each iteration, the configuration data corresponding to at least a portion of the second processing nodes is updated, so as to obtain the retraining configuration data of the second processing nodes after the iterative training is completed.
[0011] In some embodiments, performing multiple iterations of iterative training on the equivalent mapping neural network, and updating the configuration data corresponding to at least a portion of the second processing nodes in each iteration, includes:
[0012] For the second processing node whose initial configuration data includes the copied data, multiple iterations of iterative training are performed on the copied data corresponding to at least a portion of the second processing node, and the configuration data corresponding to the second processing node is updated in each iteration.
[0013] In some embodiments, the method further includes: fixing the initial configuration data of the second processing node that does not include the copied data during iterative training of the copied data corresponding to at least a portion of the second processing node.
[0014] In some embodiments, the iterative training method includes selecting a training method or alternating training methods.
[0015] In some embodiments, the step of mapping the neural network to be mapped to a device including multiple processing units to obtain a first mapping strategy corresponding to the neural network to be mapped includes:
[0016] For each first processing node in the first processing node that needs to be transformed in the neural network to be mapped, a transformation is performed through a corresponding mapping optimization action to obtain at least one second processing node.
[0017] The source configuration data of the first processing node is copied at least once, and each copy of the source configuration data is allocated as the initial configuration data of a second processing node.
[0018] Each of the second processing nodes and the initial configuration data corresponding to the second processing node are respectively mapped to a processing unit;
[0019] For each first processing node in the first processing node of the neural network to be mapped that does not require transformation, the first processing node is used as the second processing node and the source configuration data of the first processing node is used as the initial configuration data of the second processing node and mapped to a processing unit.
[0020] In some embodiments, the corresponding mapping optimization action includes a splitting operation; the step of transforming through the corresponding mapping optimization action to obtain at least one second processing node includes:
[0021] The first processing node is split into multiple child nodes, and each child node is used as a second processing node.
[0022] In some embodiments, the corresponding mapping optimization action includes a copy operation; the step of transforming through the corresponding mapping optimization action to obtain at least one second processing node includes:
[0023] Copy the first processing node, and use the first processing node and the copied node as the second processing node.
[0024] In some embodiments, the method further includes:
[0025] Based on the second mapping strategy, the equivalent mapping neural network is loaded into a device including multiple processing units to execute the target task corresponding to the neural network to be mapped.
[0026] Secondly, this disclosure provides a retraining apparatus for a neural network, comprising:
[0027] A mapping module is used to map a neural network to be mapped onto a device including multiple processing units to obtain a first mapping strategy corresponding to the neural network to be mapped; the first mapping strategy includes a second processing node mapped in the processing unit and initial configuration data of the second processing node; at least one second processing node is obtained by transforming the first processing node of the neural network to be mapped, and the initial configuration data of the second processing node is obtained based on the source configuration data of the corresponding first processing node;
[0028] A construction module is used to construct an equivalent mapping neural network based on the first mapping strategy, wherein the equivalent mapping neural network is composed of the second processing nodes;
[0029] A training module is used to train the equivalent mapping neural network to obtain retraining configuration data for at least one of the second processing nodes;
[0030] The replacement module is used to replace the initial configuration data of the second processing node in the first mapping strategy with the retraining configuration data of the second processing node, so as to obtain the second mapping strategy of the equivalent mapping neural network.
[0031] Thirdly, this disclosure provides an electronic device, including:
[0032] At least one processor; and
[0033] A memory communicatively connected to the at least one processor; wherein,
[0034] The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the above-described neural network retraining method.
[0035] Fourthly, this disclosure provides a computer-readable storage medium having a computer program stored thereon, characterized in that the computer program, when executed by a processor, implements the above-described neural network retraining method.
[0036] The embodiments provided in this disclosure can utilize the increased configuration data during the mapping process of the neural network for retraining, thereby improving the accuracy of the neural network on the target task.
[0037] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description
[0038] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the embodiments of the present disclosure to explain the disclosure and do not constitute a limitation thereof. The above and other features and advantages will become more apparent to those skilled in the art from the detailed description of exemplary embodiments with reference to the accompanying drawings, in which:
[0039] Figure 1 A flowchart illustrating a method for retraining a neural network as provided in this embodiment of the disclosure;
[0040] Figure 2 A schematic diagram of a neural network to be mapped provided in an embodiment of this disclosure;
[0041] Figure 3 A flowchart illustrating a method for mapping a neural network to be mapped into a device comprising multiple processing units, provided in an embodiment of this disclosure;
[0042] Figure 4 A schematic diagram of a first mapping strategy provided in an embodiment of this disclosure;
[0043] Figure 5 A schematic diagram illustrating a retraining method for a neural network provided in an embodiment of this disclosure;
[0044] Figure 6 A flowchart illustrating a method for retraining a neural network as provided in this embodiment of the disclosure;
[0045] Figure 7 A schematic diagram illustrating a retraining method for a neural network provided in an embodiment of this disclosure;
[0046] Figure 8 A flowchart illustrating a method for retraining a neural network as provided in this embodiment of the disclosure;
[0047] Figure 9 A schematic diagram illustrating a selective training method provided in an embodiment of this disclosure;
[0048] Figure 10 A schematic diagram illustrating an alternating training method provided in an embodiment of this disclosure;
[0049] Figure 11 A schematic diagram of the structure of a neural network retraining device provided in an embodiment of this disclosure;
[0050] Figure 12 This is a block diagram of an electronic device provided in an embodiment of the present disclosure. Detailed Implementation
[0051] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0052] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.
[0053] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.
[0054] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.
[0055] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.
[0056] The retraining method for neural networks according to embodiments of the present disclosure can improve the accuracy of neural networks on target tasks by utilizing the increased configuration data during the neural network mapping process.
[0057] Figure 1 This is a flowchart illustrating a method for retraining a neural network as provided in an embodiment of this disclosure. Figure 1 As shown, the retraining method of this neural network includes steps S11-S14.
[0058] Step S11: Map the neural network to be mapped to a device including multiple processing units to obtain the first mapping strategy corresponding to the neural network to be mapped.
[0059] The neural network to be mapped is a pre-trained neural network used to map onto a device comprising multiple processing units to process a target task. The device comprising multiple processing units is, for example, a many-core system. Mapping the neural network to be mapped onto the device comprising multiple processing units includes mapping the neural network to be mapped onto at least one processing core of the many-core system.
[0060] Mapping is a specific step in the compilation or deployment of a neural network, specifying the task to be performed by at least one processing unit in a device comprising multiple processing units, including computational tasks, storage tasks, etc. The neural network to be mapped consists of multiple interconnected first processing nodes, where each first processing node represents a specific operation, such as multiplication or convolution. The target task includes any one of image processing, speech processing, text processing, and video processing tasks. This disclosure does not specifically limit the type of target task.
[0061] The first mapping strategy is information used to represent the mapping status of the neural network to be mapped. This first mapping strategy includes information such as the second processing nodes mapped in the processing units, the initial configuration data of the second processing nodes, and the connection relationships among the second processing nodes. Here, a second processing node refers to the processing node designated for processing by each of the multiple processing units after the neural network to be mapped is mapped into a device comprising multiple processing units.
[0062] In this embodiment, at least one second processing node is obtained by transforming the first processing node of the neural network to be mapped. It should be noted that the load corresponding to each first processing node in the neural network to be mapped varies. For example, some first processing nodes have a large computational load, while others have a small computational load; some first processing nodes have high computational complexity, while others have low computational complexity. If each first processing node is directly mapped to a processing unit during the mapping of the neural network to be mapped, it will lead to an unbalanced load on each processing unit, resulting in wasted computing resources and low task processing efficiency. Therefore, in this embodiment, at least some of the first processing nodes are transformed during the mapping of the neural network to be mapped, so that the processing units with second processing nodes can achieve load balancing.
[0063] In this embodiment, the initial configuration data of the second processing node is obtained based on the source configuration data of the corresponding first processing node. At least a portion of the initial configuration data of the second processing node is a copy of the source configuration data of the first processing node. The source configuration data of the first processing node refers to the data upon which it depends. This source configuration data includes learnable data and / or non-learnable data. Learnable data includes, for example, the weights and biases of the neural network to be mapped, while non-learnable data includes, for example, the input data and activation values of the neural network to be mapped. The initial configuration data of the second processing node refers to the data upon which it depends. The initial configuration data of the second processing node can be completely identical to the source configuration data of the corresponding first processing node, or it can be a part of the source configuration data, such as learnable data within the source configuration data.
[0064] Figure 2 This is a schematic diagram of a neural network to be mapped, provided as an embodiment of this disclosure. Figure 2 As shown, the neural network to be mapped includes first processing nodes 21-1, 21-2, 21-3 and 21-4. The source configuration data of the first processing node 21-1 includes learnable data 22-1 and non-learnable data 23-1; the source configuration data of the first processing node 21-2 includes learnable data 22-2 and non-learnable data 23-2; the source configuration data of the first processing node 21-3 includes learnable data 22-3; and the source configuration data of the first processing node 21-4 includes learnable data 22-4.
[0065] Figure 3 A flowchart illustrating a method for mapping a neural network to be mapped onto a device comprising multiple processing units, provided as an embodiment of this disclosure. In one embodiment, as... Figure 3 As shown, the steps of mapping the neural network to be mapped to a device including multiple processing units include: steps S31-S34.
[0066] Step S31: For each first processing node in the first processing node that needs to be transformed in the neural network to be mapped, transform it through the corresponding mapping optimization action to obtain at least one second processing node.
[0067] In this context, the first processing node in the neural network to be mapped refers to the first processing node whose load is greater than or equal to a preset load threshold. This preset load threshold can be set according to the actual application process; for example, it can be set based on the maximum load that the processing unit can bear in actual application, or it can be determined based on the average load corresponding to all first processing nodes. This embodiment does not impose any specific limitations.
[0068] Mapping optimization actions include splitting operations, copying operations, and other operations.
[0069] In one embodiment, when the mapping optimization action corresponding to the first processing node is a splitting operation, the step of transforming the first processing node by the corresponding mapping optimization action to obtain at least one second processing node includes: splitting the first processing node into multiple child nodes, and treating each child node as a second processing node.
[0070] By splitting the first processing node into multiple child nodes, the load corresponding to the first processing node can be shared among the child nodes. Each child node can then be considered a second processing node, effectively improving the load balancing of the processing units mapped to that second processing node. For example, if the first processing node performs a convolution operation, it can be split into two child nodes: one for multiplication and one for addition, with each child node serving as a second processing node.
[0071] In another embodiment, when the mapping optimization action corresponding to the first processing node is a copy operation, the step of transforming through the corresponding mapping optimization action to obtain at least one second processing node includes: copying the first processing node and using the first processing node and the copied node as the second processing node.
[0072] Here, a duplicate node refers to a processing node that is identical to the first processing node, obtained by copying the first processing node. In this embodiment, copying the first processing node and using both the first processing node and the duplicated node as the second processing node can effectively ensure that each processing unit mapped to the second processing node can share the task, improve load balancing, and accelerate task processing efficiency.
[0073] Step S32: Copy the source configuration data of the first processing node at least once, and assign each copy of the source configuration data as the initial configuration data of a second processing node.
[0074] In this context, the source configuration data of the first processing node refers to the data that the first processing node depends on, including learnable data and / or non-learnable data. The initial configuration data of the second processing node refers to the data that the second processing node depends on. In this step, the second processing node is obtained by splitting or copying the first processing node. Therefore, the second processing node may still depend on the data of the corresponding first processing node. Thus, the source configuration data of the first processing node is copied at least once, and each copy of the source configuration data is allocated as the initial configuration data of one second processing node to effectively avoid data access conflicts caused by the increase in the number of nodes after mapping.
[0075] In some embodiments, there is a type of second processing node among the second processing nodes. This type of second processing node has little or no dependence on the unlearnable data in the source configuration data. For this type of second processing node, only the learnable data in the source configuration data of the corresponding first processing node can be copied, and the learnable data can be allocated as the initial configuration data of this type of second processing node.
[0076] Step S33: Map each second processing node and the initial configuration data corresponding to the second processing node into a processing unit.
[0077] Step S34: For each first processing node in the first processing node of the neural network to be mapped that does not require transformation, map the first processing node as a second processing node and the source configuration data of the first processing node as the initial configuration data of the second processing node into a processing unit.
[0078] In this context, the first processing node in the neural network to be mapped that does not require transformation refers to the processing node whose corresponding load is less than a preset load threshold. This preset load threshold can be set according to the actual application process; for example, it can be set based on the load that the processing unit can bear in actual application, or it can be determined based on the average load corresponding to all first processing nodes. This embodiment does not impose any specific limitations.
[0079] The method provided in this embodiment for mapping a neural network to be mapped to a device comprising multiple processing units improves the load balancing level of the processing units and the operating efficiency of the neural network compared to the traditional method of directly mapping each first processing node in the neural network to be mapped to a processing unit. In this embodiment, after mapping the neural network to be mapped to a device comprising multiple processing units, a first mapping strategy corresponding to the neural network to be mapped can be obtained.
[0080] Figure 4This is a schematic diagram of a first mapping strategy provided by an embodiment of the present disclosure. In one embodiment, the mapping strategy is... Figure 2 The neural network to be mapped shown is mapped onto a device that includes multiple processing units, and the results can be obtained as follows: Figure 4 The first mapping strategy is shown. (Refer to...) Figure 4 The first mapping strategy includes the second processing node mapped in the processing unit, the initial configuration data of the second processing node, the initial configuration data including learnable data and / or non-learnable data, and the connection relationship of each second processing node.
[0081] like Figure 4 As shown, the second processing node 42-1 mapped in processing unit 41-1 and the second processing node 42-2 mapped in processing unit 41-2 are... Figure 2 The first processing node 21-1 of the neural network to be mapped is obtained by splitting it. The initial configuration data 43-1 of the second processing node 42-1 and the initial configuration data 43-2 of the second processing node 42-2 are based on... Figure 2 The source configuration data of the first processing node 21-1 is obtained. This initial configuration data 43-1 includes non-learnable data 43-11 and learnable data 43-12, and this initial configuration data 43-2 includes non-learnable data 43-21 and learnable data 43-22. The second processing node 42-3 mapped in the processing unit 41-4 is obtained from... Figure 2 The first processing node 21-2 of the neural network to be mapped is obtained directly; the second processing node 42-4 mapped in processing unit 41-5 and the second processing node 42-5 mapped in processing unit 41-8 are the first processing node 21-3 and its copy node of the neural network to be mapped, respectively. The initial configuration data 43-4 of the second processing node 42-4 and the initial configuration data 43-5 of the second processing node 42-5 are obtained based on the source configuration data of the first processing node 21-3. Both the initial configuration data 43-4 and 43-5 are learnable data; the second processing node 42-6 mapped in processing unit 41-6 and the second processing node 42-7 mapped in processing unit 41-9 are obtained by splitting the first processing node 21-4 of the neural network to be mapped. The initial configuration data 43-6 of the second processing node 42-6 and the initial configuration data 43-7 of the second processing node 42-7 are obtained based on the source configuration data of the first processing node 21-4.
[0082] In this embodiment, by mapping the neural network to be mapped onto a device including multiple processing units to obtain the first mapping strategy corresponding to the neural network, the operating efficiency of the neural network can be improved, but it is difficult to improve the neural network's ability on the target task, such as accuracy and precision. In order to improve the neural network's ability on the target task, after obtaining the first mapping strategy corresponding to the neural network to be mapped, the following steps S12-S14 are executed in this embodiment.
[0083] Step S12: Construct an equivalent mapping neural network based on the first mapping strategy.
[0084] The equivalent mapping neural network is a neural network that is consistent with the neural network to be mapped after the mapping optimization action transformation, and the equivalent mapping neural network is composed of a second processing node.
[0085] Figure 5 This is a schematic diagram of a neural network retraining method provided in an embodiment of this disclosure.
[0086] In one implementation, such as Figure 5 As shown, the neural network 51 to be mapped is mapped to a device including multiple processing units to obtain the first mapping strategy 52 corresponding to the neural network to be mapped, and an equivalent mapping neural network 53 is constructed based on the first mapping strategy 52.
[0087] The neural network 51 to be mapped includes two first processing nodes, whose source configuration data are W1 and W2, respectively, where W1 and W2 are both learnable data. The first mapping strategy 52 includes second processing nodes mapped in the processing units. The second processing nodes mapped in the two processing units in the first column are obtained by splitting the first first processing node of the neural network 51 to be mapped. The initial configuration data of these two second processing nodes are W1 and W2, respectively. 11 and W 12 The W 11 and W 12 Based on W1. It should be noted that the identifier in this embodiment is set for the convenience of explaining the scheme, and the identifier does not limit any configuration data. The second processing node mapped in the second processing unit of the second column is directly obtained from the second first processing node of the neural network 51 to be mapped, and the initial configuration data of the second processing node is W2.
[0088] It should be noted that in the first mapping strategy, the number of post-processing nodes in the neural network to be mapped increases, and the corresponding configuration data also increases. Therefore, the equivalent mapped neural network built based on the first mapping strategy has more learnable data than the neural network to be mapped. That is, compared with the neural network to be mapped, the equivalent mapped neural network has a larger network capacity, stronger learning ability, and enhanced potential capabilities on the target task.
[0089] Step S13: Train the equivalent mapping neural network to obtain retraining configuration data for at least one second processing node.
[0090] In one possible implementation, the equivalent mapping neural network can be trained under the same training conditions as the neural network to be mapped. These training conditions include datasets or task environments, which can be virtual environments, such as mazes, office environments, or grid environments used in reinforcement learning tasks.
[0091] This disclosure does not limit the equivalent mapping neural network to being trained under the same training conditions as the neural network to be mapped. Any training condition that can improve the accuracy or precision of the equivalent mapping neural network constructed by the first mapping strategy after retraining can be used as a condition for retraining the equivalent mapping neural network.
[0092] Retraining configuration data refers to the new values of the configuration data for the second processing node learned during the training of the equivalent mapping neural network.
[0093] In this embodiment, the equivalent mapping neural network has more processing nodes and correspondingly more configuration data compared to the neural network to be mapped, meaning the equivalent mapping neural network has more learnable data. Therefore, the increased learnable data can be used to improve the accuracy of the equivalent mapping neural network on the target task, i.e., to train the equivalent mapping neural network to obtain retraining configuration data for at least one second processing node.
[0094] like Figure 5 As shown, after training the equivalent mapping neural network 53, a trained equivalent mapping neural network 54 is obtained. In this trained equivalent mapping neural network 54, the configuration data of the second processing node is changed from the initial configuration data W. 11 W 12 W2 and W2 were respectively updated to the retraining configuration data W' 11 W' 12 W'2, the retrained configuration data is the new value of the configuration data learned during the training process. Therefore, the equivalent mapping neural network 54 after training has higher accuracy on the target task compared with the equivalent mapping neural network 53 before training.
[0095] Step S14: Replace the initial configuration data of the second processing node in the first mapping strategy with the retraining configuration data of the second processing node to obtain the second mapping strategy of the equivalent mapping neural network.
[0096] The second mapping strategy is information used to represent the mapping status of the neural network to be mapped.
[0097] like Figure 5 As shown, the equivalent mapping neural network 54 includes retrained configuration data for at least one second processing node. The retrained configuration data of the second processing node is used to replace the initial configuration data of that second processing node in the first mapping strategy to obtain a second mapping strategy 55 for the equivalent mapping neural network. Compared to the first mapping strategy 52, the configuration data of the second processing node in this second mapping strategy 55 is changed from the initial configuration data W. 11 W 12 W2 and W2 were respectively updated to the retraining configuration data W' 11 W' 12 、W'2.
[0098] This disclosure provides a method for retraining a neural network. First, the neural network to be mapped is mapped onto a device including multiple processing units to obtain a first mapping strategy corresponding to the neural network to be mapped. The first mapping strategy includes second processing nodes mapped in the processing units and initial configuration data of the second processing nodes. At least one second processing node is obtained by transforming the first processing node of the neural network to be mapped, and the initial configuration data of the second processing node is obtained based on the source configuration data of the corresponding first processing node. Then, an equivalent mapping neural network is constructed based on the first mapping strategy. Finally, the equivalent mapping neural network is trained to obtain retraining configuration data of at least one second processing node, and the retraining configuration data of the second processing node is used to replace the initial configuration data of the second processing node in the first mapping strategy. This method can effectively stimulate the potential capabilities of the equivalent mapping neural network in the target task and improve the accuracy of the equivalent mapping neural network in the target task.
[0099] In one embodiment, the step of training the equivalent mapping neural network to obtain retrained configuration data for at least one second processing node includes: performing multiple iterations of iterative training on the equivalent mapping neural network, and updating the configuration data corresponding to at least a portion of the second processing nodes in each iteration, so as to obtain the retrained configuration data for the second processing nodes after the iterative training is completed.
[0100] Among them, at least some of the second processing nodes include all or some of the second processing nodes.
[0101] This disclosure provides a detailed description of the at least some second processing nodes, including both all and some of the second processing nodes, through the following embodiments. The description will be conducted under the same training conditions as the neural network to be mapped.
[0102] Figure 6 A flowchart illustrating a method for retraining a neural network as provided in this disclosure. In one embodiment, as... Figure 6 As shown, the step of training the equivalent mapping neural network under the same training conditions as the neural network to be mapped to obtain retraining configuration data for at least one second processing node includes: step S61.
[0103] Step S61: Under the same training conditions as the neural network to be mapped, perform multiple iterations of iterative training on the equivalent mapping neural network, and update the configuration data corresponding to each second processing node in each iteration, so as to obtain the retraining configuration data of the second processing node after the iterative training is completed.
[0104] Figure 7 This diagram illustrates a retraining method for a neural network provided in an embodiment of this disclosure. In one implementation, under the same training conditions as the neural network to be mapped, the neural network is retrained... Figure 5 The equivalent mapping neural network constructed in the process performs multiple iterations of iterative training. Figure 7 Taking iterative training, which includes three iterations, as an example, in the first iteration, the configuration data corresponding to the second processing node is changed by W. 11 W 12 W2 and W2 were updated to In the second iteration, the configuration data corresponding to the second processing node is updated as follows: In the third iteration, the configuration data corresponding to the second processing node is updated to... in, Configure retraining data for the second processing node.
[0105] The retraining method for neural networks provided in this embodiment performs multiple iterations of iterative training on the equivalent mapping neural network, and updates the configuration data corresponding to each second processing node in each iteration, which can effectively improve the accuracy of the trained equivalent mapping neural network on the target task.
[0106] Figure 8 A flowchart illustrating a method for retraining a neural network as provided in this disclosure. In one embodiment, as... Figure 8 As shown, the step of training the equivalent mapping neural network under the same training conditions as the neural network to be mapped to obtain retrained configuration data for at least one second processing node includes: step S81.
[0107] Step S81: Under the same training conditions as the neural network to be mapped, perform multiple iterations of iterative training on the equivalent mapping neural network, and update the configuration data corresponding to a portion of the second processing nodes in each iteration, so as to obtain the retraining configuration data of the second processing nodes after the iterative training is completed.
[0108] In this embodiment, the configuration data corresponding to a portion of the second processing nodes is updated in each iteration, rather than updating the configuration data corresponding to every second processing node. This reduces the amount of configuration data involved in the training process, speeds up the training process, and improves training efficiency.
[0109] In this embodiment, the method for updating the configuration data corresponding to a portion of the second processing nodes in each iteration includes either selecting a training method or alternating training methods.
[0110] The training method refers to pre-selecting a portion of the second processing nodes before performing multiple iterations of iterative training on the equivalent mapping neural network. In each iteration, only the configuration data corresponding to the pre-selected portion of the second processing nodes is updated, while the configuration data corresponding to the remaining second processing nodes is fixed. This "fixed" means that the data is neither trained nor updated.
[0111] Alternating training refers to dividing the second processing node obtained by transforming (splitting or copying) the first processing node into multiple groups. In each iteration, the configuration data corresponding to the processing node of one group is selected for updating. After multiple iterations of iterative training, each group is selected the same number of times.
[0112] Figure 9 This diagram illustrates a selective training method provided by an embodiment of the present disclosure. In one implementation, under the same training conditions as the neural network to be mapped, the training is performed on... Figure 5 The equivalent mapping neural network constructed in the process performs multiple iterations of iterative training. Figure 9 Taking iterative training, which includes three iterations, as an example, a second processing node 91 is pre-selected. In the first iteration, the configuration data corresponding to the second processing node 91 is changed by W. 11 Updated to The configuration data of the second processing node 92 and the second processing node 93 are fixed; in the second iteration, the configuration data corresponding to the second processing node 91 is updated to... The configuration data of the second processing node 92 and the second processing node 93 are fixed; in the third iteration, the configuration data corresponding to the second processing node 91 is updated to... in, Configure retraining data for the second processing node 91.
[0113] Figure 10 This diagram illustrates an alternating training method provided by an embodiment of the present disclosure. In one implementation, under the same training conditions as the neural network to be mapped, the training is performed on... Figure 5 The equivalent mapping neural network constructed in the process performs multiple iterations of iterative training. Figure 10 Taking iterative training consisting of four iterations as an example, the second processing nodes 91 and 92, obtained by transforming (splitting or copying) the first processing node, are divided into two groups. In the first iteration, the configuration data of the second processing nodes 91 and 93 are changed by W. 11 W2 and W2 were updated to The configuration data of the second processing node 92 is fixed; in the second iteration, the configuration data corresponding to the second processing node 92 and the second processing node 93 are updated respectively. The configuration data of the second processing node 91 is fixed; in the third iteration, the configuration data of the second processing node 91 and the second processing node 93 are updated to... The configuration data of the second processing node 92 is fixed; in the fourth iteration, the configuration data corresponding to the second processing node 92 and the second processing node 93 are updated respectively. The configuration data for the second processing node 91 is fixed. Among them, Configure retraining data for the second processing node.
[0114] It should be noted that this embodiment only provides an exemplary description of alternating training. There are other training methods that can achieve alternating training. For example, in the first and second iterations, the configuration data of the second processing node 91 is fixed while the configuration data of the second processing nodes 92 and 93 are updated. In the third and fourth iterations, the configuration data of the second processing node 92 is fixed while the configuration data of the second processing nodes 91 and 93 are updated.
[0115] In this embodiment, both selecting the training method and alternating training methods can reduce the amount of configuration data involved in the training process, accelerate the training process, and improve training efficiency. In practical applications, an appropriate training method can be selected according to the specific application scenario; this embodiment does not impose any limitations.
[0116] It should also be noted that, since the potential capability of the equivalent mapping neural network in the target task is brought about by the configuration data corresponding to the processing nodes increased by the transformation, in some embodiments, in the step of performing multiple iterations of iterative training on the equivalent mapping neural network and updating the configuration data corresponding to at least a portion of the second processing nodes in each iteration, the at least a portion of the second processing nodes may preferably include the second processing nodes obtained by transforming (splitting or copying) the first processing node, while the configuration data of the remaining second processing nodes is fixed, so as to speed up the training process and improve training efficiency.
[0117] In one embodiment, performing multiple iterations of iterative training on an equivalent mapping neural network, and updating the configuration data corresponding to at least a portion of the second processing nodes in each iteration, includes: for second processing nodes whose initial configuration data includes copied data, performing multiple iterations of iterative training on the copied data corresponding to at least a portion of the second processing nodes, and updating the configuration data corresponding to the second processing nodes in each iteration. Wherein, "at least a portion of the second processing nodes" includes all or a portion of the second processing nodes.
[0118] In one implementation, during iterative training of the replicated data corresponding to at least a portion of the second processing node, the initial configuration data of the second processing node, excluding the replicated data, is fixed.
[0119] In some embodiments, for a second processing node whose initial configuration data includes replicated data, multiple iterations of iterative training are performed on the replicated data corresponding to each second processing node, and the configuration data corresponding to each second processing node is updated in each iteration. For a second processing node whose initial configuration data does not include replicated data, the initial configuration data corresponding to each second processing node is fixed during the multiple iterations of iterative training.
[0120] For a detailed iterative process, please refer to the aforementioned document. Figure 6 and Figure 7 The embodiments shown are not described in detail here.
[0121] In other embodiments, for a second processing node whose initial configuration data includes copied data, multiple iterations of iterative training are performed on the copied data corresponding to a portion of the second processing node, and the configuration data corresponding to a portion of the second processing node is updated in each iteration; for the remaining second processing nodes, the initial configuration data corresponding to each of the remaining second processing nodes is fixed during the multiple iterations of iterative training, the remaining nodes refer to the remaining second processing nodes whose initial configuration data includes copied data, and the second processing nodes whose initial configuration data does not include copied data.
[0122] The iterative training of the replicated data corresponding to a portion of the second processing node can be performed using either a selective training method or an alternating training method. For detailed iterative processes, please refer to the preceding text. Figure 8-10 The embodiments shown are not described in detail here.
[0123] In one embodiment, the retraining method for the neural network further includes: loading an equivalent mapping neural network into a device comprising multiple processing units based on a second mapping strategy to execute the target task corresponding to the neural network to be mapped.
[0124] The target task includes any one of the following: image processing task, speech processing task, text processing task, and video processing task.
[0125] In the neural network retraining method provided in this embodiment, since the trained equivalent mapping neural network is more capable of performing the target task than the neural network to be mapped, the equivalent mapping neural network is loaded into a device including multiple processing units based on the second mapping strategy to execute the target task corresponding to the neural network to be mapped, which can improve the accuracy of the execution result obtained after executing the target task.
[0126] It is understood that the various method embodiments mentioned above in this disclosure can be combined with each other to form combined embodiments without violating the principle and logic. Due to space limitations, this disclosure will not elaborate further. Those skilled in the art will understand that in the above methods of specific implementation, the specific execution order of each step should be determined by its function and possible internal logic.
[0127] Figure 11 This is a schematic diagram of the structure of a neural network retraining device provided in an embodiment of this disclosure. Figure 11 As shown, the retraining device for the neural network includes: a mapping module 111, a construction module 112, a training module 113, and a replacement module 114.
[0128] The mapping module 111 is used to map the neural network to be mapped to a device including multiple processing units to obtain a first mapping strategy corresponding to the neural network to be mapped; the first mapping strategy includes a second processing node mapped in the processing unit and initial configuration data of the second processing node; at least one second processing node is obtained by transforming the first processing node of the neural network to be mapped, and the initial configuration data of the second processing node is obtained based on the source configuration data of the corresponding first processing node.
[0129] The construction module 112 is used to construct an equivalent mapping neural network based on the first mapping strategy, wherein the equivalent mapping neural network is composed of the second processing node.
[0130] Training module 113 is used to train the equivalent mapping neural network to obtain retraining configuration data for at least one of the second processing nodes.
[0131] Replacement module 114 is used to replace the initial configuration data of the second processing node in the first mapping strategy with the retraining configuration data of the second processing node, so as to obtain the second mapping strategy of the equivalent mapping neural network.
[0132] This disclosure provides a retraining apparatus for a neural network. A mapping module maps the neural network to be mapped onto a device comprising multiple processing units to obtain a first mapping strategy corresponding to the neural network. The first mapping strategy includes second processing nodes mapped in the processing units and initial configuration data for the second processing nodes. At least one second processing node is obtained by transforming a first processing node of the neural network to be mapped, and the initial configuration data of the second processing node is obtained based on the source configuration data of the corresponding first processing node. A construction module constructs an equivalent mapping neural network based on the first mapping strategy. A training module trains the equivalent mapping neural network to obtain retraining configuration data for at least one second processing node. A replacement module replaces the initial configuration data of the second processing node in the first mapping strategy with the retraining configuration data of the second processing node. This effectively stimulates the potential capabilities of the equivalent mapping neural network in the target task and improves the accuracy of the equivalent mapping neural network in the target task.
[0133] Figure 12 This is a block diagram of an electronic device provided in an embodiment of the present disclosure.
[0134] Reference Figure 12 This disclosure provides an electronic device, which includes: at least one processor 1201; at least one memory 1202; and one or more I / O interfaces 1203 connected between the processor 1201 and the memory 1202; wherein the memory 1202 stores one or more computer programs that can be executed by the at least one processor 1201, and the one or more computer programs are executed by the at least one processor 1201 to enable the at least one processor 1201 to perform the above-described neural network retraining method.
[0135] This disclosure also provides a computer-readable storage medium storing a computer program thereon, wherein the computer program, when executed by a processor / processor core, implements the aforementioned neural network retraining method. The computer-readable storage medium may be volatile or non-volatile.
[0136] This disclosure also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code, wherein when the computer-readable code is run in a processor of an electronic device, the processor in the electronic device executes the above-described neural network retraining method.
[0137] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).
[0138] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules, or other data). Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.
[0139] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.
[0140] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages such as Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.
[0141] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.
[0142] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.
[0143] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.
[0144] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.
[0145] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.
[0146] Example embodiments have been disclosed herein, and while specific terminology has been used, it is for illustrative purposes only and should be construed as such, and is not intended to be limiting. In some instances, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in connection with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in connection with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this disclosure as set forth by the appended claims.
Claims
1. A method for retraining a neural network, characterized in that, include: The neural network to be mapped is mapped to a device including multiple processing units to obtain a first mapping strategy corresponding to the neural network to be mapped; The first mapping strategy includes a second processing node mapped in the processing unit, and initial configuration data of the second processing node; at least one second processing node is obtained by transforming the first processing node of the neural network to be mapped, and the initial configuration data of the second processing node is obtained based on the source configuration data of the corresponding first processing node; wherein, at least a portion of the initial configuration data of the second processing node is a copy of the source configuration data of the first processing node; An equivalent mapping neural network is constructed based on the first mapping strategy, wherein the equivalent mapping neural network is composed of the second processing node; Train the equivalent mapping neural network to obtain retrained configuration data for at least one of the second processing nodes; The retraining configuration data of the second processing node is used to replace the initial configuration data of the second processing node in the first mapping strategy to obtain the second mapping strategy of the equivalent mapping neural network. The step of obtaining the second processing node includes: For each first processing node in the first processing node that needs to be transformed in the neural network to be mapped, a transformation is performed through a corresponding mapping optimization action to obtain at least one second processing node. The first processing node that needs to be transformed in the neural network to be mapped refers to the first processing node whose corresponding load is greater than or equal to a preset load threshold. The preset load threshold is set according to the maximum load that the processing unit can bear in actual application. The mapping optimization actions include splitting and copying operations; When the mapping optimization action corresponding to the first processing node is a splitting operation, the step of transforming through the corresponding mapping optimization action to obtain at least one second processing node includes: The first processing node is split into multiple child nodes so that each child node shares the load corresponding to the first processing node, and each child node is used as a second processing node. When the mapping optimization action corresponding to the first processing node is a copy operation, the step of transforming through the corresponding mapping optimization action to obtain at least one second processing node includes: Copy the first processing node, and use the first processing node and the copied node as the second processing node.
2. The retraining method according to claim 1, characterized in that, The step of training the equivalent mapping neural network to obtain retraining configuration data for at least one of the second processing nodes includes: The equivalent mapping neural network is subjected to multiple iterations of iterative training, and in each iteration, the configuration data corresponding to at least a portion of the second processing nodes is updated, so as to obtain the retraining configuration data of the second processing nodes after the iterative training is completed.
3. The retraining method according to claim 2, characterized in that, The process of performing multiple iterations of iterative training on the equivalent mapping neural network, and updating the configuration data corresponding to at least a portion of the second processing nodes in each iteration, includes: For the second processing node whose initial configuration data includes the copied data, multiple iterations of iterative training are performed on the copied data corresponding to at least a portion of the second processing node, and the configuration data corresponding to the second processing node is updated in each iteration.
4. The retraining method according to claim 3, characterized in that, The method further includes: During iterative training of the replicated data corresponding to at least a portion of the second processing node, the initial configuration data of the second processing node, excluding the replicated data, is fixed.
5. The retraining method according to claim 3 or 4, characterized in that, The iterative training methods include selective training or alternating training.
6. The retraining method according to claim 1, characterized in that, The step of mapping the neural network to be mapped onto a device including multiple processing units to obtain a first mapping strategy corresponding to the neural network to be mapped includes: The source configuration data of the first processing node is copied at least once, and each copy of the source configuration data is allocated as the initial configuration data of a second processing node. Each of the second processing nodes and the initial configuration data corresponding to the second processing node are respectively mapped to a processing unit; For each first processing node in the first processing node of the neural network to be mapped that does not require transformation, the first processing node is used as the second processing node and the source configuration data of the first processing node is used as the initial configuration data of the second processing node and mapped to a processing unit.
7. The retraining method according to claim 1, characterized in that, The method further includes: Based on the second mapping strategy, the equivalent mapping neural network is loaded into a device including multiple processing units to execute the target task corresponding to the neural network to be mapped.
8. A retraining device for a neural network, characterized in that, A method for retraining a neural network as described in any one of claims 1-7, comprising: A mapping module is used to map a neural network to be mapped onto a device including multiple processing units to obtain a first mapping strategy corresponding to the neural network to be mapped; the first mapping strategy includes a second processing node mapped in the processing unit and initial configuration data of the second processing node; at least one second processing node is obtained by transforming the first processing node of the neural network to be mapped, and the initial configuration data of the second processing node is obtained based on the source configuration data of the corresponding first processing node; A construction module is used to construct an equivalent mapping neural network based on the first mapping strategy, wherein the equivalent mapping neural network is composed of the second processing nodes; A training module is used to train the equivalent mapping neural network to obtain retraining configuration data for at least one of the second processing nodes; The replacement module is used to replace the initial configuration data of the second processing node in the first mapping strategy with the retraining configuration data of the second processing node, so as to obtain the second mapping strategy of the equivalent mapping neural network.
9. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, the one or more computer programs being executed by the at least one processor to enable the at least one processor to perform the retraining method of the neural network as described in any one of claims 1-7.
10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the retraining method of the neural network as described in any one of claims 1-7.