Method for dividing neural network, electronic device and storage medium
Patent Information
- Application Number
- CN202110853104.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-27
- Publication Date
- 2026-10-30
- Estimated Expiration
- 2041-07-27
AI Technical Summary
然而，在多个区域的某些区域中可能存在同一路径被划分到两个或两个以上的区域的情况，这使得同一路径下的两个或两个以上的区域在进行数据传输或合并时可能会产生数据丢失的情况
[0027]本申请公开的神经网络划分的方法、电子设备及计算机可读存储介质，能使划分的m个区域中同一路径的算子之间进行数据传输或合并得到更好的匹配，确保数据安全。
Smart Images
Figure CN115700594B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of electronic communication technology, and in particular to a method for neural network partitioning, an electronic device, and a storage medium. Background Technology
[0002] In modern electronic communication technologies, an increasing number of devices require increasingly complex processing of real-time inputs from the real world, such as industrial robots, autonomous vehicles, and mobile communication devices. These tasks largely fall under the umbrella of machine learning, involving mostly vector or matrix operations with extremely high parallelism. In common neural network algorithms, the neural network model can be divided into multiple regions, which can then operate in parallel. However, in some regions, the same path may be divided into two or more regions, potentially leading to data loss during data transmission or merging between these regions along the same path. Summary of the Invention
[0003] In view of the above, it is necessary to provide a method for neural network partitioning, an electronic device, and a computer-readable storage medium.
[0004] This application provides a method for neural network partitioning, the method comprising:
[0005] Obtain a neural network model, wherein the neural network model includes n operators;
[0006] Scan all operator subgroups in the neural network model;
[0007] The neural network model is divided into m regions;
[0008] Rescan all operator subgroups in the neural network model to identify corrupted operator subgroups;
[0009] Analyze the input and output of each operator in the corrupted operator subgroup to identify specific categories of operators; and
[0010] Adjust operators of specific categories to adjust the partitioned regions.
[0011] In one possible implementation, analyzing the input and output of each operator in the corrupted operator subgroup to identify operators of a specific category includes:
[0012] Identifying first-class operators whose inputs and outputs are inconsistent; and
[0013] Statistically analyze the second type of operators corresponding to the inputs of the first type of operators.
[0014] In one possible implementation, the first type of operator for identifying inconsistencies between input and output includes:
[0015] Analyze the input and output of each operator in the damaged operator subgroup, identify operators whose input and output are located in different regions, and classify these operators as first-class operators.
[0016] In one possible implementation, the second type of operator corresponding to the input of the first type of operator includes:
[0017] Analyze the operators corresponding to the inputs of each operator in the first type of operators, and classify these operators as the second type of operators.
[0018] In one possible implementation, the operator for adjusting a specific category to adjust the divided region includes:
[0019] The output of the second type of operator is adjusted to the region corresponding to the output of the first type of operator, so as to adjust the divided region.
[0020] In one possible implementation, each operator includes one or more inputs and one or more outputs, with the output of the preceding operator serving as the input of the following operator, the output of one operator serving as the input of one or more operators, and the input of one operator coming from the outputs of one or more operators.
[0021] In one possible implementation, scanning all operator subgroups in the neural network model includes:
[0022] The n operators in the neural network model are divided into m regions, where m is an integer of two or more, and each region includes a corresponding number of the operators.
[0023] In one possible implementation, rescanning all operator subgroups in the neural network model to identify corrupted operator subgroups includes:
[0024] Rescan all operator subgroups in the neural network model and identify operator subgroups located in more than one of the regions as corrupted operator subgroups.
[0025] This application also provides an electronic device, which includes a processor and a memory, wherein the processor is used to implement the neural network partitioning method when executing a computer program stored in the memory.
[0026] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the neural network partitioning method.
[0027] The neural network partitioning method, electronic device, and computer-readable storage medium disclosed in this application enable data transmission or merging of operators on the same path in the partitioned m regions to achieve better matching and ensure data security. Attached Figure Description
[0028] Figure 1 This is a flowchart of a preferred embodiment of a neural network partitioning method disclosed in this application.
[0029] Figure 2 This is a schematic diagram of an exemplary neural network model disclosed in this application.
[0030] Figure 3 This is a schematic diagram of an exemplary neural network model of a segmented region disclosed in this application.
[0031] Figure 4 This is a schematic diagram of the structure of an electronic device according to a preferred embodiment of the method for implementing neural network partitioning disclosed in this application.
[0032] Explanation of main component symbols
[0033] Neural Network Model 100
[0034] Operators Op1 to Op15
[0035] First operator subgroup 110
[0036] Second operator subgroup 120
[0037] Third operator subgroup 130
[0038] Area 1, T1
[0039] Second Zone T2
[0040] Electronic devices 4
[0041] Memory 41
[0042] Processor 42
[0043] Computer Program 43
[0044] The following detailed description, in conjunction with the accompanying drawings, will further illustrate this application. Detailed Implementation
[0045] To make the objectives, technical solutions, and advantages of this application clearer, the application will be described in detail below with reference to the accompanying drawings and specific embodiments.
[0046] like Figure 1The diagram shown is a flowchart of a preferred embodiment of the neural network partitioning method of this application. Depending on different requirements, the order of the steps in this flowchart can be changed, and some steps can be omitted.
[0047] The neural network partitioning method is applied to one or more electronic devices 4 (see reference). Figure 4 In this context, the electronic device 4 is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.
[0048] The electronic device 4 can be any electronic product that can interact with the user, such as a personal computer, tablet computer, smartphone, personal digital assistant (PDA), game console, interactive network television (IPTV), smart wearable device, etc.
[0049] The electronic device 4 may also include network devices and / or user devices. The network devices include, but are not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of hosts or network servers.
[0050] The network in which the electronic device 4 is located includes, but is not limited to, the Internet, wide area network, metropolitan area network, local area network, and virtual private network (VPN).
[0051] It should be noted that if the modules / units integrated in the electronic device 4 are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program code can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM).
[0052] For example, Figure 4 The illustrated electronic device 4 includes a memory 41 and a processor 42. The memory 41 stores a computer program 43. The processor 41, by running or executing the computer program 43 stored in the memory 42 and by accessing data stored in the memory 42, enables the processor 41 to implement a neural network partitioning method executed by the electronic device 4 and to control the memory 41 and various parts of the electronic device 4. The processor 42 can connect to the memory 41 and various parts of the electronic device 4 using various interfaces and buses.
[0053] S1. Obtain a neural network model, wherein the neural network model includes n operators.
[0054] In this embodiment, the neural network model can be a neural network model obtained directly from the network source, or it can be a neural network model obtained from the network source but optimized. Optimizing the neural network model can be understood as performing operations such as operator fusion, network pruning, model quantization, and network segmentation on the neural network model.
[0055] In this embodiment, the neural network model includes n operators, namely Op1, Op2, ..., Opn. Each operator includes one or more inputs and one or more outputs. For adjacent operators, the output of the preceding operator can be used as the input of the following operator. The output of one operator can be used as the input of one or more operators, and the input of one operator can come from the outputs of one or more operators.
[0056] S2. Scan all operator subgroups in the neural network model.
[0057] In this embodiment, the neural network model includes multiple operator subgroups, and each operator subgroup includes multiple interconnected operators. In this embodiment, all operator subgroups in the neural network model can be scanned using algorithms such as Directed Acyclic Graph (DAG), Depth-First Search (DFS), and / or Breadth-First Search (BFS).
[0058] For example, Figure 2 The neural network model 100 shown includes 15 operators, namely Op1, Op2, ..., Op15. By scanning the neural network model 100, three operator subgroups are obtained, namely the first operator subgroup 110, the second operator subgroup 120, and the third operator subgroup 130.
[0059] In this embodiment, an operator subgroup can be identified as Lx(f,e), where Lx is the series, f is the starting operator of the operator subgroup, and e is the ending operator of the operator subgroup. Scanning the neural network model 100 yields three operator subgroups identified as L1(Op1,Op3), L1(Op4,Op15), and L2(Op12,Op14), respectively.
[0060] S3, Divide the neural network model into m regions.
[0061] In this embodiment, the multiple operators in the neural network model are divided into m regions, where m is an integer of two or more, such that each region includes a considerable number of the operators.
[0062] For example, Figure 3 The neural network model 100 is divided into two regions: a first region T1 and a second region T2. Operators Op1, Op2, ..., Op7 are located in the first region T1, and operators Op8, Op9, ..., Op15 are located in the second region T2. Both the first region T1 and the second region T2 contain a roughly equal number of these operators.
[0063] S4. Rescan all operator subgroups in the neural network model to identify corrupted operator subgroups.
[0064] In this embodiment, all operator subgroups in the neural network model are rescanned to identify whether any operator subgroups are located in more than one region. That is, when an operator subgroup has some operators located in one partitioned region and other operators located in another partitioned region. In this embodiment, the neural network model can be executed by multiple processors simultaneously.
[0065] For example, Figure 3 In the three operator subgroups of the neural network model 100 shown, the first operator subgroup 110 is located in the first region T1, that is, all operators Op1, Op2, and Op3 in the first operator subgroup 110 are located in the first region T1; the third operator subgroup 130 is located in the second region T2, that is, all operators Op12, Op13, and Op14 in the third operator subgroup 130 are located in the second region T2; the second operator subgroup 120 is located in both the first region T1 and the second region T2, that is, some operators in the second operator subgroup 120, such as Op4, Op5, ..., are located in the first region T1, and some operators, such as Op9, Op10, ..., are located in the second region T2. Therefore, the second operator subgroup 120 is identified as a corrupted operator subgroup.
[0066] In this embodiment, the neural network model 100 is executed simultaneously by two processors, wherein the first region T1 and the second region T2 are each executed by one processor. Furthermore, the execution time for each processor to execute its corresponding region is the same to reduce waiting time and achieve load balancing.
[0067] S5. Analyze the input and output of each operator in the damaged operator subgroup, and identify the first type of operator whose input and output are inconsistent.
[0068] In this embodiment, the input and output of each operator in the damaged operator subgroup are analyzed to identify operators whose input and output are located in different regions. That is, when the output of an operator in the damaged operator subgroup is in one divided region, while the input comes from another divided region, such operators are classified as first-type operators.
[0069] For example, Figure 3In the second operator subgroup 120 of the neural network model 100 identified as a corrupted operator subgroup, the input and output of each operator Op4, Op6, ... are analyzed. Specifically, the output of operator Op4 is in the first region T1; the output of operator Op6 is in the first region T1; the output of operator Op7 is in the first region T1; the input of operator Op8 is in the first region T1, and the output is in the second region T2; the input of operator Op9 is in the first region T1, and the output is in the second region T2; the input of operator Op11 is in the first region T1, and the output is in the second region T2; the input of operator Op15 is in the first region T1, and the output is in the second region T2. Other operators whose input and output are clearly in the same region are not listed here. Therefore, it can be concluded that the input regions and output regions of operators Op8, Op9, Op11, and Op15 are inconsistent. That is, the inputs of operators Op8, Op9, Op11, and Op15 come from another region. Therefore, operators Op8, Op9, Op11, and Op15 are classified as first-class operators.
[0070] S6. Calculate the second type of operator corresponding to the input of the first type of operator.
[0071] In this embodiment, the operator corresponding to the input of each operator in the first type of operator is analyzed, and this type of operator is counted as the second type of operator.
[0072] For example, Figure 3 In the first type of operators shown, the input of operator Op8 comes from the output of operator Op7; the inputs of operators Op9 and Op11 come from the output of operator Op4; and the input of operator Op15 comes from the output of operator Op6. Therefore, operators Op4, Op6, and Op7 are classified as second type of operators.
[0073] S7. Adjust the divided areas.
[0074] In this embodiment, the output of the second type of operator is adjusted to the region corresponding to the output of the first type of operator, thereby adjusting the divided region.
[0075] For example, Figure 3 The outputs of the second type of operators Op4, Op6, and Op7 are adjusted to the region corresponding to the outputs of the first type of operators Op8, Op9, Op11, and Op15, which is the second region T2. The first region T1 and the second region T2 are adjusted accordingly.
[0076] Therefore, data transfer or merging between operators on the same path in the two adjusted regions can achieve better matching and ensure data security.
[0077] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements any of the steps described above for dividing a neural network.
[0078] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, methods, and computer-readable storage media can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and other division methods may be used in actual implementation.
[0079] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.
[0080] Furthermore, the functional modules in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in the form of hardware plus software functional modules.
[0081] It will be apparent to those skilled in the art that this application is not limited to the details of the exemplary embodiments described above, and that it can be implemented in other specific forms without departing from the spirit or essential characteristics of this application. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of this application is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be embraced within this application. No reference numerals in the claims should be construed as limiting the scope of the claims. Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a system claim may also be implemented by a single unit or device in software or hardware. The terms "second class" are used to indicate names and do not indicate any particular order.
[0082] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of this application without departing from the spirit and scope of the technical solutions of this application.
Claims
1. A method for neural network partitioning, applied in electronic devices, characterized in that, The method for partitioning the neural network includes: Obtain a neural network model, wherein the neural network model includes n operators; Scan all operator subgroups in the neural network model; The neural network model is divided into m regions, where m is an integer of two or more, such that each region includes a similar number of operators, and the neural network model is configured to be executed simultaneously by m processors, with each region assigned to one processor for execution, and each processor executing the corresponding region taking the same amount of time to reduce waiting time and achieve load balancing. Rescan all operator subgroups in the neural network model and identify operator subgroups located in more than one of the regions as corrupted operator subgroups; Identify a first type of operator whose input and output are inconsistent; and statistically analyze the second type of operator corresponding to the input of the first type of operator; and Adjust operators of specific categories to adjust the divided regions, so that data transmission between operators on the same path after adjustment can achieve better matching.
2. The method for neural network partitioning according to claim 1, characterized in that, The first type of operator for identifying inconsistencies between input and output includes: Analyze the input and output of each operator in the damaged operator subgroup, identify operators whose input and output are located in different regions, and classify these operators as first-class operators.
3. The method for neural network partitioning according to claim 2, characterized in that, The second type of operator corresponding to the input of the first type of operator includes: Analyze the operators corresponding to the inputs of each operator in the first type of operators, and classify these operators as the second type of operators.
4. The method for neural network partitioning according to claim 3, characterized in that, The operators that adjust specific categories to adjust the divided regions include: The output of the second type of operator is adjusted to the region corresponding to the output of the first type of operator, so as to adjust the divided region.
5. The method for neural network partitioning according to claim 1, characterized in that, Each of the operators includes one or more inputs and one or more outputs. For adjacent operators, the output of the preceding operator serves as the input of the following operator. The output of one operator serves as the input of one or more operators. The input of one operator comes from the outputs of one or more operators.
6. An electronic device, characterized in that, The electronic device includes a processor and a memory, the processor being configured to execute a computer program stored in the memory to implement the neural network partitioning method as described in any one of claims 1 to 5.
7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores at least one instruction, which, when executed by a processor, implements the neural network partitioning method as described in any one of claims 1 to 5.