A data processing method, an electronic device, and a storage medium

Abstract

The application relates to the technical field of artificial intelligence, and discloses a data processing method, an electronic device and a storage medium. In the data processing method, part of connected processing layers in a model is divided into a processing layer set, and parameters corresponding to all processing layers in the processing layer set are divided into a parameter set, thereby obtaining a plurality of parameter sets. When a multi-core NPU performs data processing, the split information in the storage can be read, and based on the split information, different parameter sets corresponding to different processing layer sets can be loaded simultaneously by using a plurality of cores in the multi-core NPU, and input data of the processing layer set can be processed respectively. In this way, by simultaneously using a plurality of cores in the multi-core NPU to perform data processing, the waste of computing power resources of the multi-core NPU can be reduced under the condition that the input and output data before and after model splitting are the same.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a data processing method, electronic device, and storage medium. Background Technology

[0002] Generally speaking, by deploying the model on a multi-core embedded neural network processing unit (NPU), the kernels in the multi-core NPU can process the input data and obtain the output data by loading the parameters corresponding to each layer in the model. Figure 1 A partial structural diagram of a recurrent neural network model is shown, such as... Figure 1 As shown, the recurrent neural network model may include an input layer 110, a hidden layer 120, an output layer 130, etc.

[0003] However, in existing data processing schemes, due to the strong coupling and low parallelism between model layers, multi-core NPUs generally use a single core to load the parameters corresponding to all layers in the model to process the input data, which results in a waste of the computing resources of multi-core NPUs. Summary of the Invention

[0004] To address the problem of wasted computing resources in multi-core embedded neural network model processors, embodiments of this application provide a data processing method, an electronic device, and a storage medium.

[0005] The first aspect of this application provides a data processing method. The electronic device includes a first processor, which includes multiple cores. The method includes: acquiring first input data of a first model; dividing multiple processing layers in the first model into multiple processing layer sets matching the segmentation information based on the segmentation information of the first model, wherein processing layers in the same processing layer set satisfy valid segmentation conditions, and the valid segmentation conditions are related to the running performance of the first model; and using the cores corresponding to each processing layer set to process each part of the first input data to be processed by each processing layer set.

[0006] Based on the above scheme, by using multiple cores in a multi-core NPU for data processing simultaneously, the waste of computing resources of the multi-core NPU can be reduced while ensuring that the input and output data are the same before and after model segmentation.

[0007] It is understood that the first processor can be a multi-core embedded neural network model processor. The first model is a neural network model. The first input data can include image data, text data, and audio data. Processing each sub-data can include image feature extraction from image data, text feature extraction from text data, and audio feature extraction from audio data, resulting in pixels, characters, audio signals, etc.

[0008] It can be understood that information partitioning can divide partially connected processing layers in a model into a set of processing layers, and then divide the parameters corresponding to all processing layers in that set into a parameter set, resulting in multiple parameter sets. These multiple parameter sets represent multiple sets of parameters that can process data in parallel.

[0009] In one possible implementation of the first aspect above, the segmentation information is obtained by: determining a first processing layer and a second processing layer from multiple processing layers in the first model, wherein the first processing layer and the second processing layer are connected; determining whether the first processing layer and the second processing layer satisfy the valid segmentation condition; corresponding to the first processing layer and the second processing layer satisfying the valid segmentation condition, dividing the first processing layer and the second processing layer into a first processing layer set; corresponding to the first processing layer and the second processing layer not satisfying the valid segmentation condition, dividing the first processing layer into a first processing layer set, and dividing the second processing layer into a second processing layer set.

[0010] For example, for a processing layer labeled 9 in the model, the processing layer labeled 9 and the processing layer labeled 8 can be placed into a temporary processing layer set, i.e., a temporary subgraph as described below. A processor loads the parameters corresponding to all processing layers in the temporary processing layer set and processes the input data of the processing layer set (i.e., the output data of the processing layer labeled 7). The performance metrics for processing the input data and the redundancy rate for repeated processing of some data are obtained. When the performance metrics for processing the input data are higher than the performance metric threshold, and the redundancy rate for repeated processing of data is lower than the redundancy rate threshold (e.g., a redundancy rate of 0.2 is lower than the redundancy rate threshold of 0.6), the processing layer labeled 9 and the processing layer labeled 8 can be classified into the same processing layer set.

[0011] In one possible implementation of the first aspect above, the method further includes: determining a third processing layer from multiple processing layers in the first model, wherein the third processing layer is connected to the first processing layer; determining whether the first processing layer, the second processing layer, and the third processing layer satisfy the effective segmentation condition; and dividing the first processing layer, the second processing layer, and the third processing layer into a set of first processing layers corresponding to the first processing layer satisfying the effective segmentation condition.

[0012] In one possible implementation of the first aspect above, the effective segmentation condition includes at least one of the following: when the first processor loads the parameters corresponding to all processing layers in the same processing layer set and processes the part of the data to be processed in the same processing layer set of the first input data, the performance index is higher than the performance index threshold; when the first processor loads the parameters corresponding to all processing layers in the same processing layer set and processes the part of the data to be processed in the same processing layer set of the first input data, the redundancy rate of repeated processing of the part of the data is lower than the redundancy rate threshold.

[0013] For example, a maximum redundancy rate, or redundancy rate threshold, can be set to 0.6. When the redundancy rate for partially repeated data processing is below 0.6, it can be determined that the first and second processing layers meet the effective partitioning conditions. When the redundancy rate for partially repeated data processing is above 0.6, it can be determined that the first and second processing layers do not meet the effective partitioning conditions.

[0014] In one possible implementation of the first aspect above, the kernel corresponding to the processing layer set includes a first kernel and a second kernel. The kernel corresponding to each processing layer set is used to process each part of the first input data to be processed by each processing layer set, including: using the first kernel to process the first sub-data in the first input data to be processed by the first processing layer set, and using the second kernel to process the second sub-data in the first input data to be processed by the first processing layer set.

[0015] For example, the first processor includes a first core and a second core. The model may include 10 processing layers (such as...). Figure 3 The layers identified as 0-9 can be configured in the memory to divide the processing layers identified as 0, 1, 2, 5, 6, and 7 into the first processing layer set, the processing layers identified as 3 and 4 into the second processing layer set, and the processing layers identified as 8 and 9 into the third processing layer set.

[0016] When processing data, the first core of the multi-core NPU can load the first parameter set corresponding to the first processing layer set to process the input data of the first processing layer set (i.e., all or part of the input data of the model). Simultaneously, the second core of the multi-core NPU loads the second parameter set corresponding to the second processing layer set to process the input data of the second processing layer set (i.e., all or part of the input data of the model). After the first core loads the first parameter set corresponding to the first processing layer set and processes the input data of the first processing layer set, the third parameter set corresponding to the third processing layer set can be loaded to process the input data of the third processing layer set (i.e., the output data of the first and second processing layer sets). Alternatively, the second core of the multi-core NPU can load the third parameter set corresponding to the third processing layer set, or other cores of the multi-core NPU besides the first and second cores can load the third parameter set corresponding to the third processing layer set to process the input data of the third processing layer set (i.e., the output data of the first and second processing layer sets).

[0017] In one possible implementation of the first aspect above, using the first kernel to process the first sub-data in the first input data to be processed by the first processing layer set includes: using the first kernel to load the first parameter set corresponding to all layers in the first processing layer set and processing the first sub-data in the first input data; and using the second kernel to process the second sub-data in the first input data to be processed by the first processing layer set includes: using the second kernel to load the second parameter set corresponding to all layers in the second processing layer set and processing the second sub-data in the first input data.

[0018] In one possible implementation of the first aspect described above, the method further includes: storing the first sub-data in a first internal storage space of the first kernel, and storing the second sub-data in a second internal storage space of the second kernel.

[0019] It is understandable that while storing the first sub-data in the first internal storage space of the first kernel, the first output sub-data obtained by the first processing of the first sub-data can also be stored in the first internal storage space of the first kernel.

[0020] In this embodiment, by storing the data corresponding to the first kernel in the internal storage space of the first kernel, compared to processing the entire first input data by a single kernel and storing it in the internal storage space of a single kernel, the storage pressure on a single kernel can be alleviated. Furthermore, compared to storing it in external memory, data can be read directly from the internal memory of the kernel without having to read data from external memory via a bus. This can alleviate the storage pressure on a single kernel, reduce the number of accesses to external memory, improve data processing efficiency, and reduce data read latency.

[0021] In one possible implementation of the first aspect above, the first processor is a multi-core embedded neural network model processor, the first model is a neural network model, and the first input data includes image data, text data, and audio data.

[0022] In a second aspect, this application provides an electronic device, including: a memory for storing instructions executed by one or more processors of the electronic device, and a processor, which is one of the one or more processors of the electronic device, for executing the data processing method mentioned in this application.

[0023] Thirdly, this application provides a readable storage medium storing instructions that, when executed on an electronic device, cause the electronic device to perform the data processing method mentioned in this application. Attached Figure Description

[0024] Figure 1 Based on some examples of this application, a partial structural schematic diagram of a recurrent neural network model is shown;

[0025] Figure 2 Based on some examples of this application, a schematic diagram of an application scenario is shown;

[0026] Figure 3 Based on some examples of this application, a schematic diagram of segmentation information for segmenting the processing layer in a model is shown;

[0027] Figure 4 Based on some examples of this application, a flowchart of a data processing method is shown;

[0028] Figure 5 Based on some examples of this application, a flowchart of a model segmentation method is shown;

[0029] Figure 6 Based on some examples of this application, a flowchart of a method for determining a set of target layers is shown;

[0030] Figure 7 Based on some examples of this application, a schematic diagram of the structure of a model is shown;

[0031] Figure 8 Based on some examples of this application, a schematic diagram of the structure of an electronic device is shown. Detailed Implementation

[0032] The illustrative embodiments of this application include, but are not limited to, a data processing method, an electronic device, and a storage medium.

[0033] It is understood that the data processing methods mentioned in the embodiments of this application can be used in neural network models, which can be applied in many fields, such as text processing, image processing, audio processing, and autonomous driving. Figure 2 As shown, in some specific implementations, server 210 can perform segmentation processing on neural network model 211 and send the segmented neural network model 211 to terminal 220, so that terminal 220 can execute the data processing method mentioned in the embodiments of this application based on the segmented neural network model 211.

[0034] To address the aforementioned problems, this application discloses a data processing method. In this method, segmentation information for dividing parameters in a model is stored in memory. For example, partially connected processing layers in the model are divided into a processing layer set, and the parameters corresponding to all processing layers in this set are divided into a parameter set, resulting in multiple parameter sets. These multiple parameter sets are multiple parameter sets that can process data in parallel. When a multi-core NPU performs data processing, the segmentation information in memory can be read, and based on this information, multiple cores in the multi-core NPU can simultaneously load different parameter sets corresponding to different processing layer sets, and process the input data of each processing layer set separately. Thus, by simultaneously using multiple cores in the multi-core NPU for data processing, the waste of computing resources in the multi-core NPU can be reduced while ensuring that the input and output data are the same before and after model segmentation.

[0035] Figure 3 This diagram illustrates the segmentation information for dividing the processing layers in a model, such as... Figure 3 As shown, the model can include 10 processing layers (such as...) Figure 3 The processing layers identified by layer 0-9 can be configured in the memory to divide the processing layers identified by layer 0, 1, 2, 5, 6, and 7 into the first processing layer set, the processing layers identified by layer 3 and 4 into the second processing layer set, and the processing layers identified by layer 8 and 9 into the third processing layer set.

[0036] based on Figure 3The segmentation information shown can be used to load parameters corresponding to all processing layers in the first processing layer set using the first core of the multi-core NPU, and simultaneously load parameters corresponding to all processing layers in the second processing layer set using the second core of the multi-core NPU. After the first core has loaded the parameters corresponding to all processing layers in the first processing layer set and processed the data, the first core of the multi-core NPU can load parameters corresponding to all processing layers in the third processing layer set.

[0037] In some optional instances, the specific method for splitting the parameters in the model can be:

[0038] Based on the first processing layer (any processing layer) in the model, create a first processing layer set, and add the second processing layer connected to the first processing layer to this first processing layer set. If the first processing layer set satisfies the valid partitioning condition, then the first and second processing layers are grouped into a single processing layer set, i.e., the first processing layer set. If the first processing layer set does not satisfy the valid partitioning condition, then create a second processing layer set based on the second processing layer, and add both the first and second processing layers to the first and second processing layer sets, respectively. Then, using the same method, the remaining processing layers in the model can be added to either the first or second processing layer set, or a new processing layer set can be created, until all processing layers in the model are in any processing layer set. Finally, the parameters corresponding to all processing layers in each processing layer set are grouped into a single parameter set.

[0039] In some optional instances, effective splitting criteria are related to the model's runtime performance. For example, effective splitting criteria may include the processor loading parameters corresponding to all processing layers in the processing layer set, the performance metrics when processing the data being processed being higher than a performance metric threshold, and the redundancy rate of repeatedly processing some data being lower than a redundancy rate threshold.

[0040] For ease of explanation, the processing layer will be referred to as the layer below.

[0041] The data processing methods mentioned in the embodiments of this application are described below. Figure 4 A flowchart illustrating a data processing method is shown, such as... Figure 4 As shown, the data processing method may include:

[0042] 401: Store the segmentation information in memory as a configuration file.

[0043] It can be understood that partitioning information refers to the information used to partition the parameters in a model. This partitioning information can be information that divides partially connected layers in the model into a layer set, and then partitions the parameters corresponding to all layers in that layer set into a parameter set, also known as subgraph topology information. Multiple parameter sets are multiple parameter sets that can be processed in parallel. When processing data on a multi-core NPU, the partitioning information can be read from memory, and based on this information, multiple cores in the multi-core NPU can simultaneously load different parameter sets corresponding to different layer sets, and process the input data of each layer set separately. In this way, by simultaneously using multiple cores in the multi-core NPU for data processing, the waste of computing resources of the multi-core NPU can be reduced while ensuring that the input and output data are the same before and after model partitioning.

[0044] For example, such as Figure 3 As shown, the model can include 10 layers (e.g. Figure 3 The layer identifiers (0-9) can be configured in memory to group layers 0, 1, 2, 5, 6, and 7 into the first layer set, layers 3 and 4 into the second layer set, and layers 8 and 9 into the third layer set. This subgraph topology information can then be stored in memory as a configuration file.

[0045] In some optional instances, the partitioning information can be stored as a configuration file in the internal storage of the multi-core NPU, or it can be stored as a configuration file in external memory, such as Double Data Rate (DDR) memory.

[0046] 402: The configuration file is read from memory while processing the data.

[0047] It is understandable that when processing data, a multi-core NPU can read configuration files from its internal storage or from external storage.

[0048] 403: Based on the segmentation information in the configuration file, multiple cores in a multi-core processor are used to process the input data to obtain the output data.

[0049] It is understandable that after reading the configuration file, based on the partitioning information in the configuration file, i.e. the subgraph topology information, multiple cores in the multi-core NPU can simultaneously load the parameters corresponding to all layers in different layer sets, and process the input data of the layer sets respectively.

[0050] As mentioned above Figure 3In some optional instances, after reading the configuration file, the first core of the multi-core NPU can load the first parameter set corresponding to the first layer set to process the input data of the first layer set (i.e., all or part of the input data of the model). Simultaneously, the second core of the multi-core NPU loads the second parameter set corresponding to the second layer set to process the input data of the second layer set (i.e., all or part of the input data of the model). After the first core loads and processes the input data of the first layer set, the third parameter set corresponding to the third layer set can be loaded to process the input data of the third layer set (i.e., the output data of the first and second layers). Alternatively, the second core of the multi-core NPU can load the third parameter set corresponding to the third layer set, or other cores in the multi-core NPU besides the first and second cores can load the third parameter set corresponding to the third layer set to process the input data of the third layer set (i.e., the output data of the first and second layers).

[0051] The data processing methods mentioned in the embodiments of this application are described below. Figure 5 A flowchart illustrating a model segmentation method is shown, such as... Figure 5 As shown, the model segmentation method may include:

[0052] 501: Determine the set of layers to be processed from the model.

[0053] It is understandable that the set of layers to be processed can include at least one layer in the model, such as the output layer or convolutional layer in a neural network model. In some optional instances, all layers of the model can be traversed. If the current layer being traversed is the output layer of the model, it can be considered as a layer to be processed. If the current layer is determined by the judgment module to be an unsuitable layer for segmentation, its parent layer can be considered as a layer to be processed. The output data of the parent layer serves as the input data of the current layer.

[0054] In some optional implementations, the decision module may store a list of supported splits, which can include multiple layer types. If the layer type corresponding to the current layer exists in the list of supported splits, the current layer is determined to be a suitable layer for splitting. If the layer type corresponding to the current layer does not exist in the list of supported splits, the current layer is determined to be an unsuitable layer for splitting.

[0055] In some optional instances, all layers of the model can be traversed. If the current layer encountered is the model's output layer, or if the current layer is determined by the decision module to be a suitable layer for splitting, then the current layer can be designated as the root layer. Thus, by traversing all layers of the model, a set of root layers can be obtained, denoted as root nodes. Then, the root nodes can be traversed to determine whether the current root layer encountered is a suitable layer for splitting.

[0056] If the result is yes, the current root layer is treated as a layer to be processed, i.e., a pooling layer, denoted as `base_merge_node`. If the result is no, it is further determined whether the parent layer of the current root layer is a suitable layer for splitting. If the result is yes, the parent layer of the current root layer is treated as a layer to be processed, i.e., a pooling layer, denoted as `base_merge_node`. Thus, all `base_merge_node`s (excluding duplicates) can be used as the set of layers to be processed, i.e., the set of pooling layers, denoted as `base_merge_nodes`.

[0057] In this embodiment of the application, by first obtaining the root-level layer set and then determining whether the root-level layer in the root-level layer set is a suitable layer for segmentation, the accuracy of obtaining the layer to be processed as the root-level layer can be guaranteed, and the accuracy of subsequent segmentation of the layers in the model can be improved.

[0058] 502: Determine the index layer from the set of layers to be processed.

[0059] It is understandable that the index layer can be any layer in the set of layers to be processed, denoted as mp.

[0060] In some optional instances, the set of layers to be processed can be copied to a copy set of layers, i.e., a new set, denoted as rest_merge_modes, and the index layer mp can be determined from the new set rest_merge_modes.

[0061] In some optional instances, a matching layer set, denoted as `match nodes`, can be defined to record which layers have been split (i.e., which layers have been assigned to the same layer set). This `match nodes` set can be initialized to empty. Simultaneously, a valid subgraph set, denoted as `valid subgraphs`, can be defined to record which layer sets have been split (i.e., multiple layers are assigned to the same layer set). This valid subgraph set can also be initialized to empty.

[0062] 503: Determine whether the index layer belongs to the matching layer set. If the result is no, proceed to step 504; otherwise, proceed to step 502.

[0063] In some optional instances, the layer identifier of the index layer can be matched with the layer identifiers in the matching layer set. If the layer identifier of the index layer is not found in the layer identifiers of the matching layer set, it can be determined that the index layer does not belong to the matching layer set, and the process proceeds to step 504, which determines the target layer set based on the index layer and all its parent layers. If the layer identifier of the index layer is found in the layer identifiers of the matching layer set, it can be determined that the index layer belongs to the matching layer set, and the process proceeds to step 502, which redetermines the index layer from the unprocessed layer set and repeats the step of determining whether the index layer belongs to the matching layer set until the redetermined index layer belongs to the matching layer set.

[0064] 504: Determine the target layer set based on the index layer and all its parent layers.

[0065] It's understandable that for a segmented model, if the input data is segmented in the H or W directions (where H represents the input data in the vertical direction and W represents the input data in the horizontal direction), there might be overlap in the input data segments. This could lead to the model repeatedly processing parts of the input data, such as convolutional layers with kernels larger than the stride. Furthermore, if the input data segments overlap, the output data will also overlap. Here, the ratio of the computational cost of repeatedly processing overlapping data to the total computational cost can be defined as the redundancy rate, or overlap. The maximum allowed redundancy rate, or redundancy rate threshold, is denoted as overlap_thr. Optionally, the maximum redundancy rate, or redundancy rate threshold, can be preset to 0.6.

[0066] It can be understood that the target layer set refers to a set of layers that divide multiple connections in the model into a single set, which can be placed into a set of valid subgraphs. Furthermore, the processor loads the parameters corresponding to all layers in this set, and the performance when processing the input data of the target layer set, as well as the redundancy rate of repeated processing of some data, both meet the requirements. For example, the performance metrics when processing the input data of the target layer set are higher than the performance metric threshold, and the redundancy rate of repeated processing of some data is lower than the redundancy rate threshold.

[0067] In some optional instances, a subgraph, denoted as `subgraph`, can be defined to record which layers can be divided into the same set of layers. The index layer `mp` determined from the set of layers to be processed is taken as the first element in the subgraph, i.e., the first layer. Furthermore, a candidate layer set, denoted as `candidate nodes`, can be defined to record which layers are candidate layers for the index layer. Candidate layers for the index layer can be all parent layers of the index layer, where the input data of the parent layers is the output data of the index layer.

[0068] It is understandable that we can iterate through all the candidate layers in the candidate node set until the candidate node set is empty, thus obtaining the target layer set.

[0069] The following details the specific method for obtaining the target layer set by iterating through all layers in the candidate layer set. Figure 6 A flowchart illustrating a method for determining a set of target layers is shown, such as... Figure 6 As shown, the method for determining the target layer set includes:

[0070] 601: Determine the candidate layer to be processed from the candidate layer set of the index layer.

[0071] It is understandable that the candidate layer to be processed can be any layer in the candidate layer set, denoted as a candi node.

[0072] 602: Determine whether the candidate layer to be processed meets the partitioning conditions.

[0073] Determine whether the candidate layer to be processed meets the partitioning conditions. If the result is yes, proceed to step 603; otherwise, proceed to step 601.

[0074] It is understandable that the partitioning criteria can be that the candidate layer node is not in the matching layer set match nodes, not in the unprocessed layer set base_merge_nodes, and not in the subgraph.

[0075] In some optional instances, the layer identifier of the candidate layer to be processed can be matched with the layer identifier in the matching layer set, the layer identifier of the candidate layer to be processed can be matched with the layer identifier in the set of layers to be processed, and the layer identifier of the candidate layer to be processed can be matched with the layer identifier in the subgraph.

[0076] In some optional instances, if there is no layer identifier for the candidate layer to be processed in the layer identifiers of the matching layer set, and there is no layer identifier for the candidate layer to be processed in the layer identifiers of the layer set to be processed, and there is no layer identifier for the candidate layer to be processed in the layer identifiers of the subgraph, it can be determined that the candidate layer to be processed meets the partitioning condition, and then proceed to step 603, that is, put the index layer and the candidate layer to be processed into the temporary subgraph.

[0077] In some alternative instances, if the layer identifier of the candidate layer to be processed exists in the layer identifiers of the matching layer set, or if the layer identifier of the candidate layer to be processed exists in the layer identifiers of the layer set to be processed, or if the layer identifier of the candidate layer to be processed does not exist in the layer identifiers of the subgraph, it can be determined that the candidate layer to be processed does not meet the partitioning condition. Then, proceed to step 601, that is, redetermine the candidate layer to be processed from the candidate layer set, and repeat the step of determining whether the candidate layer to be processed meets the partitioning condition until the redetermined candidate layer to be processed meets the partitioning condition.

[0078] 603: Place the index layer and the candidate layer to be processed into a temporary subgraph, and determine whether the temporary subgraph meets the valid splitting conditions.

[0079] Place the index layer and the candidate layer to be processed into a temporary subgraph, and determine whether the temporary subgraph meets the valid splitting conditions. If the result is yes, proceed to step 604; otherwise, proceed to step 605.

[0080] It is understandable that valid partitioning conditions may include the processor loading parameters corresponding to all layers in the temporary subgraph, the performance of processing the input data of the temporary subgraph, and the redundancy rate of repeated processing of some data meeting the requirements. For example, the performance index of processing the input data of the temporary subgraph is higher than the performance index threshold, and the redundancy rate of repeated processing of some data is lower than the redundancy rate threshold.

[0081] In some optional instances, a temporary subgraph, called subgraph_tmp, can be defined to record which layers are temporarily divided into a set of layers.

[0082] In some optional instances, the processor can be used to load the parameters corresponding to all layers in the temporary subgraph to obtain the performance metrics when processing the input data of the temporary subgraph and the redundancy rate of repeated processing of some data. When the performance metrics when processing the input data of the temporary subgraph are higher than the performance metric threshold and the redundancy rate of repeated processing of some data is lower than the redundancy rate threshold, it can be determined that the temporary subgraph meets the valid segmentation conditions. Then, proceed to step 604, that is, add the candidate node to be processed into the subgraph, add the parent layer of the candidate node to be processed into the candidate node set, remove the candidate node to be processed from the candidate node set, and repeat step 601 until the candidate node set is empty.

[0083] In some alternative instances, if the performance metrics when processing the input data of the temporary subgraph are lower than the performance metric threshold, or if the redundancy rate of repeated processing of some data is higher than the redundancy rate threshold, it can be determined that the temporary subgraph does not meet the effective splitting conditions. Then, proceed to step 605, that is, put the candidate layer node to be processed into the replication layer set rest_merge_modes, delete the candidate layer node to be processed from the candidate layer set candidate nodes, and repeat step 601 until the candidate layer set is empty.

[0084] 604: Add the candidate layer to be processed to the subgraph, add the parent layer of the matching layer to be processed to the candidate layer set, remove the candidate layer to be processed from the candidate layer set, and go to step 601 until the candidate layer set is empty.

[0085] In some optional instances, when a temporary subgraph is determined to meet the valid splitting conditions, the candidate node to be processed can be formally added to the subgraph. At the same time, the candidate node to be processed can be matched with the layer set, and the parent layer of the candidate node to be processed can be added to the candidate layer set candidate nodes. Meanwhile, the candidate node to be processed can be removed from the candidate layer set candidate nodes.

[0086] 605: Add the candidate layer to be processed to the copy layer set, and remove the candidate layer to be processed from the candidate layer set.

[0087] Add the candidate layer to be processed to the copy layer set, remove the candidate layer to be processed from the candidate layer set, and repeat step 601 until the candidate layer set is empty.

[0088] In some optional instances, when it is determined that the temporary subgraph does not meet the valid splitting conditions, the candidate node to be processed can be added to the set of replicated layers rest_merge_modes, and the candidate node to be processed can be removed from the set of candidate nodes.

[0089] 606: Determine the subgraph as the target layer set.

[0090] It is understandable that this subgraph can be identified as a valid subgraph and added to the set of valid subgraphs.

[0091] The above data segmentation method will be explained in detail below with a specific example. Figure 7 A schematic diagram of the structure of a model is shown.

[0092] In some specific implementations, all layers of the model can be traversed, and layers that are suitable for splitting (such as output layers, convolutional layers, etc.) can be determined as the pooling layer set base_merge_nodes. The pooling layer set base_merge_nodes can include 10 pooling layers with layer identifiers from 0 to 9.

[0093] If the current layer being traversed is layer with layer identifier 9, and layer with layer identifier 9 is not in the matching layer set `matchnodes`, then layer with layer identifier 9 in the merged layer set `base_merge_nodes` is used as the index layer `mp`, and the index layer with layer identifier 9 is placed into the temporary subgraph `subgraph`, as follows. Figure 7 The dashed box shown in figure a uses a processor to load the parameters corresponding to all layers in the temporary subgraph, processes the input data of the temporary subgraph (i.e., the output data of the layer with layer label 8), obtains the performance index when processing the input data and the redundancy rate of repeated processing of some data. When the performance index when processing the input data is higher than the performance index threshold and the redundancy rate of repeated processing of some data is lower than the redundancy rate threshold (e.g., the redundancy rate is 0, which is lower than the redundancy rate threshold of 0.6), the index layer with layer label 9 can be formally put into the subgraph, and the parent layer of the layer with layer label 9 is used as a candidate layer in the candidate node set of the index layer mp. For example, the parent layer with layer label 8 is used as a candidate layer in the candidate node set of the index layer with layer label 9.

[0094] Then, the candidate layer with layer identifier 8 can be used as the candidate layer to be processed (candi node). If the candidate layer with layer identifier 8 is not in the matching layer set (match nodes), not in the base_merge_nodes set (to be processed), and not in the subgraph, the index layer (mp) with layer identifier 9 and the candidate layer with layer identifier 8 are placed into the temporary subgraph (subgraph_tmp), as follows. Figure 7 The dashed box shown in b represents the process of loading parameters corresponding to all layers in the temporary subgraph using a processor. The input data (i.e., the output data of layer 7) of the temporary subgraph is processed, and performance metrics and redundancy rates for repeated processing of some data are obtained. When the performance metrics for processing the input data are higher than the performance metric threshold, and the redundancy rate for repeated processing is lower than the redundancy rate threshold (e.g., a redundancy rate of 0.2 is lower than the redundancy rate threshold of 0.6), the candidate layer with layer 8 can be formally added to the subgraph. Simultaneously, the parent layer of layer 8 (i.e., layer 7) is added to the candidate node set of the index layer with layer 9, and the candidate layer with layer 8 is removed from the candidate node set. Finally, the candidate layer with layer 8 can be added to the match node set.

[0095] Next, the candidate layer with layer identifier 7 can be used as the candidate layer to be processed (candi node). If the candidate layer to be processed (candi node) with layer identifier 7 is not in the matching layer set (match nodes), not in the base_merge_nodes set (to be processed), and not in the subgraph, then the index layer (mp) with layer identifier 9, the candidate layer with layer identifier 8, and the candidate layer to be processed (candi node) with layer identifier 7 are placed into the temporary subgraph (subgraph_tmp), as follows. Figure 7The dashed box shown in c represents the process of loading parameters corresponding to all layers in the temporary subgraph using a processor. The input data (i.e., the output data of layer 6) is processed, and performance metrics and redundancy rates for repeated processing of some data are obtained. If the performance metrics are lower than a performance threshold, or the redundancy rate for repeated processing is higher than a redundancy rate threshold (e.g., a redundancy rate of 0.65 is higher than the redundancy rate threshold of 0.6), the candidate layer with layer 7 is removed from the candidate node set, and the layer with layer 7 is added to the repeated layer set `rest_merge_node`. At this point, the candidate node set is empty, resulting in a target layer set, including layers 9 and 8. This target layer set is then added to the valid subgraph.

[0096] At the same time, the layer identified as 7 can be re-established as an index layer (mp), and the index layer identified as 7 can be placed into a temporary subgraph (subgraph), as follows. Figure 7 The dashed box shown in d is used to load the parameters corresponding to all layers in the temporary subgraph using a processor. The input data of the temporary subgraph (i.e., the output data of layer with layer label 6) is processed to obtain the performance index when processing the input data and the redundancy rate of repeated processing of some data. When the performance index when processing the input data is higher than the performance index threshold and the redundancy rate of repeated processing of data is lower than the redundancy rate threshold (e.g., the redundancy rate is 0, which is lower than the redundancy rate threshold of 0.6), the index layer mp with layer label 7 can be formally put into the subgraph. The parent layer of the layer with layer label 7 is used as the candidate layer in the candidate node set of the index layer mp. For example, the parent layer with layer label 6 is used as the candidate layer in the candidate node set of the index layer with layer label 7.

[0097] Then, the candidate layer with layer identifier 6 can be used as the candidate layer to be processed (candi node). If the candidate layer with layer identifier 6 is not in the matching layer set (match nodes), not in the base_merge_nodes set (to be processed), and not in the subgraph, the index layer (mp) with layer identifier 7 and the candidate layer with layer identifier 6 are placed into the temporary subgraph (subgraph_tmp), as follows. Figure 7The dashed box shown in e represents the process of loading parameters corresponding to all layers in the temporary subgraph using a processor. The input data (i.e., the output data of layer 5) of the temporary subgraph is processed, and performance metrics and redundancy rates for repeated processing of some data are obtained. When the performance metrics for processing the input data are higher than the performance metric threshold, and the redundancy rate for repeated processing is lower than the redundancy rate threshold (e.g., a redundancy rate of 0.2 is lower than the redundancy rate threshold of 0.6), the candidate layer with layer 6 can be formally added to the subgraph. Simultaneously, the parent layer of layer 6 (i.e., layer 5) is added to the candidate node set of the index layer with layer 7, and the candidate layer with layer 6 is removed from the candidate node set. Finally, the candidate layer with layer 6 can be added to the match node set.

[0098] Next, the candidate layer with layer identifier 5 can be used as the candidate layer to be processed (candi node). If the candidate layer to be processed (candi node) with layer identifier 5 is not in the matching layer set (match nodes), not in the base_merge_nodes set (to be processed), and not in the subgraph, then the index layer (mp) with layer identifier 7, the candidate layer with layer identifier 6, and the candidate layer to be processed (candi node) with layer identifier 5 are placed into the temporary subgraph (subgraph_tmp), as follows. Figure 7 The dashed box shown in f is used to load the parameters corresponding to all layers in the temporary subgraph using a processor. The input data of the temporary subgraph (i.e., the output data of layer with layer label 4) is processed to obtain the performance index when processing the input data and the redundancy rate of repeated processing of some data. When the performance index when processing the input data is higher than the performance index threshold and the redundancy rate of repeated processing of data is lower than the redundancy rate threshold (e.g., the redundancy rate is 0.5, which is lower than the redundancy rate threshold of 0.6), the candidate layer with layer label 5 can be formally added to the subgraph. At the same time, the parent layer of the layer with layer label 5 (i.e., the layer with layer label 4) is added to the candidate layer set candidate nodes of the index layer with layer label 7, and the candidate layer with layer label 5 is removed from the candidate layer set candidate nodes. The candidate layer with layer label 5 can also be added to the matching layer set matchnodes.

[0099] Then, the candidate layer with layer identifier 4 can be used as the candidate layer to be processed (candi node). If the candidate layer to be processed (candi node) with layer identifier 4 is not in the matching layer set (match nodes), not in the base_merge_nodes set (to be processed), and not in the subgraph, the index layer (mp) with layer identifier 7, the candidate layers with layer identifiers 6 and 5, and the candidate layer to be processed (candi node) with layer identifier 4 are placed into the temporary subgraph (subgraph_tmp), as follows. Figure 7 The dashed box shown in g is used to load the parameters corresponding to all layers in the temporary subgraph using a processor. The input data of the temporary subgraph (i.e., the output data of the layer with layer identifier 3) is processed to obtain the performance index when processing the input data and the redundancy rate of repeated processing of some data. When the performance index when processing the input data is lower than the performance index threshold and the redundancy rate of repeated processing of data is higher than the redundancy rate threshold (e.g., the redundancy rate is 0.7, which is higher than the redundancy rate threshold of 0.6), the candidate layer with layer identifier 4 can be removed from the candidate layer set candidate nodes, and the candidate layer with layer identifier 4 can be added to the repeated layer set rest_merge_node.

[0100] At the same time, the layer identified as 4 can be re-established as an index layer mp, and the index layer identified as 4 can be placed into a temporary subgraph, such as... Figure 7 The dashed box shown in h is used to load the parameters corresponding to all layers in the temporary subgraph using a processor. The input data of the temporary subgraph (i.e., the output data of the layer with layer label 3) is processed to obtain the performance index when processing the input data and the redundancy rate of repeated processing of some data. When the performance index when processing the input data is higher than the performance index threshold and the redundancy rate of repeated processing of data is lower than the redundancy rate threshold (e.g., the redundancy rate is 0.3, which is lower than the redundancy rate threshold of 0.6), the candidate layer to be processed with layer label 4 can be formally put into the subgraph. The parent layer of the layer with layer label 3 is used as the candidate layer in the candidate node set of the index layer mp. For example, the parent layer of the layer with layer label 3 is used as the candidate layer in the candidate node set of the index layer with layer label 4.

[0101] In addition, when formally placing the candidate layer with layer identifier 6 into the subgraph, the other parent layer of the layer with layer identifier 6 (i.e., the layer with layer identifier 2) can be added to the candidate node set of the index layer with layer identifier 7.

[0102] Next, the candidate layer with layer identifier 2 can be used as the candidate layer to be processed (candi node). If the candidate layer to be processed with layer identifier 2 is not in the matching layer set (match nodes), not in the base_merge_nodes set (to be processed), and not in the subgraph, then the index layer (mp) with layer identifier 7, the candidate layer with layer identifier 6, and the candidate layers to be processed with layer identifier 2 and layer identifier 5 are placed into the temporary subgraph (subgraph_tmp), as follows. Figure 7 The dashed box shown in i represents the process of loading parameters corresponding to all layers in the temporary subgraph using a processor. The input data of the temporary subgraph (i.e., the output data of layer 1 and layer 4) is processed. Performance metrics for processing the input data and the redundancy rate for repeated processing of some data are obtained. When the performance metrics for processing the input data are higher than the performance metric threshold, and the redundancy rate for repeated processing is lower than the redundancy rate threshold (e.g., a redundancy rate of 0.53 is lower than the redundancy rate threshold of 0.6), the candidate layer with layer 2 can be formally added to the subgraph. Simultaneously, the parent layer of layer 2 (i.e., layer 1) is added to the candidate node set of the index layer with layer 7, and the candidate layer with layer 2 is removed from the candidate node set. Finally, the candidate layer with layer 2 can be added to the match node set.

[0103] Then, the candidate layer with layer identifier 1 can be used as the candidate layer to be processed (candi node). If the candidate layer to be processed (candi node) with layer identifier 1 is not in the matching layer set (match nodes), not in the base_merge_nodes set (to be processed), and not in the subgraph, the index layer (mp) with layer identifier 7, the candidate layers with layer identifiers 6, 5, and 2, and the candidate layer to be processed (candi node) with layer identifier 1 are placed into the temporary subgraph (subgraph_tmp), as follows: Figure 7The dashed box shown in j represents the process of loading parameters corresponding to all layers in the temporary subgraph using a processor. The input data (i.e., the output data of the layer with layer identifier 0) of the temporary subgraph is processed. Performance metrics for processing the input data and the redundancy rate for repeated processing of some data are obtained. When the performance metrics for processing the input data are higher than the performance metric threshold, and the redundancy rate for repeated processing of data is lower than the redundancy rate threshold (e.g., a redundancy rate of 0.53 is lower than the redundancy rate threshold of 0.6), the candidate layer with layer identifier 1 can be formally added to the subgraph. Simultaneously, the parent layer of the layer with layer identifier 1 (i.e., the layer with layer identifier 0) is added to the candidate node set of the index layer with layer identifier 7, and the candidate layer with layer identifier 1 is removed from the candidate node set. Finally, the candidate layer with layer identifier 1 can be added to the match node set.

[0104] Then, the candidate layer with layer identifier 0 can be used as the candidate layer candi node to be processed. If the candidate layer candi node with layer identifier 0 is not in the matching layer set match nodes, not in the base_merge_nodes set of layers to be processed, and not in the subgraph, the index layer mp with layer identifier 7, the candidate layers with layer identifiers 6, 5, 2, and 1, and the candidate layer to be processed with layer identifier 0 are placed into the temporary subgraph subgraph_tmp, as follows. Figure 7 The dashed box shown in k represents the processing logic. The processor loads the parameters corresponding to all layers in the temporary subgraph and processes the input data (i.e., part or all of the model's data). It obtains the performance metrics for processing the input data and the redundancy rate for repeated processing of some data. When the performance metrics for processing the input data are higher than the performance metric threshold, and the redundancy rate for repeated processing is lower than the redundancy rate threshold (e.g., a redundancy rate of 0.53 is lower than the redundancy rate threshold of 0.6), the candidate layer with layer identifier 0 can be formally added to the subgraph, and the candidate layer with layer identifier 0 can be removed from the candidate layer set (candidate nodes). The candidate layer with layer identifier 0 can also be added to the matching layer set (match nodes). At this point, the candidate layer set (candidate nodes) is empty, resulting in a target layer set, which includes target layers with layer identifiers of 7, 6, 5, 2, 1, and 0. This target layer set is then added to the valid subgraph.

[0105] Then, the candidate layer with layer identifier 3 can be used as the candidate layer candi node to be processed. If the candidate layer candi node with layer identifier 3 is not in the matching layer set match nodes, not in the base_merge_nodes set of layers to be processed, and not in the subgraph, the index layer mp with layer identifier 4 and the candidate layer with layer identifier 3 are placed into the temporary subgraph subgraph_tmp, as follows. Figure 7 The dashed box shown in Figure 1 represents the process of loading parameters corresponding to all layers in the temporary subgraph using a processor. This process processes the input data (part or all of the model's data) of the temporary subgraph, obtaining performance metrics and redundancy rates for repeated processing of some data. If the performance metrics are higher than a performance threshold and the redundancy rate for repeated processing is lower than a redundancy rate threshold (e.g., a redundancy rate of 0.4 is lower than the redundancy rate threshold of 0.6), the candidate layer identified as layer 3 can be formally added to the subgraph, removed from the candidate node set, and added to the match node set. At this point, the candidate node set is empty, resulting in a target layer set, including layers identified as 4 and 3. This target layer set is then added to the valid subgraph.

[0106] In this way, multiple valid subgraphs can be obtained, namely, the target layer set including layer identifiers 0, 1, 2, 5, 6, and 7, the target layer set including layer identifiers 3 and 4, and the target layer set including layer identifiers 8 and 9. Then, the parameters corresponding to all layers in each target layer set can be divided into a parameter set, resulting in multiple parameter sets.

[0107] Furthermore, the data processing method mentioned in this application's embodiments, by setting partitioning information in the memory to divide the parameters in the model, for example, dividing the partially connected processing layers in the model into a processing layer set, and dividing the parameters corresponding to all processing layers in this processing layer set into a parameter set, resulting in multiple parameter sets. These multiple parameter sets are multiple parameter sets that can process data in parallel. When the multi-core NPU performs data processing, multiple cores in the multi-core NPU simultaneously load different parameter sets corresponding to different processing layer sets, process the input data of each processing layer set, and store the parameters, input data, and output data of each processing set into the internal storage space of different cores. Thus, when processing data, data is read directly from the internal memory of the core, eliminating the need to read data from external memory via a bus. This alleviates the storage pressure on a single core, reduces the number of accesses to external memory, improves data processing efficiency, reduces data read latency, and reduces power consumption.

[0108] The hardware structure of electronic devices is described below. Figure 8 As shown, Figure 8 A schematic diagram of the hardware structure of an electronic device is shown. It is understood that the electronic device of this application can be a server, desktop computer, handheld computer, laptop computer, or other electronic device. The structure of the electronic device will be described below using a server as an example.

[0109] In one embodiment, the server may include one or more processors 801, system control logic 802 connected to at least one of the processors 801, system memory 803 connected to the system control logic 802, non-volatile memory (NVM) 804 connected to the system control logic 802, and input / output (I / O) devices 805 and network interfaces 806 connected to the system control logic 802.

[0110] In some embodiments, processor 801 may include one or more single-core or multi-core processors. In some embodiments, processor 801 may include any combination of general-purpose processors and special-purpose processors (e.g., graphics processors, application processors, baseband processors, etc.). In embodiments where the server employs an eNB (Evolved Node B) or RAN (Radio Access Network) controller, processor 801 may be configured to perform various conforming embodiments.

[0111] In some embodiments, system control logic 802 may include any suitable interface controller to provide any suitable interface to at least one of the processors 801 and / or any suitable device or component communicating with system control logic 802.

[0112] In some embodiments, system control logic 802 may include one or more memory controllers to provide an interface to system memory 803. System memory 803 may be used to load and store data and / or instructions 8031. In some embodiments, the server's memory may include any suitable volatile memory, such as suitable dynamic random access memory (DRAM).

[0113] The non-volatile memory (NVM) 804 may include one or more tangible, non-transitory computer-readable media for storing data and / or instructions. In some embodiments, the non-volatile memory (NVM) 804 may include any suitable non-volatile memory such as flash memory and / or any suitable non-volatile storage device, such as at least one of an HDD (Hard Disk Drive), a CD (Compact Disc) drive, and a DVD (Digital Versatile Disc) drive.

[0114] Non-volatile memory (NVM) 804 may include a portion of storage resources on the device on which the server is installed, or it may be accessible by the device, but is not necessarily part of the device. For example, non-volatile memory (NVM) 804 may be accessed over a network via network interface 806.

[0115] Specifically, system memory 803 and non-volatile memory (NVM) 804 may each include temporary and permanent copies of instructions. Instructions may include instructions that, when executed by at least one of the processors 801, cause the server to implement the data processing methods mentioned in the embodiments of this application. In some embodiments, instructions, hardware, firmware, and / or their software components may additionally / alternatively be located in system control logic 802, network interface 806, and / or processor 801.

[0116] Network interface 806 may include a transceiver for providing a radio interface to the server, thereby enabling communication with any other suitable device (such as a front-end module, antenna, etc.) via one or more networks. In some embodiments, network interface 806 may be integrated into other components of the server. For example, network interface 806 may be integrated into at least one of processor 801, system memory 803, non-volatile memory (NVM) 804, and firmware device (not shown) with instructions, wherein when at least one of processor 801 executes the instructions, the server implements the data processing methods mentioned in the embodiments of this application.

[0117] The network interface 806 may further include any suitable hardware and / or firmware to provide a multiple-input multiple-output radio interface. For example, the network interface 806 may be a network adapter, a wireless network adapter, a telephone modem, and / or a wireless modem.

[0118] In one embodiment, at least one of the processors 801 may be packaged together with the logic of one or more controllers for system control logic 802 to form a system-in-package (SiP). In another embodiment, at least one of the processors 801 may be integrated on the same die with the logic of one or more controllers for system control logic 802 to form a system-on-a-chip (SoC).

[0119] The server may further include an input / output (I / O) device 805. The I / O device 805 may include a user interface enabling users to interact with the server; the peripheral component interface is designed to allow peripheral components to also interact with the server. In some embodiments, the server further includes sensors for determining at least one type of environmental condition and location information related to the server.

[0120] In some embodiments, the user interface may include, but is not limited to, a display (e.g., a liquid crystal display, a touch screen display, etc.), a speaker, a microphone, one or more cameras (e.g., a still image camera and / or a video camera), a flashlight (e.g., a light-emitting diode flash), and a keyboard.

[0121] In some embodiments, the peripheral component interface may include, but is not limited to, a non-volatile memory port, an audio jack, and a power interface.

[0122] In some embodiments, the sensor may include, but is not limited to, a gyroscope sensor, an accelerometer, a proximity sensor, an ambient light sensor, and a positioning unit. The positioning unit may also be part of or interact with the network interface 806 to communicate with components of the positioning network, such as Global Positioning System (GPS) satellites.

[0123] The above describes the possible hardware structures of electronic devices. It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine certain components, or split certain components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of both.

[0124] In the accompanying drawings, some structural or methodological features may be shown in a specific arrangement and / or order. However, it should be understood that such a specific arrangement and / or order may not be necessary. Rather, in some embodiments, these features may be arranged in a manner and / or order different from that shown in the illustrative drawings. Furthermore, the inclusion of structural or methodological features in a particular figure does not imply that such features are required in all embodiments, and in some embodiments, these features may be omitted or may be combined with other features.

[0125] The various embodiments of the mechanisms disclosed in this application can be implemented in hardware, software, firmware, or a combination of these implementation methods. Embodiments of this application can be implemented as computer programs or program code executable on a programmable system, the programmable system including at least one processor, a storage system (including volatile and non-volatile memory and / or storage elements), at least one input device, and at least one output device.

[0126] Program code can be applied to input instructions to execute the functions described in this application and generate output information. The output information can be applied to one or more output devices in a known manner. For the purposes of this application, the processing system includes any system having a processor such as, for example, a digital signal processor (DSP), a microcontroller, an application-specific integrated circuit (ASIC), or a microprocessor.

[0127] The program code can be implemented using a high-level procedural language or an object-oriented programming language to communicate with the processing system. Assembly language or machine language can also be used when needed. In fact, the mechanisms described in this application are not limited to any particular programming language. In either case, the language can be a compiled language or an interpreted language.

[0128] In some cases, the disclosed embodiments may be implemented in hardware, firmware, software, or any combination thereof. The disclosed embodiments may also be implemented as instructions carried or stored thereon on one or more temporary or non-temporary machine-readable (e.g., computer-readable) storage media, which may be read and executed by one or more processors. For example, the instructions may be distributed via a network or through other computer-readable media. Therefore, machine-readable media may include any mechanism for storing or transmitting information in a machine-readable (e.g., computer-readable) form, including but not limited to floppy disks, optical disks, CD-ROMs, magneto-optical disks, read-only memory (ROM), random access memory (RAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic cards or optical cards, flash memory, or tangible machine-readable storage for transmitting information (e.g., carrier waves, infrared signals, digital signals, etc.) using the Internet in the form of electrical, optical, acoustic, or other propagation signals. Therefore, machine-readable media include any type of machine-readable medium suitable for storing or transmitting electronic instructions or information in a machine-readable (e.g., computer-readable) form.

[0129] It should be noted that all units / modules mentioned in the device embodiments of this application are logical units / modules. Physically, a logical unit / module can be a physical unit / module, a part of a physical unit / module, or a combination of multiple physical units / modules. The physical implementation of these logical units / modules themselves is not the most important factor; the combination of functions implemented by these logical units / modules is the key to solving the technical problems proposed in this application. Furthermore, to highlight the innovative aspects of this application, the above-described device embodiments of this application have not introduced units / modules that are not closely related to solving the technical problems proposed in this application. This does not mean that the above-described device embodiments do not contain other units / modules.

[0130] It should be noted that in the examples and description of this patent, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0131] Although this application has been illustrated and described with reference to certain preferred embodiments thereof, those skilled in the art will understand that various changes in form and detail may be made thereto without departing from the scope of this application.

Claims

1. A data processing method for electronic devices, characterized in that, The electronic device includes a first processor, which includes multiple cores; And the method includes: Obtain the first input data of the first model; Based on the segmentation information of the first model, the multiple processing layers in the first model are segmented into multiple processing layer sets that match the segmentation information. The processing layers in the same processing layer set satisfy the effective segmentation condition, which is related to the running performance of the first model. The kernels corresponding to each processing layer set are used to process each part of the first input data that needs to be processed by each processing layer set; The valid segmentation conditions include at least one of the following: When the first processor loads the parameters corresponding to all processing layers in the same processing layer set and processes the portion of data in the first input data that needs to be processed by the same processing layer set, the performance index is higher than the performance index threshold. When the first processor loads the parameters corresponding to all processing layers in the same processing layer set and processes a portion of the data in the first input data that needs to be processed by the same processing layer set, the redundancy rate of repeated processing of the partial data is lower than the redundancy rate threshold. The redundancy rate refers to the ratio of the computational amount of repeated processing of overlapping input data after the first input data is segmented to the total computational amount. The step of dividing the multiple processing layers in the first model into multiple processing layer sets that match the division information includes: A set of layers to be processed is determined from the first model, the set of layers to be processed including at least one layer in the first model; Determine an index layer from the set of layers to be processed, wherein the index layer is any one of the processing layers in the set of layers to be processed; Determine whether the index layer belongs to the matching layer set, which is used to record the processing layers that have been split; If the index layer does not belong to the matching layer set, the plurality of processing layer sets are determined based on the index layer and all its parent layers. The determination of the multiple processing layer sets based on the index layer and all its parent layers includes: All parent layers of the index layer are determined as the candidate layer set of the index layer; The candidate layer to be processed is determined from the candidate layer set of the index layer; Determine whether the candidate layer to be processed meets the partitioning conditions; If the candidate layer to be processed meets the partitioning conditions, the index layer and the candidate layer to be processed are placed into a temporary subgraph, and it is determined whether the temporary subgraph meets the valid partitioning conditions. If the temporary subgraph satisfies the effective segmentation condition, the candidate layer to be processed is placed into the subgraph, the parent layer of the candidate layer to be processed is placed into the candidate layer set, and the candidate layer to be processed is deleted from the candidate layer set. The temporary subgraph is defined as a set of processing layers.

2. The method according to claim 1, characterized in that, The segmentation information is obtained through the following methods: A first processing layer and a second processing layer are determined from multiple processing layers in the first model, and the first processing layer and the second processing layer are connected. Determine whether the first processing layer and the second processing layer satisfy the effective segmentation condition; If the first processing layer and the second processing layer satisfy the effective segmentation condition, the first processing layer and the second processing layer are divided into a first processing layer set; If the first processing layer and the second processing layer do not meet the effective segmentation conditions, the first processing layer is divided into a first processing layer set, and the second processing layer is divided into a second processing layer set.

3. The method according to claim 2, characterized in that, The method further includes: A third processing layer is determined from the plurality of processing layers in the first model, and the third processing layer is connected to the first processing layer; Determine whether the first processing layer, the second processing layer, and the third processing layer satisfy the valid segmentation condition; If the first processing layer, the second processing layer, and the third processing layer satisfy the effective segmentation condition, the first processing layer, the second processing layer, and the third processing layer are divided into a first processing layer set.

4. The method according to claim 2, characterized in that, The kernels corresponding to the processing layer set include a first kernel and a second kernel. The step of using the kernels corresponding to each processing layer set to process each portion of the first input data required by each processing layer set includes: Using the first kernel, the first sub-data in the first input data that the first processing layer set needs to process is processed, and The second kernel is used to process the second sub-data in the first input data that the first processing layer set needs to process.

5. The method according to claim 4, characterized in that, The step of using the first kernel to process the first sub-data in the first input data required for processing by the first processing layer set includes: The first kernel loads the first parameter set corresponding to all layers in the first processing layer set, processes the first sub-data in the first input data, and... The step of using the second kernel to process the second sub-data in the first input data that the first processing layer set needs to process includes: The second kernel loads the second parameter set corresponding to all layers in the second processing layer set, and processes the second sub-data in the first input data.

6. The method according to claim 4, characterized in that, The method further includes: The first sub-data is stored in the first internal storage space of the first kernel, and The second sub-data is stored in the second internal storage space of the second kernel.

7. The method according to any one of claims 1-6, characterized in that, The first processor is a multi-core embedded neural network model processor. The first model is a neural network model. The first input data includes image data, text data, and audio data.

8. An electronic device, characterized in that, include: A memory for storing instructions executed by one or more processors of the electronic device, and a processor, being one of one or more processors of the electronic device, for performing the data processing method according to any one of claims 1-7.

9. A readable storage medium, characterized in that, The readable storage medium stores instructions that, when executed on an electronic device, cause the electronic device to perform the data processing method according to any one of claims 1-7.

Images