Model processing module and electronic equipment

Abstract

The invention discloses a model processing module and an electronic device. The model processing module comprises a target memory used for storing a model set of a to-be-operated target model layer in a processing model and a to-be-processed input data set of the target model layer; the first access control module is used for obtaining first data of a first data type from the target memory, wherein the first data type is one of model data or input data; the second access control module is used for obtaining second data of a second data type from the target memory, wherein the second data type is one of model data or input data; the third access control module is used for obtaining third data of the second data type from the target memory; the first processor is used for determining first output data of the target model layer based on the first data and the second data; the second processor is used for determining second output data of the target model layer based on the first data and the third data.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular, to a model processing module and an electronic device. Background Art

[0002] It has become increasingly common to use artificial intelligence chips such as neural network processors to perform data processing related to artificial intelligence models such as neural network models.

[0003] However, currently, the efficiency of using artificial intelligence chips to process data related to artificial intelligence models is relatively low. Summary of the Invention

[0004] On the one hand, this application provides a model processing module, including:

[0005] A target memory for storing a model set of a target model layer to be run in a processing model and an input data set to be processed by the target model layer, the model set including at least one model data, and the input data set including at least one input data;

[0006] A first access control module for obtaining first data of a first data type from the target memory, the first data type being one of model data or input data;

[0007] A second access control module for obtaining second data of a second data type from the target memory, the second data type being one of model data or input data, and the second data type being different from the first data type;

[0008] A third access control module for obtaining third data of the second data type from the target memory;

[0009] A first processor for determining first output data of the target model layer based on the first data and the second data;

[0010] A second processor for determining second output data of the target model layer based on the first data and the third data.

[0011] In a possible implementation, in the first mode, the third access control module is used to obtain the third data, and the second processor can obtain the first data from the first access control module and the third data from the third access control module;

[0012] The model processing module further includes: a fourth access control module;

[0013] In the second mode, the fourth access control module is used to obtain fourth data of the first data type from the target memory, and the second processor can obtain the second data from the second access control module and obtain the fourth data from the fourth access control module, and determine third output data of the target model layer based on the second data and the fourth data.

[0014] In yet another possible implementation, the first data is first model data, and the first model data belongs to the at least one model data;

[0015] The second data is first input data, and the first input data belongs to the at least one input data;

[0016] In the first mode, the third data is second input data, and the second input data belongs to the at least one input data;

[0017] In the second mode, the fourth data is second model data, and the second model data belongs to the at least one model data.

[0018] In yet another possible implementation, the model processing module further includes: a mode configuration module, configured to obtain a mode indication instruction sent by a controller, where the mode indication instruction is used to indicate that the model processing module is in the first mode or the second mode; in response to the mode indication instruction being an instruction to start the first mode, indicating that the second processor can access the first access control module and the third access control module; in response to the mode indication instruction being an instruction to start the second mode, indicating that the second processor can access the second access control module and the fourth access control module.

[0019] In yet another possible implementation, in the first mode, the target memory is used to store a model set of the target model layer to be run in the processing model and a first input data subset to be processed by the target model layer in a first storage area corresponding to a first processor, and store a second input data subset to be processed by the target model layer in a second storage area corresponding to a second processor, where the first input data subset and the second input data subset constitute the input data set;

[0020] In the first mode, the first access control module is used to obtain first model data from the model set in the first storage area;

[0021] In the first mode, the second access control module is used to obtain first input data from the first input data subset in the first storage area;

[0022] In the first mode, the third access control module is configured to obtain second input data from a second input data subset in the second storage area.

[0023] In another possible implementation, in the second mode, the target memory is configured to store a first sub-model set of a target model layer to be run in the processing model and an input data set to be processed by the target model layer in a first storage area corresponding to a first processor, and store a second sub-model set of the target model layer in a second storage area corresponding to a second processor, where the first sub-model set and the second sub-model set constitute a model set of the target model layer;

[0024] In the second mode, the first access control module is configured to obtain first model data from the first sub-model set;

[0025] In the second mode, the second access control module is configured to obtain first input data from an input data set in the first storage area;

[0026] In the second mode, the fourth access control module is configured to obtain second model data from the second sub-model set in the second storage area.

[0027] In another possible implementation, the first access control module includes: a first access storage controller, a first first-in-first-out module, and a first parameter receiving controller;

[0028] The second access control module includes: a second access storage controller, a second first-in-first-out module, and a second parameter receiving controller;

[0029] The third access control module includes: a third access storage controller, a third first-in-first-out module, and a third parameter receiving controller;

[0030] The fourth access control module includes: a fourth access storage controller, a fourth first-in-first-out module, and a fourth parameter receiving controller;

[0031] In the first mode, the second processor can obtain the first data from the first access storage controller through the fourth first-in-first-out module and the fourth parameter receiving controller;

[0032] In the second mode, the second processor can obtain the second data from the second access storage controller through the third first-in-first-out module and the third parameter receiving controller.

[0033] In another possible implementation, the model processing module further includes: a first output processing module and a second output processing module;

[0034] The first processor is further configured to output the first output data to the first output processing module; and determine the target data dimension corresponding to the model set in the next model layer after the target model layer in the processing model, and send the target data dimension to the first output processing module;

[0035] The first output processing module is configured to process the first output data into first target output data with the target data dimension;

[0036] The second processor is further configured to output the second output data to the second output processing module; and determine the target data dimension corresponding to the model set in the next model layer, and send the target data dimension to the second output processing module;

[0037] The second output processing module is configured to process the second output data into second target output data with the target data dimension.

[0038] In another possible implementation manner, the first output processing module is specifically configured to construct a first all-zero matrix with the target data dimension, and based on the positions of the respective data in the first output data corresponding to the first all-zero matrix, fill the respective data in the first output data into the corresponding positions in the first all-zero matrix to obtain first target output data;

[0039] The first output processing module is specifically configured to construct a second all-zero matrix with the target data dimension, and based on the positions of the respective data in the second output data corresponding to the second all-zero matrix, fill the respective data in the second output data into the corresponding positions in the second all-zero matrix to obtain second target output data.

[0040] In another aspect, the present application further provides an electronic device, including at least: a model processing module;

[0041] The model processing module includes:

[0042] A target memory for storing a model set of a target model layer to be run in the processing model and an input data set to be processed by the target model layer, where the model set includes at least one model data, and the input data set includes at least one input data;

[0043] A first access control module for obtaining first data of a first data type from the target memory, where the first data type is one of model data or input data;

[0044] A second access control module, configured to obtain second data of a second data type from the target memory, where the second data type is one of model data or input data, and the second data type is different from the first data type;

[0045] A third access control module, configured to obtain third data of the second data type from the target memory;

[0046] A first processor, configured to determine first output data of the target model layer based on the first data and the second data;

[0047] A second processor, configured to determine second output data of the target model layer based on the first data and the third data. Description of the Drawings

[0048] In combination with the accompanying drawings and with reference to the following specific embodiments, the above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic, and the original and elements are not necessarily drawn to scale.

[0049] Figure 1 It is a schematic structural diagram of a composition of a model processing module provided by the present application;

[0050] Figure 2 It is another schematic structural diagram of a composition of a model processing module provided by the present application;

[0051] Figure 3 It shows an example diagram of an input data set required by the target model layer and an output data set obtained based on the target model layer;

[0052] Figure 4 It shows an example diagram of a first input data subset and a second input data subset split from the input data set;

[0053] Figure 5 It shows an example diagram of the operation of the processor in the model processing module on the input data and the model data;

[0054] Figure 6 It shows another schematic structural diagram of a composition of the model processing module provided by the present application;

[0055] Figure 7 It shows the processing of the 6 6-dimensional first output data into An example diagram of the implementation principle of the dimensional first target output data;

[0056] Figure 8Shows a schematic diagram of the composition architecture of the model processing module in an application scenario;

[0057] Figure 9 Shows a schematic diagram of a composition structure of the electronic device provided by the present application. Detailed implementation manners

[0058] The embodiments of the present application will be described below with reference to the accompanying drawings in the embodiments of the present application. The terms used in the embodiments part of the present application are only used to explain the specific embodiments of the present application, rather than intended to limit the present application. Those of ordinary skill in the art will know that with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0059] The terms "first", "second", etc. in the specification, claims and above-mentioned drawings of the present application are used to distinguish similar objects, and do not have to be used to describe a specific order or sequence. It should be understood that such terms can be interchanged under appropriate circumstances, which is only a way of distinction adopted when describing objects with the same attributes in the embodiments of the present application. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, so that a process, method, system, product or device including a series of units does not have to be limited to those units, but may include other units not clearly listed or inherent to these process, method, product or device.

[0060] The model processing module of the present application will be introduced below. In the present application, the model processing module may be a chip for processing artificial intelligence-related computing tasks, or a part of an integrated chip such as a system on chip (SOC) or an artificial intelligence chip for processing artificial intelligence-related computing tasks, and there is no limitation thereto.

[0061] Such as Figure 1 , shows a schematic diagram of a composition structure of the model processing module provided by the present application.

[0062] It can be seen from Figure 1 that the model processing module includes: a target memory 101, a first access control module 102, a second access control module 103, a third access control module 104, a first processor 105, and a second processor 106.

[0063] Among them, the target memory 101 is used to store the model set of the target model layer to be run in the processing model and the input data set to be processed by the target model layer.

[0064] Among them, the processing model can be a neural network model or other artificial intelligence models with multiple hierarchical structures. Depending on the actual task inference scenario, the processing model will also vary, and there is no restriction on this.

[0065] It can be understood that the processing model generally consists of at least one model module, and each model module is a model layer. When performing task inference based on the processing model, it is necessary to sequentially input the input data into each model module of the processing model for processing. In this application, for the sake of distinction, the model module that the processing model currently needs to run is called the target model layer. For example, if the processing model is a convolutional neural network model and the convolutional neural network includes multiple convolutional layers, then it is necessary to sequentially perform computational processing on the input data based on each convolutional layer. Therefore, the currently running convolutional layer is the target model layer.

[0066] Among them, the model set includes the set of model data participating in the calculation in the target model layer of the processing model. Therefore, this model set includes at least one model data. For example, the model data in the model set can include parameter data such as the weight matrix of the target model layer. Correspondingly, this model set is also a data matrix.

[0067] Among them, the input data set includes at least one input data. The input data set is the input data that needs to be processed by the target model layer. For example, if the target model layer is the first model layer of the processing model, then the input data set is the set of input data that needs to be input into the processing model. If the target model layer is an intermediate model layer of the processing model, then the input data set corresponding to the target model layer can be the data set output after computational processing by the previous model layer of this target model layer in this processing model. In this application, the input data set that needs to be processed by the processing model can be vector data corresponding to images or texts or other forms of data matrices, and there is no specific restriction.

[0068] In this application, the specific form of this target memory can have multiple possibilities, and there is no restriction on this. For example, this target memory can be a Static Random-Access Memory (SRAM).

[0069] The first access control module 102 is used to obtain the first data of the first data type from the target memory. Among them, the first data type is used to represent that the data belongs to model data or input data. Therefore, the first data category can be one of the two data types of model data or input data. For example, in the case where the first data type is model data, the first data belongs to the input data in the model set.

[0070] The second access control module 103 is configured to obtain second data of a second data type from the target memory. Wherein, the second data type is used to characterize that the data belongs to model data or input data. Therefore, the second data type belongs to one of the two data types of model data or input data. The second data type is different from the first data type.

[0071] For example, when the first data type is model data, the second data type is input data. Correspondingly, the second data belongs to the input data in the input data set.

[0072] The third access control module 104 is configured to obtain third data of the second data type from the target memory.

[0073] In this application, the third data and the second data are different data of the same data type. For example, when the second data type is model data, the second data and the third data are different model data in the model set.

[0074] In this application, the first access control module, the second access control module, and the third access control module are directly or indirectly connected to the target memory for data.

[0075] Wherein, the first processor 105 is configured to determine first output data of the target model layer based on the first data and the second data.

[0076] The second processor 106 is configured to determine second output data of the target model layer based on the first data and the third data.

[0077] Wherein, the first processor and the first access control module and the second access control module can be directly connected, or indirectly connected through other data transmission modules, etc., or connected or disconnected under the control of a register, etc., without specific limitation. Similarly, the second processor can be directly or indirectly connected to the first access control module and the third access control module.

[0078] In this application, both the first processor and the second processor are processors or processor cores capable of running an artificial intelligence model, and there is no limitation on the processor type of the first processor and the second processor. For example, the first processor and the second processor can be a Neural Network Processing Unit (NPU) or an NPU core, and can also be a CPU, a CPU core, or other types of processing units, without specific limitation.

[0079] It can be understood that since the time-consuming for data reading from the target memory is relatively long, the data reading efficiency of reading the model data of the target model layer to be run by the processing model and the required input data from the target memory is an important factor restricting the processing efficiency of the processor for performing arithmetic processing on the data related to the target model layer. In this application, two processors for running the processing model are provided in the model processing module. In this way, the two processors can read the data of the second data type required by each of them from the target memory in parallel to improve the data reading efficiency.

[0080] Moreover, considering that the data reading bandwidth supported by the target memory is fixed, in this application, the first processor and the second processor can share the first data of the first data type obtained by the first access control module, that is, while transmitting the first data obtained by the first access control module to the first processor, the first data can be mirrored to the second processor. Thus, there is no need for the two processors to separately obtain the first data from the target memory, and naturally, the amount of data to be read from the target memory can be reduced, and the number of data reading times can be reduced.

[0081] Among them, the first data type can be set according to actual needs. For example, if the first data type is input data, then through the first access control module, while transmitting the input data to the first processor, the corresponding input data can be mirrored to the second processor. Correspondingly, the input data of the first processor and the second processor is the same, but the model data is different. Another example is that if the first data type is model data, then it is equivalent to transmitting the model data obtained by the first access control module to the first processor while mirroring the model data to the second processor. Therefore, the input data of the first processor and the second processor is different, but the model data is the same.

[0082] Since the first processor and the second processor both obtain the first data of the first data type from the target memory through the first access control module, the data of the first data type that the first processor and the second processor need to process is the same. However, the first processor and the second processor will obtain different data of the second data type through different access control modules, so that the first processor and the second processor can be respectively responsible for performing arithmetic processing on the first data of the first data type and the different data of the second data type, and further enabling the first processor and the second processor to synchronously execute the arithmetic between different input data and model data.

[0083] It can be understood that during the process of processing input data based on the target model layer, it is necessary to sequentially perform arithmetic operations on each model data of the target model layer and each input data. For example, it is necessary to perform multiplication and addition operations on each model data in the target model layer and each input data respectively to obtain the output data after the target model layer processes the input data set. Based on this, in the case where the first data of the first data type in the two processors is the same, by separately executing the operation of the first data and different data of the second data type by the two processors, it is natural that each model data can be finally corresponding to each input data through the two processors. That is, the relevant data calculations required by the target model layer can be completed by the two processors.

[0084] It can be understood that the first processor and the second processor may include at least one arithmetic unit. For example, when the first processor and the second processor are NPUs, each NPU may include 16 Multiply Accumulate (MAC) units. Based on this, the first data in this application may include at least one first array, and the number of arrays is the same as the number of arithmetic units in the first processor (or the second processor), and the dimension of each array is based on the number of bits supported by the arithmetic unit. For example, if each arithmetic unit supports the operation between 32-bit data, then each array is 32-bit data. Similarly, the second data includes at least one second array, the third data includes at least one third array, and the number of second arrays in the second data and the number of third arrays in the third data are both the same as the number of first arrays in the first data.

[0085] As can be seen from the above, the model processing module of the present application has a first processor and a second processor. Since the first data of the first data type obtained by each processor is the same, while the data of the second data type is different, the two processors can respectively undertake the arithmetic processing of different data of the first data and the second data type, and jointly complete the data arithmetic processing related to the target model layer in the processing model. On this basis, since the two processors can simultaneously read different data of the second data type from the target memory through different access control modules, the data can be read from the target memory more efficiently. Moreover, considering the limitation of the data reading bandwidth of the target memory, both the first processor and the second processor in the present application can obtain the first data of the first data type read by the first access control module from the target memory, so that the first type of data required by the two first processors only needs to be read from the target memory once, which naturally can reduce the amount of data read from the target memory and improve the data reading efficiency. By improving the data reading efficiency of the two processors, the situation where the arithmetic processing of the data related to the processing model cannot be performed in time due to the low data reading rate can be reduced, and naturally the processing efficiency of the model processing module for performing the data calculation related to the processing model can be improved.

[0086] It can be understood that in the present application, the first data type and the second data type can be set according to actual needs, so that the input data or model data read by the first access control module can be shared between the first processor and the second processor.

[0087] In order to be able to more flexibly control the data types of the data that can be shared between the first processor and the second processor, in the present application, the model processing module may further include a fourth access control module. As Figure 2 , it shows another schematic structural diagram of the model processing module provided by the present application.

[0088] From Figure 2 it can be seen that in addition to including the target memory 101, the first access control module 102, the second access control module 103, the third access control module 104, the first processor 105 and the second processor 106, the model processing module further includes: a fourth access control module 107.

[0089] On this basis, the model processing module in the present application supports two modes.

[0090] In the first mode, the third access control module is used to obtain the third data, and the second processor can obtain the first data from the first access control module and the third data from the third access control module. Correspondingly, in the first mode, the second processor can determine the second output data of the target model layer based on the first data and the third data.

[0091] It can be seen that in the first mode, the first processor and the second processor share the first data of the first data type, but the data of the second data type obtained by the first processor and the second processor is different.

[0092] In the first mode, the fourth access control module may not work, and the second processor does not read data from the target memory through the fourth access control module.

[0093] In the second mode, the fourth access control module 107 is used to obtain the fourth data of the first data type from the target memory. And in the second mode, the second processor can obtain the second data from the second access control module and the fourth data from the fourth access control module, and determine the third output data of the target model layer based on the second data and the fourth data.

[0094] Wherein, the fourth access control module may have a data connection with the second processor directly or indirectly through a bus or other devices.

[0095] It can be understood that in order to enable the second processor to communicate with the first access control module or the second access control module in different modes, there is a direct or indirect data connection between the second processor and the first access control module and the second access control module. Of course, the data connection path between the second processor and the first access control module and the second access control module is a controllable data connection path that can be controlled to be connected or disconnected.

[0096] To distinguish from the second output data determined by the second processor in the first mode, the output data of the target model layer determined by the second processor based on the second data and the fourth data in this application is referred to as the third output data.

[0097] It can be understood that in the second mode, the third access control module does not work. In this case, the data of the first data type obtained by the first processor and the second processor is different, but the first processor and the second processor can share the second data of the second data type.

[0098] Among them, there are many possible specific implementations for the control model processing module to be in the first mode or the second mode, and there is no specific limitation.

[0099] In a possible implementation manner, as Figure 2 shown, the model processing module may further include: a mode configuration module 108.

[0100] Among them, the mode configuration module 108 is configured to obtain a mode indication instruction sent by a controller, and the mode indication instruction is used to indicate that the model processing module is in a first mode or a second mode. Correspondingly, in response to the mode indication instruction being an instruction to start the first mode, the mode configuration module instructs the second processor to be able to access the first access control module and the third access control module; in response to the mode indication instruction being an instruction to start the second mode, it instructs the second processor to be able to access the second access control module and the fourth access control module.

[0101] Among them, the controller can be a processor or a control device outside the model processing module that can send control instructions to the model processing module, and there is no specific limitation. For example, if both the first processor and the second processor are NPUs, the controller can be a Central Processing Unit (CPU), etc., and there is no limitation on this.

[0102] The mode configuration module can directly or indirectly establish a communication connection with the first processor and the second processor through a bus or other communication means.

[0103] Among them, the mode configuration module can be used to control and manage the working modes, resource scheduling, and configuration parameters of the first processor and the second processor. For example, the mode configuration module can be a control register (ControlRegister) in the model processing module.

[0104] It can be understood that on the premise that the first data type and the second data type are fixed, by switching between the first mode and the second mode, the data type of the data that the first processor and the second processor can share can be changed, so that it is possible to more flexibly select whether to mirror and share the input data or the model data according to the data volume of the input data and the model data in the target model layer.

[0105] For example, in a possible implementation manner, the first data type is model data. On this basis, the first data is the first model data, and the first model data belongs to at least one model data in the model set of the target model layer. Correspondingly, the second data type is input data. Based on this, the second data is the first input data, and the first input data belongs to at least one input data in the input data set.

[0106] On this basis, in the first mode, the third data obtained by the third access control module from the target memory is the second input data, and the second input data belongs to at least one input data in the input data set.

[0107] In the second mode, the fourth data obtained by the fourth access control module from the target memory is second model data, and the second model data belongs to at least one model data in the model set of the target model layer.

[0108] In this possible implementation, by Figure 2 It can be seen that in the first mode, after the first access control module obtains the first model data, while transmitting the first model data to the first processor, it can also mirror the first model data to the second processor, without the second processor reading the first model data from the target memory through the fourth access control module, thus avoiding the situation where the two processors repeatedly read the first model data from the target memory, and naturally improving the data reading efficiency. Correspondingly, in the second mode, after the second access control module obtains the first input data, while transmitting the first input data to the first processor, it will also mirror the first input data to the second processor, so that the second processor does not need to read the first input data from the target memory through the third access control module.

[0109] As described above, the first model data may include at least one first array, the number of the first arrays is related to the number of arithmetic units in the first processor, and the first array is an array composed of model data. The number of bits of the model data in the first array is related to the bit width supported by the first processor. For specific details, please refer to the relevant introduction of the first array above and will not be elaborated here. Similarly, the first input data may include at least one second array, the second data is an array composed of input data, the first input data may include at least one third array, the third array is an array composed of input data, and the third array is different from the second array. The second model data may include at least one fourth array, the fourth array is an array composed of model data, and the fourth array is not completely the same as the model data in the first array. The number of bits of the data in the second array, the third array and the fourth array is the same as that in the first array, and will not be elaborated here.

[0110] It can be understood that in the first mode, the first processor and the second processor need to obtain different data of the second data type from the target memory through different data access modules. In order to enable the two processors to accurately obtain different data belonging to the second data type through different data access modules, the controller mentioned above can also pre-split the data of the second data type into two first type data subsets, and store these two first type data subsets into the first memory corresponding to the first processor and the second storage area corresponding to the second processor in the target memory respectively.

[0111] Similarly, in the second mode, the electronic device can also split the data of the first data type into two subsets of the second type data, and store these two subsets of the second type data into the first memory corresponding to the first processor and the second storage area corresponding to the second processor in the target memory respectively.

[0112] For example, taking the first data type as model data for illustration:

[0113] In the first mode, the first processor and the second processor can share the first model data. Therefore, the input data set can be split into a first input data subset and a second input data subset by a controller outside the model control module. The first input subset is stored in the first storage area corresponding to the first processor in the target memory, and the second input subset is stored in the second memory corresponding to the second processor in the target memory. Among them, the first input data subset and the second input data subset are not exactly the same, and the first input data subset and the second input data subset can form the input data set.

[0114] Among them, the purpose of the controller splitting the input data set is to input different parts of the input data set into the first processor and the second processor for arithmetic processing respectively.

[0115] For the specific implementation method of the controller splitting the input data set, it can be controlled by the corresponding segmentation algorithm according to actual needs, as long as it is ensured that each input data subset switched out can perform arithmetic operations with the model data in the target model layer, and there is no specific limitation.

[0116] For the sake of easy understanding, combined with Figure 3 and Figure 4 , a simple illustration is given with an example.

[0117] Such as Figure 3 shows an example diagram of the input data set required by the target model layer and the output data set obtained based on the target model layer. Figure 4 shows Figure 3 an example diagram of the first input data subset and the second input data subset split from the input data set in

[0118] In Figure 3 and Figure 4 both the input data set and the output data set are data matrices. Taking to represent the data dimension, in Figure 3 the left side of the arrow is the input data set that needs to be processed based on the target model layer, and the data dimension of this input data set is The data, where 4 corresponds to the length (or height), 5 corresponds to the width, and 8 corresponds to the depth. Therefore, this input data set corresponds to data of 0 - 159 bits. Correspondingly, the data dimension obtained by performing arithmetic processing on the input data set based on the target model layer is The output data set, and the size of the output data set is 96 bits.

[0119] On the basis of Figure 3 , assuming that the data dimension of the first model data read by each arithmetic unit in the first processor and the second processor is , then Figure 3 The input data set in can be switched to Figure 4 The first input data subset and the second input data subset shown.

[0120] From Figure 4 it can be seen that the data dimension of the first input data subset is , so the first input data subset is data of 120 bits. To ensure that data can be taken out from the second input data subset for arithmetic processing such as matrix multiplication with the first model data, therefore, the length and width of the second input data subset should not be less than . Based on this, the second input data subset can have partial data overlap with the first input data subset. For example, Figure 4 The dark gray part in is the overlapping data part between the first input data subset and the second input data subset. On this basis, the data dimension of the second input data subset is . Correspondingly, the input data in the first input subset is operated with the model data in the model set of the target model layer through the first processor to obtain Figure 4 The first output data in, and similarly, the second output data can be obtained through the second processor.

[0121] The first output data and the second output data actually constitute Figure 3 The output data in. For example, Figure 4 The first output data in is data of 64 bits, and the data dimension is , while the second output data is data of 32 bits, and the data dimension is . Therefore, by combining the first output data and the second output data, Figure 3 The output data with the dimension in is obtained.

[0122] Of course, Figure 3 This is just a simple example, and this application does not limit the specific switching method and control of how the controller outside the model processing module divides the input data set.

[0123] On the above basis, in the first mode, the target memory is used to store, in the first storage area corresponding to the first processor, the model set of the target model layer to be run in the processing model and the first input data subset to be processed by the target model layer, and store, in the second storage area corresponding to the second processor, the second input data subset to be processed by the target model layer.

[0124] Correspondingly, in the first mode, the first access control module is used to obtain first model data from the model set in the first storage area of the target memory; the second access control module is used to obtain first input data from the first input data subset in the first storage area; and the third access control module is used to obtain second input data from the second input data subset in the second storage area of the target memory.

[0125] Still taking the first data type as model data as an example, in the second mode, the first processor and the second processor can share the first input data. Therefore, the model set can be split into a first sub-model set and a second sub-model set by a controller other than the model control module, and the first sub-model set and the second sub-model set are respectively stored in the first memory and the second memory. Among them, the first sub-model set and the second sub-model set can form the model set of the target model layer. Based on this, in the second mode, the target memory is used to store, in the first storage area corresponding to the first processor, the first sub-model set of the target model layer to be run in the processing model and the input data set to be processed by the target model layer, and store, in the second storage area corresponding to the second processor, the second sub-model set of the target model layer.

[0126] Correspondingly, in the second mode, the first access control module is used to obtain first model data from the first sub-model set in the first storage area; the second access control module is used to obtain first input data from the input data set in the first storage area; and the fourth access control module is used to obtain second model data from the second sub-model set in the second storage area.

[0127] It can be understood that in this application, controlling the model processing module to enter the first mode or the second mode can be pre-specified or selected manually according to needs. In particular, usually, compared with the data volume in the input data set corresponding to the target model layer, the data volume of the model set of the target model layer is relatively small. Therefore, in most cases, the cache spaces of the arithmetic units of the first processor and the second processor can accommodate the model data in the model set of the target model layer.

[0128] Based on this, when the first data type is model data, if the data volume of the model set in the target model layer is not greater than the cache space of the arithmetic units in the first processor (or the second processor), the model set does not need to be split. Therefore, the controller can control the model processing module to be in the first mode. That is to say, when the first data type is model data, the mode indication instruction for starting the first mode can indicate that the data volume of the model set in the target model layer is not greater than the cache space of the arithmetic units in the first processor. On the contrary, the mode indication instruction for starting the second mode can indicate that the data volume of the model set in the target model layer is greater than the cache space of the arithmetic units in the first processor.

[0129] For the sake of easy understanding, the process of parallelly executing the data operations of the target model layer by the first processor and the second processor will be described below in combination with Figure 5 for illustration. Figure 5 Fig. shows an example diagram of the processor in the model processing module performing operations on the input data and the model data.

[0130] In Figure 5 , it is taken as an example that the processor includes 8 arithmetic units and the arithmetic unit is a MAC unit. For the sake of easy description, Figure 5 only introduces the operation process executed by a single processor. It is assumed that the input data processed by the two processors is different, but the model data is the same. Then, except for the different input data in different processors, the operation processing process is the same.

[0131] Since each data in the input data set needs to perform multiplication and addition operations with each model data in the model set, therefore, in Figure 5 , the input data of each MAC unit in the input processor can be the same, but the model data processed by each MAC unit is different.

[0132] On this basis, the input data of can be taken out from the input data set. The input data is 32 bits. As shown in Figure 5 , the input data required for each MAC unit in the input processor is the data covered by the green part area in the input data set. Correspondingly, 8 copies of model data need to be taken out from the model set of the target model layer. Each copy of model data is also data. Therefore, each copy of model data is also 32-bit data. As shown in Figure 5 , multiple three-dimensional data matrices of different colors represent different model data.

[0133] Correspondingly, each MAC unit of the processor performs matrix multiplication and addition operations on the input data and model data. Finally, the output data output by these 8 MAC units can form the output data obtained by performing operations on the input data in the green part and different model data in the target model layer. In Figure 5 the different layers of the output data on the right side in are of different colors, and each layer corresponding to each color represents the data result obtained by performing matrix multiplication and addition operations on the model data corresponding to that color and the input data.

[0134] It can be seen that the input data processed by the 8 MAC units is the same, and the processed model data is different. However, each MAC unit can independently perform matrix multiplication and addition operations, and finally the data results output by each MAC unit can be combined into output data.

[0135] Correspondingly, if there are two processors, then the input data processed by each MAC unit in these two processors is the same, but the processed model data is different. In this way, the matrix multiplication and addition operations between the input data in the green part and 18 different portions of model data are completed in parallel by the two processors.

[0136] It can be understood that after completing the processing of the input data in the green part of the input data set and all model data in all model sets, the next input data to be processed can continue to be taken out from the input data set, and then the above operations are repeated until each input data in the input data set has completed the multiplication and addition operations with each model data in the model set. Figure 5 For any access control module mentioned in this application, there can be various possible specific forms of this access control module, which are not specifically limited.

[0137] In a possible implementation manner, in order to enable the data obtained by the access control module to be input into each arithmetic unit of the processor in a first-in, first-out manner, in this application, each access control module may include: an access storage controller, a first-in, first-out (FIFO) module, and a parameter receiving controller.

[0138] Among them, the access storage controller is responsible for reading data from the target memory to transfer the data from the target memory to the processor.

[0139] The first-in, first-out module is used to cache the data read by the access storage controller from the target memory in sequence, so that the processor can obtain data according to the first-in, first-out principle.

[0140] The parameter receiving controller is used to take out the data cached in the first-in, first-out module and transmit it to the processor to ensure the correctness and timing of data calculation.

[0141] The parameter receiving controller is used to take out the data cached in the first-in, first-out module and transmit it to the processor to ensure the correctness and timing of data calculation.

[0142] As shown in Figure 6 , a schematic diagram of another composition structure of the model processing module provided by the present application is shown.

[0143] In Figure 6 , the first access control module includes: a first access storage controller 1021, a first first-in-first-out module 1022, and a first parameter receiving controller 1023.

[0144] The second access control module includes: a second access storage controller 1031, a second first-in-first-out module 1032, and a second parameter receiving controller 1033.

[0145] The third access control module includes: a third access storage controller 1041, a third first-in-first-out module 1042, and a third parameter receiving controller 1043.

[0146] The fourth access control module includes: a fourth access storage controller 1071, a fourth first-in-first-out module 1072, and a fourth parameter receiving controller 1073.

[0147] On this basis, in the first mode, the second processor 106 can obtain the first data from the first access storage controller 1021 through the fourth first-in-first-out module 1072 and the fourth parameter receiving controller 1073. For example, when the first data type is model data, the first data is the first model data.

[0148] In the second mode, the second processor 106 can obtain the second data from the second access storage controller 1031 through the third first-in-first-out module and the third parameter receiving controller.

[0149] For example, in addition to establishing a data connection with the first first-in-first-out module 1022, the first access storage controller 1021 can also establish a data connection with the fourth first-in-first-out module 1042. On this basis, after obtaining the first data of the first data type from the target memory, the first access storage controller can not only transmit the first data to the first processor through the first first-in-first-out module, but also mirror and transmit the first data to the fourth first-in-first-out module, so that the second processor can also obtain the first data.

[0150] Similarly, in addition to being able to establish a data connection with the second first-in-first-out module 1032, the second access storage controller 1031 can also establish a data connection with the third first-in-first-out module 1042, so that the second processor can also obtain the second data read by the second access storage controller from the target memory.

[0151] Among them, for Figure 6For information such as the functions of other component modules, reference may be made to the relevant descriptions in the foregoing embodiments, which will not be elaborated herein.

[0152] It can be understood that in any of the foregoing embodiments of the present application, the output data obtained by processing the input data set based on the target model layer is actually the input data of the next model layer of the target model layer. However, the first output data and the second output data obtained by the first processor and the second processor may have inconsistent dimensions with the target data corresponding to the model data in the next model layer, resulting in the inability to directly perform operations on the second output data and the second output data with the model data in the next model layer.

[0153] Based on this, the model processing module may further include: a first output processing module and a second output processing module.

[0154] On this basis, the first processor is further configured to output the first output data to the first output processing module; and determine the target data dimension corresponding to the model set in the next model layer after the target model layer in the processing model, and send the target data dimension to the first output processing module. Correspondingly, the first output processing module is configured to process the first output data into the first target output data with the target data dimension.

[0155] Similarly, the second processor is further configured to output the second output data to the second output processing module; and determine the target data dimension corresponding to the model set in the next model layer, and send the target data dimension to the second output processing module. The second output processing module is configured to process the second output data into the second target output data with the target data dimension.

[0156] Wherein, the next model layer is the next model layer after the target model layer in the processing model. The model set of the next model layer includes at least one model parameter participating in the calculation in the next model layer, such as model data such as weight data in the next model layer. For the sake of distinction, the model set in the target model layer may also be referred to as the first model set, and the model set of the next model layer may be referred to as the second model set.

[0157] Further, in order to enable the processor to read the relevant data required for the next model layer to perform calculations from the target memory when processing the next model layer, the first output processing module is further configured to store the first target output data in the target memory, for example, store the first target output data in the first storage area of the target memory. Similarly, the second output processing module is further configured to store the second target output data in the target memory, for example, store the second target output data in the second storage area of the target memory.

[0158] For ease of understanding, take Figure 6Taking the model processing module shown as an example, from Figure 6 It can be seen that the first processor 105 can also be connected to a first output processing module 109, and the second processor 106 is also connected to a second output processing module 110.

[0159] In addition, there is a direct or indirect data connection channel between the first output processing module 109 and the second output processing module 110 and the target memory 101. On this basis, the first output processing module can store the first target output data in the target memory, and the second output processing module can also store the second target output data in the target memory.

[0160] Specifically, when the model processing module can have a first mode and a second mode, in the first mode and the second mode, the first processor outputs first output data to the first output processing module, and correspondingly, the first output processing module also processes the first target transmission data. However, in the first mode and the second mode, the specific operations of the second processor and the second output processing module will be different.

[0161] Specifically, in the first mode, the second processor outputs second output data to the second output processing module and sends the target data dimension to the second output processing module; correspondingly, the second output processing module is used to process the second output data into second target output data with the target data dimension.

[0162] In the second mode, the first processor is further used to output third output data to the second output processing module, and determine the target data dimension corresponding to the model set in the next model layer, and send the target data dimension to the second output processing module; correspondingly, the second output processing module is used to process the third output data into third target output data with the target data dimension.

[0163] In this application, since both the first output processing module and the second output processing module are hardware, this application can use the hardware to align the output data with the data dimension in the model set in the next model layer, so as to effectively utilize the characteristics of the hardware itself and more efficiently achieve data dimension alignment.

[0164] In a possible implementation manner, the first output processing module is specifically used to construct a first all-zero matrix of the target data dimension, and based on the positions of the data in the first output data corresponding to the first all-zero matrix, fill the data in the first output data into the corresponding positions in the first all-zero matrix to obtain the first target output data. It can be seen that the first target output data is data of the target data dimension. Among them, the all-zero matrix refers to a matrix with all values at each position being zero.

[0165] Similarly, the first output processing module is specifically configured to construct a second all-zero matrix for the target data dimension, and fill each data in the second output data into the corresponding position in the second all-zero matrix based on the position of each data in the second output data corresponding to the second all-zero matrix.

[0166] It can be understood that the first output data and the second output data are also matrix data. Moreover, when the first output data is determined, the storage intervals occupied by each data in the first output data in the target memory are fixed, and the number of bits in this storage interval is the same as the number of bits corresponding to the data of the target data dimension. Therefore, it can be determined which position in the data of the target data dimension each data in the first output data corresponds to. However, since the dimension of the first output data is inconsistent with the data dimension in the model set in the next model layer, the bits corresponding to each data in the first output data are not continuous.

[0167] For ease of understanding, in combination with Figure 7 for illustration. Figure 7 shows a schematic diagram of an implementation principle of processing the first output data of the dimension into the first target output data of the

[0168] It can be understood that the first output data and the model set in the next model layer can be three-dimensional data. However, for ease of description and more intuitive understanding of the dimension alignment process, in Figure 7 the dimension alignment of two-dimensional data is introduced.

[0169] As Figure 7 shown, the first output data is data of the dimension, and the model set of the next model layer is data of the dimension. On this basis, a 64-bit data storage area is allocated for the first output data. For ease of introduction, these 64 bits are respectively called bit 0 to bit 63. Therefore, the data "1" in the first row and first column of the first output data should be located at bit 9, and the data "2" in the first row and second column is located at bit 10. And so on, the data "6" in the first row and last column of the first output data is located at bit 14. And the data "7" in the second row and first column of the first output data should be located at bit 17. It can be seen that the number "6" in the first row and last column of the first output data is not continuous with the data "7" in the second row and first column.

[0170] On this basis, since the dimension of the model set in the next model layer is dimension, therefore, the first output processing module can construct A zero matrix in a certain dimension. It can be understood that after mapping the zero matrix to the 64-bit data storage area corresponding to the first output data, the "0" in the first row and first column of the zero matrix corresponds to bit 0, the "0" in the first row and second column corresponds to bit 1, and so on. It can be seen that the data from the second row and second column to the second row and seventh column in the zero matrix corresponds to the data from the first row and first column to the last column in the first output data. Correspondingly, as Figure 7 It can be seen that by skipping the places in the zero matrix that do not need to be filled with data, the positions on the zero matrix corresponding to the data in other positions in the first output data can be determined and filled, and the zeros in the corresponding positions are replaced with the data in the corresponding positions in the first output data.

[0171] Based on this, in this application, each data in the first output data can be filled into the corresponding positions on the zero matrix, and then the target output data as shown in Figure 7 can be obtained.

[0172] Of course, if the first output data is data, and the target data dimension is , then only a zero matrix with dimension needs to be constructed. However, the process of filling the first output data into the corresponding positions in the zero matrix with dimension is similar and will not be elaborated here.

[0173] It can be understood that the above is described by taking the first output data as an example. The process of processing the second output data into the second target output data is similar and is not specifically limited.

[0174] It can be understood that after constructing a zero matrix with the target data dimension through the first output processing module or the second output processing module, only the data in the first output data and the second output data need to be filled into the corresponding positions on the zero matrix, so that the dimension alignment operation can be completed through one operation. Compared with the method of addressing each data in the first output data or the second output data through a software program and performing multiple zero-padding operations, this application greatly improves the efficiency of dimension alignment.

[0175] In this application, the specific forms of the first output processing module and the second output processing module can also have multiple possibilities and are not specifically limited.

[0176] To facilitate the understanding of the solution of this application, below, taking both the first processor and the second processor as NPUs as an example, a possible component structure of the Figure 8 shown model processing module will be described.

[0177] Figure 8 The model processing module shown can be an artificial intelligence chip carrying two NPUs. InFigure 8 Taking the processing model processed by the NPU in the middle as a convolutional model as an example, therefore, the model set of each convolutional layer in the convolutional model is a convolutional kernel, and the convolutional kernel includes the weight matrix in the convolutional layer.

[0178] From Figure 8 It can be seen that the model processing module includes two NPUs, such as Figure 8 NPU1 and NPU2 in the middle. There is an input data transmission channel and a model data transmission channel between each NPU and the SRAM memory.

[0179] Taking NPU1 as an example for illustration, the input data transmission channel between NPU1 and the SRAM memory successively includes: an input direct memory access (DMA) controller, an input first-in first-out module, and an instruction fetch controller.

[0180] Among them, the input DMA controller is a DMA controller for reading input data, which is equivalent to the access controller for reading data in the previous embodiment. The input first-in first-out module is a first-in first-out module for transmitting input data. The instruction fetch controller is equivalent to the parameter receiving controller mentioned above.

[0181] The model data transmission channel between NPU1 and the SRAM memory successively includes: a model DMA controller, a model first-in first-out module, and an instruction fetch controller. Among them, the model DMA controller is a DMA controller for reading model data, and the model first-in first-out module is a first-in first-out module for transmitting model data.

[0182] From Figure 8 It can be seen that there is also a connection between the input first-in first-out module corresponding to NPU2 and the input DMA controller corresponding to NPU1, so that NPU2 can obtain the input data read from the SRAM by the input DMA controller corresponding to NPU1. Moreover, there is also a connection between the model first-in first-out module corresponding to NPU2 and the model DMA controller corresponding to NPU1, so that NPU2 can obtain the model data read from the SRAM memory by the model DMA controller corresponding to NPU1.

[0183] In Figure 8 each NPU, there are 8 MAC units. The 8 MAC units in NPU1 are successively denoted as MAC0 - MAC7, and the 8 MAC units in NPU2 are successively denoted as MAC8 - MAC15.

[0184] Each NPU is also connected to an output processing module, and a data quantization unit and a mapping unit are also connected between each NPU and the output processing module. The data quantization unit can perform quantization processing on the output data, and the mapping unit can perform processing such as linear conversion according to the mapping relationship, without specific limitations.

[0185] The output processing module connected to each NPU is also connected to the SRAM memory, so that the output processing module can write the processed output data back to the SRAM memory.

[0186] In addition, the model processing module further includes a control register, and through this control register, NPU1 and NPU2 can be controlled to be in the first mode or the second mode.

[0187] On the other hand, the present application also provides an electronic device, such as Figure 9 , which shows a schematic diagram of a composition structure of the electronic device provided by the present application.

[0188] The electronic device at least includes: a model processing module 900;

[0189] Among them, the model processing module 900 includes:

[0190] A target memory 901, which is used to store the model set of the target model layer to be run in the processing model and the input data set to be processed by the target model layer. The model set includes at least one model data, and the input data set includes at least one input data;

[0191] A first access control module 902, which is used to obtain first data of a first data type from the target memory, and the first data type is one of model data or input data;

[0192] A second access control module 903, which is used to obtain second data of a second data type from the target memory, the second data type is one of model data or input data, and the second data type is different from the first data type;

[0193] A third access control module 904, which is used to obtain third data of the second data type from the target memory;

[0194] A first processor 905, which is used to determine first output data of the target model layer based on the first data and the second data;

[0195] A second processor 906, which is used to determine second output data of the target model layer based on the first data and the third data.

[0196] Of course, the model processing module may further include a fourth access control module 907, specifically as described in the relevant introduction of the fourth access control module above.

[0197] For the specific functions of each part in the model processing module, reference may be made to the relevant introduction in the foregoing embodiments, which will not be elaborated here.

[0198] In a possible implementation manner, the electronic device may further include: a controller 908. The controller may be a CPU or other control components, and there is no limitation thereto.

[0199] The model processing module may further include a mode configuration module 909, configured to obtain a mode indication instruction sent by the controller, where the mode indication instruction is used to indicate that the model processing module is in a first mode or a second mode; in response to the mode indication instruction being an instruction to start the first mode, indicating that the second processor can access the first access control module and the third access control module; in response to the mode indication instruction being an instruction to start the second mode, indicating that the second processor can access the second access control module and the fourth access control module.

[0200] In addition, it should be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationships between the modules indicate that they have communication connections, which may be specifically implemented as one or more communication buses or signal lines.

[0201] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware. Of course, it can also be implemented by dedicated hardware including application-specific integrated circuits, dedicated CPUs, dedicated memories, dedicated components, etc. Generally, functions completed by computer programs can be easily implemented by corresponding hardware, and the specific hardware structures for implementing the same function can also be various, such as analog circuits, digital circuits or dedicated circuits, etc. However, for the present application, software program implementation is a better embodiment in more cases. Based on such an understanding, the technical solution of the present application, in essence or the part that makes a contribution to the prior art, can be embodied in the form of a software product. The computer software product is stored in a readable storage medium, such as a floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disc of a computer, etc., and includes several instructions for causing a computer device (which can be a personal computer, training device, or network device, etc.) to execute the methods described in various embodiments of the present application.

[0202] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product.

[0203] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a dedicated computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, computer, training device or data center to another website, computer, training device or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)), etc.

Claims

1. A model processing module, comprising: A target memory for storing a model set of a target model layer to be run in a processing model and an input data set to be processed by the target model layer, the model set including at least one model data, and the input data set including at least one input data; A first access control module for obtaining first data of a first data type from the target memory, the first data type being one of model data or input data; A second access control module for obtaining second data of a second data type from the target memory, the second data type being one of model data or input data, and the second data type being different from the first data type; A third access control module for obtaining third data of the second data type from the target memory; A first processor for determining first output data of the target model layer based on the first data and the second data; A second processor for determining second output data of the target model layer based on the first data and the third data.

2. The model processing module according to claim 1, in a first mode, the third access control module is used to obtain the third data, and the second processor can obtain the first data from the first access control module and obtain the third data from the third access control module; The model processing module further comprises: A fourth access control module; In a second mode, the fourth access control module is used to obtain fourth data of the first data type from the target memory, and the second processor can obtain the second data from the second access control module and obtain the fourth data from the fourth access control module, and determine third output data of the target model layer based on the second data and the fourth data.

3. The model processing module according to claim 2, the first data is first model data, and the first model data belongs to the at least one model data; The second data is first input data, and the first input data belongs to the at least one input data; In the first mode, the third data is second input data, and the second input data belongs to the at least one input data; In the second mode, the fourth data is second model data, and the second model data belongs to the at least one model data.

4. The model processing module according to claim 2 or 3 further comprises: A mode configuration module for obtaining a mode indication instruction sent by a controller, the mode indication instruction being used to indicate that the model processing module is in the first mode or the second mode; in response to the mode indication instruction being an instruction to start the first mode, indicating that the second processor can access the first access control module and the third access control module; in response to the mode indication instruction being an instruction to start the second mode, indicating that the second processor can access the second access control module and the fourth access control module.

5. The model processing module according to claim 3, in the first mode, the target memory is configured to store, in a first storage area corresponding to a first processor, a model set of a target model layer to be run in the processing model and a first input data subset to be processed by the target model layer, and store, in a second storage area corresponding to a second processor, a second input data subset to be processed by the target model layer, where the first input data subset and the second input data subset constitute the input data set; In the first mode, the first access control module is configured to obtain first model data from the model set in the first storage area; In the first mode, the second access control module is configured to obtain first input data from the first input data subset in the first storage area; In the first mode, the third access control module is configured to obtain second input data from the second input data subset in the second storage area.

6. The model processing module according to claim 3, in the second mode, the target memory is configured to store, in a first storage area corresponding to a first processor, a first sub-model set of a target model layer to be run in the processing model and an input data set to be processed by the target model layer, and store, in a second storage area corresponding to a second processor, a second sub-model set of the target model layer, where the first sub-model set and the second sub-model set constitute the model set of the target model layer; In the second mode, the first access control module is configured to obtain first model data from the first sub-model set; In the second mode, the second access control module is configured to obtain first input data from the input data set in the first storage area; In the second mode, the fourth access control module is configured to obtain second model data from the second sub-model set in the second storage area.

7. The model processing module according to claim 2 or 3, wherein the first access control module comprises: A first access storage controller, a first first-in-first-out module, and a first parameter receiving controller; The second access control module includes: a second access storage controller, a second first-in-first-out module, and a second parameter receiving controller; The third access control module includes: a third access storage controller, a third first-in-first-out module, and a third parameter receiving controller; The fourth access control module includes: a fourth access storage controller, a fourth first-in-first-out module, and a fourth parameter receiving controller; In the first mode, the second processor can obtain the first data from the first access storage controller through the fourth first-in-first-out module and the fourth parameter receiving controller; In the second mode, the second processor can obtain the second data from the second access storage controller through the third first-in-first-out module and the third parameter receiving controller.

8. The model processing module according to claim 1 further includes: A first output processing module and a second output processing module; The first processor is further configured to output first output data to the first output processing module; and determine a target data dimension corresponding to a model set in a next model layer after the target model layer in the processing model, and send the target data dimension to the first output processing module. The first output processing module is configured to process the first output data into first target output data in the target data dimension; The second processor is further configured to output second output data to the second output processing module; and determine the target data dimension corresponding to the model set in the next model layer, and send the target data dimension to the second output processing module; The second output processing module is configured to process the second output data into second target output data in the target data dimension.

9. The model processing module according to claim 8, wherein the first output processing module is specifically configured to construct a first all-zero matrix in the target data dimension, and based on the positions of the respective data in the first output data corresponding to the first all-zero matrix, fill the respective data in the first output data into the corresponding positions in the first all-zero matrix to obtain first target output data; The first output processing module is specifically configured to construct a second all-zero matrix in the target data dimension, and based on the positions of the respective data in the second output data corresponding to the second all-zero matrix, fill the respective data in the second output data into the corresponding positions in the second all-zero matrix to obtain second target output data.

10. An electronic device, at least comprising: Model processing module; The model processing module includes: A target memory for storing a model set of a target model layer to be run in a processing model and an input data set to be processed by the target model layer, the model set including at least one model data, and the input data set including at least one input data; A first access control module for obtaining first data of a first data type from the target memory, the first data type being one of model data or input data; A second access control module for obtaining second data of a second data type from the target memory, the second data type being one of model data or input data, and the second data type being different from the first data type; A third access control module for obtaining third data of the second data type from the target memory; A first processor for determining first output data of the target model layer based on the first data and the second data; A second processor for determining second output data of the target model layer based on the first data and the third data.

Images