Model arrangement method and device, electronic equipment and storage medium

Abstract

The invention provides a model arrangement method and apparatus, an electronic device, a chip and a storage medium. The method comprises the steps of obtaining information of a virtual storage and calculation array corresponding to a storage and calculation unit on a kernel of a storage and calculation integrated chip; wherein the information comprises size information and a first number determined based on a DAC ratio and an ADC ratio, the DAC ratio is a ratio between an input dimension of the storage and calculation unit and the number of DACs, and the ADC ratio is a ratio between an output dimension of the storage and calculation unit and the number of ADCs; segmenting each layer of network weight in the target model according to the size information to obtain a plurality of blocks; respectively mapping the plurality of blocks to a first number of virtual storage and calculation arrays to obtain an arrangement scheme of the target model; wherein the arrangement scheme of the target model at least comprises virtual storage and calculation arrays which are executed in parallel, and each virtual storage and calculation array in the first number of virtual storage and calculation arrays respectively and correspondingly executes DAC conversion of each cycle number and ADC conversion of the cycle number in the storage and calculation unit.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of storage and computing integration, and particularly relates to a model arrangement method and device, an electronic device, and a storage medium. BACKGROUND

[0002] A storage and computing integration chip breaks through the bottleneck of the traditional Von Neumann architecture by deeply integrating storage and computing functions, and provides a higher computing power and higher energy efficiency solution for artificial intelligence computing.

[0003] In the prior art, each layer network in a model is deployed on a storage and computing integration chip. Taking two layer networks as an example, the second layer network takes the calculation result of the first layer network as input data, and the second layer network needs to wait for the first layer network to finish calculation before starting, which leads to low utilization of the computing power of the storage and computing integration chip and increases the calculation delay. SUMMARY

[0004] The present disclosure provides a model arrangement method and device, an electronic device, and a storage medium. A target model is divided into multiple blocks according to the size of a virtual storage and computing array, parallel computing of the virtual storage and computing array having a mapping relationship with the blocks in the front and rear layer networks can be implemented, thereby improving the utilization of the computing power of the storage and computing integration chip and reducing the calculation delay.

[0005] A first aspect embodiment of the present disclosure provides a model arrangement method. The method comprises: obtaining information of a virtual storage and computing array corresponding to a storage and computing unit on a kernel of a storage and computing integration chip; wherein the information comprises size information and a first number determined based on a DAC ratio and an ADC ratio, the DAC ratio is a ratio between an input dimension of the storage and computing unit and a number of DACs, and the ADC ratio is a ratio between an output dimension of the storage and computing unit and a number of ADCs; dividing each layer network weight in a target model according to the size information to obtain multiple blocks; and mapping the multiple blocks to the first number of virtual storage and computing arrays respectively to obtain an arrangement scheme of the target model; wherein the arrangement scheme of the target model at least comprises virtual storage and computing arrays executed in parallel, and each virtual storage and computing array in the first number of virtual storage and computing arrays corresponds to DAC conversion and ADC conversion of each cycle number in the storage and computing unit.

[0006] The DAC ratio is a ratio between an input dimension of the storage and computing unit and a number of DACs, and the ADC ratio is a ratio between an output dimension of the storage and computing unit and a number of ADCs; dividing each layer network weight in a target model according to the size information to obtain multiple blocks; and mapping the multiple blocks to the first number of virtual storage and computing arrays respectively to obtain an arrangement scheme of the target model; wherein the arrangement scheme of the target model at least comprises virtual storage and computing arrays executed in parallel, and each virtual storage and computing array in the first number of virtual storage and computing arrays corresponds to DAC conversion and ADC conversion of each cycle number in the storage and computing unit.

[0007] In some embodiments of the present disclosure, dividing each layer network weight in a target model according to the size information to obtain multiple blocks comprises: dividing each layer network weight in the target model according to the length in the size information in the longitudinal direction, and dividing each layer network weight in the target model according to the width in the size information in the transverse direction to obtain the multiple blocks.

[0008] In some embodiments of the present disclosure, the method further comprises: recording split information of the plurality of blocks; wherein the split information comprises a mode number of a split mode corresponding to each block and a layer number of a layer where each block is located; the mode number comprises a first mode number of horizontal splitting and a second mode number of vertical splitting; and determining the block arrangement scheme according to the split information of the plurality of blocks.

[0009] In some embodiments of the present disclosure, determining the block arrangement scheme according to the split information of the plurality of blocks comprises: if there is split information of a first block in the split information of the plurality of blocks, the block arrangement scheme comprises a first arrangement scheme of setting the first block as a block for parallel execution; wherein the first block comprises all blocks with the same layer number; and if there is split information of a second block in the split information of the plurality of blocks, the block arrangement scheme comprises a second arrangement scheme of setting the second block as a block for parallel execution; wherein the second block comprises all blocks with the same second mode number.

[0010] In some embodiments of the present disclosure, mapping the plurality of blocks to the first number of virtual storage-computing arrays respectively to obtain the arrangement scheme of the target model comprises: mapping the plurality of blocks to the first number of virtual storage-computing arrays respectively to obtain a plurality of array arrangement schemes; and selecting an array arrangement scheme that meets a preset condition from the plurality of array arrangement schemes as the arrangement scheme of the target model.

[0011] In some embodiments of the present disclosure, mapping the plurality of blocks to the first number of virtual storage-computing arrays respectively comprises: if the split information of the plurality of blocks comprises split information of a first block, mapping the blocks in the first arrangement scheme to corresponding virtual storage-computing arrays in the first number of virtual storage-computing arrays respectively; if the split information of the plurality of blocks comprises split information of a second block, mapping the blocks in the second arrangement scheme to corresponding virtual storage-computing arrays in the first number of virtual storage-computing arrays respectively; and if the split information of the plurality of blocks comprises split information of a third block, mapping the third block to corresponding virtual storage-computing arrays in the first number of virtual storage-computing arrays respectively; wherein the third block comprises all blocks with the same first mode number in the plurality of blocks.

[0012] In some embodiments of the present disclosure, if the split information of the plurality of blocks comprises split information of a first block, mapping the blocks in the first arrangement scheme to DACs with different cycle numbers in the first number of virtual storage-computing arrays and to DACs with different cycle numbers; if the split information of the plurality of blocks comprises split information of a second block, mapping the blocks in the second arrangement scheme to DACs with the same cycle number in the first number of virtual storage-computing arrays respectively; and if the split information of the plurality of blocks comprises split information of a third block, mapping the third block to ADCs with the same cycle number in the first number of virtual storage-computing arrays respectively.

[0013] In some embodiments of the present disclosure, the array arrangement scheme satisfying the preset condition among the plurality of array arrangement schemes is taken as the arrangement scheme of the target model, including: respectively determining the time delay of each array arrangement scheme among the plurality of array arrangement schemes; and taking the array arrangement scheme satisfying the preset condition among the plurality of array arrangement schemes as the arrangement scheme of the target model.

[0014] In some embodiments of the present disclosure, the time delay of each array arrangement scheme among the plurality of array arrangement schemes is respectively determined, including: respectively determining the first total number of times of performing ADC conversion, the second total number of times of performing DAC conversion, and the third total number of times of performing virtual memory and calculation array calculation of each array arrangement scheme among the plurality of array arrangement schemes; obtaining the first time delay of performing one ADC conversion, the second time delay of performing one DAC conversion, and the third time delay of performing one virtual memory and calculation array calculation; and determining the time delay of each array arrangement scheme among the plurality of array arrangement schemes according to the first total number of times and the first time delay, the second total number of times and the second time delay, and the third total number of times and the third time delay.

[0015] In some embodiments of the present disclosure, the first total number of times of performing ADC conversion, the second total number of times of performing DAC conversion, and the third total number of times of performing virtual memory and calculation array calculation of each array arrangement scheme among the plurality of array arrangement schemes are respectively determined, including: respectively determining whether there is parallel DAC conversion, whether there is parallel ADC conversion, and whether there is parallel virtual memory and calculation array calculation in each array arrangement scheme among the plurality of array arrangement schemes; if there is parallel DAC conversion in each array arrangement scheme, taking the parallel DAC conversion as one DAC conversion; otherwise, adding 1 to the number of times of performing DAC conversion until all DAC conversions included in each array arrangement scheme are traversed, to obtain the first total number of times; if there is parallel ADC conversion in each array arrangement scheme, taking the parallel ADC conversion as one ADC conversion; otherwise, adding 1 to the number of times of performing ADC conversion until all ADC conversions included in each array arrangement scheme are traversed, to obtain the second total number of times; and if there is parallel virtual memory and calculation array calculation in each array arrangement scheme, taking the parallel virtual memory and calculation array calculation as one virtual memory and calculation array calculation; otherwise, adding 1 to the number of times of performing virtual memory and calculation array calculation until all virtual memory and calculation array calculations included in each array arrangement scheme are traversed, to obtain the third total number of times.

[0016] In some embodiments of the present disclosure, the first total number of times of performing ADC conversion, the second total number of times of performing DAC conversion, and the third total number of times of performing virtual memory and calculation array calculation of each array arrangement scheme among the plurality of array arrangement schemes are respectively determined, including: respectively determining whether there is parallel DAC conversion, whether there is parallel ADC conversion, and whether there is parallel virtual memory and calculation array calculation in each array arrangement scheme among the plurality of array arrangement schemes; if there is parallel DAC conversion in each array arrangement scheme, taking the parallel DAC conversion as one DAC conversion; otherwise, adding 1 to the number of times of performing DAC conversion until all DAC conversions included in each array arrangement scheme are traversed, to obtain the first total number of times; if there is parallel ADC conversion in each array arrangement scheme, taking the parallel ADC conversion as one ADC conversion; otherwise, adding 1 to the number of times of performing ADC conversion until all ADC conversions included in each array arrangement scheme are traversed, to obtain the second total number of times; and if there is parallel virtual memory and calculation array calculation in each array arrangement scheme, taking the parallel virtual memory and calculation array calculation as one virtual memory and calculation array calculation; otherwise, adding 1 to the number of times of performing virtual memory and calculation array calculation until all virtual memory and calculation array calculations included in each array arrangement scheme are traversed, to obtain the third total number of times.

[0017] In some embodiments of the present disclosure, determining whether there are parallel DAC conversions, parallel ADC conversions, and parallel virtual memory-computing array computations in each of the plurality of array arrangement schemes respectively comprises: if there is a first array arrangement scheme in the plurality of array arrangement schemes, determining that there are parallel ADC conversions, parallel DAC conversions, and parallel virtual memory-computing array computations in the first array arrangement scheme; wherein the first array arrangement scheme comprises virtual memory-computing arrays corresponding to memory-computing units of different cores; if there is a second array arrangement scheme in the plurality of array arrangement schemes, determining whether there are parallel DAC conversions, parallel ADC conversions, and parallel virtual memory-computing array computations in the second array arrangement scheme; wherein the second array arrangement scheme comprises virtual memory-computing arrays corresponding to memory-computing units on the same core; and if there is a third array arrangement scheme in the plurality of array arrangement schemes, determining whether there are parallel ADC conversions, parallel DAC conversions, and parallel virtual memory-computing array computations in the third array arrangement scheme; wherein the third array arrangement scheme comprises virtual memory-computing arrays corresponding to the same memory-computing unit.

[0018] In some embodiments of the present disclosure, determining whether there are parallel ADC conversions, parallel DAC conversions, and parallel virtual memory-computing array computations in the third array arrangement scheme comprises: if there is a fourth array arrangement scheme in the third array arrangement scheme, determining that there are parallel DAC conversions and virtual memory-computing array computations, and there is no parallel ADC conversion in the fourth array arrangement scheme; wherein the fourth array arrangement scheme is a virtual memory-computing array having a mapping relationship with a fourth block that is vertically divided and has the same layer number as the plurality of blocks; if there is a fifth array arrangement scheme in the third array arrangement scheme, determining that there are parallel ADC conversions and virtual memory-computing array computations, and there is no parallel DAC conversion in the fifth array arrangement scheme; wherein the fifth array arrangement scheme is a virtual memory-computing array having a mapping relationship with a fifth block that is horizontally divided and has the same layer number as the plurality of blocks; and if there is a sixth array arrangement scheme in the third array arrangement scheme, determining that there is parallel virtual memory-computing array computation, and there are no parallel ADC conversions and DAC conversions in the sixth array arrangement scheme; wherein the third array arrangement scheme is a virtual memory-computing array having a mapping relationship with a sixth block, and the sixth block is a block in the plurality of blocks other than the fourth block and the fifth block; and wherein the sixth array arrangement scheme is an array arrangement scheme in the third array arrangement scheme other than the fourth array arrangement scheme and the fifth array arrangement scheme.

[0019] In some embodiments of the present disclosure, determining whether the second array arrangement scheme exists parallelly executed DAC conversion, whether the second array arrangement scheme exists parallelly executed ADC conversion, and whether the second array arrangement scheme exists parallelly executed virtual memory and array calculation includes: if the value of the first parameter of the same kernel is a true value, determining that the second array arrangement scheme exists parallelly executed ADC conversion, parallelly executed DAC conversion, and parallelly executed virtual memory and array calculation.

[0020] In some embodiments of the present disclosure, the array arrangement scheme in the plurality of array arrangement schemes that satisfies the preset condition is taken as the arrangement scheme of the target model, including: determining an initial array arrangement scheme from the plurality of array arrangement schemes by using a genetic algorithm; performing the i th round of operation according to the initial array arrangement scheme: randomly selecting n pairs of blocks for k times from the initial array arrangement scheme, and adjusting the arrangement scheme of the n pairs of blocks selected for the j th time in the k times; for the j th time, exchanging the blocks with adjusted arrangement scheme and the corresponding blocks in the initial array arrangement scheme, to obtain the array arrangement scheme of the j th time, until k array arrangement schemes are obtained; taking the array arrangement scheme with a smaller time delay than the initial array arrangement scheme in the k array arrangement schemes as the initial array arrangement scheme of the i+1 th round; until the initial array arrangement scheme that satisfies the preset condition is obtained, and taking the initial array arrangement scheme that satisfies the preset condition as the arrangement scheme of the target model; wherein i, j, and n are positive integers, and k is an integer greater than 1.

[0021] In some embodiments of the present disclosure, the preset condition includes at least one of the following: the time delay of the k array arrangement schemes is greater than that of the initial array arrangement scheme of the m th round; m is greater than a preset round threshold; and m is greater than i+1.

[0022] The second aspect embodiment of the present disclosure proposes an inference method, which includes: in response to receiving an inference request for a target model, scheduling a memory and computing integrated chip deploying an arrangement scheme of the target model; wherein the arrangement scheme of the target model is the arrangement scheme deployed on the memory and computing integrated chip by the method described in the first aspect; determining the computing task of each scheduling unit in the memory and computing integrated chip according to the arrangement scheme of the target model and the input and output of each operator in the target model; scheduling each scheduling unit to perform its own computing task to obtain an inference result.

[0023] In a third aspect, an embodiment of the present disclosure provides a model scheduling device, which comprises: an obtaining module configured to obtain information of a virtual storage-computing array corresponding to a storage-computing unit on a kernel of a storage-computing chip; wherein the information comprises size information and a first quantity determined based on a DAC ratio and an ADC ratio, the DAC ratio being a ratio between an input dimension of the storage-computing unit and a number of DACs, and the ADC ratio being a ratio between an output dimension of the storage-computing unit and a number of ADCs; a splitting module configured to split each layer network weight in a target model according to the size information to obtain a plurality of blocks; and a mapping module configured to map the plurality of blocks to the first quantity of virtual storage-computing arrays respectively to obtain a scheduling scheme of the target model; wherein the scheduling scheme of the target model at least comprises virtual storage-computing arrays that are executed in parallel, and each virtual storage-computing array in the first quantity of virtual storage-computing arrays corresponds to a DAC conversion of a cycle number and an ADC conversion of the cycle number in the storage-computing unit.

[0024] In a fourth aspect, an embodiment of the present disclosure provides an inference device, which comprises: a receiving module configured to receive an inference request for a target model; a scheduling module configured to schedule a storage-computing chip deploying a scheduling scheme of the target model; wherein the scheduling scheme of the target model is a scheduling scheme deployed on the storage-computing chip according to the method of the first aspect; a determining module configured to determine a computing task of each scheduling unit in the storage-computing chip according to an input and output of each operator in the target model and the scheduling scheme of the target model; and an inference module configured to schedule each scheduling unit to execute the computing task of the scheduling unit to obtain an inference result.

[0025] In a fifth aspect, an embodiment of the present disclosure provides an electronic device, which comprises: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect of the present disclosure or the method described in the second aspect of the present disclosure.

[0026] In a sixth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to perform the method described in the first aspect of the present disclosure or the method described in the second aspect of the present disclosure.

[0027] In a seventh aspect, an embodiment of the present disclosure provides a chip, which comprises one or more interfaces and one or more processors; the interfaces are configured to receive a signal from a memory of an electronic device and send the signal to the processors, the signal comprising computer instructions stored in the memory, and when the processors execute the computer instructions, the electronic device performs the method described in the first aspect of the present disclosure or the method described in the second aspect of the present disclosure.

[0028] To sum up, the model arrangement method, device, electronic equipment and storage medium provided by the present disclosure can obtain a plurality of blocks by cutting the target model according to the size of the virtual memory-computing array, can realize parallel computing of the virtual memory-computing array having a mapping relationship with the blocks in the front and rear layer networks, and thus improves the computing power utilization rate of the memory-computing chip and reduces the computing delay.

[0029] It should be understood that the foregoing general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0030] The accompanying drawings incorporated in the specification and forming a part of it illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure, and do not limit the present disclosure.

[0031] Figure 1 A flowchart of a model arrangement method provided by an embodiment of the present disclosure;

[0032] Figure 2 A schematic diagram of a memory-computing unit of a memory-computing chip provided by an embodiment of the present disclosure;

[0033] Figure 3 A schematic diagram of size information of a virtual memory-computing array and circle weights of each layer network in a target model provided by an embodiment of the present disclosure;

[0034] Figure 4 A cutting schematic diagram of weights of each layer network in a target model provided by an embodiment of the present disclosure;

[0035] Figure 5 A schematic diagram of a block arrangement scheme provided by an embodiment of the present disclosure;

[0036] Figure 6 A schematic diagram of parallel computing of each block in the front and rear layer networks provided by an embodiment of the present disclosure;

[0037] Figure 7 A schematic diagram of mapping of the blocks to a virtual memory-computing array provided by an embodiment of the present disclosure;

[0038] Figure 8 A schematic diagram of an array arrangement scheme provided by an embodiment of the present disclosure;

[0039] Figure 9 A flowchart of a reasoning method provided by an embodiment of the present disclosure;

[0040] Figure 10 A structure schematic diagram of a model arrangement device provided by an embodiment of the present disclosure;

[0041] Figure 11A reasoning device structure schematic diagram provided by an embodiment of the present disclosure;

[0042] Figure 12 An electronic device structure schematic diagram provided by an embodiment of the present disclosure;

[0043] Figure 13 A chip structure schematic diagram provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0044] Embodiments of the present disclosure are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numbers represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the accompanying drawings are exemplary and are intended to explain the present disclosure, and cannot be understood as a limitation of the present disclosure.

[0045] The memory-compute integrated chip breaks through the bottleneck of the traditional Von Neumann architecture by deeply integrating memory and computing functions, and provides a higher computing power and higher energy efficiency solution for artificial intelligence computing. In engineering implementation, the memory-compute unit of the memory-compute integrated chip needs DAC (Digital-to-Analog Converter) to convert input data into input voltage, and ADC (Analog-to-Digital Converter) to convert output current into output data, but in the memory-compute integrated chip

[0046] DAC and ADC occupy a large area, limiting the integration density of the memory-compute unit, thereby affecting the computing power of the chip. Meanwhile, the conversion process of DAC and ADC consumes energy and introduces delay, which increases the overall power consumption and computing delay of the chip, reduces the actual energy efficiency ratio, and thus there is a large gap between the actual energy efficiency ratio and the computing power of the memory-compute integrated technology and the theoretical upper limit.

[0047] Figure 1 A flowchart of a model arrangement method provided by an embodiment of the present disclosure is shown in FIG. 13. As shown in FIG. 13, the execution subject of the method is an electronic device. The method can include the following steps. Figure 1

[0048] Step 101, obtaining information of a virtual memory-compute array corresponding to a memory-compute unit on a kernel of a memory-compute integrated chip; wherein the information includes size information and a first number determined based on a DAC ratio and an ADC ratio, the DAC ratio being a ratio between an input dimension of the memory-compute unit and a number of DACs, and the ADC ratio being a ratio between an output dimension of the memory-compute unit and a number of ADCs.

[0049]

[0050] Step 102, dividing each layer network weight in a target model according to the size information to obtain a plurality of blocks.​​

[0051] Step 103, mapping the plurality of blocks to the first number of virtual memory-computing arrays respectively to obtain a scheduling scheme of the target model; wherein the scheduling scheme of the target model at least includes the virtual memory-computing arrays executed in parallel, and each of the first number of virtual memory-computing arrays corresponds to the DAC conversion and the ADC conversion of each cycle number in the memory-computing unit.

[0052] The model scheduling method provided by the present disclosure first determines the size information and the first number of the virtual memory-computing array corresponding to the memory-computing unit in the content of the memory-computing chip according to the DAC ratio and the ADC ratio; then, the network weight of each layer in the target model is divided according to the size information to obtain a plurality of blocks; and then, the plurality of blocks are mapped to the first number of virtual memory-computing arrays respectively to obtain a scheduling scheme of the target model. The scheduling scheme of the target model includes the virtual memory-computing arrays executed in parallel, which can realize the parallel computing of the virtual memory-computing arrays having the mapping relationship with the blocks in the front and rear layers of the network, thereby improving the computing power utilization rate of the memory-computing chip and reducing the computing delay.

[0053] Step 101 (i.e., obtaining the information of the virtual memory-computing array corresponding to the memory-computing unit in the kernel of the memory-computing chip; wherein the information includes the size information and the first number determined based on the DAC ratio and the ADC ratio, the DAC ratio is the ratio between the input dimension of the memory-computing unit and the number of DACs, and the ADC ratio is the ratio between the output dimension of the memory-computing unit and the number of ADCs) will be described in detail below.

[0054] Before deploying the target model on the memory-computing chip, the model scheduling method further includes: configuring the parameters of the memory-computing chip; and determining the information of the virtual memory-computing array based on the ADC ratio and the DAC ratio.

[0055] The present disclosure is an automatic software tool chain for a heterogeneous memory-computing chip, which involves functional modules such as software tool chain compiler model scheduling and model computing flow scheduling.

[0056] The memory-computing software tool chain compiler can view the scheduling result of the model, and in addition, the software tool chain runtime system has related interfaces for querying the model scheduling deployment array information and the memory-computing array usage.

[0057] It should be noted that the related parameters of the ADC and the DAC in the memory-computing chip are configured in the software tool chain.

[0058] In one example, configuring the parameters of the memory-computing chip includes: configuring the chip design parameters related to model scheduling in the scheduling software.

[0059] In one example, the above parameters can include at least one of the following:

[0060] The number of chips (CHIP (chip)_NUM (number)) of the memory-compute integrated chip;

[0061] The number of cores (core_NUM) of the memory-compute integrated chip;

[0062] The number of memory-compute units of each core;

[0063] Whether the memory-compute units of the same core can be calculated in parallel, when the value of the first parameter of the core is true, the memory-compute units of the core support parallel calculation; wherein the first parameter is used to represent the parallelizable feature of the memory-compute units of the same core.

[0064] The size information of each memory-compute unit, including: length (i.e. the number of rows of the memory-compute unit) and width (i.e. the number of columns of the memory-compute unit);

[0065] The ADC ratio is ADC_RATIO (ratio): 1, and the DAC ratio is DAC_RATIO: 1;

[0066] The delay of ADC conversion (i.e. the delay of performing one ADC conversion) is DELAY (delay)_ADC, the delay of DAC conversion (i.e. the delay of performing one DAC conversion) is DELAY_DAC, and the delay of performing virtual memory-compute array (i.e. virtual XB) calculation (i.e. the delay of performing one virtual XB calculation) is DELAY_virtualXB.

[0067] Some embodiments of the present disclosure, the multiple rows of input of the memory-compute unit on the core of the memory-compute integrated chip (i.e. the input dimension of the memory-compute unit) are matched with one DAC (i.e. the DAC ratio), and the multiple rows of output (i.e. the output dimension of the memory-compute unit) are matched with one ADC (i.e. the ADC ratio), so that when performing model inference, the input data is converted by DAC in batches, and the output current is converted by ADC in batches to output data, which reduces the power consumption of the memory-compute integrated chip.

[0068] In Figure 2 , the ratio between one DAC and the number of rows of memory-compute units (the maximum input dimension of a single memory-compute unit) is 1:4, and the ratio between one ADC and the number of columns of memory-compute units (i.e. the maximum output dimension of a single memory-compute unit) is 1:4.

[0069] For example, the ratio of the DAC of the memory-compute integrated chip to the input dimension, and the ratio of the ADC to the output dimension is: 1:2, and the first number corresponding to a single memory-compute unit is 2x2=4 virtual memory-compute arrays.

[0070] Optionally, the information of the virtual memory-computing array can include at least one of the following:

[0071] The number of virtual memory-computing arrays of the same core: ADC_RATIO * DAC_RATIO, wherein ADC_RATIO is the number of ADC modules in a memory-computing unit, and DAC_RATIO is the number of DAC modules in a memory-computing unit. Figure 2 For example, the ADC_RATIO and the DAC_RATIO are both 2, and the first number of virtual memory-computing arrays corresponding to a memory-computing unit is 4.

[0072] The number of virtual memory-computing arrays of each core (the number of memory-computing units * ADC_RATIO * DAC_RATIO);

[0073] The number of virtual memory-computing arrays of the memory-computing chip (core_NUM * ADC_RATIO * DAC_RATIO);

[0074] The size information of the virtual memory-computing array, including: the number of rows (i.e., the number of rows of memory-computing units / ADC_RATIO), and the number of columns (i.e., the number of columns of memory-computing units / DAC_RATIO).

[0075] The naming of the virtual memory-computing array of a memory-computing chip is: virtual XB u,v,w,x (i.e., a virtual XB without mapping) u represents the number of the core where the virtual memory-computing array is located, v represents the number of the memory-computing unit where the virtual memory-computing array is located, w represents the period number of DAC conversion in the virtual memory-computing array, and x represents the period number of ADC conversion in the virtual memory-computing array.

[0076] The step 102 (i.e., splitting the network weights of each layer in the target model according to the size information to obtain a plurality of blocks) is described in detail below.

[0077] In an embodiment of the present disclosure, the network weights of each layer in the target model are split according to the size information of the virtual memory-computing array to obtain a plurality of blocks.

[0078] It should be noted that the network weights of each layer in the target model can include longitudinal splitting and / or transverse splitting of the network of each layer in the target model.

[0079] The longitudinal splitting is to split the matrix of the weights of the target model into a plurality of sub-matrices according to the output dimension, and each sub-matrix corresponds to the calculation of part of the output; the longitudinal splitting is suitable for the following scenarios: (1) the output row number (output dimension) of the memory-computing unit (or the number of ADC modules) is limited, and the output needs to be processed in batches; (2) suitable for convolution layers (such as splitting the convolution kernel according to the output).

[0080] Among them, horizontal partitioning is to divide the weight matrix into multiple sub-matrices according to the input dimension, and each sub-matrix corresponds to the calculation of part of the input. Horizontal partitioning is applicable to the following scenarios: (1) the number of input columns of the storage unit (or the number of DAC modules) is limited, and the input needs to be loaded in batches; (2) it is suitable for fully connected layers or convolutional layers with a large number of inputs.

[0081] In one example, the weights of each layer of the target model are divided into multiple blocks according to the size information, including: vertically dividing the weights of each layer of the target model according to the length in the size information, and horizontally dividing the weights of each layer of the target model according to the width in the size information, to obtain multiple blocks.

[0082] Specifically, when dividing the network weights of each layer in the target model, the network weights of each layer are vertically divided according to the length in the size information of the virtual memory array and rounded up, and the network weights of each layer are horizontally divided according to the width in the size information of the virtual memory array and rounded up, resulting in multiple blocks.

[0083] In embodiments of this disclosure, in order to analyze the relationships between blocks, the model orchestration method further includes: marking the segmentation information of each block.

[0084] For example, Block i,j,k It can be used to mark the segmentation information of each block; where i represents the layer number of the layer where the block is located, j represents the first method number of the horizontal segmentation corresponding to the block, and k represents the second method number of the vertical segmentation corresponding to the block.

[0085] The following is combined Figures 3-4 The weighting process is explained in detail.

[0086] Will Figure 3 The weights of the first layer of the network are vertically segmented according to the length in the virtual memory array's size information, and horizontally segmented according to the width in the virtual memory array's size information, to obtain the Block. 1,0,0 (like Figure 4 (As shown); Next, will Figure 3 The weights of the second layer network are vertically segmented according to the length in the virtual memory array's size information, and horizontally segmented according to the width in the virtual memory array's size information, to obtain...

[0087] Block 2,0,0 and Block 2,0,1 (like Figure 4 (As shown); Next, will Figure 3the third layer network weight in the target model is divided according to the length in the size information of the virtual storage computing array in the longitudinal direction and according to the width in the size information of the virtual storage computing array in the transverse direction to obtain Block 3,0,0 , Block 3,0,1 , Block 3,1,0 , and Block 3,1,1 (as shown in Figure 4 ).

[0088] It should be noted that the first layer network weight, the second layer network weight, and the third layer network weight can be divided in parallel, which will not be described here.

[0089] In some embodiments of the present disclosure, after the network weights of each layer in the target model are divided according to the size information of the virtual storage computing array, the model arrangement method further includes:

[0090] record the division information of the plurality of blocks; wherein the division information of the plurality of blocks includes at least one of the following: a mode number of a division mode corresponding to each block, a layer number of a layer where each block is located; the mode number includes a first mode number of transverse division and a second mode number of longitudinal division;

[0091] determine a block arrangement scheme according to the division information of the plurality of blocks.

[0092] Embodiments of the present disclosure represent the relationship between blocks (such as parallel execution relationship, non-parallel execution relationship, etc.) through division information, and arrange blocks that meet the parallel execution relationship into blocks that are executed in parallel (i.e., block arrangement scheme).

[0093] In one example, determining a block arrangement scheme according to the division information of the plurality of blocks includes: if there is division information of a first block in the division information of the plurality of blocks, the block arrangement scheme includes a first arrangement scheme of setting the first block as a block that is executed in parallel; wherein the first block includes all blocks with the same layer number.

[0094] Specifically, all blocks (i.e., first block) with the same i in the target model are arranged to obtain a first arrangement scheme, which includes a scheme of setting the first block as a block that is executed in parallel.

[0095] In Figure 5 , the blocks with the same i are blocks of the same layer network, such as Block 2,0,0 and Block 2,0,1 are blocks with a parallel execution relationship, Block 3,0,0 and Block 3,0,1 are blocks with a parallel execution relationship.

[0096] In one example, determining the block arrangement scheme according to the split information of the plurality of blocks comprises: if there is split information of a second block in the split information of the plurality of blocks, the block arrangement scheme comprises a second arrangement scheme of arranging the second block as a block executed in parallel; wherein the second block comprises all blocks with the same second mode number.

[0097] For example, a second block (i.e., a block with the same i and j in the plurality of blocks) split longitudinally in the target model is arranged as a block executed in parallel to obtain a second block arrangement scheme.

[0098] In Figure 5 , the block with the same i and j is a second block split transversely, such as Block 2,0,0 Block 2,0,1 is a block having a parallel execution relationship. 3,0,0 Block 3,0,1 is a block having a parallel execution relationship.

[0099] In Figure 5 , the split information of Block1 (i.e., Block 1,0,0 ) comprises: the layer number of the layer where Block1 is located is 1 (i.e., the first layer network), the first mode number of transverse split is 0, and the second mode number of longitudinal split is 0. The split information of Block2 (i.e., Block 2,0,0 ) comprises: the layer number of the layer where Block2 is located is 2 (i.e., the second layer network), the first mode number of transverse split is 0, and the second mode number of longitudinal split is 0. The split information of Block3 (i.e., Block 2,0,1 ) comprises: the layer number of the layer where Block3 is located is 2 (i.e., the second layer network), the first mode number of transverse split is 0, and the second mode number of longitudinal split is 1. The split information of Block4 (i.e., Block 3,0,0 ) comprises: the layer number of the layer where Block4 is located is 3 (i.e., the third layer network), the first mode number of transverse split is 0, and the second mode number of longitudinal split is 0. The split information of Block5 (i.e., Block 3,0,1 ) comprises: the layer number of the layer where Block5 is located is 3 (i.e., the third layer network), the first mode number of transverse split is 0, and the second mode number of longitudinal split is 1. The split information of Block6 (i.e., Block 3,1,0 ) comprises: the layer number of the layer where Block6 is located is 3 (i.e., the third layer network), the first mode number of transverse split is 1, and the second mode number of longitudinal split is 0. The split information of Block7 (i.e., Block 3,1,1) includes: the layer number of the layer where Block 7 is located is 3 (i.e., the third layer network), the first mode number of the transverse splitting is 1, and the second mode number of the longitudinal splitting is 1.

[0100] The Blocks 1 to 7 are arranged according to the splitting information of the Blocks 1 to 7, and a block arrangement scheme as shown in FIG. 3 is obtained. Figure 5 The Blocks 2 and 3 are parallelly executed blocks, the Blocks 4 and 5 are parallelly executed blocks, the Blocks 6 and 7 are parallelly executed blocks, the Blocks 4 and 6 are transversely split Blocks and are mapped to a virtual storage-computing array corresponding to the same storage-computing unit to automatically complete summation, and the Blocks 5 and 7 are transversely split Blocks and are mapped to a virtual storage-computing array corresponding to the same storage-computing unit to automatically complete summation.

[0101] It should be noted that the calculation results of the blocks with the same i and j are part of the overall calculation results of the storage-computing integrated chip, and the blocks with the same i and j values are arranged into the second block arrangement scheme, so that the calculation results of the blocks with the same i and j values do not need to be calculated twice, the computing power utilization rate of the storage-computing integrated chip is improved, and the calculation delay is reduced.

[0102] The embodiments of the present disclosure can be applied to a fully connected operator with large size information of weights, and the calculation scheduling of an entire layer network is refined to the calculation scheduling of a virtual storage-computing array. Figure 6 In the embodiments of the present disclosure, the second layer network depends on the output results of the first layer network as input, and the Blocks 1 and 2 can be executed in parallel or separately after being mapped to the virtual storage-computing array, while the calculation input of the Blocks 3 and 4 only depends on the output results of the Block 1, and the calculation input of the Blocks 5 and 6 only depends on the output results of the Block 2. Compared with the prior art in which the second layer network calculation is started after the first layer network calculation is completed, the embodiments of the present disclosure can be started without waiting for the first layer network calculation to be completed, and it can be seen that the calculation control flow is refined to the virtual storage-computing array for scheduling, which can improve the calculation parallelism, improve the computing power utilization rate of the storage-computing integrated chip, and reduce the network calculation delay.

[0103] The step 103 (i.e., mapping the plurality of blocks to the first number of virtual storage-computing arrays to obtain an arrangement scheme of the target model; wherein the arrangement scheme of the target model at least includes virtual storage-computing arrays executed in parallel, and each virtual storage-computing array in the first number of virtual storage-computing arrays corresponds to DAC conversion and ADC conversion of each cycle number in the storage-computing unit) will be described in detail below.

[0104] Based on the number of blocks after the weights of each layer of the target model are divided, determine the second number of virtual XB (in-memory array) computations that the target model needs to perform during inference (e.g., if the target model is divided into 7 blocks, the target model needs to perform 7 virtual XB computations during inference (i.e., perform 7 virtual XB computations)), and determine the second number of virtual XB computations to be performed from the first number of virtual in-memory arrays.

[0105] exist Figure 7 In this process, four virtual memory arrays are determined based on the ADC and DAC ratios, namely, virtual XB. 00 Virtual XB 01 Virtual XB 10 And Virtual XB 11 Block1 and Block2 are respectively mapped to virtual XB 00 The above shows that during inference of the target model, virtual XB computation corresponding to Block 1 and virtual XB computation corresponding to Block 2 are performed; Block 3 is mapped to virtual XB. 01 The above yields the parameters required to be executed when the target model is executed.

[0106] Block 3 corresponds to the virtual XB calculation, and Block 4 and Block 5 are respectively mapped to virtual XB. 10 The above shows that during inference of the target model, virtual XB computation corresponding to Block 4 and virtual XB computation corresponding to Block 5 are performed; Block 6 and Block 7 are mapped to virtual XB respectively. 11 The above yields the results of the execution of the target model during inference.

[0107] The virtual XB computation corresponding to Block 6 is performed in accordance with the virtual XB computation corresponding to Block 7.

[0108] It should be noted that, based on the requirement of deploying the model on the in-memory computing chip, a second number of virtual XB computations can be determined from a first number of virtual in-memory arrays; for example, if the requirement is to deploy a compact model on the in-memory computing chip to maximize resource utilization, cores and in-memory units can be selected sequentially based on the second number; if the requirement is latency, such as the lowest possible computation latency, more cores can be used to execute virtual XB computations in parallel.

[0109] In some examples of this disclosure, multiple blocks are mapped to a first number of virtual memory arrays to obtain an arrangement scheme for the target model, including: mapping multiple blocks to a first number of virtual memory arrays to obtain multiple array arrangement schemes; and using the array arrangement scheme that meets the preset conditions among the multiple array arrangement schemes as the arrangement scheme for the target model.

[0110] In one example, mapping the plurality of blocks to the first number of virtual storage-computing arrays respectively can include: mapping the plurality of blocks one by one to corresponding virtual storage-computing arrays respectively.

[0111] For example, Block 1,0,0 has a mapping relationship with virtual XB u0,v0,w0,x0 has a mapping relationship with virtual XB 2,0,0 has a mapping relationship with virtual XB u1,v1,w1,x1 has a mapping relationship with virtual XB 2,0,1 has a mapping relationship with virtual XB u2,v2,w2,x2 has a mapping relationship with virtual XB 3,0,0 has a mapping relationship with virtual XB u3,v3,w3,x3 has a mapping relationship with virtual XB 3,0,1 has a mapping relationship with virtual XB u5,v5,w5,x5 has a mapping relationship with virtual XB 3,1,0 has a mapping relationship with virtual XB u4,v4,w4,x4 has a mapping relationship with virtual XB 3,1,1 has a mapping relationship with virtual XB u6,v6,w6,x6 .

[0112] In another example, according to the relationship between the blocks in the plurality of blocks (i.e., the split information), the plurality of blocks are mapped to virtual storage-computing arrays supporting the relationship, for example, blocks that are executed in parallel are mapped to cores of virtual storage-computing arrays that support parallel execution of the virtual storage-computing arrays; or, according to the requirements of deploying a model on a storage-computing integrated chip, such as minimum latency, the plurality of blocks are mapped to virtual storage-computing arrays of a plurality of cores to execute the virtual storage-computing arrays in parallel, so as to ensure that the latency is minimized.

[0113] The mapping relationship between the plurality of blocks and the first number of virtual storage-computing arrays is selected according to the split information in detail as follows.

[0114] In one example, mapping the plurality of blocks to the first number of virtual storage-computing arrays respectively includes:

[0115] If the split information of the first block is included in the split information of the plurality of blocks, each block in the first arrangement scheme is mapped to a corresponding virtual storage-computing array in the first number of virtual storage-computing arrays respectively.

[0116] Here, when the split information of the first block is included in the split information of the plurality of blocks, the mapping relationship between the plurality of blocks and the first number of virtual storage-computing arrays is the mapping relationship corresponding to the first arrangement scheme.

[0117] If the split information of the second block is included in the split information of the plurality of blocks, each block in the second arrangement scheme is mapped to a corresponding virtual storage-computing array in the first number of virtual storage-computing arrays respectively.

[0118] Here, when the split information of the second block is included in the split information of the plurality of blocks, the mapping relationship between the plurality of blocks and the second number of virtual memory-computing arrays is a mapping relationship corresponding to the second arrangement scheme.

[0119] If the split information of the third block is included in the plurality of split information, the third block is respectively mapped to the corresponding virtual memory-computing array in the first number of virtual memory-computing arrays; wherein the third block is all blocks with the same first mode number in the plurality of blocks.

[0120] Wherein, the third block can be a horizontally split block, that is, for Block i,j,k If i and k are the same, the calculation result of each block in the third block is only a partial weight multiplication and addition result, and the third block can be arranged in the corresponding virtual memory-computing array of the same memory-computing unit to automatically complete summation. Compared with performing summation calculation on the calculation result of each block in the third block again, the calculation power utilization rate of the chip is improved, and the calculation delay is reduced.

[0121] In addition, it should be noted that the matrix multiplication and accumulation can be performed by corresponding hardware or software in the memory-computing unit, and the accumulation calculation is realized at the same time as the matrix multiplication and accumulation is completed.

[0122] The following takes 8 blocks and 8 virtual XB calculations during inference of the target model as an example.

[0123] In Figure 8 , the virtual XB un,vn,wn,xn is used to mark the Block i,j,k with the serial number n, and the virtual XB i,j,k is used to mark the Block 1,0,0 with the serial number n+1. 2,0,0 The serial number n of the Block 2,0,1 is 3, the serial number n of the Block 3,0,0 is 4, the serial number n of the Block 3,0,1 is 5, the serial number n of the Block 3,1,0 is 6, the serial number n of the Block 3,1,1 is 7, and during mapping, the Block 1,0,0 is mapped to perform virtual XB u0,v0,w0,x0 calculation, the Block 2,0,0 is mapped to perform virtual XB u1,v1,w1,x1 calculation, the Block 2,0,1 is mapped to perform virtual XB u2,v2,w2,x2 calculation, the Block 3,0,0 is mapped to perform virtual XB u3,v3,w3,x3 calculation, the Block 3,0,1 is mapped to perform virtual XB u5,v5,w5,x5Compute, Block 3,1,0 mapped to execute virtual XB u4,v4,w4,x4 Compute, Block 3,1,1 mapped to execute virtual XB u6,v6,w6,x6 Compute.

[0124] In Figure 8 the virtual XB u1,v1,w1,x1 with the virtual XB u2,v2,w2,x2 for parallel execution of virtual XB computation, the virtual

[0125] XB u4,v4,w4,x4 with the virtual XB u6,v6,w6,v6 for parallel execution of virtual XB computation, the virtual XB u3,v3,w3,v3 with the virtual

[0126] XB u5,v5,w5,v5 for parallel execution of virtual XB computation, the execute virtual XB u3,v3,w3,v3 Compute with the virtual XB u4,v4,w4,v4 Compute to automatically complete summation, the execute virtual XB u5,v5,w5,v5 Compute with the execute virtual XB u6,v6,w6,v6 Compute to automatically complete summation.

[0127] Here, the orchestration scheme of the target model includes at least parallel execution of virtual XB u1,v1,w1,x1 with the virtual

[0128] XB u2,v2,w2,x2 , parallel execution of virtual XB u4,v4,w4,x4 with the virtual XB u6,v6,w6,v6 , and parallel execution of virtual

[0129] XB u3,v3,w3,v3 with the virtual XB u5,v5,w5,v5 .

[0130] If the split information of the plurality of blocks includes the split information of the first block, the blocks in the first orchestration scheme are respectively mapped to corresponding virtual compute array in the first number of virtual compute arrays, specifically including: respectively mapping each block in the first orchestration scheme to the DAC with different cycle numbers in the first number of virtual compute arrays, and on the DAC with different cycle numbers.

[0131] If the split information of the plurality of blocks includes the split information of the second block, the blocks in the second orchestration scheme are respectively mapped to corresponding virtual compute array in the first number of virtual compute arrays, specifically including: respectively mapping each block in the second orchestration scheme to the DAC with the same cycle number in the first number of virtual compute arrays.

[0132] If the split information of the third block is included in the plurality of split information, the third block is respectively mapped to the corresponding virtual memory-computing array in the first number of virtual memory-computing arrays, specifically including: respectively mapping the third block to the ADCs with the same period number in the third virtual memory-computing array.

[0133] Embodiments of the present disclosure, based on the virtual XB parallel computing characteristics for computing scheduling (i.e., parallel scheduling during inference):

[0134] For example, for the virtual memory-computing arrays within the same memory-computing array, i.e., when the virtual XB u,v,w When the u and v values of x are the same, the characteristics of whether different blocks can be computed in parallel are different.

[0135] The first kind: for the blocks with the same input data (i.e., the blocks vertically split in the same network layer, corresponding to the second block described above), the virtual memory-computing arrays are arranged to the DACs with the same period number.

[0136] The second kind: for the blocks whose output data need to be accumulated (i.e., the blocks horizontally split in the same network layer, corresponding to the third block described above), the virtual memory-computing arrays are arranged to the ADCs with the same period number.

[0137] The third kind: for the blocks whose input data are not the same and whose output data are not accumulated (corresponding to the first block described above), when the blocks are arranged to the virtual XB i,j,k with different period numbers, i.e., the blocks with different i (different operators) or the same i but different j and k (the input and output of the same network layer are all irrelevant), are mapped to the virtual XB u,v,w,x with the same u and v but different w and x.

[0138] The memory-computing integrated chip is innovated from the aspects of materials, devices, computing paradigm, architecture, etc., uses the storage unit to complete the computing function to realize zero distance of storage and computing, which can greatly improve the chip computing power and energy efficiency level, and is expected to become a higher computing power and higher energy efficiency artificial intelligence computing chip localization solution. However, in engineering implementation, each memory-computing unit needs to have a DAC digital-analog conversion module and an ADC analog-digital conversion module, and the DAC and ADC modules occupy a large area of the memory-computing integrated chip, reduce the integration density of the memory-computing unit, and greatly affect the chip computing power. At the same time, it also contributes to most of the power consumption and computing delay of the memory-computing integrated chip, so that the actual energy efficiency ratio and computing power of the memory-computing integrated technology have a great gap with the theoretical upper limit.

[0139] In the related art, when the software tool chain of the storage-computing integrated chip arranges the model weight, it is only based on the number and size of the chip storage-computing array to arrange, without combining with the ADC and DAC hardware design, and ignores the influence of different arrangement schemes on the DAC and ADC delay. The scheme of the present disclosure proposes a model weight arrangement method combining DAC, ADC and storage-computing array hardware design. The method reduces the DAC and ADC conversion times in the model inference process, thereby reducing the time delay and power consumption caused by the DAC and ADC process.

[0140] Here, according to the DAC ratio and the ADC ratio, the DAC and the ADC of the same cycle number in the storage-computing unit are selected as the logical subarray, also known as the virtual storage-computing array. When arranging the model weight, the weight is arranged according to the logical subarray, and the number of operator logical subarrays is minimized, thereby reducing the DAC and ADC conversion delay in the calculation process.

[0141] In the Figure 2 , the cycle number corresponds to cycle1, cycle2, cycle3 and cycle4 cycle numbers, the ADC and DAC of cycle1 are taken as a virtual storage-computing array, the ADC and DAC of cycle2 are taken as a virtual storage-computing array, the ADC and DAC of cycle3 are taken as a virtual storage-computing array, and the ADC and DAC of cycle4 are taken as a virtual storage-computing array, thereby obtaining a first number of virtual storage-computing arrays.

[0142] In some embodiments of the present disclosure, the array arrangement scheme in the plurality of array arrangement schemes that meets the preset condition is taken as the arrangement scheme of the target model, including: respectively determining the delay of each array arrangement scheme in the plurality of array arrangement schemes; and taking the array arrangement scheme in the plurality of array arrangement schemes that meets the preset condition as the arrangement scheme of the target model.

[0143] In this embodiment, after determining the delay of each array arrangement scheme in the plurality of array arrangement schemes, the array arrangement scheme in the plurality of array arrangement schemes that meets the preset condition is taken as the arrangement scheme of the target model.

[0144] In one example, determining the time delay of each array arrangement scheme in the plurality of array arrangement schemes respectively includes: determining the first total number of times of performing ADC conversion, the second total number of times of performing DAC conversion, and the third total number of times of performing virtual XB calculation for each array arrangement scheme in the plurality of array arrangement schemes respectively; obtaining the first time delay of performing one ADC conversion, the second time delay of performing one DAC conversion, and the third time delay of performing one virtual XB calculation; and determining the time delay of each array arrangement scheme in the plurality of array arrangement schemes according to the first total number and the first time delay, the second total number and the second time delay, and the third total number and the third time delay.

[0145] In this embodiment, the time delay of each array arrangement scheme includes the total time delay of performing DAC, the total time delay of performing virtual XB calculation, and the total time delay of performing ADC, specifically:

[0146] The time delay of each virtual storage and calculation array = the total time delay of performing DAC + the total time delay of performing virtual XB calculation + the total time delay of performing ADC = the first total number of times of performing DAC conversion * the time delay of performing one DAC conversion + the third total number of times of performing virtual XB calculation * the time delay of performing one virtual XB calculation + the second total number of times of performing ADC conversion * the time delay of performing ADC conversion.

[0147] In the related art, the current storage and calculation integrated chip software tool chain only considers the storage and calculation array utilization and array calculation time delay when deploying and arranging the model weight, without considering the influence of different arrangement schemes on DAC and ADC time delay. Among them, the time delay and power consumption caused by DAC and ADC is one of the most important factors that cause the computing power and energy efficiency ratio of the storage and calculation integrated chip to be far away from the theoretical upper limit.

[0148] The present disclosure reduces the number of DAC conversions and ADC conversions in the model inference process, thereby reducing the time delay and power consumption caused by DAC conversion and ADC conversion.

[0149] In one example, determining the first total number of times of performing ADC conversion, the second total number of times of performing DAC conversion, and the third total number of times of performing virtual XB calculation in each array arrangement scheme in the plurality of array arrangement schemes respectively comprises: determining whether there is parallel DAC conversion, whether there is parallel ADC conversion, and whether there is parallel virtual XB calculation in each array arrangement scheme in the plurality of array arrangement schemes respectively; if there is parallel DAC conversion in each array arrangement scheme, regarding the parallel DAC conversion as one DAC conversion; otherwise, adding 1 to the number of times of performing DAC conversion until all DAC conversions included in each array arrangement scheme are traversed to obtain the first total number; if there is parallel ADC conversion in each array arrangement scheme, regarding the parallel ADC conversion as one ADC conversion; otherwise, adding 1 to the number of times of performing ADC conversion until all ADC conversions included in each array arrangement scheme are traversed to obtain the second total number; if there is parallel virtual XB calculation in each array arrangement scheme, regarding the parallel virtual XB calculation as one virtual XB calculation; otherwise, adding 1 to the number of times of performing virtual XB calculation until all virtual XB calculations included in each array arrangement scheme are traversed to obtain the third total number.

[0150] In one example, determining whether there is parallel DAC conversion, whether there is parallel ADC conversion, and whether there is parallel virtual XB calculation in each array arrangement scheme in the plurality of array arrangement schemes respectively comprises:

[0151] If there is a first array arrangement scheme in the plurality of array arrangement schemes, determining whether there is parallel ADC conversion, parallel DAC conversion, and parallel virtual XB calculation in the first array arrangement scheme; wherein the first array arrangement scheme comprises virtual memory-computing arrays corresponding to memory-computing units of different cores; if there is a second array arrangement scheme in the plurality of array arrangement schemes, determining whether there is parallel DAC conversion, whether there is parallel ADC conversion, and whether there is parallel virtual XB calculation in the second array arrangement scheme; wherein the second array arrangement scheme comprises virtual memory-computing arrays corresponding to memory-computing units on the same core; if there is a third array arrangement scheme in the plurality of array arrangement schemes, determining whether there is parallel ADC conversion, parallel DAC conversion, and parallel virtual XB calculation in the third array arrangement scheme; wherein the third array arrangement scheme comprises virtual memory-computing arrays corresponding to the same memory-computing unit.

[0152] In this example, determining whether the second array arrangement scheme has parallelly executed DAC conversion, whether the second array arrangement scheme has parallelly executed ADC conversion, and whether the second array arrangement scheme has parallelly executed virtual XB computation can include: if the first parameter of the same core is true, determining that the second array arrangement scheme has parallelly executed ADC conversion, parallelly executed DAC conversion, and the second array arrangement scheme includes parallelly executed virtual XB computation.

[0153] Specifically, for the virtual memory-computing array of different cores (i.e., the first virtual memory-computing array scheme), the DAC, ADC and virtual XB computation are all parallelly executable among each other. That is, when the virtual XB u,v,w,x computation with different u values can be executed in parallel.

[0154] For the virtual memory-computing array of different memory-computing units of the same core (i.e., the second virtual memory-computing array scheme), that is, when the virtual XB u,v,w,x computation with different v values can be executed in parallel.

[0155] For the virtual memory-computing array corresponding to the same memory-computing unit (the third virtual memory-computing array scheme), it is determined whether the third array arrangement scheme has parallelly executed ADC conversion, parallelly executed DAC conversion, and parallelly executed virtual XB computation.

[0156] In one example, determining whether parallel ADC conversion, parallel DAC conversion, and parallel virtual XB computation exist in the third array orchestration scheme includes: if a fourth array orchestration scheme exists in the third array orchestration scheme, then determining that the fourth array orchestration scheme contains parallel DAC conversion and virtual XB computation, but does not contain parallel ADC conversion; wherein, the fourth array orchestration scheme is a virtual memory array with a mapping relationship to the fourth block, which has the same layer number as multiple blocks and is vertically partitioned; if a fifth array orchestration scheme exists in the third array orchestration scheme, then determining that the fifth array orchestration scheme contains parallel ADC conversion and virtual XB computation. And there is no parallel execution of DAC conversion; wherein, the fifth array arrangement scheme is a virtual memory array with the same layer number as the middle layer of multiple blocks and with a mapping relationship to the fifth block using horizontal partitioning; if there is a sixth array arrangement scheme in the third array arrangement scheme, then it is determined that there is parallel execution of virtual XB computation in the sixth array arrangement scheme, and there is no parallel execution of ADC conversion and DAC conversion; wherein, the third array arrangement scheme is a virtual memory array with a mapping relationship to the sixth block, and the sixth block is a block other than the fourth and fifth blocks among multiple blocks; wherein, the sixth array arrangement scheme is an array arrangement scheme other than the fourth and fifth array arrangement schemes in the third array arrangement scheme.

[0157] In embodiments of this disclosure, for virtual in-memory arrays (i.e., the third array arrangement scheme) corresponding to the same in-memory unit, the following cases can be used to determine whether parallel ADC conversion, parallel DAC conversion, and parallel virtual XB computation exist:

[0158] The first scheme corresponds to the fourth array arrangement: For blocks with the same input data (i.e., vertically divided within the same network layer), when mapped to the virtual memory array of the ADC with the same period number, that is, when the block... i,j,k The i and j are the same, and the virtual XB that maps to the same u, v, and w is also the same. u,v,w,x At the same time, DAC and virtual XB computation can be performed in parallel (i.e., the same virtual XB computations for u, v, and w can be performed in parallel), but ADC cannot be parallelized.

[0159] The second scheme corresponds to the fifth array arrangement: For blocks where output data needs to be accumulated (i.e., blocks horizontally divided within the same network layer), when mapped to the virtual memory array of the ADC with the same period number, i.e., the blocks... i,j,k The i and k are the same, and the virtual XB that maps to the same u, v, and x is also the same. u,v,w,x At that time, ADC and virtual XB calculations can be performed in parallel (i.e., the same virtual XB calculations for u, v, and x can be performed in parallel), and the step of accumulation can be omitted. However, DAC cannot be parallelized.

[0160] The third corresponding sixth array arrangement scheme, for the input data not the same, output data not accumulated Block, mapped to the different cycle number of ADC and the different cycle number of DAC virtual XB time, namely Block i,j,k i is different (different operators, one operator corresponds to one layer network) or i is the same but j and k are different (the same layer network input and output are irrelevant), mapped to the virtual XB u,v,w,x u and v are the same, w and x are different, because the input and output are not conflicting, it is possible to perform parallel virtual XB calculation (i.e., perform parallel virtual XB calculation of u and v are the same, w and x are different), but DAC and ADC are not parallel.

[0161] Embodiments of the present disclosure determine the first total number of ADC conversions, the second total number of DAC conversions, and the third total number of virtual XB calculations performed in each array arrangement scheme based on the above rules, and in combination with the time delay of the memory-compute chip for performing one ADC conversion, the time delay of the memory-compute chip for performing one DAC conversion, and the time delay of the memory-compute chip for performing one virtual XB calculation, the time delay of each array arrangement scheme in the plurality of array arrangement schemes can be calculated. Compared with related technologies, the ADC conversion, the DAC conversion, and the virtual XB calculation that can be performed in parallel can be reduced, thereby improving the computing power utilization rate of the memory-compute chip and reducing the time delay of the memory-compute chip.

[0162] In some embodiments of the present disclosure, the following method can be used to determine the array arrangement scheme that meets the preset condition from the plurality of array arrangement schemes, and the array arrangement scheme that meets the preset condition is used as the arrangement scheme of the target model.

[0163] The first method: traverse the plurality of array arrangement schemes to determine the array arrangement scheme that meets the preset condition in the plurality of array arrangement schemes.

[0164] For this method, the following optimization can be performed: (1) Assuming that the time delay of the chip for performing summation calculation on part of the calculation results exceeds the DAC time delay, for a Block (i.e., a Block i,j,k with the same i and k values) that is horizontally divided by an operator, it is optimal to combine them together (up to the DAC ratio) and arrange them on the virtual memory-compute array of the cycle number of the ADC of the same memory-compute array (which can eliminate the output result summation calculation). (2) For the case where a plurality of Blocks (i.e., Blocks i,j,k), since the input data of these Blocks are the same, it is optimal to combine them together and schedule to the virtual processing-in-memory array of the same number of DACs in the same cycle.

[0165] The second way: assuming the number of Blocks after model cutting is N, there are N! scheduling schemes based on N selected virtual processing-in-memory arrays. It can be seen that the traversal scheme is suitable for the case where N is not too large, otherwise the compiler of the processing-in-memory chip will cause the time delay to rise sharply when scheduling the target model.

[0166] Embodiments of the present disclosure adopt a genetic algorithm to find a relatively excellent scheduling scheme. First, a scheduling scheme is randomly selected as the initial array scheduling scheme of the genetic algorithm, that is, the first generation of parent schemes. Then, on the basis of this scheme, n pairs of Blocks are randomly selected K times to exchange the virtual processing-in-memory arrays (for the above optimization scheme, there may be a case of scheduling the Blocks as a whole, for this case, the Blocks to be exchanged can be multiple independent Blocks). Then, the scheduling scheme with the lowest time delay is found from the K results, and this is used as the parent scheme of the new generation. Then, on the basis of this parent scheme, n pairs of Blocks are randomly selected K times to exchange the virtual processing-in-memory arrays. This is repeated until the time delay of all schemes in the new generation is not smaller than the parent scheme of this round, or the difference is negligible, or the number of iterations reaches a certain number. Compared with the traversal scheme, this scheme can find a relatively excellent scheduling scheme at a lower cost.

[0167] In one example, the array scheduling scheme that meets the preset condition from the plurality of array scheduling schemes is used as the scheduling scheme of the target model, including: determining an initial array scheduling scheme from the plurality of array scheduling schemes by using a genetic algorithm; performing the i-th round of operation according to the initial array scheduling scheme: randomly selecting n pairs of Blocks k times from the initial array scheduling scheme, and adjusting the scheduling scheme of the n pairs of Blocks selected in the j-th time in the k times; for the j-th time, exchanging the Blocks with the adjusted scheduling scheme and the corresponding Blocks in the initial array scheduling scheme, to obtain the array scheduling scheme of the j-th time, until k array scheduling schemes are obtained; taking the array scheduling scheme with a smaller time delay than the initial array scheduling scheme from the k array scheduling schemes as the initial array scheduling scheme of the i+1-th round; until the initial array scheduling scheme that meets the preset condition is obtained, and taking the initial array scheduling scheme that meets the preset condition as the scheduling scheme of the target model; wherein i, j, and n are positive integers, and k is an integer greater than 1.

[0168] In one example, the preset condition includes at least one of the following:

[0169] The time delay of the k array arrangement schemes is greater than that of the initial array arrangement scheme of the mth round.

[0170] m is greater than a preset round threshold.

[0171] m is greater than i+1.

[0172] Figure 9 A flowchart of a reasoning method provided by an embodiment of the present disclosure is shown in FIG. 1. Figure 9 As shown in FIG. 1, the execution subject of the method is an electronic device. The method can include the following steps.

[0173] Step 901, in response to receiving a reasoning request for a target model, scheduling a storage-computing integrated chip deploying an arrangement scheme of the target model; wherein the arrangement scheme of the target model is Figure 1 The model arrangement method provided by the embodiment shown in FIG. 1 is deployed on the arrangement scheme of the storage-computing integrated chip.

[0174] Step 902, determining the computing task of each scheduling unit in the storage-computing integrated chip according to the arrangement scheme of the target model and the input and output of each operator in the target model.

[0175] Step 903, scheduling each scheduling unit to execute its own computing task to obtain a reasoning result.

[0176] In some embodiments of the present disclosure, the first parameter of the core of the storage-computing integrated chip is a true value, and the scheduling unit is a storage-computing unit.

[0177] In some embodiments of the present disclosure, the core in the storage-computing integrated chip is a scheduling unit.

[0178] In some embodiments of the present disclosure, one computing task includes: the computing task of a plurality of blocks horizontally cut in the arrangement scheme of the target model; or the computing task of a plurality of blocks vertically cut in the arrangement scheme of the target model.

[0179] When the arrangement scheme of the weight Block to the virtual storage-computing array is determined, the reasoning and computing control flow of the model based on the virtual storage-computing array is also determined. Based on the computing control flow of the model, combined with the input and output buffers of each operator, a computing control flow based on the storage-computing integrated chip can be formed, and the core of the storage-computing integrated chip is obviously a self-scheduling computing unit. For the storage-computing integrated chip with the first parameter being a true value (i.e., the storage-computing integrated chip that can also independently compute), the storage-computing array is also a self-scheduling computing unit, and then the computing control flow of each storage-computing array needs to be determined.

[0180] The calculation control flow of the compute unit of the memory-compute integrated chip includes a calculation task list. For a memory-compute integrated chip with a core as a minimum self-scheduling compute unit, one calculation task of one core should include the following contents (for the case that the minimum compute unit is a memory-compute array, the calculation task information is the same except that the memory-compute array id is not required), and then each core (or memory-compute array) executes according to its own calculation task list.

[0181] The memory-compute array information of the task includes: the id of the memory-compute array, the input range, and the output range.

[0182] Input data: data storage location and size.

[0183] The period number of the DAC. For a traditional arrangement scheme, the period number of the DAC calculated by each Block is converted from 0 to DAC_RATIO-1 for DAC_RATIO times. For the arrangement scheme based on the present disclosure, the number of analog-digital conversion of each Block based on the virtual memory-compute array is 1, and only the period number of the DAC of the virtual memory-compute array arranged by the Block needs to be given. Of course, for an operator with a large weight size, the weight may be cut horizontally multiple times, and these Blocks may finally be arranged to the same physical memory-compute array (the Blocks cut horizontally are arranged to the same period number of the ADC of the same physical memory-compute array, and the summation calculation step can be omitted), and for this case, the calculation of multiple Blocks is combined into one calculation task, and the task contains multiple period numbers of the DAC.

[0184] The period number different ADC. For a traditional arrangement scheme, the period number different ADC calculated by each Block is converted from 0 to ADC_RATIO-1 for ADC_RATIO times. For the arrangement scheme based on the present disclosure, the number of analog-digital conversion of each Block based on the virtual memory-compute array is 1, and only the period number different ADC of the virtual memory-compute array arranged by the Block needs to be given. Similarly, for the case that the weight of an operator with a large weight size may be cut vertically multiple times, and these Blocks may finally be arranged to the same physical memory-compute array with the same period number of the DAC (the Blocks cut vertically are arranged to the same period number of the DAC of the same physical memory-compute array, and the multiple times of analog-digital conversion of the same input data can be omitted), for this case, the calculation of multiple Blocks is combined into one calculation task, and the task contains multiple ADCs with the same period number.

[0185] Output data: data storage location and size.

[0186] For the foregoing method embodiments, in order to simplify the description, they are all described as a series of actions. However, those skilled in the art should know that this disclosure is not limited to the described order of actions, because according to this disclosure, some steps may be performed in other orders or simultaneously.

[0187] Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by this disclosure.

[0188] Figure 10 This is a block diagram of a model arrangement device 1000 provided in an embodiment of this disclosure. Figure 10 As shown, the device 1000 includes: an acquisition module 1001, used to acquire information about the virtual in-memory arrays corresponding to the in-memory computing units on the core of the in-memory computing chip; wherein the information includes size information and a first quantity determined based on DAC ratio and ADC ratio, the DAC ratio being the ratio between the input dimension of the in-memory computing unit and the number of DACs, and the ADC ratio being the ratio between the output dimension of the in-memory computing unit and the number of ADCs; a segmentation module 1002, used to segment the network weights of each layer in the target model according to the size information to obtain multiple blocks; and a mapping module 1003, used to map the multiple blocks to the first number of virtual in-memory arrays respectively to obtain an orchestration scheme for the target model; wherein the orchestration scheme for the target model includes at least virtual in-memory arrays executed in parallel, and each virtual in-memory array in the first number of virtual in-memory arrays corresponds to the DAC conversion and ADC conversion performed in each cycle number in the in-memory computing unit.

[0189] In summary, the model orchestration device proposed in this disclosure first determines the size information and a first quantity of the virtual in-memory array corresponding to the in-memory computing unit on the content of the in-memory computing chip based on the DAC ratio and ADC ratio by the acquisition module 1001. Then, the segmentation module 1002 segments the network weights of each layer in the target model according to the size information to obtain multiple blocks. Next, the mapping module 1003 maps the multiple blocks to the first quantity of virtual in-memory arrays to obtain the orchestration scheme of the target model. The orchestration scheme of the target model includes virtual in-memory arrays that are executed in parallel, which can realize the parallel computing of virtual in-memory arrays that have a mapping relationship with the blocks in the preceding and following layers of the network, thereby improving the computing power utilization of the in-memory computing chip and reducing the computing latency.

[0190] In some embodiments, the segmentation module 1002 is specifically used to: vertically segment the network weights of each layer in the target model according to the length in the size information, and horizontally segment the network weights of each layer in the target model according to the width in the size information, to obtain multiple blocks.

[0191] In some embodiments, the apparatus further comprises a recording module configured to record split information of the plurality of blocks; wherein the split information comprises a mode number of a split mode corresponding to each block and a layer number of a layer where each block is located; the mode number comprises a first mode number of a horizontal split and a second mode number of a vertical split; and the determining module is configured to determine the block arrangement scheme according to the split information of the plurality of blocks.

[0192] In some embodiments, the determining module is specifically configured to: if the split information of the plurality of blocks includes split information of a first block, the block arrangement scheme comprises a first arrangement scheme of setting the first block as a block for parallel execution; wherein the first block comprises all blocks with the same layer number; and if the split information of the plurality of blocks includes split information of a second block, the block arrangement scheme comprises a second arrangement scheme of setting the second block as a block for parallel execution; wherein the second block comprises all blocks with the same second mode number.

[0193] In some embodiments, the mapping module 1003 comprises a mapping unit configured to map the plurality of blocks to a first number of virtual storage-computing arrays respectively to obtain a plurality of array arrangement schemes; and a selecting unit configured to select an array arrangement scheme that meets a preset condition from the plurality of array arrangement schemes as the arrangement scheme of the target model.

[0194] In some embodiments, the mapping unit is specifically configured to: if the split information of the plurality of blocks includes split information of a first block, map the blocks in the first arrangement scheme to corresponding virtual storage-computing arrays in the first number of virtual storage-computing arrays respectively; if the split information of the plurality of blocks includes split information of a second block, map the blocks in the second arrangement scheme to corresponding virtual storage-computing arrays in the first number of virtual storage-computing arrays respectively; and if the split information of the plurality of blocks includes split information of a third block, map the third block to corresponding virtual storage-computing arrays in the first number of virtual storage-computing arrays respectively; wherein the third block comprises all blocks with the same first mode number in the plurality of blocks.

[0195] In some embodiments, the mapping unit is specifically configured to: if the split information of the plurality of blocks includes split information of a first block, map the blocks in the first arrangement scheme to DACs with different cycle numbers in the first number of virtual storage-computing arrays respectively; if the split information of the plurality of blocks includes split information of a second block, map the blocks in the second arrangement scheme to DACs with the same cycle number in the first number of virtual storage-computing arrays respectively; and if the split information of the plurality of blocks includes split information of a third block, map the third block to DACs with the same cycle number in the first number of virtual storage-computing arrays respectively.

[0196] In some embodiments, as a unit, comprising: a determining sub-unit, configured to respectively determine a time delay of each array arrangement scheme in a plurality of array arrangement schemes; and as a sub-unit, configured to take an array arrangement scheme meeting a preset condition in the plurality of array arrangement schemes as an arrangement scheme of a target model.

[0197] In some embodiments, the determining sub-unit is specifically configured to: respectively determine a first total number of times of performing ADC conversion, a second total number of times of performing DAC conversion, and a third total number of times of performing virtual memory and calculation array calculation in each array arrangement scheme in the plurality of array arrangement schemes; obtain a first time delay of performing one ADC conversion, a second time delay of performing one DAC conversion, and a third time delay of performing one virtual memory and calculation array calculation; and determine the time delay of each array arrangement scheme in the plurality of array arrangement schemes according to the first total number of times and the first time delay, the second total number of times and the second time delay, and the third total number of times and the third time delay.

[0198] In some embodiments, the determining sub-unit is specifically configured to: respectively determine whether there is parallel DAC conversion, whether there is parallel ADC conversion, and whether there is parallel virtual memory and calculation array calculation in each array arrangement scheme in the plurality of array arrangement schemes; if there is parallel DAC conversion in each array arrangement scheme, take the parallel DAC conversion as one DAC conversion; otherwise, add 1 to the number of times of performing DAC conversion until all DAC conversions included in each array arrangement scheme are traversed, to obtain a first total number of times; if there is parallel ADC conversion in each array arrangement scheme, take the parallel ADC conversion as one ADC conversion; otherwise, add 1 to the number of times of performing ADC conversion until all ADC conversions included in each array arrangement scheme are traversed, to obtain a second total number of times; and if there is parallel virtual memory and calculation array calculation in each array arrangement scheme, take the parallel virtual memory and calculation array calculation as one virtual memory and calculation array calculation; otherwise, add 1 to the number of times of performing virtual memory and calculation array calculation until all virtual memory and calculation array calculations included in each array arrangement scheme are traversed, to obtain a third total number of times.

[0199] In some embodiments, the determining sub-unit is specifically configured to: respectively determine whether there is parallel DAC conversion, whether there is parallel ADC conversion, and whether there is parallel virtual memory and calculation array calculation in each array arrangement scheme in the plurality of array arrangement schemes; if there is parallel DAC conversion in each array arrangement scheme, take the parallel DAC conversion as one DAC conversion; otherwise, add 1 to the number of times of performing DAC conversion until all DAC conversions included in each array arrangement scheme are traversed, to obtain a first total number of times; if there is parallel ADC conversion in each array arrangement scheme, take the parallel ADC conversion as one ADC conversion; otherwise, add 1 to the number of times of performing ADC conversion until all ADC conversions included in each array arrangement scheme are traversed, to obtain a second total number of times; and if there is parallel virtual memory and calculation array calculation in each array arrangement scheme, take the parallel virtual memory and calculation array calculation as one virtual memory and calculation array calculation; otherwise, add 1 to the number of times of performing virtual memory and calculation array calculation until all virtual memory and calculation array calculations included in each array arrangement scheme are traversed, to obtain a third total number of times.

[0200] In some embodiments, the determining subunit is specifically configured to: if there is a first array arrangement scheme in the plurality of array arrangement schemes, determine that there are parallelly executed ADC conversions, parallelly executed DAC conversions, and parallelly executed virtual memory-computing array calculations in the first array arrangement scheme; wherein the first array arrangement scheme comprises virtual memory-computing arrays corresponding to memory-computing units of different cores; if there is a second array arrangement scheme in the plurality of array arrangement schemes, determine whether there are parallelly executed DAC conversions, whether there are parallelly executed ADC conversions, and whether there are parallelly executed virtual memory-computing array calculations in the second array arrangement scheme; wherein the second array arrangement scheme comprises virtual memory-computing arrays corresponding to memory-computing units on the same core; and if there is a third array arrangement scheme in the plurality of array arrangement schemes, determine whether there are parallelly executed ADC conversions, parallelly executed DAC conversions, and parallelly executed virtual memory-computing array calculations in the third array arrangement scheme; wherein the third array arrangement scheme comprises virtual memory-computing arrays corresponding to the same memory-computing unit.

[0201] In some embodiments, the determining subunit is specifically configured to: if there is a fourth array arrangement scheme in the third array arrangement scheme, determine that there are parallelly executed DAC conversions and virtual memory-computing array calculations, and there are no parallelly executed ADC conversions in the fourth array arrangement scheme; wherein the fourth array arrangement scheme is a virtual memory-computing array having a mapping relationship with a fourth block that is vertically split and has the same layer number in the plurality of blocks; if there is a fifth array arrangement scheme in the third array arrangement scheme, determine that there are parallelly executed ADC conversions and virtual memory-computing array calculations, and there are no parallelly executed DAC conversions in the fifth array arrangement scheme; wherein the fifth array arrangement scheme is a virtual memory-computing array having a mapping relationship with a fifth block that is horizontally split and has the same layer number in the plurality of blocks; and if there is a sixth array arrangement scheme in the third array arrangement scheme, determine that there are parallelly executed virtual memory-computing array calculations, and there are no parallelly executed ADC conversions and DAC conversions in the sixth array arrangement scheme; wherein the third array arrangement scheme is a virtual memory-computing array having a mapping relationship with a sixth block, and the sixth block is a block other than the fourth block and the fifth block in the plurality of blocks; and wherein the sixth array arrangement scheme is an array arrangement scheme other than the fourth array arrangement scheme and the fifth array arrangement scheme in the third array arrangement scheme.

[0202] In some embodiments, the determining subunit is specifically configured to: if a value of a first parameter of the same core is a true value, determine that there are parallelly executed ADC conversions, parallelly executed DAC conversions, and parallelly executed virtual memory-computing array calculations in the second array arrangement scheme.

[0203] In some embodiments, as a unit, it is specifically used for: determining an initial array arrangement scheme from multiple array arrangement schemes using a genetic algorithm; performing the i-th round of operations according to the initial array arrangement scheme: randomly selecting n pairs of blocks k times from the initial array arrangement scheme, and adjusting the arrangement scheme of the n pairs of blocks selected j times in each of the k times; for the j-th time, exchanging the blocks after the arrangement scheme adjustment with the corresponding blocks in the initial array arrangement scheme to obtain the array arrangement scheme for the j-th time, until k array arrangement schemes are obtained; taking the array arrangement scheme with a latency less than the initial array arrangement scheme among the k array arrangement schemes as the initial array arrangement scheme for the (i+1)-th round; until an initial array arrangement scheme that meets the preset conditions is obtained, and taking the initial array arrangement scheme that meets the preset conditions as the arrangement scheme of the target model; where i, j, and n are positive integers, and k is an integer greater than 1.

[0204] In some embodiments, the preset conditions include at least one of the following: the latency of all k array orchestration schemes is greater than that of the initial array orchestration scheme in the m-th round; m is greater than a preset round number threshold; where m is greater than i+1.

[0205] Figure 11 This is a block diagram of an inference device 1100 provided in an embodiment of this disclosure. (See diagram below.) Figure 11 As shown, the device 1100 includes: a receiving module 1101 for receiving inference requests for a target model; a scheduling module 1102 for scheduling an in-memory computing chip that deploys an orchestration scheme for the target model; a determining module 1103 for determining the computational tasks of each scheduling unit in the in-memory computing chip based on the orchestration scheme of the target model and the inputs and outputs of each operator in the target model; and an inference module 1104 for scheduling each scheduling unit to execute its own computational tasks and obtain inference results.

[0206] In some embodiments, the first parameter of the kernel of the in-memory computing chip is a true value, and the scheduling unit is an in-memory computing unit.

[0207] In some embodiments, the kernel in the in-memory computing chip is a scheduling unit.

[0208] In some embodiments, a computational task includes: a computational task of multiple blocks horizontally divided in the orchestration scheme of the target model; or a computational task of multiple blocks vertically divided in the orchestration scheme of the target model.

[0209] Figure 12 This is a block diagram of an electronic device 1200 for implementing the above-described model orchestration method or inference method, provided as an embodiment of the present disclosure.

[0210] Based on the hardware implementation of the above program modules, and in order to implement the method of this disclosure embodiment, this disclosure embodiment also provides an electronic device, such as... Figure 12As shown, the electronic device 1200 includes:

[0211] a communication interface 1201, capable of information interaction with other devices;

[0212] a processor 1202 connected with the communication interface 1201 to realize information interaction with other devices, for running a computer program, executing the method provided by one or more technical solutions described above;

[0213] a memory 1203, on which a computer program is stored.

[0214] Specifically, the processor 1202 can obtain information of a virtual storage and calculation array corresponding to a storage and calculation unit on a kernel of a storage and calculation integrated chip; wherein, the information includes size information and a first number determined based on a DAC ratio and an ADC ratio, the DAC ratio is a ratio between an input dimension of the storage and calculation unit and a number of DACs, and the ADC ratio is a ratio between an output dimension of the storage and calculation unit and a number of ADCs; each layer network weight in a target model is divided according to the size information to obtain a plurality of blocks; the plurality of blocks are respectively mapped to the first number of virtual storage and calculation arrays to obtain an arrangement scheme of the target model; wherein, the arrangement scheme of the target model at least includes virtual storage and calculation arrays executed in parallel, and each virtual storage and calculation array in the first number of virtual storage and calculation arrays respectively corresponds to DAC conversion and ADC conversion of each cycle number in the storage and calculation unit.

[0215] It should be noted that the specific processing process of the processor 1202 can be understood with reference to the above method.

[0216] Of course, in actual application, each component in the electronic device 1200 is coupled together through the bus system 1204. It can be understood that the bus system 1204 is used to realize the connection and communication between the components. The bus system 1204 includes not only a data bus, but also a power bus, a control bus and a status signal bus. However, for the purpose of clear illustration, all kinds of buses are marked as the bus system 1204 in Figure 12 .

[0217] The memory 1203 in the embodiment of the present disclosure is used to store various types of data to support the operation of the electronic device 1200. Examples of these data include: any computer program used for operation on the electronic device 1200.

[0218] The method disclosed by the embodiments of the present disclosure can be applied to the processor 1202 or implemented by the processor 1202. The processor 1202 can be an integrated circuit chip having a signal processing capability. In the implementation process, each step of the above method can be completed by an integrated logic circuit or an instruction in the form of software in the processor 1202. The first processor 1202 described above can be a general processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The processor 1202 can implement or execute the disclosed methods, steps and logic block diagrams in the embodiments of the present disclosure. The general processor can be a microprocessor or any conventional processor, etc. In combination with the steps of the method disclosed in the embodiments of the present disclosure, the steps can be directly embodied as a hardware coding processor to execute, or be executed by a combination of hardware and software modules in the coding processor. The software module can be located in a storage medium, and the storage medium is located in the memory 1203. The processor 1202 reads the information in the memory 1203 and combines the hardware to complete the steps of the above method.

[0219] In exemplary embodiments, the electronic device 1200 can be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, micro-controllers, microprocessors, or other electronic elements for executing the above-described methods.

[0220] In exemplary embodiments, a non-transitory computer-readable storage medium including instructions, such as the memory 1204 including instructions, is also provided. The above-described methods can be executed by the processor 1220 of the electronic device 1200 by executing the instructions. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0221] The embodiments of the present disclosure also propose a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to make a computer execute the method described in the above embodiments of the present disclosure.

[0222] The embodiments of the present disclosure also propose a chip, such as Figure 13As shown, the chip includes a processor 1310 and an interface 1320. Among them, the number of the processor 1310 can be one or more, and the number of the interface 1320 can be multiple. The interface circuit is used to receive a signal from the memory of the electronic device, and send a signal to the processor. The signal includes computer instructions stored in the memory, and when the processor executes the computer instructions, the electronic device executes the method described in the above embodiments of the present disclosure.

[0223] It should be noted that, for the convenience of description, only the parts related to the application are shown in the drawings. The embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0224] It should be understood that the "system", "device", "unit" and / or "module" used in the present disclosure is a method for distinguishing different components, elements, parts, sections or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0225] As shown in the present disclosure and claims, unless the context clearly indicates otherwise, "one", "a", "an" and / or "the" do not refer to the singular, but also include the plural. Generally, the terms "comprise" and "include" only indicate that the steps and elements explicitly identified are included, and these steps and elements do not constitute an exclusive list, and the method or device can also include other steps or elements. The element defined by the statement "comprises a" does not exclude the presence of another identical element in the process, method, product or device comprising the element.

[0226] Among them, in the description of the embodiments of the present disclosure, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this paper is only a description of the association between the associated objects, which means that there can be three kinds of relationships, for example, A and / or B, which can mean: A exists alone, A and B exist together, and B exists alone. In addition, in the description of the embodiments of the present disclosure, "multiple" means two or more than two.

[0227] Hereinafter, the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include one or more of the features.

[0228] Flow diagrams have been used in the present disclosure to illustrate the operation of systems in accordance with embodiments of the present disclosure. It will be understood that the operations of the preceding or following examples do not necessarily have to be performed in the precise order described. Rather, various steps can be handled in an interleaved or reverse order. Additionally, other operations can be added or removed as appropriate.

[0229] In the description of the specification, reference has been made to the description of "one embodiment", "some embodiments", "an exemplary embodiment", "an example", "a specific example" or "some examples" etc. which means that a specific feature, structure, material or characteristic described in connection with the embodiment or example is included in at least one embodiment or example of the application. Descriptions of the above terms in the specification do not necessarily refer to the same embodiment or example. Also, the specific features, structures, materials, or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0230] Any process or method descriptions or blocks in flow diagrams in this specification, and elsewhere, can be understood as representing modules, segments, or portions of code which include one or more executable instructions for implementing specific logical functions or steps in the process. Alternate implementations are included within the scope of the preferred embodiments of the present application in which additional functionality can be added or further steps can be provided before, during, or after the described process steps. Embodiments of the present application may, for example, be facilitated by one or more machines or processors, which can be implemented as embodying the functions or steps described.

[0231] The logic and / or steps represented in flow diagrams or otherwise described herein, for example, can be considered as a sequence of instructions to implement logic functions, and can be embodied in any computer-readable medium for use by an instruction execution system, apparatus, or device, such as a computer-based system, a system including a processing module, or other system that can fetch the instructions from the instruction execution system, apparatus, or device and execute the instructions. In the context of this specification, a "computer-readable medium" can be any means that can contain, store, communicate, propagate or transport the program for use by or in connection with the instruction execution system, apparatus, or device. The computer-readable medium can be a machine-readable storage device (control method), a portable computer diskette (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber device, and a portable compact disc read-only memory (CD-ROM). In addition, the computer-readable medium can even be paper or other suitable medium upon which the program can be printed, as the program can be electronically captured, for example, by optically scanning the paper or other suitable medium, then electronically converted into a form that can be further processed by a computer. For example, the program can be transmitted from the website, server, or other remote sources using a modem, which is connected to a telephone line, and used to electronically capture the program.

[0232] It should be understood that each of the elements of the embodiments of the present application can be implemented in hardware, software, firmware, or a combination thereof. In the above-described embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented in hardware, and as in another embodiment, any one or a combination of the following technologies known in the art can be used: discrete logic circuitry having logic gates for implementing logic functions on data signals, application specific integrated circuits having appropriate combinational logic gates, programmable gate arrays (PGA), field programmable gate arrays (FPGA), etc.

[0233] Those skilled in the art of the present technology can understand that all or part of the steps carried out by the above-described embodiment methods can be completed by programs instructing related hardware, and the programs can be stored in a computer-readable storage medium. When the programs are executed, they include one or a combination of the steps of the method embodiments.

[0234] In addition, each function unit in each embodiment of the present application can be integrated in one processing module, or each unit can exist physically separately, or two or more units can be integrated in one module. The integrated module can be realized in the form of hardware, or in the form of software function module. When the integrated module is realized in the form of software function module and sold or used as an independent product, it can also be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk or an optical disk, etc.

[0235] Although the embodiments of the present application have been shown and described above, it should be understood by those skilled in the art that the above embodiments are exemplary and cannot be understood as limiting the present application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the present application.

Claims

1. A model orchestration method, characterized in that, The method comprises: obtaining information of a virtual memory-computing array corresponding to a memory-computing unit on a kernel of a memory-computing integrated chip; wherein the information comprises size information and a first quantity determined based on a DAC ratio and an ADC ratio, the DAC ratio being a ratio between an input dimension of the memory-computing unit and a number of DACs, and the ADC ratio being a ratio between an output dimension of the memory-computing unit and a number of ADCs; dividing each layer network weight in a target model according to the size information to obtain a plurality of blocks; mapping the plurality of blocks to the first quantity of virtual memory-computing arrays respectively to obtain an arrangement scheme of the target model; wherein the arrangement scheme of the target model at least comprises virtual memory-computing arrays executed in parallel, and each virtual memory-computing array in the first quantity of virtual memory-computing arrays corresponds to DAC conversion and ADC conversion of a cycle number in the memory-computing unit.

2. The method of claim 1, wherein, The method further comprises: recording the size information of the plurality of blocks; wherein the size information comprises at least one of the following: a way number of a corresponding division way of each block, and a layer number of a layer where each block is located; the way number comprises a first way number of horizontal division and a second way number of vertical division; 3. The method according to claim 1 or 2, characterized in that, determining a block arrangement scheme according to the size information of the plurality of blocks. The method further comprises: if the size information of the plurality of blocks includes size information of a first block, the block arrangement scheme comprises a first arrangement scheme of setting the first block as a block executed in parallel; wherein the first block comprises all blocks with the same layer number; 4. The method of claim 3, wherein, if the size information of the plurality of blocks includes size information of a second block, the block arrangement scheme comprises a second arrangement scheme of setting the second block as a block executed in parallel; wherein the second block comprises all blocks with the same second way number. The method further comprises: mapping the plurality of blocks to the first quantity of virtual memory-computing arrays respectively to obtain a plurality of array arrangement schemes; 5. The method according to claim 1 or 4, characterized in that, taking an array arrangement scheme meeting a preset condition in the plurality of array arrangement schemes as the arrangement scheme of the target model. The method further comprises: if the size information of the plurality of blocks includes size information of a first block, mapping the block in the first arrangement scheme to a corresponding virtual memory-computing array in the first quantity of virtual memory-computing arrays respectively; 6. The method of claim 5, wherein, if the size information of the plurality of blocks includes size information of a second block, mapping the block in the second arrangement scheme to a corresponding virtual memory-computing array in the first quantity of virtual memory-computing arrays respectively. ​ ​ If the splitting information of the plurality of blocks includes the splitting information of the third block, the third block is respectively mapped to the corresponding virtual memory and calculation array in the first number of virtual memory and calculation arrays; wherein the third block is all blocks with the same first mode number in the plurality of blocks.

7. The method of claim 6, wherein, If the splitting information of the plurality of blocks includes the splitting information of the first block, the blocks in the first arrangement scheme are mapped to the DACs with different cycle numbers in the first number of virtual memory and calculation arrays, and on the DACs with different cycle numbers; If the splitting information of the plurality of blocks includes the splitting information of the second block, the blocks in the second arrangement scheme are respectively mapped to the DACs with the same cycle number in the first number of virtual memory and calculation arrays; If the splitting information of the plurality of blocks includes the splitting information of the third block, the third block is respectively mapped to the corresponding virtual memory and calculation array in the first number of virtual memory and calculation arrays; wherein the third block is all blocks with the same first mode number in the plurality of blocks.

8. The method of claim 5, wherein, The method comprises: respectively determining the time delay of each array arrangement scheme in the plurality of array arrangement schemes; determining the array arrangement scheme that meets the preset condition in the plurality of array arrangement schemes as the arrangement scheme of the target model.

9. The method of claim 8, wherein, The method comprises: respectively determining the first total number of times of executing ADC conversion, the second total number of times of executing DAC conversion, and the third total number of times of executing virtual memory and calculation array calculation of each array arrangement scheme in the plurality of array arrangement schemes; obtaining the first time delay of executing ADC conversion once, the second time delay of executing DAC conversion once, and the third time delay of executing virtual memory and calculation array calculation once; determining the time delay of each array arrangement scheme in the plurality of array arrangement schemes according to the first total number of times and the first time delay, the second total number of times and the second time delay, and the third total number of times and the third time delay.

10. The method of claim 9, wherein, The method comprises: respectively determining whether there is parallel DAC conversion, whether there is parallel ADC conversion, and whether there is parallel virtual memory and calculation array calculation in each array arrangement scheme in the plurality of array arrangement schemes; if there is parallel DAC conversion in each array arrangement scheme, the parallel DAC conversion is regarded as one DAC conversion; otherwise, the number of times of executing DAC conversion is increased by 1 until all DAC conversions included in each array arrangement scheme are traversed, and the first total number of times is obtained; if there is parallel ADC conversion in each array arrangement scheme, the parallel ADC conversion is regarded as one ADC conversion; otherwise, the number of times of executing ADC conversion is increased by 1 until all ADC conversions included in each array arrangement scheme are traversed, and the second total number of times is obtained; If there is a parallel executed virtual storage and calculation array calculation in each array arrangement scheme, the parallel executed virtual storage and calculation array calculation is taken as a virtual storage and calculation array calculation; otherwise, the number of times of executing the virtual storage and calculation array calculation is added by 1 until all virtual storage and calculation array calculations included in each array arrangement scheme are traversed, and the third total number is obtained.

11. The method of claim 10, wherein, Determine whether there is parallel DAC conversion, whether there is parallel executed ADC conversion, and whether there is parallel executed virtual storage and calculation array calculation in each array arrangement scheme in the plurality of array arrangement schemes, including: If there is a first array arrangement scheme in the plurality of array arrangement schemes, it is determined that there is parallel executed ADC conversion, parallel executed DAC conversion, and parallel executed virtual storage and calculation array calculation in the first array arrangement scheme; wherein the first array arrangement scheme includes virtual storage and calculation array corresponding to storage and calculation units of different cores; If there is a second array arrangement scheme in the plurality of array arrangement schemes, it is determined whether there is parallel executed DAC conversion, whether there is parallel executed ADC conversion, and whether there is parallel executed virtual storage and calculation array calculation in the second array arrangement scheme; wherein the second array arrangement scheme includes virtual storage and calculation array corresponding to storage and calculation units on the same core; If there is a third array arrangement scheme in the plurality of array arrangement schemes, it is determined whether there is parallel executed ADC conversion, parallel executed DAC conversion, and parallel executed virtual storage and calculation array calculation in the third array arrangement scheme; Wherein the third array arrangement scheme includes virtual storage and calculation array corresponding to the same storage and calculation unit.

12. The method of claim 11, wherein, The determination of whether there is parallel executed ADC conversion, parallel executed DAC conversion, and parallel executed virtual storage and calculation array calculation in the third array arrangement scheme includes: If there is a fourth array arrangement scheme in the third array arrangement scheme, it is determined that there is parallel executed DAC conversion and virtual storage and calculation array calculation, and there is no parallel executed ADC conversion in the fourth array arrangement scheme; wherein the fourth array arrangement scheme is a virtual storage and calculation array having a mapping relationship with a fourth block that is vertically divided and has the same layer number in the plurality of blocks; If there is a fifth array arrangement scheme in the third array arrangement scheme, it is determined that there is parallel executed ADC conversion and virtual storage and calculation array calculation, and there is no parallel executed DAC conversion in the fifth array arrangement scheme; wherein the fifth array arrangement scheme is a virtual storage and calculation array having a mapping relationship with a fifth block that is horizontally divided and has the same layer number in the plurality of blocks; If there is a sixth array arrangement scheme in the third array arrangement scheme, it is determined that there is parallel executed virtual storage and calculation array calculation, and there is no parallel executed ADC conversion and DAC conversion in the sixth array arrangement scheme; wherein the third array arrangement scheme is a virtual storage and calculation array having a mapping relationship with the sixth block, and the sixth block is a block in the plurality of blocks other than the fourth block and the fifth block; The sixth array arrangement scheme is an array arrangement scheme other than the fourth array arrangement scheme and the fifth array arrangement scheme in the third array arrangement scheme.

13. The method of claim 10, wherein, The determining whether the second array arrangement scheme has parallelly executed DAC conversion, whether the second array arrangement scheme has parallelly executed ADC conversion, and whether the second array arrangement scheme has parallelly executed virtual memory and array calculation comprises: If the value of the first parameter of the same kernel is a true value, it is determined that the second array arrangement scheme has parallelly executed ADC conversion, parallelly executed DAC conversion, and parallelly executed virtual memory and array calculation.

14. The method of claim 5, wherein, The array arrangement scheme meeting the preset condition from the plurality of array arrangement schemes is used as the arrangement scheme of the target model, which comprises: An initial array arrangement scheme is determined from the plurality of array arrangement schemes by using a genetic algorithm; According to the initial array arrangement scheme, an i-th round of operation is performed: n pairs of blocks are randomly selected k times from the initial array arrangement scheme, and the arrangement scheme of the n pairs of blocks selected in the j-th time of the k times is adjusted; For the j-th time, the blocks with the adjusted arrangement scheme are exchanged with the corresponding blocks in the initial array arrangement scheme to obtain an array arrangement scheme in the j-th time, until k array arrangement schemes are obtained; An array arrangement scheme with a smaller time delay than the initial array arrangement scheme from the k array arrangement schemes is used as an initial array arrangement scheme in an (i+1)-th round; Until an initial array arrangement scheme meeting the preset condition is obtained, and the initial array arrangement scheme meeting the preset condition is used as the arrangement scheme of the target model; Wherein, i, j, n are positive integers, and k is an integer greater than 1.

15. The method of claim 14, wherein, The preset condition comprises at least one of the following: The time delay of the k array arrangement schemes is greater than that of the initial array arrangement scheme in the m-th round; m is greater than a preset round threshold value; Wherein, m is greater than i+1.

16. An inference method, comprising: The method comprises: In response to receiving an inference request for a target model, a scheduling arrangement scheme of a storage-computing integrated chip deploying the target model is scheduled; wherein the arrangement scheme of the target model is the arrangement scheme deployed on the storage-computing integrated chip by the method in any one of claims 1-15; According to the arrangement scheme of the target model and the input and output of each operator in the target model, the calculation task of each scheduling unit in the storage-computing integrated chip is determined; Scheduling each scheduling unit to perform its own calculation task to obtain an inference result.

17. The method of claim 16, wherein, The first parameter of the kernel of the storage-computing integrated chip is a true value, and the scheduling unit is a storage-computing unit.

18. The method of claim 16, wherein, The kernel in the storage-computing integrated chip is a scheduling unit.

19. The method of claim 18, wherein, One calculation task comprises: a calculation task of a plurality of blocks horizontally divided in the arrangement scheme of the target model; or a calculation task of a plurality of blocks vertically divided in the arrangement scheme of the target model.

20. A model orchestration apparatus, comprising: The device comprises: An acquisition module is configured to acquire information of a virtual memory and array corresponding to a storage-computing unit on a kernel of a storage-computing integrated chip; wherein the information comprises size information and a first quantity determined based on a DAC ratio and an ADC ratio, the DAC ratio being a ratio between an input dimension of the storage-computing unit and a number of DACs, and the ADC ratio being a ratio between an output dimension of the storage-computing unit and a number of ADCs. The cutting module is configured to cut each layer network weight in the target model according to the size information to obtain a plurality of blocks. The mapping module is configured to map the plurality of blocks to the first number of virtual memory-computing array respectively to obtain the arrangement scheme of the target model; wherein the arrangement scheme of the target model at least includes a virtual memory-computing array performing in parallel, and each virtual memory-computing array in the first number of virtual memory-computing arrays respectively corresponds to the DAC conversion and the ADC conversion of the cycle number.

21. An inference apparatus characterized by comprising: The device comprises: The receiving module is configured to receive an inference request for a target model; The scheduling module is configured to schedule a memory-computing integrated chip deploying an arrangement scheme of the target model; wherein the arrangement scheme of the target model is the arrangement scheme deployed on the memory-computing integrated chip by the method in any one of claims 1-15; The determining module is configured to determine a computing task of each scheduling unit in the memory-computing integrated chip according to the arrangement scheme of the target model and the input and output of each operator in the target model; The inference module is configured to schedule each scheduling unit to perform its own computing task to obtain an inference result.

22. An electronic device, comprising: One or more processors; A storage device in communication connection with the one or more processors, and one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method in any one of claims 1-19. The computer instructions are used to make the computer execute the method in any one of claims 1-19.

23. A non-transitory computer-readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to make the computer execute the method in any one of claims 1-19.

24. A chip, characterized by The computer instructions are used to make the computer execute the method in any one of claims 1-19. The computer instructions are used to make the computer execute the method in any one of claims 1-19.

Images