Data prefetching method and related apparatus

Abstract

A data prefetching method and a related apparatus, which are applied to the technical field of artificial intelligence. The data prefetching method is applied to a first network model. The first network model comprises a first operator. The data prefetching method comprises: a first data prefetching apparatus acquiring the first network model (S201); generating first information on the basis of the first network model (S202); sending the first information to a second data prefetching apparatus (S203); correspondingly, the second data prefetching apparatus acquiring the first information; and on the basis of the first information, prefetching, in a first prefetching mode, data corresponding to the first operator (S204). The first information comprises the first prefetching mode corresponding to the first operator. The method can improve the hit rate at which data is acquired in the computation process of the first operator, thereby reducing the length of a data transfer path, reducing the data transfer time, improving data transfer efficiency, and improving the inference performance of the first network model.

Description

Data prefetching method and related apparatus TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence (AI) technology, and in particular to a data prefetching method and related apparatus. BACKGROUND

[0002] The development of AI is changing the world at an unprecedented pace, and its influence has penetrated into all walks of life. Under the background of the rapid development of current AI technology, the actual deployment of AI network models has become a key step to realize their practical value. In the deployment process of AI network models, it is usually necessary to transfer data from double data rate synchronous dynamic random access memory (DDR SDRAM) (hereinafter referred to as DDR) to the local buffer memory of the neural network processing unit (NPU) for network model inference.

[0003] Currently, the commonly used data transfer strategy uses cache to improve data transfer performance, and usually pre-fetches data beyond the current needs of the NPU to reduce future access delay.

[0004] However, the data transfer efficiency of this data transfer strategy is low, resulting in low inference performance of the network model.

[0005] SUMMARY

[0006] The embodiments of the present application provide a data prefetching method and related apparatus, which can improve data transfer efficiency and thus improve the inference performance of the network model.

[0007] In a first aspect, the embodiments of the present application provide a data prefetching method applied to a first network model, the first network model comprising a first operator; the data prefetching method comprising:

[0008] obtaining the first network model;

[0009] generating first information based on the first network model, the first information comprising a first prefetching mode corresponding to the first operator.

[0010] In the embodiments of the present application, a data prefetching method is provided, which can obtain a first network model and obtain a first prefetching mode corresponding to a first operator in the first network model based on the first network model, so as to realize subsequent prefetching of data corresponding to the first operator in the first prefetching mode.

[0011] The current data transfer strategy uses a cache to improve data transfer performance, usually by pre-fetching a data amount exceeding the current demand of the NPU to reduce future access delay, but does not consider the pre-fetching characteristics of each different operator in NPU calculation. Even if a data amount exceeding the current demand of the NPU is pre-fetched, it may not be the data required for the operator calculation, so the hit rate of data acquisition during the operator calculation is low, data needs to be transferred from the DDR, the data transfer path length is long, the data transfer time is long, the data transfer efficiency is low, and the inference performance of the network model is low.

[0012] According to the embodiments of the present application, the data is pre-fetched by combining the pre-fetching mode corresponding to the pre-fetching characteristics of each different operator in the network model, so that the data pre-fetched by the pre-fetching mode corresponding to the pre-fetching characteristics of the operator is the data required for the operator calculation, which can improve the hit rate of data acquisition during the operator calculation, thereby reducing the data transfer path length, reducing the data transfer time, improving the data transfer efficiency, and improving the inference performance of the network model.

[0013] In a possible implementation, the first information is used to indicate that the data corresponding to the first operator is pre-fetched in the first pre-fetching mode.

[0014] In the present embodiment, the first information generated based on the first network model is used to indicate that the data corresponding to the first operator is pre-fetched in the first pre-fetching mode. The data pre-fetched by the first pre-fetching mode corresponding to the pre-fetching characteristics of the first operator is the data required for the first operator calculation, which can improve the hit rate of data acquisition during the first operator calculation, thereby improving the data transfer efficiency and improving the inference performance of the first network model.

[0015] In a possible implementation, the method further includes:

[0016] Based on the first network model, second information is generated, and the second information includes a first step length required to pre-fetch the data corresponding to the first operator in the first pre-fetching mode.

[0017] In the present embodiment, the data pre-fetching method can also obtain, based on the first network model, a first step length required to pre-fetch the data corresponding to the first operator in the first pre-fetching mode, thereby realizing subsequent pre-fetching of the data corresponding to the first operator in the first pre-fetching mode across the first step length. The data pre-fetched by the first pre-fetching mode corresponding to the pre-fetching characteristics of the first operator across the first step length is the data required for the first operator calculation, which can improve the hit rate of data acquisition during the first operator calculation, thereby improving the data transfer efficiency and improving the inference performance of the first network model.

[0018] Optionally, the first prefetching manner can include, but is not limited to, two categories: one is skip prefetching, and the other is continuous prefetching. The continuous prefetching refers to prefetching data blocks continuous to the current acquired data block, and the skip prefetching refers to prefetching data blocks spaced from the current acquired data block. The skip prefetching can further include fixed-step skip prefetching, non-fixed-step skip prefetching, linear skip prefetching, non-linear skip prefetching, etc., and the embodiments of the present application do not limit this. Optionally, the first prefetching manner can further include diagonal prefetching and other prefetching strategies, and the embodiments of the present application do not limit this.

[0019] Optionally, in the case that the first prefetching manner is skip prefetching, the second information is generated based on the first network model.

[0020] Optionally, in the case that the first prefetching manner is continuous prefetching, the second information can be generated or not, and the embodiments of the present application do not limit this.

[0021] Optionally, in the case that the first prefetching manner is continuous prefetching, if the second information is generated, the information of the first step included in the second information can be empty or 0, or can be an illegal value or other data form used to indicate that the data block to be prefetched is continuous to the current acquired data block, and the embodiments of the present application do not limit this.

[0022] In a possible implementation, the second information is used to indicate that the data corresponding to the first operator is prefetched by the first prefetching manner across the first step.

[0023] In the embodiment, the second information generated based on the first network model is used to indicate that the data corresponding to the first operator is prefetched by the first prefetching manner across the first step. The data prefetched by the first prefetching manner across the first step according to the prefetching characteristic of the first operator is the data required for the calculation of the first operator, which can improve the hit rate of data acquisition in the calculation process of the first operator, thereby improving the data carrying efficiency and the inference performance of the first network model.

[0024] In a possible implementation, the first information and / or the second information is carried in an offline model corresponding to the first network model, or the first information and / or the second information is carried in a first file.

[0025] In the embodiment, the generated first information and / or second information can be carried in the offline model corresponding to the first network model. Alternatively, the first information and / or second information can be embedded in the offline model corresponding to the first network model as parameters of the first operator. Alternatively, the generated first information and / or second information can exist as a first file (information) independent of the offline model, and the embodiments of the present application do not limit this.

[0026] In a possible implementation, the first information is generated based on the first network model, including:

[0027] The type of the first operator is obtained based on the first network model.

[0028] The first pre-fetching mode corresponding to the first operator is determined based on the type of the first operator and a correspondence between operator types and data pre-fetching modes.

[0029] In the embodiment, a possible specific implementation of determining the pre-fetching mode corresponding to the operator is provided. Specifically, the type of the first operator in the first network model is obtained based on the first network model, and the first pre-fetching mode corresponding to the first operator is determined based on the correspondence between the operator and the data pre-fetching mode and the type of the first operator. Through the embodiments of the present application, the first pre-fetching mode corresponding to the first operator is determined in combination with the pre-fetching characteristics of the first operator in the network model, so that the data pre-fetched through the first pre-fetching mode is all the data required for the calculation of the first operator, which can improve the hit rate of data acquisition in the calculation process of the first operator, thereby improving the data carrying efficiency and the inference performance of the first network model.

[0030] In a possible implementation, the correspondence between the operator types and the data pre-fetching modes is determined by a protocol or pre-configuration.

[0031] In the embodiment, the correspondence between the operator types and the data pre-fetching modes is determined by a protocol or pre-configuration, and the embodiments of the present application do not limit this.

[0032] In a possible implementation, the second information is generated based on the first network model, including:

[0033] The data format of the data block required by the first operator is obtained based on the first network model.

[0034] The first step size required for pre-fetching the data corresponding to the first operator in the first pre-fetching mode is determined based on the data format of the data block required by the first operator and the first pre-fetching mode.

[0035] In the embodiment, a possible implementation of determining a step size required for pre-fetching data corresponding to an operator in a pre-fetch manner is provided. Specifically, first, a data format of a data block required by a first operator in a first network model is obtained based on a first network model. Then, based on the data format of the data block required by the first operator and a first pre-fetch manner corresponding to the first operator, a first step size required for pre-fetching data corresponding to the first operator in the first pre-fetch manner is determined. Through the embodiment, the data format of the data block required in the calculation process of the first operator in the first network model and the first pre-fetch manner corresponding to the pre-fetch characteristic of the first operator are combined to determine the first step size required for pre-fetching data corresponding to the first operator in the first pre-fetch manner. As a result, the data pre-fetched by the first pre-fetch manner corresponding to the pre-fetch characteristic of the first operator across the first step size is the data required for the calculation of the first operator. The hit rate of obtaining data in the calculation process of the first operator is improved, thereby improving the data carrying efficiency and the inference performance of the first network model.

[0036] In a possible implementation, the first operator at least includes any one of the following:

[0037] a convolution Conv operator, a pooling Pooling operator, a matrix multiplication Matmul operator, a reshaping Reshape operator, a rectified linear Relu operator, an addition Add operator, a multiplication Mul operator, a normalized exponential Softmax operator, and a slicing Slice operator.

[0038] In the embodiment, the first operator in the first network model can include but is not limited to any one of the above types of operators. It can be understood that the above convolution Conv operator, the pooling Pooling operator, the matrix multiplication Matmul operator, and the reshaping Reshape operator belong to the jump pre-fetch operator type, and the above rectified linear Relu operator, the addition Add operator, the multiplication Mul operator, the normalized exponential Softmax operator, and the slicing Slice operator belong to the continuous pre-fetch (next-cache) operator type.

[0039] Optionally, the embodiment can further include other operators of the above types or other types of operators, and the like. The embodiment does not limit this.

[0040] In a possible implementation, the first pre-fetch manner at least includes any one of the following:

[0041] pre-fetching a data block continuous to a currently obtained data block, and pre-fetching a data block spaced from the currently obtained data block.

[0042] In a possible implementation, the first pre-fetch manner comprises pre-fetching data blocks that are spaced apart from a currently fetched data block, and the data format of the data block required by the first operator comprises N, C, H and W, and the information of the first stride comprises H and W.

[0043] N is the number of data blocks, and C, H and W are the format sizes of the data blocks.

[0044] In this embodiment, when the first pre-fetch manner comprises skip pre-fetching (i.e., the pre-fetched data block is spaced apart from the currently fetched data block), and the data format of the data block required by the first operator comprises information of N, C, H and W, it can be determined that the information of the first stride comprises information of H and W. According to the embodiment of the present application, the data format (N, C, H and W) of the data block required in the calculation process of the first operator in the first network model, and the first pre-fetch manner (skip pre-fetching) corresponding to the pre-fetching characteristic of the first operator are combined to determine the first stride (H and W) that needs to be crossed to pre-fetch the data corresponding to the first operator in the first pre-fetch manner, so that the data pre-fetched by the first pre-fetch manner corresponding to the pre-fetching characteristic of the first operator across the first stride is all the data required for the calculation of the first operator, which can improve the hit rate of data acquisition in the calculation process of the first operator, thereby improving the data carrying efficiency and the inference performance of the first network model.

[0045] In a possible implementation, the first pre-fetch manner comprises pre-fetching data blocks that are continuous with a currently fetched data block, and the information of the first stride is empty or 0.

[0046] In a second aspect, an embodiment of the present application provides a data pre-fetching method applied to a first network model, the first network model comprising a first operator, and the data pre-fetching method comprising:

[0047] obtaining first information, the first information comprising a first pre-fetch manner corresponding to the first operator;

[0048] based on the first information, pre-fetching data corresponding to the first operator in the first pre-fetch manner.

[0049] In the embodiment of the present application, a data pre-fetching method is provided, which can obtain first information and pre-fetch data corresponding to a first operator in a first network model in a first pre-fetch manner based on the first information, thereby reducing the data carrying path length, reducing the data carrying time consumption, improving the data carrying efficiency, and improving the inference performance of the network model.

[0050] The current data transfer strategy uses a cache to improve data transfer performance, usually by pre-fetching data exceeding the current demand of the NPU to reduce future access delay, but does not consider the pre-fetching characteristics of each different operator in NPU calculation. Even if data exceeding the current demand of the NPU is pre-fetched, it may not be the data required for the operator calculation, so the hit rate of data acquisition during the operator calculation is low, data needs to be transferred from the DDR, the data transfer path length is long, the time consumption is high, the data transfer efficiency is low, and the inference performance of the network model is low.

[0051] According to the embodiments of the present application, the data is pre-fetched by combining the pre-fetching mode corresponding to the pre-fetching characteristics of each different operator in the network model, so that the data pre-fetched by the pre-fetching mode corresponding to the pre-fetching characteristics of the operator is the data required for the operator calculation, which can improve the hit rate of data acquisition during the operator calculation, thereby reducing the data transfer path length, reducing the data transfer time consumption, improving the data transfer efficiency, and improving the inference performance of the network model.

[0052] In a possible implementation, the method further includes:

[0053] obtaining second information, the second information including a first step length required for pre-fetching data corresponding to the first operator in the first pre-fetching mode;

[0054] The pre-fetching data corresponding to the first operator in the first pre-fetching mode based on the first information includes:

[0055] Pre-fetching the data corresponding to the first operator in the first pre-fetching mode based on the first information and the second information and across the first step length.

[0056] In the present embodiment, a possible specific implementation of pre-fetching data is provided, specifically, the data pre-fetching method can further obtain second information, and pre-fetch the data corresponding to the first operator in the first pre-fetching mode based on the first information and the second information and across the first step length. It can be understood that the data pre-fetched by the first pre-fetching mode corresponding to the pre-fetching characteristics of the first operator across the first step length is the data required for the first operator calculation, which can improve the hit rate of data acquisition during the first operator calculation, thereby reducing the data transfer path length, reducing the data transfer time consumption, improving the data transfer efficiency, and improving the inference performance of the first network model.

[0057] Optionally, the first prefetching manner can include, but is not limited to, two categories: one is skip prefetching, and the other is continuous prefetching. The continuous prefetching refers to prefetching data blocks that are continuous to the data block currently acquired, and the skip prefetching refers to prefetching data blocks that are spaced from the data block currently acquired. The skip prefetching can further include fixed-step skip prefetching, non-fixed-step skip prefetching, linear skip prefetching, non-linear skip prefetching, and the like, which are not limited by embodiments of the present application. Optionally, the first prefetching manner can further include diagonal prefetching and other prefetching strategies, which are not limited by embodiments of the present application.

[0058] Optionally, in the case that the first prefetching manner is skip prefetching, the data is prefetched based on the first information and the second information.

[0059] Optionally, in the case that the first prefetching manner is continuous prefetching, the data can be prefetched based on the first information and the second information, or can be prefetched based on only the first information, which is not limited by embodiments of the present application.

[0060] Optionally, in the case that the first prefetching manner is continuous prefetching, if the data is prefetched based on the first information and the second information, the information of the first step included in the second information can be empty, or 0, or other data forms for indicating that the prefetched data block is continuous to the data block currently acquired, which is not limited by embodiments of the present application.

[0061] In a possible embodiment, the method further includes:

[0062] receiving a first instruction, the first instruction being used for requesting a first data block, the first data block including a data block required by the first operator to perform a first operation;

[0063] in response to the first instruction, acquiring and sending the first data block;

[0064] the first data is prefetched in the first prefetching manner based on the first information, including:

[0065] a second data block is prefetched in the first prefetching manner based on the first information, the second data block including a data block required by the first operator to perform a second operation, the second operation being a next operation of the first operation.

[0066] In the embodiment, a possible implementation of pre-fetching data is provided, specifically, in the case of receiving a first instruction, in response to the first instruction, a data block required by a first operator for a first operation is acquired and sent, it can be understood that at this time, the data is not pre-fetched, so the data block required by the first operator for the first operation needs to be carried and sent from the DDR, the data carrying path length is relatively long, the time consumption is relatively large, and the data carrying efficiency is relatively low. In this case, the data block required by the first operator for a second operation can be pre-fetched in a first pre-fetching manner based on first information, the second operation being a subsequent operation of the first operation, it can be understood that at this time, the data is pre-fetched, so the data block required by the first operator for the second operation does not need to be carried and sent from the DDR when the first operator performs the second operation, but can directly hit the data block required for the second operation in the pre-fetched data, the hit rate of the data acquired in the first operator calculation process can be improved, thereby the data carrying path length can be reduced, the data carrying time consumption can be reduced, the data carrying efficiency can be improved, and the inference performance of the first network model can be improved.

[0067] In a possible implementation, the method further includes:

[0068] receiving a first instruction, the first instruction being used to request a first data block, the first data block including a data block required by the first operator for a first operation;

[0069] in response to the first instruction, acquiring and sending the first data block;

[0070] the pre-fetching, in the first pre-fetching manner and across the first step length, of the data corresponding to the first operator based on the first information and the second information includes:

[0071] pre-fetching, in the first pre-fetching manner and across the first step length, a second data block based on the first information and the second information, the second data block including a data block required by the first operator for a second operation, the second operation being a subsequent operation of the first operation, and a storage address of the second data block being apart from a storage address of the first data block by the first step length.

[0072] In the embodiment, a possible implementation of pre-fetching data is provided. Specifically, in the case of receiving a first instruction, the data block required for the first operator to perform the first operation is acquired and sent in response to the first instruction. It can be understood that the data is not pre-fetched at this time, and therefore the data block required for the first operator to perform the first operation needs to be carried and sent from the DDR. The data carrying path is relatively long, the time consumption is relatively high, and the data carrying efficiency is relatively low. In this case, the data block required for the second operation of the first operator can be pre-fetched based on the first information and the second information in the first pre-fetching manner across the first step length. The second operation is the next operation of the first operation. It can be understood that the data is pre-fetched at this time, and therefore the data block required for the second operation of the first operator does not need to be carried and sent from the DDR when the second operation is performed. Instead, the data block required for the second operation can be directly hit in the pre-fetched data. The hit rate of the data acquired in the calculation process of the first operator can be improved, so that the data carrying path length can be reduced, the data carrying time consumption can be reduced, the data carrying efficiency can be improved, and the inference performance of the first network model can be improved.

[0073] In a possible implementation, the method further includes:

[0074] receiving a second instruction, the second instruction being used to request the second data block;

[0075] sending the second data block in response to the second instruction.

[0076] In the embodiment, since the data block required for the second operation of the first operator has been pre-fetched based on the first information or based on the first information and the second information, in the case of receiving the second instruction, the data block required for the second operation can be directly hit and sent in the pre-fetched data in response to the second instruction. Through the embodiment of the present application, the hit rate of the data acquired in the calculation process of the first operator can be improved, so that the data carrying path length can be reduced, the data carrying time consumption can be reduced, the data carrying efficiency can be improved, and the inference performance of the first network model can be improved.

[0077] In a possible implementation, the first information and / or the second information is carried in an offline model corresponding to the first network model; or the first information and / or the second information is carried in a first file.

[0078] In the embodiment, the first information and / or the second information acquired above can be carried in an offline model corresponding to the first network model. Alternatively, the first information and / or the second information can be embedded in the offline model corresponding to the first network model as parameters of the first operator. Alternatively, the first information and / or the second information acquired above can exist as a first file (information) independent of the offline model, and the present application does not limit this.

[0079] In a possible implementation, the first operator includes at least any of the following:

[0080] a convolution Conv operator, a pooling Pooling operator, a matrix multiplication Matmul operator, a reshaping Reshape operator, a rectified linear Relu operator, an addition Add operator, a multiplication Mul operator, a normalized exponential Softmax operator, and a slicing Slice operator.

[0081] In this embodiment, the first operator in the first network model can include, but is not limited to, any of the above types of operators. It can be understood that the above convolution Conv operator, the pooling Pooling operator, the matrix multiplication Matmul operator, and the reshaping Reshape operator belong to the operator type of jump prefetching, and the above rectified linear Relu operator, the addition Add operator, the multiplication Mul operator, the normalized exponential Softmax operator, and the slicing Slice operator belong to the operator type of continuous prefetching (next-cache).

[0082] Optionally, the embodiments of the present application can further include other operators of the above types or other types of operators, and the like, which are not limited by the embodiments of the present application.

[0083] In a possible implementation, the first prefetching manner includes at least any of the following:

[0084] prefetching data blocks continuous to the currently acquired data block, and prefetching data blocks spaced from the currently acquired data block.

[0085] In a possible implementation, the first prefetching manner includes prefetching data blocks spaced from the currently acquired data block, and the data format of the data block required by the first operator includes N, C, H, and W, and the information of the first step size includes H and W.

[0086] wherein N is the number of data blocks, and C, H, and W are the format size of the data block.

[0087] In the embodiment, when the first prefetch manner includes skip prefetching (i.e., the prefetched data block is spaced from the currently acquired data block), and the data format of the data block required by the first operator includes information of N, C, H, and W, it can be determined that the information of the first step length includes information of H and W. According to the embodiment, the data format (N, C, H, and W) of the data block required in the calculation process of the first operator in the first network model and the first prefetch manner (skip prefetching) corresponding to the prefetch characteristic of the first operator are combined to determine the first step length (H and W) required for the first operator to prefetch data corresponding to the first prefetch manner, so that the data prefetched by the first prefetch manner corresponding to the prefetch characteristic of the first operator across the first step length is all data required for the calculation of the first operator, the hit rate of acquiring data in the calculation process of the first operator can be improved, the data carrying efficiency can be improved, and the inference performance of the first network model can be improved.

[0088] In a possible implementation, the first prefetch manner includes prefetching a data block continuous to a currently acquired data block, and the information of the first step length is empty or 0.

[0089] In a third aspect, an embodiment of the present application provides a data prefetching apparatus, which comprises units for performing the method in any one of the first aspect.

[0090] In a possible design, the data prefetching apparatus is applied to a first network model, and the first network model comprises a first operator. The apparatus comprises:

[0091] a communication unit configured to acquire the first network model;

[0092] a processing unit configured to generate first information based on the first network model, the first information comprising a first prefetch manner corresponding to the first operator.

[0093] The processing unit and the communication unit described with respect to the third aspect and any possible implementation perform the steps as described with respect to the first aspect and the corresponding implementation.

[0094] The technical effects brought by the third aspect and any possible implementation can be referred to the introduction of the technical effects of the first aspect and the corresponding implementation.

[0095] In a fourth aspect, an embodiment of the present application provides a data prefetching apparatus, which comprises units for performing the method in any one of the second aspect.

[0096] In a possible design, the data prefetching apparatus is applied to a first network model, and the first network model comprises a first operator. The apparatus comprises:

[0097] a communication unit, configured to acquire first information, wherein the first information comprises a first prefetch mode corresponding to the first operator;

[0098] a processing unit, configured to prefetch data corresponding to the first operator in the first prefetch mode based on the first information.

[0099] The processing unit and the communication unit according to the fourth aspect and any possible implementation of the fourth aspect perform the steps as described with reference to the second aspect and the corresponding implementation.

[0100] The technical effects brought by the fourth aspect and any possible implementation of the fourth aspect can refer to the introduction of the technical effects of the second aspect and the corresponding implementation.

[0101] Optionally, in the data prefetching apparatus according to any one of the third aspect to the fourth aspect and any possible implementation of the third aspect to the fourth aspect,

[0102] In an implementation, the data prefetching apparatus is a data prefetching device. When the data prefetching apparatus is the data prefetching device, the communication unit can be a transceiver, or an input / output interface; and the processing unit can be at least one processor. Optionally, the transceiver can be a transceiver circuit. Optionally, the input / output interface can be an input / output circuit.

[0103] In another implementation, the data prefetching apparatus is a chip (system) or a circuit used in a data prefetching device. When the data prefetching apparatus is the chip (system) or the circuit used in the data prefetching device, the communication unit can be a communication interface (input / output interface), an interface circuit, an output circuit, an input circuit, a pin or related circuit on the chip (system) or the circuit; and the processing unit can be at least one processor, a processing circuit or a logic circuit.

[0104] In the fifth aspect, an embodiment of the present application provides a data prefetching apparatus, which comprises a processor. The processor is coupled with a memory and is configured to execute instructions in the memory to implement the method according to any one of the first aspect to the second aspect and any possible implementation of the first aspect to the second aspect. Optionally, the data prefetching apparatus further comprises the memory. Optionally, the data prefetching apparatus further comprises a communication interface, and the processor is coupled with the communication interface.

[0105] In the sixth aspect, an embodiment of the present application provides a chip, which comprises a logic circuit and a communication interface. The communication interface is configured to receive information or send information; and the logic circuit is configured to receive information or send information through the communication interface, so that the chip executes the method according to any one of the first aspect to the second aspect and any possible implementation of the first aspect to the second aspect.

[0106] In a seventh aspect, an embodiment of the present application provides a computer readable storage medium, which is used to store a computer program (also referred to as code or instruction); when the computer program is run on a computer, the method in any of the first aspect to the second aspect and any possible implementation manner is implemented.

[0107] In an eighth aspect, an embodiment of the present application provides a computer program product, which includes: a computer program (also referred to as code or instruction); when the computer program is run, the computer executes the method in any of the first aspect to the second aspect and any possible implementation manner.

[0108] In a ninth aspect, an embodiment of the present application provides a data prefetching system, which includes: a first data prefetching device and a second data prefetching device; wherein the first data prefetching device is used to execute the method in the first aspect and any possible implementation manner, and the second data prefetching device is used to execute the method in the second aspect and any possible implementation manner.

[0109] In a tenth aspect, an embodiment of the present application provides a mobile terminal, which includes at least one data prefetching device as described in the third aspect, or a data prefetching device as described in the fourth aspect, or a data prefetching device as described in the fifth aspect, or a chip as described in the sixth aspect, or a data prefetching system as described in the ninth aspect.

[0110] Optionally, the mobile terminal can be a vehicle, for example, a vehicle used in any possible scenario such as a car, a truck, an aircraft, a drone, a slow transport vehicle, a space vehicle, or a ship, and the like, and the present application is not limited thereto.

[0111] Optionally, the mobile terminal is used to implement the method described in any of the first aspect to the second aspect or any possible implementation manner.

[0112] In addition, in the process of executing the method in any of the first aspect to the second aspect and any possible implementation manner, the process of sending information and / or receiving information in the above method can be understood as the process of outputting information by the processor, and / or the process of receiving input information by the processor. When the information is output, the processor can output the information to the transceiver (or communication interface or sending module) so as to be transmitted by the transceiver. After the information is output by the processor, it can also need to be processed in other ways before reaching the transceiver. Similarly, when the processor receives the input information, the transceiver (or communication interface or sending module) receives the information and inputs it to the processor. Furthermore, after the transceiver receives the information, the information can need to be processed in other ways before being input to the processor.

[0113] Based on the above principles, for example, the sending information mentioned in the foregoing method can be understood as processor output information. For another example, the receiving information can be understood as processor receiving input information.

[0114] Optionally, for the transmitting, sending and receiving operations involved by the processor, if no special description is made, or if it is not contrary to the actual role or inherent logic in the related description, it can be more generally understood as processor output and receiving, input and the like.

[0115] Optionally, in the process of executing the method of any one of the first aspect to the second aspect and any possible implementation manner, the processor can be a processor specially used for executing the method, or a processor executing the method by executing computer instructions in a memory, such as a general-purpose processor. The memory can be a non-transitory memory, such as a read-only memory (ROM), which can be integrated on the same chip as the processor, or can be separately arranged on different chips. The type of the memory and the arrangement manner of the memory and the processor are not limited in the embodiments of the present application.

[0116] In a possible implementation manner, the at least one memory is located outside the device.

[0117] In another possible implementation manner, the at least one memory is located inside the device.

[0118] In another possible implementation manner, part of the at least one memory is located inside the device, and another part of the at least one memory is located outside the device.

[0119] In the present application, the processor and the memory can also be integrated into one device, that is, the processor and the memory can also be integrated together. BRIEF DESCRIPTION OF DRAWINGS

[0120] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments of the present application. Obviously, the drawings described below are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor on the basis of these drawings.

[0121] FIG. 1 is a schematic diagram of an architecture of a data prefetching system provided by an embodiment of the present application;

[0122] FIG. 2 is a schematic diagram of a data prefetching method provided by an embodiment of the present application;

[0123] FIG. 3 is a schematic diagram of a cache configuration table and a step configuration table according to an embodiment of the present application;

[0124] FIG. 4A is a schematic diagram of a pre-fetch data block according to an embodiment of the present application;

[0125] FIG. 4B is a schematic diagram of another pre-fetch data block according to an embodiment of the present application;

[0126] FIG. 5 is a flowchart of a data pre-fetch strategy according to an embodiment of the present application;

[0127] FIG. 6 is a schematic diagram of a data pre-fetch apparatus according to an embodiment of the present application;

[0128] FIG. 7 is a schematic diagram of an electronic device according to an embodiment of the present application;

[0129] FIG. 8 is a schematic diagram of a chip according to an embodiment of the present application. DETAILED DESCRIPTION

[0130] In order to make the objects, technical solutions and advantages of the present application clearer, the embodiments of the present application will be described below with reference to the drawings.

[0131] The terms "first" and "second" and the like in the specification of the present application, claims, and drawings are used to distinguish different objects, and are not used to describe a particular order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device, etc. that includes a series of steps or units is not limited to the listed steps or units, but can optionally include other steps or units not listed, or can optionally include other steps or units inherent to the process, method, product, or device, etc.

[0132] In this document, "embodiment" means that the specific features, structures, or characteristics described in connection with the embodiment can be included in at least one embodiment of the present application. The appearance of this phrase in various places in the specification does not necessarily mean that the same embodiment is referred to each time, nor does it mean that other embodiments are mutually exclusive or alternative to the embodiment. It is explicitly and implicitly understood by those skilled in the art that, unless otherwise specified and logically contradictory, the terms and / or descriptions between various embodiments are consistent and can be mutually referred to, and the technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0133] It should be understood that in the present application, "at least one" refers to one or more, "multiple" refers to two or more, "at least two" refers to two or three and three or more, and "and / or" is used to describe the association relationship of the associated objects, which means that there can be three relationships, for example, "A and / or B" can mean: only A, only B, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects before and after it. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0134] It should be noted that in the present application, "indication" can include direct indication, indirect indication, display indication, and implicit indication. When describing that certain indication information is used to indicate A, it can be understood that the indication information carries A, directly indicates A, or indirectly indicates A.

[0135] In the present application, the information indicated by the indication information is referred to as the to-be-indicated information. In the specific implementation process, there are many ways to indicate the to-be-indicated information, for example, but not limited to, the to-be-indicated information can be directly indicated, such as the to-be-indicated information itself or the index of the to-be-indicated information. The to-be-indicated information can also be indirectly indicated by indicating other information, where the other information and the to-be-indicated information have an association relationship. The to-be-indicated information can also be indicated only by a part of the to-be-indicated information, and the other part of the to-be-indicated information is known or agreed in advance. For example, the indication of a specific information can also be achieved by means of the arrangement order of each information agreed in advance (for example, the protocol stipulates), thereby reducing the indication overhead to a certain extent. The to-be-indicated information can be sent together as a whole, or can be sent separately into multiple sub-information, and the sending period and / or sending time of these sub-information can be the same or different. The specific sending method is not limited by the present application. The sending period and / or sending time of these sub-information can be pre-defined, for example, pre-defined according to the protocol, or can be configured by the transmitting end device by sending configuration information to the receiving end device.

[0136] It should be noted that in the present application, "sending" can be understood as "output", and "receiving" can be understood as "input". "Sending information to A" means that A is the destination of the information transmission, and A is not limited to direct sending on the air interface. "Sending information to A" includes directly sending information to A, and also includes indirectly sending information to A through a transmitter, so "sending information to A" can also be understood as "outputting information to A". Similarly, "receiving information from A" means that the source of the information is A, which includes directly receiving information from A, and also includes indirectly receiving information from A through a receiver, so "receiving information from A" can also be understood as "inputting information from A".

[0137] The present application provides a data prefetching method and related device, which is applied to the field of artificial intelligence (AI) technology, such as data prefetching in AI network model operation. In order to better understand the technical solutions of the present application, the related terms and concepts that may be involved in the embodiments of the present application will be introduced first.

[0138] AI: is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. AI is a very broad science, including robots, language recognition, image recognition, natural language processing, expert systems, machine learning, computer vision, etc.

[0139] The development of AI is changing the world at an unprecedented speed, and its influence has penetrated into various industries. However, to fully realize the potential of AI, the key is to transform these advanced algorithms and models from theoretical research into practical applications, which requires the actual deployment of AI network models. In the context of the rapid development of current AI technology, the actual deployment of AI network models has become a key step to realize their practical value.

[0140] AI network models are composed of various operators, which can be divided into two categories according to the continuity of data in memory: 1. Operators whose calculation data are not continuous in memory (such as convolution operation conv, etc.); 2. Operators whose calculation data are continuous in memory (such as addition operation add, etc.).

[0141] In the deployment process of AI network models, it is usually necessary to move data from DDR to the local buffer memory of the neural network processing unit (NPU) for network model inference.

[0142] Currently, the commonly used data moving strategy uses cache to improve data moving performance, and usually reduces future access delay by pre-fetching data beyond the current needs of the NPU.

[0143] However, the data transfer strategy does not fully consider the specific needs of different operators for data prefetching during execution. Even if an amount of data beyond the current needs of the NPU is prefetched, it may not be the data required for the operator calculation, resulting in a low hit rate for data acquisition during the operator calculation, the need to transfer data from the DDR, a long data transfer path length, a long time consumption, a low data transfer efficiency, and low inference performance of the network model.

[0144] In view of this, the present application provides an architecture of a data prefetching system and a new data prefetching method based on the architecture, which is applied to the field of AI technology, such as data prefetching in AI network model operation. The data is prefetched by combining the prefetching mode corresponding to the prefetching characteristics of each different operator in the network model, so that the data prefetched by the prefetching mode corresponding to the prefetching characteristics of the operator is the data required for the operator calculation, which can improve the hit rate for data acquisition during the operator calculation, thereby reducing the data transfer path length, reducing the data transfer time consumption, improving the data transfer efficiency, and improving the inference performance of the network model.

[0145] Please refer to FIG. 1, which is a schematic diagram of an architecture of a data prefetching system provided by an embodiment of the present application.

[0146] As shown in FIG. 1, the architecture of the data prefetching system mainly includes but is not limited to:

[0147] The parsing module, the cache module, the main memory DDR, and the NPU local memory.

[0148] After the network model is deployed, the parsing module is used to parse the deployed network model to obtain each operator included in the network model, and generate a cache configuration table and a step configuration table based on the types of each operator.

[0149] The cache configuration table records the cache prefetching strategy most suitable for each operator in the network model according to its own prefetching characteristics, that is, it can be understood that the cache prefetching strategy can maximize the benefits of the operator. The step configuration table records the step length that needs to be crossed when actually performing data prefetching with the cache prefetching strategy in the cache configuration table, that is, it can be understood that performing data prefetching by crossing the step length information in the step configuration table with the cache prefetching strategy in the cache configuration table can maximize the benefits of the operator. It can be understood that there is a corresponding relationship between each operator in the network model, the cache prefetching strategy in the cache configuration table, and the step length information in the step configuration table.

[0150] The analysis module transmits the generated cache configuration table and step configuration table to the cache module. During network model inference, the cache module dynamically prefetches data from the main memory DDR to the cache module according to the incoming cache configuration table and step configuration table, and then transmits the data to the NPU local memory to participate in operator calculation in the network model.

[0151] Optionally, the cache module described above can be integrated into the NPU or can be a separate module independent of the NPU, and the embodiments of the present application do not limit this.

[0152] Through the architecture of the data prefetching system in the embodiments of the present application, data can be prefetched in combination with the prefetching manner corresponding to the prefetching characteristics of each different operator in the network model and cached in the cache module. Thus, the data prefetched by the prefetching manner corresponding to the prefetching characteristics of the operator is the data required for the calculation of the operator. The hit rate of data acquisition in the cache module during the calculation of the operator can be improved, thereby reducing the data transfer path length and data transfer time, improving data transfer efficiency, and improving the inference performance of the network model.

[0153] Based on the architecture of the data prefetching system shown in FIG. 1, the present application further provides a new data prefetching method, which will be described below in conjunction with FIGS. 2 to 5.

[0154] Please refer to FIG. 2, which is a flowchart of a data prefetching method provided by an embodiment of the present application. The data prefetching method is applied in the field of AI technology, such as data prefetching in AI network model operation. Specifically, the data prefetching method is applied to a first network model, and the first network model includes a first operator. The data prefetching method includes but is not limited to the following steps:

[0155] S201: A first data prefetching device acquires a first network model.

[0156] Optionally, the first network model can be a general AI network model. Specifically, the general model can be an onnx, tensorflow, or caffe model, etc. The embodiments of the present application do not limit this.

[0157] S202: The first data prefetching device generates first information based on the first network model.

[0158] The first information includes a first prefetching manner corresponding to the first operator.

[0159] Optionally, the first information is used to indicate that the data corresponding to the first operator is prefetched in the first prefetching manner.

[0160] It can be understood that the first information is used to instruct the second data prefetching device to prefetch data corresponding to the first operator in the first prefetching manner.

[0161] S203: The first data prefetching device sends the first information to the second data prefetching device, and correspondingly, the second data prefetching device receives the first information.

[0162] S204: The second data prefetching device prefetches data corresponding to the first operator in the first prefetching manner based on the first information.

[0163] It can be understood that the first data prefetching device in the embodiment of the application can be a device carrying a processor / chip that can be used to execute computer execution instructions, or can be a processor / chip that can be used to execute computer execution instructions. Alternatively, the first data prefetching device can be an electronic device or a processor / chip in an electronic device. Illustratively, the first data prefetching device can also be the parsing module in the above-mentioned FIG. 1, which is used to execute the data prefetching method in the embodiment of the application to improve the data carrying efficiency and thus improve the inference performance of the network model.

[0164] It can be understood that the second data prefetching device in the embodiment of the application can be a device carrying a processor / chip that can be used to execute computer execution instructions, or can be a processor / chip that can be used to execute computer execution instructions. Alternatively, the second data prefetching device can be an electronic device or a processor / chip in an electronic device. Illustratively, the second data prefetching device can also be the cache module in the above-mentioned FIG. 1, which is used to execute the data prefetching method in the embodiment of the application to improve the data carrying efficiency and thus improve the inference performance of the network model.

[0165] Alternatively, the data prefetching device and the data prefetching method in the embodiment of the application can be applied to scenarios involving NPU computing in fields including but not limited to the field of autonomous driving, the field of AI, the field of robots, the field of medical imaging, etc., specifically, the optimization of data prefetching strategy in the process of AI network model calculation. In the field of autonomous driving, it can be specifically applied to a vehicle-mounted system, the vehicle carrying the vehicle-mounted system is an intelligent driving vehicle, and the terminal device can be replaced, which can include but is not limited to vehicles such as commercial vehicles, passenger vehicles, trains, industrial vehicles (such as forklifts, trailers, tractors, etc.), engineering vehicles (such as excavators, bulldozers, cranes, etc.), robots, etc., which are not limited in the embodiment of the application.

[0166] Alternatively, the above-mentioned first prefetching manner can include but is not limited to two categories:

[0167] One is jump prefetching, and the other is continuous prefetching (next-cache).

[0168] wherein, the continuous prefetching refers to prefetching data blocks continuous to the currently acquired data block, and the skip prefetching refers to prefetching data blocks spaced from the currently acquired data block.

[0169] Optionally, the skip prefetching can further include fixed-step skip prefetching, non-fixed-step skip prefetching, linear skip prefetching, non-linear skip prefetching, etc., and the embodiments of the present application do not make any limitation in this regard.

[0170] Optionally, the first prefetching mode can further include, but not limited to, diagonal prefetching and other prefetching strategies, and the embodiments of the present application do not make any limitation in this regard.

[0171] Optionally, the first operator can include, but not limited to, any one of the following types of operators:

[0172] convolution Conv operator, pooling Pooling operator, matrix multiplication Matmul operator, reshaping Reshape operator, rectified linear Relu operator, addition Add operator, multiplication Mul operator, normalized exponential Softmax operator, and slicing Slice operator.

[0173] It can be understood that the convolution Conv operator, the pooling Pooling operator, the matrix multiplication Matmul operator, and the reshaping Reshape operator belong to the operator type of skip prefetching, and the rectified linear Relu operator, the addition Add operator, the multiplication Mul operator, the normalized exponential Softmax operator, and the slicing Slice operator belong to the operator type of continuous prefetching (next-cache).

[0174] Optionally, the first operator can further include, but not limited to, other operators of the above types or other operators of other types, etc., and the embodiments of the present application do not make any limitation in this regard.

[0175] It can be understood that in the data prefetching method in the embodiments of the present application, the first network model can be acquired, and the first prefetching mode corresponding to the first operator in the first network model can be obtained based on the first network model, so that the data corresponding to the first operator can be subsequently prefetched in the first prefetching mode.

[0176] The current data carrying strategy uses cache to improve the data carrying performance, and usually reduces the future access delay by prefetching data exceeding the current demand of NPU, but does not consider the prefetching characteristics of each different operator in NPU calculation. Even if the data exceeding the current demand of NPU is prefetched, it may not be the data required by the operator, so the hit rate of the data acquired in the calculation process of the operator is low, the data carrying path length is long, the time consumption is large, the data carrying efficiency is low, and the inference performance of the network model is low.

[0177] By the embodiment of the present application, the data is pre-fetched by combining the pre-fetching manner corresponding to the pre-fetching characteristics of each different operator in the network model, so that the data pre-fetched by the pre-fetching manner corresponding to the pre-fetching characteristics of the operator is the data required for the calculation of the operator, which can improve the hit rate of data acquisition in the calculation process of the operator, thereby reducing the data transfer path length, reducing the data transfer time consumption, improving the data transfer efficiency, and improving the inference performance of the network model.

[0178] In a possible embodiment, the first data pre-fetching device in the step S202 generates the first information based on the first network model, which can be implemented by the following manner:

[0179] The first data pre-fetching device obtains the type of the first operator based on the first network model.

[0180] The first data pre-fetching device determines the first pre-fetching manner corresponding to the first operator based on the type of the first operator and the correspondence between the operator type and the data pre-fetching manner.

[0181] It can be understood that the type of the first operator in the first network model is obtained based on the first network model first, and then the first pre-fetching manner corresponding to the first operator is determined based on the correspondence between the operator and the data pre-fetching manner and the type of the first operator.

[0182] Optionally, the correspondence between the operator type and the data pre-fetching manner can be specified by a protocol or pre-configured, and the embodiment of the present application does not limit this.

[0183] By the embodiment of the present application, the first pre-fetching manner corresponding to the first operator in the network model is determined based on the pre-fetching characteristics of the first operator, which can make the data pre-fetched by the first pre-fetching manner be the data required for the calculation of the first operator, and can improve the hit rate of data acquisition in the calculation process of the first operator, thereby improving the data transfer efficiency and improving the inference performance of the first network model.

[0184] In a possible embodiment, the data pre-fetching method can further perform the following steps, but is not limited to:

[0185] The first data pre-fetching device generates second information based on the first network model.

[0186] The second information includes the first step length required for pre-fetching the data corresponding to the first operator in the first pre-fetching manner.

[0187] Optionally, the second information is used to indicate the pre-fetching of the data corresponding to the first operator in the first pre-fetching manner by crossing the first step length.

[0188] It can be understood that the second information is used to indicate that the second data prefetching device pre-fetches data corresponding to the first operator in the first pre-fetching manner by the first step length.

[0189] Optionally, in the case that the first pre-fetching manner is skip pre-fetching, the second data prefetching device generates the second information based on the first network model.

[0190] Optionally, in the case that the first pre-fetching manner is continuous pre-fetching, the second data prefetching device can not generate the second information, or can generate the second information, and the embodiments of the present application do not limit this.

[0191] Optionally, in the case that the first pre-fetching manner is continuous pre-fetching, if the second data prefetching device generates the second information, the information of the first step length included in the second information can be empty, or 0, or other data forms for indicating that the pre-fetched data block is continuous with the currently acquired data block, and the embodiments of the present application do not limit this.

[0192] Optionally, the first information and / or the second information can be carried in the offline model corresponding to the first network model.

[0193] Optionally, the first information and / or the second information can be embedded in the offline model corresponding to the first network model as parameters of the first operator. Alternatively, the first information and / or the second information can exist as a first file (information) independent of the offline model, and the embodiments of the present application do not limit this.

[0194] Optionally, the offline model corresponding to the first network model can be obtained by model conversion processing of the first network model.

[0195] For example, in a mobile data center (MDC) positioned as an intelligent driving computing platform, the offline model (om) is obtained by model conversion processing of the first network model.

[0196] Through the embodiments of the present application, the data pre-fetched by the first pre-fetching manner corresponding to the pre-fetching characteristics of the first operator by the first step length is the data required for the calculation of the first operator, which can improve the hit rate of data acquisition in the calculation process of the first operator, thereby improving the data carrying efficiency and improving the inference performance of the first network model.

[0197] Optionally, the first data prefetching device generates the second information based on the first network model, which can be implemented by the following manner:

[0198] The first data prefetching apparatus obtains, based on the first network model, a data format of a data block required by the first operator.

[0199] The first data prefetching apparatus determines, based on the data format of the data block required by the first operator and the first prefetching manner, a first step size required for the first operator to cross in prefetching data corresponding to the first operator in the first prefetching manner.

[0200] It can be understood that the data format of the data block required by the first operator in the first network model is first obtained based on the first network model, and then the first step size required for the first operator to cross in prefetching data corresponding to the first operator in the first prefetching manner is determined based on the data format of the data block required by the first operator and the first prefetching manner corresponding to the prefetching characteristic of the first operator.

[0201] Through the embodiments of the present application, the first step size required for the first operator to cross in prefetching data corresponding to the first operator in the first prefetching manner is determined by combining the data format of the data block required by the first operator in the first network model and the first prefetching manner corresponding to the prefetching characteristic of the first operator, so that the data prefetched by the first operator in the first prefetching manner can be the data required by the first operator for calculation, the hit rate of data acquisition in the calculation process of the first operator can be improved, the data carrying efficiency can be improved, and the inference performance of the first network model can be improved.

[0202] It can be understood that the first operator in the first network model, the first prefetching manner in the first information, and the first step size information in the second information have the following corresponding relationship:

[0203] Based on the prefetching characteristic of the first operator in the first network model, the first prefetching manner corresponding to the first operator can be determined.

[0204] Based on the data format of the data block required by the first operator in the first network model and the first prefetching manner corresponding to the first operator, the first step size required for the first operator to cross in prefetching data corresponding to the first operator in the first prefetching manner can be determined.

[0205] Optionally, the corresponding relationship among the above three can refer to FIG. 3, which is a schematic diagram of a cache configuration table and a step size configuration table provided by an embodiment of the present application.

[0206] As shown in FIG. 3, a network model is first deployed, which can be a general AI network model. The general model can be an onnx, tensorflow or caffe model, etc., which is not limited by the embodiments of the present application.

[0207] After the network model is deployed, the parsing module can obtain parameters of each operator in the network model by parsing the deployed network model, and the main obtained parameters include but are not limited to the type of each operator and the data format of the data block required by each operator.

[0208] It can be understood that the parsing module internally has a mapping table of operator types and prefetch strategies, which records the mapping relationship between each operator type and the prefetch strategy, and the mapping relationship can be specified by a protocol or pre-configured and determined, and the embodiments of the application do not limit this.

[0209] For example, the network model includes the following operator types: convolution Conv2D operator, rectified linear Relu operator, reshaping Reshape operator, addition Add operator, pooling Pooling operator, format conversion TransData operator, and multiplication Mul operator.

[0210] Among them, the convolution Conv2D operator, the reshaping Reshape operator, and the pooling Pooling operator correspond to the jump prefetch strategy, and the rectified linear Relu operator, the addition Add operator, and the multiplication Mul operator correspond to the continuous prefetch (next_cache) strategy.

[0211] For another example, the data format (N, C, H, W) of the data block required by the first convolution Conv2D operator is (1, 16, 64, 32), and the data format (N, C, H, W) of the data block required by the second rectified linear Relu operator is (1, 16, 64, 32).

[0212] Among them, N is the number of data blocks, and C, H, and W are the format size of the data block.

[0213] The parsing module can generate a cache configuration table based on the type of each operator.

[0214] Among them, the cache configuration table records the cache prefetch strategy most suitable for each operator in the network model according to its prefetch characteristics, that is, it can be understood that the use of the cache prefetch strategy can maximize the benefit of the operator.

[0215] For example, the first convolution Conv2D operator most suitably adopts jump prefetch according to its prefetch characteristics, that is, it can be understood that the use of the jump prefetch strategy can maximize the benefit of the operator, and accordingly, the cache configuration table records 1. The second rectified linear Relu operator most suitably adopts continuous prefetch (next_cache) according to its prefetch characteristics, that is, it can be understood that the use of the continuous prefetch (next_cache) strategy can maximize the benefit of the operator, and accordingly, the cache configuration table records 0.

[0216] Optionally, the skip prefetch strategy can also be represented by 00, the next_cache strategy can also be represented by 01, or other data forms can also be used to represent the skip prefetch strategy and the next_cache strategy respectively, and the embodiments of the present application do not limit this.

[0217] It can be understood that the cache configuration table can be regarded as the first information described above.

[0218] Based on the data format of the data block required by each operator and the cache configuration table, the parsing module can generate a step configuration table.

[0219] In the step configuration table, the step information that needs to be crossed when actually performing data prefetching according to the cache prefetch strategy in the cache configuration table is recorded, that is, it can be understood that the data prefetching by crossing the step information in the step configuration table according to the cache prefetch strategy in the cache configuration table can maximize the benefit of the operator.

[0220] If the skip prefetch strategy is used, the data of H and W in the data format (N, C, H, W) of the data block required by the operator is saved in the step configuration table; if the next_cache strategy is used, the corresponding data in the step configuration table is empty (None).

[0221] For example, the step information (H, W) that needs to be crossed when actually prefetching the data corresponding to the first convolution Conv2D operator according to the skip prefetch strategy is (64, 32), and the step information that needs to be crossed when actually prefetching the data corresponding to the second rectified linear Relu operator according to the next_cache strategy is empty (None).

[0222] Optionally, the step information corresponding to the next_cache strategy can be empty (None), can also be 0, or an illegal value or other data form used to represent that the prefetched data block is continuous with the currently acquired data block, and the embodiments of the present application do not limit this.

[0223] It can be understood that the step configuration table can be regarded as the second information described above.

[0224] Optionally, when the parsing module parses the network model, the network model can also be synchronized into the model conversion process to obtain a corresponding offline model, and the offline model is used for deployment and reasoning on an NPU (MDC device, etc.).

[0225] Optionally, in the MDC that is an intelligent driving computing platform, the om (offline model) model is obtained by converting the network model.

[0226] Optionally, the cache configuration table and the step configuration table obtained by the parsing module parsing the network model can be embedded in the offline model corresponding to the network model as parameters of each operator. Alternatively, the first information and / or the second information can also exist as a first file (information) independent of the offline model, and the embodiments of the present application do not limit this.

[0227] It can be understood that during inference of the network model, the cache module can dynamically prefetch data from the main memory DDR into the cache module according to the incoming cache configuration table and step configuration table, and then transmit the data into the NPU local memory to participate in the operator calculation in the network model.

[0228] Optionally, the first prefetching manner includes skip prefetching, that is, a data block spaced from the currently acquired data block is prefetched, and the data format of the data block required by the first operator includes N, C, H, and W, and the information of the first step includes H and W.

[0229] Wherein, N is the number of data blocks, and C, H, and W are the format size of the data blocks.

[0230] It can be understood that when the first prefetching manner includes skip prefetching (that is, the prefetched data block is spaced from the currently acquired data block), and the data format of the data block required by the first operator includes information of N, C, H, and W, it can be determined that the information of the first step includes information of H and W.

[0231] For details, refer to FIGS. 4A and 4B, which are schematic diagrams of prefetched data blocks provided by the embodiments of the present application.

[0232] FIG. 4A is a schematic diagram of a data block input to the first operator, and FIG. 4B is a schematic diagram of the actual storage location of the data block input to the first operator.

[0233] As shown in FIGS. 4A and 4B, the data format of the data block input to the first operator includes N, C, H, and W, wherein N is the number of data blocks input to the first operator, which is 1 in this case, and C, H, and W are the format size of the data blocks. If the currently acquired first data block includes data point 1, data point 2, data point 3, and data point 4, and the prefetched second data block includes data point 5, data point 6, data point 7, and data point 8, the step information required to span the prefetched second data block includes H and W. Similarly, if the currently acquired first data block includes data point 1, data point 2, data point 3, and data point 4, and the prefetched third data block includes data point 9, data point 10, data point 11, and data point 12, the step information required to span the prefetched third data block includes H and W.

[0234] For example, the calculation process of the step information can be as follows: n+i×W+j×(H×W)

[0235] wherein H, W in the formula are H, W included in the data format described above, n represents the number of data points required to be crossed by a data point compared to the first data point in the row where the data point is located, i represents the number of rows (i.e. W) required to be crossed by the pre-fetch data point compared to the current acquisition data point, and j represents the number of surfaces (i.e. HxW) required to be crossed by the pre-fetch data point compared to the current acquisition data point.

[0236] For example, the number of data points required to be crossed by the data point 5 in the pre-fetch second data block compared to the first data point in the row where the data point 5 is located is 2, and at this time n = 2, the number of rows required to be crossed by the data point 5 in the pre-fetch second data block compared to the data point 1 in the current acquisition first data block is 2, and at this time i = 2, and the number of surfaces required to be crossed is 0, and at this time j = 0. Therefore, the step length information required to be crossed by the data point 5 in the pre-fetch second data block compared to the data point 1 in the current acquisition first data block includes 2 + 2xW.

[0237] The number of data points required to be crossed by the data point 6 in the pre-fetch second data block compared to the first data point in the row where the data point 6 is located is 3, and at this time n = 3, the number of rows required to be crossed by the data point 6 in the pre-fetch second data block compared to the data point 1 in the current acquisition first data block is 2, and at this time i = 2, and the number of surfaces required to be crossed is 0, and at this time j = 0. Therefore, the step length information required to be crossed by the data point 6 in the pre-fetch second data block compared to the data point 1 in the current acquisition first data block includes 3 + 2xW.

[0238] The number of data points required to be crossed by the data point 7 in the pre-fetch second data block compared to the first data point in the row where the data point 7 is located is 2, and at this time n = 2, the number of rows required to be crossed by the data point 7 in the pre-fetch second data block compared to the data point 1 in the current acquisition first data block is 3, and at this time i = 3, and the number of surfaces required to be crossed is 0, and at this time j = 0. Therefore, the step length information required to be crossed by the data point 7 in the pre-fetch second data block compared to the data point 1 in the current acquisition first data block includes 2 + 3xW.

[0239] The number of data points required to be crossed by the data point 8 in the pre-fetch second data block compared to the first data point in the row where the data point 8 is located is 3, and at this time n = 3, the number of rows required to be crossed by the data point 8 in the pre-fetch second data block compared to the data point 1 in the current acquisition first data block is 3, and at this time i = 3, and the number of surfaces required to be crossed is 0, and at this time j = 0. Therefore, the step length information required to be crossed by the data point 8 in the pre-fetch second data block compared to the data point 1 in the current acquisition first data block includes 3 + 3xW.

[0240] For example, the data point 9 in the third data block to be prefetched needs to cross 4 data points compared to the first data point in the row where the data point 9 is located, n = 4, needs to cross 0 rows compared to the data point 1 in the first data block to be acquired currently, i = 0, and needs to cross 1 surface, j = 1. Therefore, the step length information needed for the data point 9 in the third data block to be prefetched to cross the data point 1 in the first data block to be acquired currently includes 4 + H x W.

[0241] The data point 10 in the third data block to be prefetched needs to cross 5 data points compared to the first data point in the row where the data point 10 is located, n = 5, needs to cross 0 rows compared to the data point 1 in the first data block to be acquired currently, i = 0, and needs to cross 1 surface, j = 1. Therefore, the step length information needed for the data point 10 in the third data block to be prefetched to cross the data point 1 in the first data block to be acquired currently includes 5 + H x W.

[0242] The data point 11 in the third data block to be prefetched needs to cross 4 data points compared to the first data point in the row where the data point 11 is located, n = 4, needs to cross 1 row compared to the data point 1 in the first data block to be acquired currently, i = 1, and needs to cross 1 surface, j = 1. Therefore, the step length information needed for the data point 11 in the third data block to be prefetched to cross the data point 1 in the first data block to be acquired currently includes 4 + W + H x W.

[0243] The data point 12 in the third data block to be prefetched needs to cross 5 data points compared to the first data point in the row where the data point 12 is located, n = 5, needs to cross 1 row compared to the data point 1 in the first data block to be acquired currently, i = 1, and needs to cross 1 surface, j = 1. Therefore, the step length information needed for the data point 12 in the third data block to be prefetched to cross the data point 1 in the first data block to be acquired currently includes 5 + W + H x W.

[0244] By combining the data format (N, C, H, W) of the data block required in the first operator calculation process in the first network model and the first prefetching manner (skip prefetching) corresponding to the prefetching characteristic of the first operator, the first step (H, W) required for the first operator to prefetch data corresponding to the first prefetching manner is determined, so that the data prefetched by the first prefetching manner corresponding to the prefetching characteristic of the first operator across the first step is the data required for the calculation of the first operator, the hit rate of data acquisition in the calculation process of the first operator can be improved, thereby the data carrying efficiency can be improved, and the inference performance of the first network model can be improved.

[0245] Optionally, the first prefetching manner includes continuous prefetching, that is, prefetching a data block continuous to the currently acquired data block, and the information of the first step is empty or 0, or can be an illegal value or other data form for indicating that the prefetched data block is continuous to the currently acquired data block, and the embodiments of the present application do not limit this.

[0246] In a possible embodiment, in the case where the first data prefetching device generates the second information, the data prefetching method can further perform the following steps, but is not limited to the following steps:

[0247] The first data prefetching device sends the second information to the second data prefetching device, and correspondingly, the second data prefetching device receives the second information.

[0248] Correspondingly, in this case, the second data prefetching device in step S204 prefetches data corresponding to the first operator in the first prefetching manner based on the first information, which can be specifically implemented in the following manner:

[0249] The second data prefetching device prefetches data corresponding to the first operator in the first prefetching manner across the first step based on the first information and the second information.

[0250] It can be understood that in the data prefetching method in the embodiments of the present application, the second information can also be acquired, and data corresponding to the first operator can be prefetched in the first prefetching manner across the first step based on the first information and the second information.

[0251] It can be understood that the data prefetched by the first prefetching manner corresponding to the prefetching characteristic of the first operator across the first step is the data required for the calculation of the first operator, which can improve the hit rate of data acquisition in the calculation process of the first operator, thereby reducing the data carrying path length, reducing the data carrying time consumption, improving the data carrying efficiency, and improving the inference performance of the first network model.

[0252] Optionally, specifically in the case where the first prefetching manner is skip prefetching, the second data prefetching device prefetches data based on the first information and the second information.

[0253] Optionally, when the first prefetching method is continuous prefetching, the second data prefetching device may prefetch data based on the first information and the second information, or it may prefetch data based only on the first information. This application embodiment does not limit this.

[0254] Optionally, if the first prefetching method is continuous prefetching, and the second data prefetching device prefetches data based on the first information and the second information, the first step length information included in the second information can be empty, or 0, or an illegal value or other data form used to indicate that the prefetched data block is continuous with the currently acquired data block. This application embodiment does not limit this.

[0255] In one possible embodiment, data is prefetched based on first information, or data is prefetched based on first information and second information. For both cases, the data prefetching method may further perform the following steps, including but not limited to:

[0256] Scenario 1 (based on prefetched data from the first piece of information):

[0257] The first network model sends a first instruction to the second data prefetching device, and the second data prefetching device receives the first instruction accordingly.

[0258] The first instruction is used to request a first data block, which includes the data block required for the first operator to perform a first operation.

[0259] In response to the first instruction, the second data prefetching device acquires the first data block and sends the first data block to the first network model, and the first network model receives the first data block accordingly.

[0260] Understandably, no data is prefetched at this time, so the second data prefetching device needs to move and send the first data block required by the first operator to perform the first operation from the DDR. The data moving path is long, takes a lot of time, and has low data moving efficiency.

[0261] Furthermore, the second data device will also prefetch the second data block based on the aforementioned first information using a first prefetching method.

[0262] The second data block includes the data block required for the first operator to perform the second operation, which is the operation following the first operation.

[0263] It can be understood that the data is prefetched at this time, so that the second data block required for the second operation of the first operator does not need to be carried and sent from the DDR again when the first operator performs the second operation, but can hit the second data block required for the second operation directly in the prefetched data, which can improve the hit rate of data acquisition in the first operator calculation process, thereby reducing the data carrying path length, reducing the data carrying time consumption, improving the data carrying efficiency, and improving the inference performance of the first network model.

[0264] Optionally, the first network model sends a second instruction to the second data prefetching device, and correspondingly, the second data prefetching device receives the second instruction.

[0265] The second instruction is used to request the second data block.

[0266] Optionally, the second data prefetching device sends the second data block to the first network model in response to the second instruction, and correspondingly, the first network model receives the second data block.

[0267] It can be understood that since the second data block required for the second operation of the first operator has been prefetched based on the first information, in the case of receiving the second instruction, the second instruction can be responded to, and the data block required for the second operation can be hit and sent directly in the prefetched data, thereby reducing the data carrying path length, reducing the data carrying time consumption, improving the data carrying efficiency, and improving the inference performance of the first network model.

[0268] Case two (prefetch data based on first information and second information):

[0269] The first network model sends a first instruction to the second data prefetching device, and correspondingly, the second data prefetching device receives the first instruction.

[0270] The first instruction is used to request the first data block, and the first data block includes a data block required for the first operation of the first operator.

[0271] The second data prefetching device acquires the first data block in response to the first instruction, and sends the first data block to the first network model, and correspondingly, the first network model receives the first data block.

[0272] It can be understood that the data is not prefetched at this time, so the second data prefetching device needs to carry and send the first data block required for the first operation of the first operator from the DDR, the data carrying path length is relatively long, the time consumption is relatively large, and the data carrying efficiency is relatively low.

[0273] And the second data device will also prefetch the second data block across the first step length in the first prefetching manner based on the above first information and second information.

[0274] The second data block includes a data block required by the first operator to perform the second operation, the second operation is a subsequent operation of the first operation, and a storage address of the second data block is separated from a storage address of the first data block by a first step length.

[0275] It can be understood that the data is prefetched at this time, so that the second data block required by the first operator to perform the second operation does not need to be carried and sent from the DDR again when the first operator performs the second operation, but the second data block required by the second operation can be directly hit in the prefetched data, the hit rate of data acquisition in the first operator calculation process can be improved, the data carrying path length can be reduced, the data carrying time consumption can be reduced, the data carrying efficiency can be improved, and the inference performance of the first network model can be improved.

[0276] Optionally, the first network model sends a second instruction to the second data prefetching device, and correspondingly, the second data prefetching device receives the second instruction.

[0277] The second instruction is used to request the second data block.

[0278] Optionally, the second data prefetching device sends the second data block to the first network model in response to the second instruction, and correspondingly, the first network model receives the second data block.

[0279] It can be understood that since the second data block required by the first operator to perform the second operation has been prefetched based on the first information and the second information, in the case of receiving the second instruction, the second instruction can be responded to, and the data block required by the second operation can be directly hit and sent in the prefetched data, so that the data carrying path length can be reduced, the data carrying time consumption can be reduced, the data carrying efficiency can be improved, and the inference performance of the first network model can be improved.

[0280] Optionally, the execution steps in the above two cases can also be referred to FIG. 5, which is a flow diagram of a data prefetching strategy provided by an embodiment of the present application.

[0281] As shown in FIG. 5, after obtaining the first information (which can also be understood as the above-mentioned cache configuration table) and the second information (which can also be understood as the above-mentioned step configuration table), the cache module will switch to use different data prefetching strategies to dynamically prefetch data from the main memory DDR. Specifically as follows:

[0282] The NPU sends a first instruction to the cache module, and correspondingly, the cache module receives the first instruction.

[0283] The first instruction is used to request the first data block, and the first data block includes a data block required by the first operator to perform the first operation.

[0284] The cache module acquires the first data block in response to the first instruction and sends the first data block to the NPU. Accordingly, the NPU receives the first data block.

[0285] It can be understood that at this time, the data is not prefetched, and therefore the cache module needs to carry and send the first data block required for the first operation of the first operator from the DDR, the data carrying path length is relatively long, the data carrying time is relatively long, and the data carrying efficiency is relatively low.

[0286] In addition, the cache module also pre-fetches a second data block in the first prefetch manner based on the first information or pre-fetches the second data block in the first prefetch manner across a first step length based on the first information and the second information.

[0287] The second data block includes a data block required for a second operation of the first operator, the second operation is a subsequent operation of the first operation, and a storage address of the second data block is separated from a storage address of the first data block by the first step length.

[0288] It can be understood that at this time, the data is prefetched, and therefore the second data block required for the second operation of the first operator does not need to be carried and sent from the DDR during the second operation of the first operator, but can be directly hit in the prefetched data, which can improve the hit rate of data acquisition during the calculation of the first operator, thereby reducing the data carrying path length, reducing the data carrying time, improving the data carrying efficiency, and improving the inference performance of the NPU.

[0289] Optionally, the NPU sends a second instruction to the cache module, and accordingly, the cache module receives the second instruction.

[0290] The second instruction is used to request the second data block.

[0291] Optionally, the cache module sends the second data block to the NPU in response to the second instruction, and accordingly, the NPU receives the second data block.

[0292] It can be understood that since the second data block required for the second operation of the first operator has been prefetched, in the case of receiving the second instruction, the second instruction can be responded to, the data block required for the second operation is directly hit and sent in the prefetched data, thereby reducing the data carrying path length, reducing the data carrying time, improving the data carrying efficiency, and improving the inference performance of the NPU.

[0293] It can be understood that, compared with the data prefetching process shown by the solid line in FIG. 5, the data prefetching process shown by the dashed line has a much shorter data carrying path length, can greatly reduce the data carrying time, improve the data carrying efficiency, and improve the inference performance of the NPU.

[0294] The above describes the method of the embodiments of the present application in detail. The following provides an apparatus for implementing any one of the methods of the embodiments of the present application, for example, an apparatus including units (or means) for implementing each step performed by the device in any one of the above methods.

[0295] Referring to FIG. 6, FIG. 6 is a structural schematic diagram of a data prefetching apparatus provided by the embodiments of the present application.

[0296] As shown in FIG. 6, the data prefetching apparatus 60 can include a communication unit 601 and a processing unit 602. The communication unit 601 and the processing unit 602 can be software, hardware, or a combination of software and hardware.

[0297] The communication unit 601 can implement a sending function and / or a receiving function, and the communication unit 601 can also be described as a transceiving unit. The communication unit 601 can also be a unit integrating an acquisition unit and a sending unit, where the acquisition unit is configured to implement a receiving function, and the sending unit is configured to implement a sending function. Alternatively, the communication unit 601 can be configured to receive information sent by another apparatus, and can also be configured to send information to another apparatus.

[0298] In a possible design, the data prefetching apparatus 60 can correspond to the first data prefetching apparatus in the method embodiment shown in FIG. 2, and the data prefetching apparatus 60 can be an electronic device or a chip in an electronic device. The data prefetching apparatus 60 can include units for performing operations performed by the first data prefetching apparatus in the method embodiment shown in FIG. 2, and each unit in the data prefetching apparatus 60 is configured to implement operations performed by the first data prefetching apparatus in the method embodiment shown in FIG. 2. The data prefetching apparatus 60 is applied to a first network model, and the first network model includes a first operator. The units are described as follows:

[0299] The communication unit 601 is configured to acquire the first network model.

[0300] The processing unit 602 is configured to generate first information based on the first network model, where the first information includes a first prefetching manner corresponding to the first operator.

[0301] The steps performed by the communication unit 601 and the processing unit 602 described in the design can refer to the implementation corresponding to the first data prefetching device in the method embodiment shown in FIG. 2.

[0302] The technical effects brought by the implementation performed by the communication unit 601 and the processing unit 602 described in the design can refer to the introduction of the technical effects of the method embodiment shown in FIG. 2.

[0303] In another possible design, the data prefetching device 60 can correspond to the second data prefetching device in the method embodiment shown in FIG. 2. The data prefetching device 60 can be an electronic device or a chip in an electronic device. The data prefetching device 60 can include units for performing the operations performed by the second data prefetching device in the method embodiment shown in FIG. 2, and each unit in the data prefetching device 60 is respectively configured to implement the operation performed by the second data prefetching device in the method embodiment shown in FIG. 2. The data prefetching device 60 is applied to a first network model, and the first network model includes a first operator. The units are described as follows:

[0304] The communication unit 601 is configured to obtain first information, and the first information includes a first prefetching manner corresponding to the first operator.

[0305] The processing unit 602 is configured to prefetch data corresponding to the first operator in the first prefetching manner based on the first information.

[0306] The steps performed by the communication unit 601 and the processing unit 602 described in the design can refer to the implementation corresponding to the second data prefetching device in the method embodiment shown in FIG. 2.

[0307] The technical effects brought by the implementation performed by the communication unit 601 and the processing unit 602 described in the design can refer to the introduction of the technical effects of the method embodiment shown in FIG. 2.

[0308] According to the embodiments of the present application, each unit in the apparatus shown in FIG. 6 can be combined into one or several other units or split into more units with smaller functions to implement the same operation, without affecting the implementation of the technical effects of the embodiments of the present application. The above units are divided based on logical functions. In actual applications, the function of one unit can also be implemented by multiple units, or the functions of multiple units can be implemented by one unit. In other embodiments of the present application, the electronic device can also include other units, and these functions can also be implemented by other units in actual applications, and can be implemented by multiple units.

[0309] It should be noted that the implementation of each unit can also correspond to the description of the corresponding method embodiment of the method shown in FIG. 2.

[0310] In the data prefetching apparatus 60 described in FIG. 6, the data carrying efficiency can be improved, thereby improving the inference performance of the network model.

[0311] For the case that the above-mentioned data prefetching apparatus 60 is an electronic device, refer to the structural schematic diagram of the electronic device shown in FIG. 7.

[0312] It should be understood that the electronic device 70 shown in FIG. 7 is only an example, and the electronic device of the embodiments of the present application can also include other components, or include components similar in function to the components in FIG. 7, or not all the components in FIG. 7.

[0313] The electronic device 70 includes a transceiver interface 701 and at least one processor 702.

[0314] The electronic device 70 can correspond to the data prefetching apparatus. The transceiver interface 701 is used for transceiving signals, and the at least one processor 702 executes program instructions, so that the electronic device 70 implements the corresponding processes of the method performed by the corresponding device in the above-mentioned method embodiment.

[0315] In a possible design, the electronic device 70 can correspond to the first data prefetching apparatus in the method embodiment shown in FIG. 2, for example, the electronic device 70 can be the first data prefetching apparatus, or can be a chip in the first data prefetching apparatus. The electronic device 70 can include components for performing the operations performed by the first data prefetching apparatus in the above-mentioned method embodiment, and each component in the electronic device 70 is respectively for implementing the operations performed by the first data prefetching apparatus in the above-mentioned method embodiment. Specifically, it can be as follows:

[0316] Obtaining the first network model;

[0317] Based on the first network model, generating first information, the first information includes a first prefetching manner corresponding to the first operator.

[0318] The transceiver interface 701 and the at least one processor 702 described in this design perform the steps, which can be referred to the implementation corresponding to the first data prefetching apparatus in the method embodiment shown in FIG. 2.

[0319] The technical effects brought by the implementation of the transceiver interface 701 and the at least one processor 702 described in this design can be referred to the introduction of the technical effects corresponding to the method embodiment shown in FIG. 2.

[0320] In a possible design, the electronic device 70 can correspond to the second data prefetching apparatus in the method embodiment shown in FIG.2, and can be the second data prefetching apparatus or a chip in the second data prefetching apparatus. The electronic device 70 can include components for performing operations performed by the second data prefetching apparatus in the method embodiment, and each component in the electronic device 70 is respectively configured to implement the operations performed by the second data prefetching apparatus in the method embodiment. Specifically, the electronic device 70 can include the following components:

[0321] obtaining first information, the first information including a first prefetching manner corresponding to the first operator;

[0322] based on the first information, prefetching data corresponding to the first operator in the first prefetching manner.

[0323] The transceiver interface 701 and the at least one processor 702 described in this design perform steps, which can refer to the implementation corresponding to the second data prefetching apparatus in the method embodiment shown in FIG.2.

[0324] The technical effects brought by the implementation performed by the transceiver interface 701 and the at least one processor 702 described in this design can refer to the introduction of the technical effects of the method embodiment shown in FIG.2.

[0325] In the electronic device 70 described in FIG.7, the data carrying efficiency can be improved, and thus the inference performance of the network model can be improved.

[0326] For the case that the data prefetching apparatus 60 is a chip or a chip system, refer to the structural schematic diagram of the chip shown in FIG.8.

[0327] As shown in FIG.8, the chip 80 includes a processor 801 and an interface 802. The number of the processor 801 can be one or more, and the number of the interface 802 can be multiple. It should be noted that the functions of the processor 801 and the interface 802 can be implemented by hardware design, software design, or a combination of hardware and software, which is not limited here.

[0328] Optionally, the chip 80 can further include a memory 803, and the memory 803 is configured to store necessary program instructions and data.

[0329] In this application, the processor 801 can be configured to call the implementation program of the data prefetching method provided by one or more embodiments of the present application in the data prefetching apparatus from the memory 803, and execute the instructions included in the program. The interface 802 can be configured to output the execution result of the processor 801. In this application, the interface 802 can be specifically configured to output each message or information of the processor 801.

[0330] The data prefetch method provided in relation to one or more embodiments of the present application can refer to the various embodiments shown in the foregoing Figure 2, and will not be described again here.

[0331] The processor in the embodiments of the present application can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0332] The memory in the embodiments of the present application is used to provide storage space, and the storage space can store data such as operating systems and computer programs. The memory includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), or compact disc read-only memory (CD-ROM).

[0333] According to the method provided in the embodiments of the present application, the embodiments of the present application also provide a computer-readable storage medium, and the computer-readable storage medium stores a computer program. When the computer program runs on one or more processors, the method shown in the foregoing Figure 2 can be implemented.

[0334] According to the method provided in the embodiments of the present application, the embodiments of the present application also provide a computer program product, and the computer program product includes a computer program. When the computer program runs on a processor, the method shown in the foregoing Figure 2 can be implemented.

[0335] The embodiments of the present application provide a data prefetch system, which includes: a first data prefetch device and a second data prefetch device; wherein the first data prefetch device is used to perform the steps performed by the first data prefetch device in any of the embodiments of the foregoing Figure 2, and the second data prefetch device is used to perform the steps performed by the second data prefetch device in any of the embodiments of the foregoing Figure 2.

[0336] The embodiment of the present application further provides a mobile terminal, which comprises at least one data prefetching device 60, or an electronic device 70, or a chip 80, or the data prefetching system.

[0337] Optionally, the mobile terminal can be a vehicle, such as a car, a truck, an aircraft, a drone, a slow transport vehicle, a spacecraft, or a ship, or any possible vehicle used in any possible scenario, and the embodiment of the present application does not make any limitation.

[0338] The embodiment of the present application further provides a processing device, comprising a processor and an interface; the processor is used for executing the method in any of the method embodiments.

[0339] It should be understood that the processing device described above can be a chip. The units in the various device embodiments and the electronic device in the method embodiments correspond completely, and the corresponding steps are performed by the corresponding modules or units, for example, the communication unit (transceiver) performs the steps of receiving or sending in the method embodiment, and the other steps except sending and receiving can be performed by the processing unit (processor). The functions of the specific units can refer to the corresponding method embodiments. The processor can be one or more.

[0340] It can be understood that the electronic device in the embodiment of the present application can perform part or all of the steps in the embodiment of the present application, and these steps or operations are only examples, and the embodiment of the present application can also perform other operations or various modifications of the operations. In addition, each step can be executed in a different order from the embodiment of the present application, and it is possible that not all operations in the embodiment of the present application are executed.

[0341] In several embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented by other means. For example, the device embodiments described above are only schematic, and the division of the units is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0342] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment of the present application.

[0343] In addition, each function unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present separately, or two or more units can be integrated in one unit.

[0344] The functions, if implemented in the form of software functional units and sold or used as independent products, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the parts that essentially contribute to the present application or the parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0345] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application.

Claims

1. A data prefetching method, characterized by, The method is applied to a first network model, and the first network model comprises a first operator. The method comprises: obtaining the first network model; 2. The method of claim 1, wherein, generating first information based on the first network model, wherein the first information comprises a first pre-fetching mode corresponding to the first operator.

3. The method according to claim 1 or 2, characterized in that, The first information is used to instruct to pre-fetch data corresponding to the first operator in the first pre-fetching mode. The method further comprises:

4. The method of claim 3, wherein, generating second information based on the first network model, wherein the second information comprises a first step length required to pre-fetch data corresponding to the first operator in the first pre-fetching mode.

5. The method according to claim 3 or 4, characterized in that, The second information is used to instruct to pre-fetch data corresponding to the first operator in the first pre-fetching mode by crossing the first step length.

6. The method according to any one of claims 1 to 5, characterized in that, The first information and / or the second information are carried in an offline model corresponding to the first network model, or the first information and / or the second information are carried in a first file. The method of generating first information based on the first network model comprises: obtaining a type of the first operator based on the first network model; 7. The method of claim 6, wherein, determining the first pre-fetching mode corresponding to the first operator based on the type of the first operator and a correspondence relationship between operator types and data pre-fetching modes.

8. The method according to any one of claims 3 to 5, characterized in that, The correspondence relationship between operator types and data pre-fetching modes is determined by a protocol or pre-configuration. The method of generating second information based on the first network model comprises: obtaining a data format of a data block required by the first operator based on the first network model; 9. The method according to any one of claims 1 to 8, characterized in that, determining the first step length required to pre-fetch data corresponding to the first operator in the first pre-fetching mode based on the data format of the data block required by the first operator and the first pre-fetching mode. The first operator at least comprises any one of the following:

10. The method according to any one of claims 1 to 9, characterized in that, a convolution (Conv) operator, a pooling (Pooling) operator, a matrix multiplication (Matmul) operator, a reshape (Reshape) operator, a rectified linear (Relu) operator, an addition (Add) operator, a multiplication (Mul) operator, a normalized exponential (Softmax) operator, and a slice (Slice) operator. The first pre-fetching mode at least comprises any one of the following:

11. The method according to any one of claims 3 to 5 or 8, characterized in that, pre-fetching a data block continuous to a currently acquired data block, and pre-fetching a data block spaced from the currently acquired data block. When the first pre-fetching mode comprises pre-fetching a data block spaced from a currently acquired data block, the data format of the data block required by the first operator comprises N, C, H, and W, and the information of the first step length comprises H and W.

12. The method according to any one of claims 3 to 5 or 8, characterized in that, N is the number of data blocks, and C, H, and W are the format size of the data blocks.

13. A data prefetching method, comprising: When the first pre-fetching mode comprises pre-fetching a data block continuous to a currently acquired data block, the information of the first step length is empty or 0. The method is applied to a first network model, and the first network model comprises a first operator. The method comprises:

14. The method of claim 13, wherein, obtaining first information, wherein the first information comprises a first pre-fetching mode corresponding to the first operator; pre-fetching data corresponding to the first operator in the first pre-fetching mode based on the first information. The method further comprises: obtaining second information, the second information comprising a first step length required for pre-fetching data corresponding to the first operator in the first pre-fetch manner; the pre-fetching, in the first pre-fetch manner, of the data corresponding to the first operator based on the first information comprises: the pre-fetching, in the first pre-fetch manner, of the data corresponding to the first operator based on the first information and the second information comprises:

15. The method of claim 13, wherein, the method further comprises: receiving a first instruction for requesting a first data block, the first data block comprising a data block required for a first operation of the first operator; in response to the first instruction, obtaining and sending the first data block; the pre-fetching, in the first pre-fetch manner, of the data corresponding to the first operator based on the first information comprises: the pre-fetching, in the first pre-fetch manner, of a second data block based on the first information, the second data block comprising a data block required for a second operation of the first operator, the second operation being a subsequent operation of the first operation.

16. The method of claim 14, wherein, the method further comprises: receiving a first instruction for requesting a first data block, the first data block comprising a data block required for a first operation of the first operator; in response to the first instruction, obtaining and sending the first data block; the pre-fetching, in the first pre-fetch manner, of the data corresponding to the first operator based on the first information and the second information comprises: the pre-fetching, in the first pre-fetch manner, of a second data block based on the first information and the second information, the second data block comprising a data block required for a second operation of the first operator, the second operation being a subsequent operation of the first operation, a storage address of the second data block being apart from a storage address of the first data block by the first step length.

17. The method according to claim 15 or 16, characterized in that the method further comprises: receiving a second instruction for requesting the second data block; in response to the second instruction, sending the second data block.

18. The method of claim 14 or 16, wherein, the first information and / or the second information is borne in an offline model corresponding to the first network model, or the first information and / or the second information is borne in a first file.

19. The method according to any one of claims 13 to 18, characterized in that, the first operator at least comprises any one of the following: a convolution (Conv) operator, a pooling (Pooling) operator, a matrix multiplication (Matmul) operator, a reshaping (Reshape) operator, a rectified linear (Relu) operator, an addition (Add) operator, a multiplication (Mul) operator, a normalized exponential (Softmax) operator, and a slicing (Slice) operator.

20. The method of any one of claims 13-19, wherein, the first pre-fetch manner at least comprises any one of the following: pre-fetching a data block continuous to a currently obtained data block, and pre-fetching a data block spaced from the currently obtained data block.

21. The method of claim 14 or 16 or 18, wherein, the first pre-fetch manner comprises pre-fetching a data block spaced from a currently obtained data block, a data format of a data block required by the first operator comprises N, C, H, and W, and information of the first step length comprises H and W; wherein N is a number of data blocks, and C, H, and W are format sizes of data blocks.

22. The method of any one of claims 14 or 16 or 18, wherein, The first prefetching mode comprises prefetching data blocks continuous to the currently acquired data block, and the information of the first step length is empty or 0.

23. A data prefetching apparatus, comprising: comprising means for performing the method of any one of claims 1 to 22.

24. A data prefetching apparatus, comprising: comprising a processor for performing the method of any one of claims 1 to 22.

25. A chip, characterized by comprising a logic circuit and an interface, the logic circuit and the interface being coupled; the interface being for inputting and / or outputting information, the logic circuit being for performing the method of any one of claims 1 to 22.

26. A data prefetching system, comprising: comprising: a first data prefetching device and a second data prefetching device; wherein the first data prefetching device is for performing the method of any one of claims 1 to 12, and the second data prefetching device is for performing the method of any one of claims 13 to 22.

27. A mobile terminal, characterized by comprising the data prefetching device of claim 23, or the data prefetching device of claim 24, or the chip of claim 25, or the data prefetching system of claim 26.

28. A computer-readable storage medium, characterized in that, The computer readable storage medium is for storing a computer program, the computer program being executed to perform the method of any one of claims 1 to 22.

29. A computer program product, characterised in that, The computer program product comprises a computer program, the computer program being executed to perform the method of any one of claims 1 to 22.

Images