Data prefetching method and device, electronic equipment and computer readable medium

Abstract

The invention discloses a data prefetching method and device, electronic equipment and a computer readable medium, and the method comprises the steps: when to-be-processed data is stored in a first storage space, and the data size of the to-be-processed data comprises size characterization values under at least two dimensions, determining a thread division description parameter corresponding to each dimension according to the data size; according to the thread division description parameters corresponding to the at least two dimensions, selecting a target dimension meeting a preset condition from the at least two dimensions; performing thread division processing on the to-be-processed data on the target dimension to obtain at least one piece of block data and thread processing time constraints corresponding to the block data; and for any block of data, the block of data is prefetched from the first storage space to the second storage space according to the thread processing time constraint corresponding to the block of data, so that the moment when the block of data is stored in the second storage space meets the thread processing time constraint corresponding to the block of data, and the prefetching effect is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data pre-fetching method, device, electronic device, and computer-readable medium. Background Art

[0002] In some application fields (for example, high-performance computing), it is usually possible to pre-fetch certain data (for example, image data, etc.) from a low-level memory type (for example, memory) to a high-level memory type (for example, cache) so that the data stored in the high-level memory type (for example, cache) can be directly used to complete corresponding tasks (for example, image processing tasks, etc.), which is conducive to improving task execution results (for example, execution efficiency, etc.).

[0003] However, due to defects in some related prefetching schemes, the prefetching effect is relatively poor, thereby affecting the task execution effect of the corresponding task. Summary of the Invention

[0004] The present application provides a data pre-fetching method, device, electronic device, and computer-readable medium.

[0005] In order to achieve the above objectives, the technical solutions provided by this application are as follows:

[0006] The present application provides a data pre-fetching method, the method comprising:

[0007] When data to be processed is stored in the first storage space, and the data size of the data to be processed includes size representation values ​​in at least two dimensions, determining thread partition description parameters corresponding to each of the dimensions based on the data size, the thread partition description parameters including the number of thread tasks and the amount of data pre-fetched by the thread tasks;

[0008] Selecting a target dimension that meets a preset condition from the at least two dimensions according to thread partition description parameters corresponding to the at least two dimensions;

[0009] Perform thread division processing on the data to be processed on the target dimension to obtain at least one block of data and a thread processing time constraint corresponding to each block of data;

[0010] For any block of data, the block of data is pre-fetched from the first storage space to the second storage space according to the thread processing time constraint corresponding to the block of data.

[0011] In one possible implementation, the preset conditions include: the number of thread tasks corresponding to the target dimension is not less than a preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the target dimension is not greater than a data amount threshold.

[0012] In a possible implementation manner, the preset condition further includes: the target dimension is located first among the at least two dimensions.

[0013] In a possible implementation manner, an arrangement position of a first dimension corresponding to the target dimension in the at least two dimensions is adjacent to an arrangement position of the target dimension in the at least two dimensions, and an arrangement position of the first dimension in the at least two dimensions is forward of an arrangement position of the target dimension in the at least two dimensions;

[0014] The preset conditions also include: the number of thread tasks corresponding to the first dimension is not less than a preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the first dimension is greater than a data amount threshold;

[0015] or,

[0016] The preset condition also includes: the number of thread tasks corresponding to the first dimension is less than a preset number threshold.

[0017] In one possible implementation, the at least two dimensions include N dimensions arranged sequentially, and the process of determining the target dimension includes:

[0018] Initialize m to 1;

[0019] Determine whether the number of thread tasks corresponding to the mth dimension is less than a preset number threshold;

[0020] If the number of thread tasks corresponding to the m-th dimension is not less than a preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the m-th dimension is not greater than a data amount threshold, then the m-th dimension is determined as the target dimension;

[0021] If the number of thread tasks corresponding to the m-th dimension is not less than the preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the m-th dimension is greater than the data amount threshold, then update m and continue to perform the step of determining whether the number of thread tasks corresponding to the m-th dimension is less than the preset number threshold;

[0022] If the number of thread tasks corresponding to the mth dimension is less than the preset number threshold, then update m and continue to execute the step of determining whether the number of thread tasks corresponding to the mth dimension is less than the preset number threshold.

[0023] In one possible implementation, the at least two dimensions include N dimensions arranged sequentially, the number of thread tasks corresponding to the (n+1)th dimension is not less than the number of thread tasks corresponding to the (n)th dimension, where n is a positive integer and n+1≤N.

[0024] The process of determining the target dimension includes:

[0025] Initialize m to 1;

[0026] Determine whether the number of thread tasks corresponding to the mth dimension is less than a preset number threshold;

[0027] If the number of thread tasks corresponding to the m-th dimension is less than the preset number threshold, update m and continue to perform the step of determining whether the number of thread tasks corresponding to the m-th dimension is less than the preset number threshold;

[0028] If the number of thread tasks corresponding to the m-th dimension is not less than a preset number threshold, determining whether the amount of pre-fetched data of the thread task corresponding to the m-th dimension is greater than a data amount threshold;

[0029] If the amount of pre-fetched data of the thread task corresponding to the m-th dimension is not greater than the data amount threshold, determining the m-th dimension as the target dimension;

[0030] If the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold, then update m and continue to execute the step of determining whether the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold.

[0031] In a possible implementation manner, updating m includes:

[0032] If the m is less than N-1, update the m;

[0033] The method further comprises:

[0034] If m is equal to N-1, the Nth dimension is determined as the target dimension.

[0035] In one possible implementation, the target processor includes a first storage space and a second storage space;

[0036] The data amount threshold is determined according to the prefetch instruction constraint of the target processor.

[0037] In one possible implementation, for any dimension, if the dimension is located first among the at least two dimensions, the number of thread tasks corresponding to the dimension is determined based on the ratio between the size representation value and the number of threads in the dimension;

[0038] If the dimension is not the first one in the at least two dimensions, the number of thread tasks corresponding to the dimension is determined based on the size representation value of at least one second dimension corresponding to the dimension, the size representation value of the dimension and the number of threads, and the arrangement position of the second dimension in the at least two dimensions is ahead of the arrangement position of the dimension in the at least two dimensions.

[0039] In one possible implementation, for any of the dimensions, if the dimension is located in a non-tail position in the at least two dimensions, the amount of data pre-fetched by the thread task corresponding to the dimension is determined based on the size representation value of at least one third dimension corresponding to the dimension, and the arrangement position of the third dimension in the at least two dimensions is later than the arrangement position of the dimension in the at least two dimensions.

[0040] In a possible implementation manner, the data to be processed is vector data;

[0041] and / or,

[0042] The data to be processed is input data of the data processing model.

[0043] In a possible implementation manner, the data processing model is an image processing model; and the data to be processed is image data.

[0044] In a possible implementation manner, the data pre-fetching method is applied to a terminal device.

[0045] In a possible implementation manner, the first storage space is a memory in a target processor, and the second storage space is a cache in the target processor.

[0046] The present application provides a data pre-fetching device, comprising:

[0047] a first determining unit configured to, when data to be processed is stored in the first storage space and the data size of the data to be processed includes size representation values ​​in at least two dimensions, determine, based on the data size, thread partition description parameters corresponding to each of the dimensions, the thread partition description parameters including the number of thread tasks and the amount of data pre-fetched by the thread tasks;

[0048] a dimension screening unit, configured to select a target dimension that meets a preset condition from the at least two dimensions according to thread partition description parameters corresponding to the at least two dimensions;

[0049] a data partitioning unit, configured to perform thread partitioning processing on the data to be processed on the target dimension to obtain at least one block of data and a thread processing time constraint corresponding to each block of data;

[0050] The data prefetching unit is configured to prefetch any block of data from the first storage space to the second storage space according to the thread processing time constraint corresponding to the block of data.

[0051] The present application provides an electronic device, the device comprising: a processor and a memory;

[0052] The memory is used to store instructions or computer programs;

[0053] The processor is used to execute the instructions or computer programs in the memory, so that the electronic device executes the data prefetching method provided in this application.

[0054] The present application provides a computer-readable medium, wherein the computer-readable medium stores instructions or computer programs. When the instructions or computer programs are executed on a device, the device executes the data prefetching method provided by the present application.

[0055] The present application provides a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the data prefetching method provided by the present application.

[0056] Compared with the prior art, this application has at least the following advantages:

[0057] In the technical solution provided by the present application, when data to be processed (for example, input data of a model, etc.) is stored in a first storage space (for example, memory), and the data size of the data to be processed includes size representation values ​​under at least two dimensions (for example, batch, height, width, depth, etc.), first, based on the data size, the thread partitioning description parameters corresponding to each dimension are determined, so that the thread partitioning description parameters corresponding to the dimension can represent some parameter information involved in thread partitioning processing of the data to be processed on the dimension (for example, the number of thread tasks and the amount of pre-fetched data of the thread tasks); then, based on the thread partitioning description parameters corresponding to the at least two dimensions, a target dimension that meets the preset conditions is selected from the at least two dimensions, so that the target dimension can represent the dimension used by the optimal thread partitioning scheme; then, the data to be processed is processed on the target dimension Thread division processing is performed to obtain at least one block of data and a thread processing time constraint corresponding to each block of data (for example, constraint information such as the block of data will be processed by the second thread when executing the third task); finally, for any block of data, according to the thread processing time constraint corresponding to the block of data, the block of data is prefetched from the first storage space to the second storage space (for example, cache), so that the moment when the second storage space stores the block of data meets the thread processing time constraint corresponding to the block of data. This can effectively ensure that the thread corresponding to the block of data can smoothly read the block of data from the second storage space, thereby effectively avoiding the defect that when the data size of the data to be processed is too large, the hit rate of the second storage space decreases (for example, the cache hit rate decreases) due to the inability to prefetch the data to be processed to the second storage space in time, thereby helping to improve the prefetching effect.

[0058] In addition, since the above thread partitioning description parameters include not only the amount of pre-fetched data of the thread tasks, but also the number of thread tasks, the thread partitioning description parameters can indicate the status of multi-thread usage when thread partitioning is performed according to a certain dimension (for example, whether multi-threading can be fully utilized), so that the target dimension determined based on the thread partitioning description parameters can indicate the dimension used to comprehensively achieve the optimal thread partitioning scheme in terms of multi-thread usage and the second storage space hit rate, and thus the data pre-fetching process completed according to the target dimension can achieve the second storage space hit rate as much as possible while ensuring full utilization of multi-threads. This can effectively overcome the defects caused by the inability to fully utilize multi-thread resources, thereby helping to improve the pre-fetching effect (for example, pre-fetching efficiency + cache hit rate).

[0059] In addition, because the data prefetching method provided by the present application can comprehensively consider the number of thread tasks corresponding to each dimension and the amount of prefetched data of the thread tasks to make the final division dimension decision, so that the target dimension obtained by the decision can represent the dimension used to comprehensively achieve the optimal thread division scheme in terms of multi-thread usage and the second storage space hit rate, the data prefetching method provided by the present application is suitable for parallel prefetching of input data of data processing models with different structures / computational amounts, which is conducive to improving the universality of the data prefetching method. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments recorded in this application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0061] Figure 1 A flowchart of a data pre-fetching method is provided for an embodiment of the present application;

[0062] Figure 2 A schematic diagram of the structure of a data pre-fetching device provided in an embodiment of the present application;

[0063] Figure 3 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] Research has found that some prefetching schemes can often directly partition thread data based on a fixed dimension, height, so that subsequent data prefetching can be performed based on the partitioning. For example, when the data size (Bacth×Height×Width×Depth) of a certain data is 1×224×224×32, the corresponding prefetching method is to prefetch 224×32 blocks of data at a time, so that the corresponding thread can subsequently perform a processing task on this block of data.

[0065] The research also found that the above-mentioned prefetching scheme has defects as shown in ①-② below.

[0066] ① When the size representation value of a certain data (for example, 1×1×1024×32) in the height dimension is relatively small, if the thread data is still divided and processed in the height direction, the multi-threading feature cannot be utilized, resulting in the inability to process the data in parallel, thereby increasing the computing delay.

[0067] ② When the data size of a certain data (for example, 1×4096×4096×32) is relatively large, if the thread data division processing is still performed in the height direction, the data size of the divided block data (for example, 4096×32) will be relatively large, which will make it impossible to use a single prefetch instruction (for example, Prefetch instruction) to complete the prefetch processing of the block data, and thus cause the prefetch time of the block data to be relatively long, thus resulting in a decrease in the cache hit rate (that is, when a thread needs to read the block data from the cache, it finds that only a small part of the block data is stored in the cache, so that the thread cannot obtain the entire block data from the cache, and thus the thread cannot continue to execute the task of processing the block data).

[0068] Based on the above research, in order to better improve the prefetching effect, the present application provides a data prefetching method, which includes: when data to be processed (for example, input data of a model, etc.) is stored in a first storage space (for example, memory), and the data size of the data to be processed includes size representation values ​​under at least two dimensions (for example, batch, height, width, depth, etc.), first determine the thread partitioning description parameters corresponding to each dimension based on the data size, so that the thread partitioning description parameters corresponding to the dimension can represent some parameter information involved in thread partitioning processing of the data to be processed on the dimension (for example, the number of thread tasks and the amount of data prefetched by the thread tasks); then, based on the thread partitioning description parameters corresponding to the at least two dimensions, select a target dimension that meets the preset conditions from the at least two dimensions, so that the target dimension can represent the dimension used by the optimal thread partitioning scheme; then, The data to be processed is thread-divided on the target dimension to obtain at least one block of data and the thread processing time constraint corresponding to each block of data (for example, the block of data will be processed by the second thread when executing the third task, etc.); finally, for any block of data, according to the thread processing time constraint corresponding to the block of data, the block of data is prefetched from the first storage space to the second storage space (for example, cache), so that the moment when the second storage space stores the block of data meets the thread processing time constraint corresponding to the block of data. This can effectively ensure that the thread corresponding to the block of data can smoothly read the block of data from the second storage space, thereby effectively avoiding the defect that when the data size of the data to be processed is too large, the hit rate of the second storage space is reduced due to the inability to prefetch the data to be processed to the second storage space in time (for example, the defect shown in ② above), thereby helping to improve the prefetching effect.

[0069] In addition, since the thread partitioning description parameters above include not only the amount of prefetched data for thread tasks, but also the number of thread tasks, the thread partitioning description parameters can indicate the status of multi-thread usage when thread partitioning is performed according to a certain dimension (for example, whether multi-threading can be fully utilized), so that the target dimension determined based on the thread partitioning description parameters can indicate the dimension used to comprehensively achieve the optimal thread partitioning scheme in terms of multi-thread usage and the second storage space hit rate, and thus the data prefetching process completed according to the target dimension can achieve the second storage space hit rate as much as possible while ensuring full utilization of multi-threads. This can effectively overcome the defects caused by the inability to fully utilize multi-thread resources (for example, the defects shown in ① above), thereby helping to improve the prefetching effect (for example, prefetching efficiency + cache hit rate).

[0070] In addition, because the data prefetching method provided by the present application can comprehensively consider the number of thread tasks corresponding to each dimension and the amount of prefetched data of the thread tasks to make the final division dimension decision, so that the target dimension obtained by the decision can represent the dimension used to comprehensively achieve the optimal thread division scheme in terms of multi-thread usage and the second storage space hit rate, the data prefetching method provided by the present application is suitable for parallel prefetching of input data of data processing models with different structures / computational amounts, which is conducive to improving the universality of the data prefetching method.

[0071] In addition, the present application does not limit the execution subject of the above data pre-fetching method. For example, any implementation of the data pre-fetching method provided in the embodiment of the present application can be applied to a device with data processing function such as a terminal device or a server. For another example, any implementation of the data pre-fetching method provided in the embodiment of the present application can also be implemented with the help of a data communication process between different devices (for example, a terminal device and a server, two terminal devices, or two servers). Among them, the terminal device can be a smart phone, a computer, a personal digital assistant (PDA) or a tablet computer, etc. The server can be a stand-alone server, a cluster server or a cloud server.

[0072] In order to help those skilled in the art better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0073] In order to better understand the technical solution provided by this application, the data prefetching method provided by this application is described below with reference to some drawings. Figure 1 As shown, the data pre-fetching method provided by the embodiment of the present application includes the following S1-S4. Figure 1 A flowchart of a data pre-fetching method is provided for an embodiment of the present application.

[0074] S1: When data to be processed is stored in the first storage space, and the data size of the data to be processed includes size representation values ​​in at least two dimensions, the thread partitioning description parameters corresponding to each dimension are determined based on the data size, and the thread partitioning description parameters include the number of thread tasks and the amount of data pre-fetched by the thread tasks.

[0075] The first storage space is used to store some data, and these data can be pre-fetched to other storage spaces. In addition, this application does not limit the first storage space. For example, in some application scenarios, the first storage space can belong to a bottom-level memory type (e.g., memory, etc.).

[0076] Based on the above content, it can be seen that in one possible implementation, the first storage space mentioned above can be the memory in the target processor. The target processor refers to a processor that needs to use the data to be processed to perform a certain data processing task; and this application does not limit the implementation of the target processor. For example, it can be implemented using any existing or future processor (such as a DSP processor, TI processor, or MTK processor, etc.).

[0077] The data to be processed refers to the data required to perform a data processing task (e.g., an image processing task, a voice processing task, a video processing task, an audio processing task, etc.); and this application does not limit the implementation method of the data to be processed. For example, when the data processing task is completed using a pre-built data processing model, the data to be processed can be the input data of the data processing model. The data processing model is used to perform data processing on the input data of the data processing model.

[0078] In addition, this application is not limited to the above data processing model. For example, in some application scenarios, the data processing model can be a machine learning model. For another example, in some application scenarios, the data processing model can be an entity such as a device / module / unit / single-chip microcomputer with data processing capabilities.

[0079] In addition, this application does not limit the implementation methods of the above data processing model. For example, in some application scenarios (for example, image processing scenarios such as target detection and image style conversion), the data processing model can be an image processing model, and the input data of the data processing model can be image data, so that the data processing model can be used to perform certain image processing (for example, target detection, image style conversion, etc.) on the image data. For another example, in some application scenarios (for example, speech processing scenarios such as speech recognition and speech translation), the data processing model can be a speech processing model, and the input data of the data processing model can be speech data, so that the data processing model can be used to perform certain speech processing (for example, speech recognition, speech translation, etc.) on the speech data.

[0080] Based on the content of the previous paragraph, it can be seen that in one possible implementation method, when the data to be processed is the input data of a data processing model, if the data processing model is an image processing model, then the data to be processed can be image data, so that the data to be processed is used to represent the image data required to use when performing a certain image processing task using the data processing model.

[0081] In addition, the present application does not limit the data type of the above-mentioned data to be processed. For example, in some application scenarios, the data to be processed may be vector data.

[0082] Based on the above two paragraphs, it can be seen that in a possible implementation mode, the above data to be processed may be a data with B i ×H i ×W i ×D i The vector data of this data size refers to the input data of the above data processing model. i Indicates the size representation value of the data to be processed in the batch dimension; H i W represents the size representation value of the data to be processed in the dimension of height (Height); i Indicates the size representation value of the data to be processed in the dimension of width (Width); D i Represents the size representation value of the data to be processed in the depth dimension. i is used to indicate that the data to be processed is the input data of the data processing model.

[0083] The data size of the data to be processed is used to describe the amount of data in the data to be processed; and the data size of the data to be processed includes size representation values ​​in at least two dimensions. Among them, the size representation value in the nth dimension is used to represent the size information presented by the data to be processed in the nth dimension; and this application does not limit the implementation method of the size representation value in the nth dimension. For example, it can adopt the above B i 、H i 、W i , or D i n is a positive integer, n≤N, N is a positive integer, and N represents the number of dimensions in the "at least two dimensions".

[0084] Based on the above content, it can be seen that in a possible implementation mode, when the above data to be processed can be a B i ×H i ×W i ×D i For vector data of this data size, the data size of the data to be processed is B i ×H i ×W i×D i This size information; and the data size of the data to be processed may include the size representation value B under the batch dimension; i , the size representation value H in the height dimension i , the size representation value W under the width dimension i , and the size representation value D in the depth dimension i .

[0085] In addition, this application does not limit the implementation of the above "at least two dimensions". For example, in some application scenarios, the "at least two dimensions" may include N dimensions arranged in sequence (for example, the four dimensions of batch, height, width, and depth arranged in sequence). Where N is a positive integer, N represents the number of dimensions in the above "at least two dimensions".

[0086] It should be noted that the "arrangement" in the above paragraph refers to placing N dimensions in a certain order so that the position information corresponding to these N dimensions is in order. For example, when the data size of the data to be processed above is in the form of a string (for example, "B i ×H i ×W i ×D i "), the nth dimension is located earlier in the string than the n+1th dimension is located in the string. Where n is a positive integer, n+1≤N, and N is a positive integer. For example, when the data size of the data to be processed above is in vector form (for example, [B i , H i , W i , D i ]), the data index n corresponding to the nth dimension in the vector is such that the nth dimension refers to the data value at the nth position in the vector, and the data index n corresponding to the nth dimension in the vector is smaller than the data index n+1 corresponding to the n+1th dimension in the vector. Here, n is a positive integer, n+1≤N, and N is a positive integer.

[0087] The thread partition description parameters corresponding to the nth dimension are used to represent some parameter information involved when performing thread partitioning processing according to the nth dimension (for example, the number of thread tasks and the amount of pre-fetched data for the thread tasks). It can be seen that in one possible implementation, the thread partition description parameters corresponding to the nth dimension can at least include the number of thread tasks corresponding to the nth dimension and the amount of pre-fetched data for the thread tasks corresponding to the nth dimension. Where n is a positive integer, n≤N.

[0088] The number of thread tasks corresponding to the nth dimension indicates how many times each thread needs to execute tasks when thread partitioning is performed along the nth dimension. This number of thread tasks corresponding to the nth dimension can thus indicate whether the data prefetching scheme implemented by thread partitioning along the nth dimension can fully utilize the multithreading feature. Here, n is a positive integer, n≤N.

[0089] In addition, this application does not limit the process of determining the number of thread tasks corresponding to the nth dimension above. For example, it may specifically include the following steps 11 and 12.

[0090] Step 11: If the nth dimension is ranked first among the at least two dimensions above, the number of thread tasks corresponding to the nth dimension is determined based on the ratio between the size representation value under the nth dimension and the number of threads.

[0091] In this application, for the nth dimension above, if the nth dimension represents the dimension that is arranged most forward among the at least two dimensions above (for example, the "batch" dimension), then the ratio between the size representation value under the nth dimension and the number of threads can be directly determined as the number of thread tasks corresponding to the nth dimension. The number of threads refers to the number of threads that can be called by the target processor during multi-threaded processing.

[0092] Step 12: If the nth dimension is not the first one among the at least two dimensions above, the number of thread tasks corresponding to the nth dimension is determined based on the size representation value of at least one second dimension corresponding to the nth dimension, the size representation value of the nth dimension, and the number of threads; the arrangement position of the second dimension in the at least two dimensions is earlier than the arrangement position of the nth dimension in the at least two dimensions.

[0093] The second dimension is used to represent a dimension that exists in the at least two dimensions above and is arranged in a position that is earlier than the arrangement position corresponding to the nth dimension. It can be seen that the at least one second dimension corresponding to the nth dimension includes the 1st dimension to the n-1th dimension.

[0094] Based on the content of the previous paragraph, it can be seen that in one possible implementation, when the at least two dimensions above include four dimensions arranged in sequence, namely batch, height, width, and depth, if the nth dimension is height, then at least one second dimension corresponding to the nth dimension includes batch; if the nth dimension is width, then at least one second dimension corresponding to the nth dimension includes batch and height; if the nth dimension is depth, then at least one second dimension corresponding to the nth dimension includes batch, height, and width.

[0095] In addition, the present application does not limit the implementation method of the above step 12. For example, it can be specifically as follows: if the nth dimension is not the first one among the at least two dimensions above, first calculate the product between the size representation value of at least one second dimension corresponding to the nth dimension and the size representation value of the nth dimension; then determine the ratio between the product and the number of threads as the number of thread tasks corresponding to the nth dimension.

[0096] Based on the relevant content of step 12 above, it can be known that for the nth dimension above, if the nth dimension represents other dimensions among the at least two dimensions above except the dimension with the earliest arrangement position (for example, the dimension of "height", the dimension of "width", or the dimension of "depth"), then the product of the size representation value under at least one second dimension corresponding to the nth dimension and the size representation value under the nth dimension can be calculated first; and then the ratio between the product and the number of threads is determined as the number of thread tasks corresponding to the nth dimension.

[0097] Based on the relevant contents of steps 11 to 12 above, it can be known that in one possible implementation, when at least two dimensions above include four dimensions arranged in sequence, namely, batch, height, width, and depth, the number of thread tasks corresponding to the batch can be determined using the following formula (1), the number of thread tasks corresponding to the height can be determined using the following formula (2), the number of thread tasks corresponding to the width can be determined using the following formula (3), and the number of thread tasks corresponding to the depth can be determined using the following formula (4).

[0098] Num B =B i / L (1)

[0099] Num H =(B i ×H i ) / L (2)

[0100] Num W =(B i ×H i ×W i ) / L (3)

[0101] Num D =(B i ×H i ×W i ×D i ) / L (4)

[0102] Where Num B Indicates the number of thread tasks corresponding to the batch; B iIndicates the size representation value of the batch in this dimension; L indicates the number of threads above; Num H Indicates the number of thread tasks corresponding to the height; H i Indicates the size representation value of the height dimension; Num W Indicates the number of thread tasks corresponding to the width; W i Indicates the size representation value under the dimension of width; Num D Indicates the number of thread tasks corresponding to the depth; D i Indicates the size representation value under the depth dimension.

[0103] The amount of prefetched data for a thread task corresponding to the nth dimension represents the amount of prefetched data that each thread needs to process for a task when thread partitioning is performed according to the nth dimension. This amount of prefetched data for a thread task corresponding to the nth dimension indicates how many prefetch instructions can be used to prefetch the data that the thread needs to process for a task. Here, n is a positive integer, n≤N.

[0104] In addition, the present application does not limit the method for determining the amount of pre-fetched data for the thread task corresponding to the nth dimension above. For example, it may specifically include the following steps 21 and 22.

[0105] Step 21: If the nth dimension is located in a non-last position among the at least two dimensions above, then determine the amount of pre-fetched data for the thread task corresponding to the nth dimension based on the size representation value of at least one third dimension corresponding to the nth dimension, and the arrangement position of the third dimension in the at least two dimensions is later than the arrangement position of the nth dimension in the at least two dimensions.

[0106] The third dimension is used to represent the dimension that exists in the at least two dimensions above and is arranged later than the arrangement position corresponding to the nth dimension. It can be seen that the at least one third dimension corresponding to the nth dimension includes the (n+1)th dimension to the Nth dimension.

[0107] Based on the content of the previous paragraph, it can be seen that in one possible implementation, when the at least two dimensions above include four dimensions arranged in sequence, namely batch, height, width and depth, if the nth dimension is a batch, then the at least one third dimension corresponding to the nth dimension may include height, width and depth; if the nth dimension is height, then the at least one third dimension corresponding to the nth dimension may include width and depth; if the nth dimension is width, then the at least one third dimension corresponding to the nth dimension may include depth.

[0108] In addition, the present application does not limit the implementation method of the above step 21. For example, the specific implementation method of step 21 may be: if the nth dimension is located in a non-tail position among the at least two dimensions above, then the product of the size representation values ​​under at least one third dimension corresponding to the nth dimension is determined as the pre-fetched data amount of the thread task corresponding to the nth dimension. For another example, the specific implementation method of step 21 may be: first, using the size representation value under at least one third dimension corresponding to the nth dimension, a data size is constructed so that the data size includes the size representation values ​​under these third dimensions arranged in sequence; then, the data size is determined as the pre-fetched data amount of the thread task corresponding to the nth dimension, so that the pre-fetched data amount of the thread task can express the size of the pre-fetched data that each thread needs to process under one task when thread division processing is performed according to the nth dimension according to the size expression method.

[0109] Based on the relevant content of step 21 above, it can be known that for the nth dimension above, if the nth dimension represents other dimensions among the at least two dimensions above except the dimension with the last arrangement position (for example, the dimension of "batch", the dimension of "height", or the dimension of "width"), then the amount of pre-fetched data of the thread task corresponding to the nth dimension can be determined based on the size representation values ​​of all third dimensions corresponding to the nth dimension.

[0110] Step 22: If the nth dimension is located at the last position among the at least two dimensions above, 1 is determined as the pre-fetched data amount of the thread task corresponding to the nth dimension.

[0111] Based on the relevant contents of steps 21 to 22 above, it can be known that in a possible implementation mode, when at least two dimensions above include four dimensions arranged in sequence, namely, batch, height, width, and depth, the amount of pre-fetched data of the thread task corresponding to the batch can be determined using the following formula (5), the amount of pre-fetched data of the thread task corresponding to the height can be determined using the following formula (6), the amount of pre-fetched data of the thread task corresponding to the width can be determined using the following formula (7), and the amount of pre-fetched data of the thread task corresponding to the depth can be determined using the following formula (8).

[0112] Size B =H i ×W i ×D i (5)

[0113] Size H =W i ×D i (6)

[0114] Size W =D i (7)

[0115] Size D =1 (8)

[0116] Where Size B Indicates the amount of data pre-fetched by the thread task corresponding to the batch; Size H Indicates the amount of prefetched data for the highly corresponding thread task; Size W Indicates the amount of prefetched data for the thread task corresponding to the width; Size D Indicates the amount of pre-fetched data by the thread task corresponding to the depth; H i Indicates the size representation value of the height dimension; W i Indicates the size representation value under the dimension of width; D i Indicates the size representation value under the depth dimension.

[0117] It should be noted that, since the pre-fetched data amount of the thread task corresponding to the last dimension in at least two dimensions is a fixed value of 1, in order to save resources, it is not necessary to calculate the pre-fetched data amount of the thread task corresponding to the last dimension (for example, there is no need to calculate the Size above). D ).

[0118] Based on the above content regarding the number of thread tasks corresponding to the nth dimension and the amount of pre-fetched data of the thread tasks corresponding to the nth dimension, it can be seen that in one possible implementation, there is a correlation relationship between these parameters as shown in the following formula (9).

[0119] Num n ×Size n =Size all / L (9)

[0120] Where Num n Indicates the number of thread tasks corresponding to the nth dimension above; Size n Indicates the amount of prefetched data by the thread task corresponding to the nth dimension above; Size all Indicates the data size of the data to be processed; L indicates the number of threads above.

[0121] Based on the relevant content of S1 above, it can be known that for the target processor, if it is desired to process the data to be processed stored in the first storage space of the target processor (for example, with B i ×H i ×W i ×D iIf pre-fetching is performed on vector data of this data size), the thread partitioning description parameters corresponding to each dimension in the data size can be calculated based on the data size of the data to be processed, so that the thread partitioning description parameters corresponding to the dimension can represent some parameter information involved when thread partitioning is performed according to the dimension (for example, the number of tasks that each thread needs to execute and the data size of the pre-fetched data that each thread needs to process under one task, etc.), so that the optimal thread partitioning scheme can be decided based on these thread partitioning description parameters later.

[0122] S2: Selecting a target dimension that meets a preset condition from the at least two dimensions according to thread partition description parameters corresponding to the at least two dimensions.

[0123] The target dimension is used to represent the partitioning direction required for the optimal thread partitioning solution determined for the above-mentioned data to be processed.

[0124] The preset conditions refer to the conditions that need to be achieved in the above-mentioned target dimensions; and this application does not limit the preset conditions. For example, they may refer to the conditions set in advance by relevant personnel based on the actual application scenario.

[0125] In fact, in order to better improve the prefetching effect, the present application also provides a possible implementation method of the above preset conditions. Under this implementation method, the preset conditions may include: the number of thread tasks corresponding to the target dimension is not less than the preset number threshold, and the amount of prefetched data of the thread tasks corresponding to the target dimension is not greater than the data amount threshold, so that with the help of this preset condition, a target dimension can be selected from at least two dimensions above to maximize the hit rate of the second storage space while fully utilizing the multi-threading characteristics.

[0126] Based on the above content, it can be seen that in one possible implementation, after obtaining the thread partition description parameters corresponding to at least two dimensions above, it can be determined whether the number of thread tasks in the thread partition description parameters corresponding to the nth dimension is less than the preset number threshold, and whether the amount of pre-fetched data of the thread tasks in the thread partition description parameters corresponding to the nth dimension is greater than the data amount threshold. When it is determined that the number of thread tasks corresponding to the nth dimension is greater than or equal to the preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the nth dimension is not greater than the data amount threshold, it can be determined that the nth dimension can achieve the maximum improvement of the second storage space hit rate while fully utilizing the multi-threading characteristics, so the nth dimension can be directly determined as the target dimension. Wherein, n is a positive integer, n≤N, N is a positive integer, and N represents the number of dimensions in the "at least two dimensions".

[0127] It should be noted that this application does not limit the method for obtaining the above-mentioned preset number threshold. For example, the preset number threshold can be manually set by relevant personnel based on actual application scenarios. In addition, this application does not limit the method for obtaining the above-mentioned data volume threshold. For example, the data volume threshold can also be manually set by relevant personnel based on actual application scenarios.

[0128] In fact, in order to better save resources (for example, time resources + human resources), the above data volume threshold can be automatically determined. Based on this, the present application also provides a possible implementation method of the data volume threshold determination process. Under this implementation method, when the above target processor includes the above first storage space and the below second storage space, the data volume threshold determination process can be specifically as follows: according to the prefetch instruction constraint of the target processor, the data volume threshold is determined so that the data volume threshold is more in line with the performance of the target processor itself. Among them, the prefetch instruction constraint refers to the constraint on the prefetch processing process due to the performance of the target processor itself; and the present application does not limit the implementation method of the prefetch instruction constraint. For example, the prefetch instruction constraint can include a byte number constraint for a single prefetch instruction in the target processor (for example, ≤8192Byte) and an instruction number constraint for consecutive prefetch instructions within the performance allowable range of the target processor (for example, the performance is best when no more than 3 prefetch instructions are consecutive).

[0129] It should be noted that the present application does not limit the implementation method of the above step of "determining the data volume threshold based on the prefetch instruction constraint of the target processor". For example, when the prefetch instruction constraint includes the byte number constraint of a single prefetch instruction in the target processor and the instruction number constraint of consecutive prefetch instructions within the performance allowable range of the target processor, the product of the maximum value involved in the byte number constraint (for example, 8192) and the maximum value involved in the instruction number constraint (for example, 3) can be used as the data volume threshold (for example, 8192×3).

[0130] In addition, in order to better improve the prefetching effect (for example, prefetching efficiency), the present application also provides a possible implementation method of the above target dimension. Under this implementation method, the target dimension can be implemented using any one of the target dimensions shown in (1) to (3) below.

[0131] (1) The target dimension is located first among the at least two dimensions above, the number of thread tasks corresponding to the target dimension is not less than a preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the target dimension is not greater than a data amount threshold.

[0132] In this application, for the dimension that is ranked most forward among the two dimensions above (for example, the "batch" dimension), if the number of thread tasks corresponding to this dimension is greater than or equal to the preset number threshold (for example, 2), and the amount of pre-fetched data of the thread tasks corresponding to this dimension is less than or equal to the data amount threshold (for example, 8192×3), then it can be determined that this dimension can achieve the maximum improvement of the hit rate of the second storage space while fully utilizing the multi-threading characteristics. Therefore, this dimension can be directly determined as the target dimension above, so that the pre-fetching processing of the above data to be processed can be completed with the help of the thread division process on this dimension.

[0133] (2) When the arrangement position of a first dimension corresponding to a target dimension in the at least two dimensions is adjacent to the arrangement position of the target dimension in the at least two dimensions, and the arrangement position of the first dimension in the at least two dimensions is earlier than the arrangement position of the target dimension in the at least two dimensions, the number of thread tasks corresponding to the first dimension is not less than a preset number threshold, the amount of pre-fetched data of the thread tasks corresponding to the first dimension is greater than the data amount threshold, the number of thread tasks corresponding to the target dimension is not less than the preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the target dimension is not greater than the data amount threshold.

[0134] The first dimension refers to a dimension that exists in the at least two dimensions above, is adjacent to the target dimension above, and is arranged before the target dimension, so that the first dimension is used to represent the dimension before the target dimension. For example, when the at least two dimensions include batch, height, width, and depth, which are arranged in sequence, if the target dimension is height, then the first dimension corresponding to the target dimension is batch; if the target dimension is width, then the first dimension corresponding to the target dimension is height; if the target dimension is depth, then the first dimension corresponding to the target dimension is width.

[0135] It can be seen that for any dimension other than the dimension with the most forward arrangement position in the above two dimensions (for example, height, width or depth), if the number of thread tasks corresponding to the previous dimension of the dimension is greater than or equal to the preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the previous dimension is greater than the data amount threshold, it can be determined that although the previous dimension can fully utilize the multi-threading characteristics, there will be a defect of decreased hit rate of the second storage space when the thread division processing is performed according to the previous dimension. Therefore, when it is determined that the number of thread tasks corresponding to the next dimension of the previous dimension (that is, the aforementioned "this dimension") is greater than or equal to the preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to this dimension is not greater than the data amount threshold, it can be determined that this dimension can achieve the purpose of improving the hit rate of the second storage space as much as possible while fully utilizing the multi-threading characteristics. Therefore, this dimension can be directly determined as the above target dimension, so that the pre-fetching processing of the above data to be processed can be completed with the help of the thread division process on this dimension.

[0136] (3) When the arrangement position of a first dimension corresponding to a target dimension in the at least two dimensions is adjacent to the arrangement position of the target dimension in the at least two dimensions, and the arrangement position of the first dimension in the at least two dimensions is earlier than the arrangement position of the target dimension in the at least two dimensions, the number of thread tasks corresponding to the first dimension is less than a preset number threshold, the number of thread tasks corresponding to the target dimension is not less than the preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the target dimension is not greater than the data amount threshold.

[0137] In the present application, for any dimension other than the dimension with the most forward arrangement position in the above two dimensions (for example, height, width or depth), if the number of thread tasks corresponding to the previous dimension of the dimension is less than the preset number threshold, it can be determined that the previous dimension cannot fully utilize the multi-threading feature. Therefore, when it is determined that the number of thread tasks corresponding to the next dimension of the previous dimension (that is, the aforementioned "this dimension") is greater than or equal to the preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the dimension is not greater than the data amount threshold, it can be determined that the dimension can achieve the purpose of improving the hit rate of the second storage space as much as possible while fully utilizing the multi-threading feature. Therefore, the dimension can be directly determined as the above target dimension, so that the pre-fetching processing of the above data to be processed can be completed with the help of the thread division process on this dimension.

[0138] Based on the relevant content of the target dimensions shown in (1) to (3) above, it can be seen that under a possible implementation method, the preset conditions satisfied by the target dimension can specifically be: the target dimension is located first among the at least two dimensions above, the number of thread tasks corresponding to the target dimension is not less than the preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the target dimension is not greater than the data amount threshold.

[0139] In another possible implementation, when the arrangement position of the first dimension corresponding to the target dimension in the at least two dimensions above is adjacent to the arrangement position of the target dimension in the at least two dimensions, and the arrangement position of the first dimension in the at least two dimensions is forward of the arrangement position of the target dimension in the at least two dimensions, the preset conditions satisfied by the target dimension may specifically be: the number of thread tasks corresponding to the first dimension is not less than the preset number threshold, the amount of pre-fetched data of the thread tasks corresponding to the first dimension is greater than the data amount threshold, the number of thread tasks corresponding to the target dimension is not less than the preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the target dimension is not greater than the data amount threshold.

[0140] In another possible implementation, when the arrangement position of the first dimension corresponding to the target dimension in the at least two dimensions above is adjacent to the arrangement position of the target dimension in the at least two dimensions, and the arrangement position of the first dimension in the at least two dimensions is forward of the arrangement position of the target dimension in the at least two dimensions, the preset conditions satisfied by the target dimension may specifically be: the number of thread tasks corresponding to the first dimension is less than the preset number threshold, the number of thread tasks corresponding to the target dimension is not less than the preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the target dimension is not greater than the data amount threshold.

[0141] In fact, in order to better improve the efficiency of determining the target dimensions above, the present application also provides a possible implementation method of the target dimension determination process. Under this implementation method, when the at least two dimensions above include N dimensions arranged in sequence, the target dimension determination process may include the following steps 31-35.

[0142] Step 31: Initialize m to 1.

[0143] Here, m is used to represent the arrangement position of the dimension being traversed.

[0144] Step 32: Determine whether the number of thread tasks corresponding to the mth dimension is less than a preset number threshold.

[0145] In this application, after obtaining the number of thread tasks corresponding to the mth dimension, it can be determined whether the number of thread tasks corresponding to the mth dimension is less than a preset number threshold, so that the judgment result can indicate whether the mth dimension can fully utilize the multi-threading characteristics.

[0146] Step 33: If the number of thread tasks corresponding to the mth dimension is not less than a preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the mth dimension is not greater than a data amount threshold, then the mth dimension is determined as the target dimension.

[0147] In the present application, for the mth dimension, if the number of thread tasks corresponding to the mth dimension is not less than the preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the mth dimension is not greater than the data amount threshold, it can be determined that the mth dimension can achieve the goal of improving the hit rate of the second storage space as much as possible while fully utilizing the multi-threading characteristics. Therefore, the mth dimension can be directly determined as the target dimension above, and the traversal process for at least two dimensions above can be ended, so that the pre-fetching processing of the above data to be processed can be completed with the help of the thread division process on the mth dimension.

[0148] Step 34: If the number of thread tasks corresponding to the mth dimension is not less than the preset number threshold, and the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold, then update m and return to continue executing the above step 32 and its subsequent steps.

[0149] It should be noted that the present application does not limit the implementation method of the step of "updating m" in the above step 34. For example, it can be implemented using the following formula (10).

[0150] m'=m+1 (10)

[0151] Where m' represents the updated m; m represents the updated m.

[0152] Based on the relevant content of step 34 above, it can be known that for the mth dimension, if the number of thread tasks corresponding to the mth dimension is not less than the preset number threshold, but the amount of pre-fetched data of the thread tasks corresponding to the mth dimension is greater than the data amount threshold, it can be determined that although the mth dimension can fully utilize the multi-threading characteristics, there will be a defect of a decreased hit rate of the second storage space when performing thread division processing according to the mth dimension. Therefore, in order to avoid this defect, the above formula (10) can be used to update m, and the above step 32 and its subsequent steps can be continued based on the updated m to start the judgment process for the next dimension.

[0153] Step 35: If the number of thread tasks corresponding to the mth dimension is less than the preset number threshold, update m and return to continue executing the above step 32 and subsequent steps.

[0154] In the present application, for the mth dimension, if the number of thread tasks corresponding to the mth dimension is less than the preset number threshold, it can be determined that the mth dimension cannot fully utilize the multi-threading feature. Therefore, in order to avoid this defect, the above formula (10) can be used to update m, and the above step 32 and its subsequent steps can be continued based on the updated m to start the judgment process for the next dimension.

[0155] Based on the relevant content of steps 31 to 35 above, it can be seen that for the at least two dimensions above, it can be determined in turn whether each dimension of the at least two dimensions can be used as a target dimension. This is conducive to improving the search effect of the target dimension, thereby helping to improve the determination effect of the optimal thread partitioning scheme, and further helping to improve the prefetching effect.

[0156] In fact, in some application scenarios, the dimension with the last arranged position among the at least two dimensions mentioned above can be regarded as the dimension used by the fallback solution, so that when it is determined that all other dimensions except the dimension with the last arranged position among the at least two dimensions are not suitable as target dimensions, no matter what state the thread partitioning description parameter corresponding to the dimension is in, the dimension with the last arranged position can be directly regarded as the target dimension.

[0157] Based on the content of the above paragraph, the present application also provides a possible implementation method of the above target dimension determination process. Under this implementation method, when the above at least two dimensions include N dimensions arranged in sequence, the target dimension determination process may include the following steps 41-47.

[0158] Step 41: Initialize m to 1.

[0159] Step 42: Determine whether the number of thread tasks corresponding to the mth dimension is less than a preset number threshold.

[0160] Step 43: If the number of thread tasks corresponding to the mth dimension is not less than a preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the mth dimension is not greater than a data amount threshold, the mth dimension is determined as the target dimension.

[0161] It should be noted that for the relevant contents of steps 41 to 43, please refer to steps 31 to 33 above respectively.

[0162] Step 44: If the number of thread tasks corresponding to the mth dimension is not less than the preset number threshold, the amount of data pre-fetched by the thread task corresponding to the mth dimension is greater than the data amount threshold, and m is less than N-1, then update m and return to continue executing the above step 42 and its subsequent steps.

[0163] In the present application, for the mth dimension, if the number of thread tasks corresponding to the mth dimension is not less than the preset number threshold, but the amount of pre-fetched data of the thread tasks corresponding to the mth dimension is greater than the data amount threshold, it can be determined that although the mth dimension can fully utilize the multi-threading characteristics, there will be a defect of a decreased hit rate of the second storage space when performing thread division processing according to the mth dimension, so that when it is determined that m is less than N-1, it can be determined that the next dimension of the mth dimension is not the dimension with the last arrangement position in the at least two dimensions above, so that it can be determined that there are other dimensions that have not been traversed in addition to the dimension with the last arrangement position in the at least two dimensions. Therefore, in order to avoid the aforementioned defect, the above formula (10) can be used to update m, and the above step 42 and its subsequent steps can be continued based on the updated m to start the judgment process for the next dimension.

[0164] Step 45: If the number of thread tasks corresponding to the mth dimension is not less than the preset number threshold, the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold, and m is equal to N-1, then the Nth dimension is determined as the target dimension.

[0165] In the present application, for the mth dimension, if the number of thread tasks corresponding to the mth dimension is not less than the preset number threshold, but the amount of pre-fetched data of the thread tasks corresponding to the mth dimension is greater than the data amount threshold, it can be determined that although the mth dimension can fully utilize the multi-threading characteristics, there will be a defect of decreased hit rate of the second storage space when performing thread division processing according to the mth dimension, so that when it is determined that m is equal to N-1, it can be determined that the next dimension of the mth dimension is the dimension with the last arrangement position among the at least two dimensions above, so that it can be determined that only the dimension with the last arrangement position among the at least two dimensions has not been traversed. At this time, based on the above-mentioned fallback solution, no matter what state the thread division description parameter corresponding to the next dimension is in, the next dimension can be directly regarded as the dimension used by the optimal thread division scheme. Therefore, in order to save resources, there is no need to continue to execute the judgment process for the next dimension, and the next dimension (that is, the Nth dimension) can be directly determined as the target dimension.

[0166] Step 46: If the number of thread tasks corresponding to the mth dimension is less than the preset number threshold, and m is less than N-1, then update m and return to continue executing the above step 42 and subsequent steps.

[0167] In the present application, for the mth dimension, if the number of thread tasks corresponding to the mth dimension is less than a preset number threshold, it can be determined that the mth dimension cannot fully utilize the multi-threading feature, so that when it is determined that m is less than N-1, it can be determined that the next dimension of the mth dimension is not the dimension with the last arranged position in the at least two dimensions above, so that it can be determined that there are other dimensions that have not been traversed in the at least two dimensions in addition to the dimension with the last arranged position. Therefore, in order to avoid the aforementioned defects, the above formula (10) can be used to update m, and the above step 42 and its subsequent steps can be continued based on the updated m to start the judgment process for the next dimension.

[0168] Step 47: If the number of thread tasks corresponding to the mth dimension is less than the preset number threshold, and m is equal to N-1, the Nth dimension is determined as the target dimension.

[0169] In the present application, for the mth dimension, if the number of thread tasks corresponding to the mth dimension is less than the preset number threshold, it can be determined that the mth dimension cannot fully utilize the multi-threading feature, so that when it is determined that m is equal to N-1, it can be determined that the next dimension of the mth dimension is the dimension with the last arrangement position among the at least two dimensions above, so that it can be determined that only the dimension with the last arrangement position among the at least two dimensions has not been traversed. At this time, based on the above-mentioned fallback solution, no matter what state the thread partition description parameter corresponding to the next dimension is in, the next dimension can be directly regarded as the dimension used by the optimal thread partitioning solution. Therefore, in order to save resources, there is no need to continue to execute the judgment process for the next dimension, and the next dimension (that is, the Nth dimension) can be directly determined as the target dimension.

[0170] Based on the relevant contents of steps 41 to 47 above, it can be known that in some application scenarios, the dimension with the last arranged position among the at least two dimensions above can be regarded as the dimension used in the fallback solution. Therefore, when it is determined that the other dimensions except the dimension with the last arranged position among the two dimensions are not suitable as target dimensions, there is no need to analyze the thread partitioning description parameters corresponding to the dimension with the last arranged position. The dimension with the last arranged position can be directly determined as the target dimension. This can effectively reduce the time-consuming process of determining the target dimension, thereby helping to improve the prefetching efficiency.

[0171] In fact, in some application scenarios, for any pair of adjacent dimensions in the at least two dimensions mentioned above, the number of thread tasks corresponding to the dimension with a later arrangement position may be at least twice the number of thread tasks corresponding to the dimension with a earlier arrangement position (for example, the multiple relationship between the number of thread tasks corresponding to the four dimensions of batch, height, width and depth arranged in sequence as shown in formulas (1)-(4) above), so that in the at least two dimensions, the number of thread tasks corresponding to the dimension with a later arrangement position is not less than the number of thread tasks corresponding to the dimension with a earlier arrangement position. Therefore, when it is determined that the number of thread tasks corresponding to the dimension with a earlier arrangement position is not less than the preset number threshold, it can be directly concluded that the number of thread tasks corresponding to all dimensions arranged later than the "dimension with an earlier arrangement position" is not less than the preset number threshold. Therefore, in order to save resources, there is no need to perform the step of comparing the number of thread tasks corresponding to these dimensions with the preset number threshold.

[0172] Based on this, the present application also provides a possible implementation method of the above target dimension determination process. Under this implementation method, when the above at least two dimensions include N dimensions arranged in sequence, and the number of thread tasks corresponding to the n+1th dimension is not less than the number of thread tasks corresponding to the nth dimension, n is a positive integer, n+1≤N, the target dimension determination process may include the following steps 51-57.

[0173] Step 51: Initialize m to 1.

[0174] It should be noted that for the relevant content of step 51, please refer to step 31 above.

[0175] Step 52: Determine whether the number of thread tasks corresponding to the mth dimension is less than a preset number threshold.

[0176] It should be noted that for the relevant content of step 52, please refer to step 32 above.

[0177] Step 53: If the number of thread tasks corresponding to the mth dimension is less than the preset number threshold, update m and continue to execute the above step 52 and subsequent steps.

[0178] It should be noted that the relevant content of step 53 is similar to the relevant content of step 35 above, and for the sake of brevity, it will not be repeated here.

[0179] Step 54: If the number of thread tasks corresponding to the mth dimension is not less than the preset number threshold, determine whether the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold.

[0180] In the present application, for the mth dimension, if the number of thread tasks corresponding to the mth dimension is greater than or equal to the preset number threshold, it can be determined that the mth dimension can fully utilize the multi-threading characteristics, so it can be further determined whether the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold, so that the judgment result can indicate whether the defect of a decreased hit rate of the second storage space will occur when thread division processing is performed according to the mth dimension.

[0181] Step 55: If the amount of pre-fetched data of the thread task corresponding to the m-th dimension is not greater than the data amount threshold, the m-th dimension is determined as the target dimension.

[0182] In the present application, for the mth dimension, if the amount of pre-fetched data of the thread task corresponding to the mth dimension is less than or equal to the data amount threshold, it can be determined that the defect of a decrease in the hit rate of the second storage space will not occur when the thread division processing is performed according to the mth dimension, so the mth dimension can be directly determined as the target dimension.

[0183] Step 56: If the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold, update m and return to continue executing the step of "determining whether the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold" in the above step 54 and its subsequent steps.

[0184] In the present application, for the mth dimension, if the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold, it can be determined that when the thread division processing is performed according to the mth dimension, a defect of a decreased hit rate of the second storage space will occur. Therefore, in order to avoid this defect, the above formula (10) can be used to update m, and based on the updated m, the step of "determining whether the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold" and its subsequent steps in the above step 54 are continued to be executed to start the judgment process for the next dimension.

[0185] Based on the relevant content of steps 51 to 56 above, it can be known that in some application scenarios, for any pair of adjacent dimensions in at least two dimensions above, the number of thread tasks corresponding to the dimension with a later arrangement position is usually greater than or equal to the number of thread tasks corresponding to the dimension with a earlier arrangement position. Therefore, after determining that the number of thread tasks corresponding to the dimension with a earlier arrangement position is not less than the preset number threshold, it can be directly concluded that the number of thread tasks corresponding to the dimension with a later arrangement position is not less than the preset number threshold, and there is no need to perform the step of comparing the number of thread tasks corresponding to the dimension with a later arrangement position with the preset number threshold. This can effectively reduce the time-consuming process of determining the target dimension, thereby helping to improve the prefetching efficiency.

[0186] In fact, in some application scenarios, the fallback approach shown in steps 41 to 47 above can also be applied to the processing shown in steps 51 to 56 above. Based on this, the present application also provides a possible implementation method of the above target dimension determination process. In this implementation method, when the above at least two dimensions include N dimensions arranged in sequence, and the number of thread tasks corresponding to the n+1th dimension is not less than the number of thread tasks corresponding to the nth dimension, n is a positive integer, and n+1≤N, the target dimension determination process can include the following steps 61 to 68.

[0187] Step 61: Initialize m to 1.

[0188] It should be noted that for the relevant content of step 61, please refer to step 31 above.

[0189] Step 62: Determine whether the number of thread tasks corresponding to the mth dimension is less than a preset number threshold.

[0190] It should be noted that for the relevant content of step 61, please refer to step 32 above.

[0191] Step 63: If the number of thread tasks corresponding to the mth dimension is less than the preset number threshold, and m is less than N-1, then update m and continue to execute the above step 62 and subsequent steps.

[0192] It should be noted that for the relevant content of step 63, please refer to step 46 above.

[0193] Step 64: If the number of thread tasks corresponding to the mth dimension is less than the preset number threshold, and m is equal to N-1, the Nth dimension is determined as the target dimension.

[0194] It should be noted that for the relevant content of step 64, please refer to step 47 above.

[0195] Step 65: If the number of thread tasks corresponding to the mth dimension is not less than the preset number threshold, determine whether the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold.

[0196] It should be noted that for the relevant content of step 65, please refer to step 54 above.

[0197] Step 66: If the amount of pre-fetched data of the thread task corresponding to the mth dimension is not greater than the data amount threshold, the mth dimension is determined as the target dimension.

[0198] It should be noted that for the relevant content of step 66, please refer to step 55 above.

[0199] Step 67: If the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold, and m is less than N-1, then update m and return to continue executing the step of "determining whether the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold" in the above step 65 and its subsequent steps.

[0200] In the present application, for the mth dimension, if the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold, it can be determined that the defect of a decreased hit rate of the second storage space will occur when the thread division processing is performed according to the mth dimension, so that when it is determined that m is less than N-1, it can be determined that the next dimension of the mth dimension is not the dimension with the last arranged position in the at least two dimensions above, and thus it can be determined that there are other dimensions that have not been traversed in addition to the dimension with the last arranged position in the at least two dimensions. Therefore, in order to avoid this defect, m can be updated using the above formula (10), and the step of "determining whether the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold" in the above step 65 and its subsequent steps are continued based on the updated m to start the judgment process for the next dimension.

[0201] Step 68: If the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold, and m is equal to N-1, the Nth dimension is determined as the target dimension.

[0202] In the present application, for the mth dimension, if the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold, it can be determined that when the thread division processing is performed according to the mth dimension, the defect of a decreased hit rate of the second storage space will occur, so that when it is determined that m is equal to N-1, it can be determined that the next dimension of the mth dimension is the dimension with the last arrangement position among the at least two dimensions above, so that it can be determined that only the dimension with the last arrangement position among the at least two dimensions has not been traversed. At this time, based on the above-mentioned fallback solution, no matter what state the thread division description parameter corresponding to the next dimension is in, the next dimension can be directly regarded as the dimension used by the optimal thread division solution. Therefore, in order to save resources, there is no need to continue to execute the judgment process for the next dimension, and the next dimension (that is, the Nth dimension) can be directly determined as the target dimension.

[0203] Based on the relevant content of steps 61 to 68 above, it can be known that in some application scenarios, the dimension with the last arranged position among the at least two dimensions above can be regarded as the dimension used by the fallback solution. Therefore, when it is determined that the other dimensions except the dimension with the last arranged position among the two dimensions are not suitable as target dimensions, there is no need to analyze the thread partitioning description parameters corresponding to the dimension with the last arranged position. The dimension with the last arranged position can be directly determined as the target dimension. This can effectively reduce the time spent in the target dimension determination process, thereby helping to improve the prefetching efficiency.

[0204] To better understand the process of determining the target dimensions above, the following example is used to illustrate.

[0205] As an example, when the above at least two dimensions include four dimensions arranged in sequence: batch, height, width, and depth, the above target dimension determination process may specifically include the following steps 71 and 72.

[0206] Step 71: If Num B ≥2, the target dimension can be selected from the four dimensions of batch, height, width and depth, and the selection process can be specifically as follows: If Size B ≤8192×3, select Batch as the target dimension; if Size B >8192×3 and Size H ≤8192×3, select Height as the target dimension; if Size H >8192×3 and Size W ≤8192×3, select width as the target dimension; if Size W >8192×3, select depth as the target dimension.

[0207] Step 72: If Num B <2, the target dimension can be selected from the three dimensions of height, width and depth, and the selection process can be implemented using the selection process shown in steps 721 and 722 below.

[0208] Step 721: If Num H ≥2, the target dimension can be selected from the three dimensions of height, width and depth, and the selection process can be specifically as follows: If Size H ≤8192×3, select Height as the target dimension; if Size H >8192×3 and Size W≤8192×3, select width as the target dimension; if Size W >8192×3, select depth as the target dimension.

[0209] Step 722: If Num H <2, the target dimension can be selected from the width and depth dimensions, and the selection process can be specifically as follows: if Num W ≥2 and Size W ≤8192×3, select width as the target dimension; if Num W ≥2 and Size W >8192×3, select depth as the target dimension; if Num W <2, select depth as the target dimension.

[0210] Based on the relevant contents of steps 71 to 72 above, it can be known that in some application scenarios, when at least two of the above dimensions include batch, height, width and depth, which are arranged in sequence, the thread partitioning scheme under these four dimensions can be traversed to determine the number of thread tasks and the amount of pre-fetched data of thread tasks corresponding to different dimensions when thread partitioning is performed in different dimensions, so that the dimension used by the optimal thread partitioning scheme (that is, the target dimension above) can be decided based on the number of thread tasks and the amount of pre-fetched data of thread tasks corresponding to these dimensions. This enables the thread partitioning scheme on the target dimension to achieve the goal of improving the hit rate of the second storage space as much as possible while making full use of the multi-threading characteristics, which is conducive to improving the pre-fetching effect.

[0211] Based on the relevant content of S2 above, it can be known that for the data to be processed stored in the first storage space above, if the data size of the data to be processed includes size characterization values ​​under at least two dimensions, then after obtaining the thread division description parameters corresponding to these dimensions, the target dimension can be determined from these dimensions based on these thread division description parameters, so that the thread division scheme on the target dimension can achieve the second storage space hit rate as much as possible while making full use of the multi-threading characteristics, which is conducive to improving the prefetching effect. In addition, the target dimension is determined by traversing the thread division scheme under each dimension involved in the data to be processed, so that the target dimension is more suitable for the data distribution characteristics of the data to be processed, so that the data prefetching method including S2 is suitable for parallel prefetching of input data of data processing models with different structures / computational amounts, which is conducive to improving the universality of the data prefetching method.

[0212] S3: Divide the data to be processed into threads on the target dimension to obtain at least one block of data and a thread processing time constraint corresponding to each block of data.

[0213] Among them, block data refers to a part of the data to be processed above. For example, when the data to be processed is a i ×H i ×W i ×D i For vector data of this data size, if the target dimension above is batch, the data size of this block of data is H i ×W i ×D i ; If the target dimension is height, the data size of the block data is W i ×D i ; If the target dimension is width, the data size of the block data is D i ; If the target dimension is depth, the data size of the block data is 1.

[0214] The thread processing time constraint corresponding to the kth block of data is used to describe when the thread corresponding to the kth block of data processes the kth block of data; and this application does not limit the implementation method of the thread processing time constraint corresponding to the kth block of data. For example, when there is a corresponding relationship between the kth block of data and the jth thread, the thread processing time constraint corresponding to the kth block of data can be specifically: the jth thread processes the kth block of data under the yth task. Wherein, k is a positive integer, k≤K, K is a positive integer, and K represents the number of block data in the above "at least one block of data"; j is a positive integer, and the jth thread refers to the thread existing in the multi-thread of the target processor above and used to process the kth block of data; y is a positive integer, and the yth task refers to the task existing in all tasks that the jth thread needs to execute and used to implement data processing for the kth block of data.

[0215] It should be noted that this application does not limit the time when the correspondence between the kth block data and the jth thread is constructed. For example, it can be determined during the thread division processing of the above-mentioned data to be processed on the target dimension.

[0216] Based on the relevant content of S3 above, it can be known that for the data to be processed stored in the first storage space above, after determining that the dimension used in the optimal thread division scheme for the data to be processed is the target dimension above, the data to be processed can be directly thread-divided on the target dimension to obtain at least one block of data and the thread processing time constraint corresponding to each block of data, so that the thread processing time constraint can indicate which thread will process the corresponding block of data when executing the task for which time, so that the block of data can be pre-fetched based on the thread processing time constraint to ensure that the block of data can be smoothly pre-fetched from the first storage space to the second storage space before being processed, which is conducive to improving the hit rate of the second storage space.

[0217] S4: For any block of data, pre-fetch the block of data from the first storage space to the second storage space according to the thread processing time constraint corresponding to the block of data.

[0218] Among them, the second storage space is used to store some data pre-fetched from the first storage space above. In addition, the present application does not limit the second storage space. For example, in some application scenarios, the second storage space may belong to a high-level memory type. In addition, the present application does not limit the association relationship between the second storage space and the first storage space. For example, in some application scenarios, the second storage space and the first storage space may belong to the same processor. For example, in other application scenarios, the processor to which the second storage space belongs is different from the processor to which the first storage space belongs.

[0219] Based on the above content, it can be seen that in one possible implementation, when the first storage space mentioned above is the memory in the target processor, the second storage space mentioned above can be the cache in the target processor. It should be noted that the relevant content of the target processor can be found above.

[0220] Based on the content of the above paragraph, it can be seen that in a possible implementation, when the first storage space above is the memory in the target processor, the second storage space above is the cache in the target processor, and the thread processing time constraint corresponding to the k-th block data is used to indicate that the j-th thread processes the k-th block data under the y-th task, the above S4 can be specifically: according to the thread processing time constraint corresponding to the k-th block data, the k-th block data is pre-fetched from the memory to the cache to ensure that the time when the k-th block data is stored in the cache is no later than the time when the j-th thread uses the k-th block data to execute the y-th task, so as to effectively ensure that when the j-th thread executes the y-th task, the k-th block data can be successfully obtained from the cache, thereby effectively improving the hit rate of the second storage space. Wherein, k is a positive integer, k≤K, K is a positive integer, and K represents the number of block data in the above "at least one block data".

[0221] Based on the relevant contents of S1 to S4 above, it can be seen that for a certain processor, if the processor includes a first storage space and a second storage space, when data to be processed (for example, input data of a model, etc.) is stored in the first storage space (for example, memory), and the data size of the data to be processed includes size representation values ​​under at least two dimensions (for example, batch, height, width, depth, etc.), first, based on the data size, the thread partitioning description parameters corresponding to each dimension are determined, so that the thread partitioning description parameters corresponding to the dimension can represent some parameter information involved in thread partitioning processing of the data to be processed in the dimension (for example, the number of thread tasks and the amount of data pre-fetched by the thread tasks); then, based on the thread partitioning description parameters corresponding to the at least two dimensions, a target dimension that meets the preset conditions is selected from the at least two dimensions, so that the target dimension can represent the dimension used by the optimal thread partitioning scheme; Then, the data to be processed is thread-divided on the target dimension to obtain at least one block of data and the thread processing time constraint corresponding to each block of data (for example, the block of data will be processed by the second thread when executing the third task, and other constraint information); finally, for any block of data, according to the thread processing time constraint corresponding to the block of data, the block of data is prefetched from the first storage space to the second storage space (for example, cache), so that the moment when the second storage space stores the block of data meets the thread processing time constraint corresponding to the block of data. This can effectively ensure that the thread corresponding to the block of data can smoothly read the block of data from the second storage space, thereby effectively avoiding the defect that when the data size of the data to be processed is too large, the hit rate of the second storage space decreases (for example, the cache hit rate decreases) due to the inability to prefetch the data to be processed to the second storage space in time, thereby helping to improve the prefetching effect.

[0222] In addition, since the above thread partitioning description parameters include not only the amount of pre-fetched data of the thread tasks, but also the number of thread tasks, the thread partitioning description parameters can indicate the status of multi-thread usage when thread partitioning is performed according to a certain dimension (for example, whether multi-threading can be fully utilized), so that the target dimension determined based on the thread partitioning description parameters can indicate the dimension used to comprehensively achieve the optimal thread partitioning scheme in terms of multi-thread usage and the second storage space hit rate, and thus the data pre-fetching process completed according to the target dimension can achieve the second storage space hit rate as much as possible while ensuring full utilization of multi-threads. This can effectively overcome the defects caused by the inability to fully utilize multi-thread resources, thereby helping to improve the pre-fetching effect (for example, pre-fetching efficiency + cache hit rate).

[0223] In addition, because the data prefetching method provided by the present application can comprehensively consider the number of thread tasks corresponding to each dimension and the amount of prefetched data of the thread tasks to make the final division dimension decision, so that the target dimension obtained by the decision can represent the dimension used to comprehensively achieve the optimal thread division scheme in terms of multi-thread usage and the second storage space hit rate, the data prefetching method provided by the present application is suitable for parallel prefetching of input data of data processing models with different structures / computational amounts, which is conducive to improving the universality of the data prefetching method.

[0224] In addition, in some application scenarios, the data prefetching method provided by the present application can be applied to a terminal device, so that the terminal device can use the data prefetching method to prefetch data stored in the terminal device's memory (for example, input data of a certain machine learning model) into the cache of the terminal device. In particular, because the data prefetching method can determine the optimal thread partitioning scheme for data of any size, the terminal device executing the data prefetching method can maximize the cache hit rate while ensuring full utilization of multiple threads, which is conducive to improving the data prefetching effect in the terminal device (for example, prefetching efficiency + cache hit rate, etc.).

[0225] Based on the data pre-fetching method provided in the embodiment of the present application, the embodiment of the present application also provides a data pre-fetching device. Figure 2 Explain and illustrate. Figure 2 This is a schematic diagram of the structure of a data pre-fetching device provided in an embodiment of the present application. It should be noted that for the technical details of the data pre-fetching device provided in an embodiment of the present application, please refer to the relevant content of the data pre-fetching method above.

[0226] like Figure 2 As shown, the data pre-fetching device 200 provided in the embodiment of the present application includes:

[0227] A first determining unit 201 is configured to, when data to be processed is stored in the first storage space and the data size of the data to be processed includes size representation values ​​in at least two dimensions, determine, based on the data size, thread partition description parameters corresponding to each of the dimensions, the thread partition description parameters including the number of thread tasks and the amount of data pre-fetched by the thread tasks;

[0228] A dimension screening unit 202 is configured to select a target dimension that meets a preset condition from the at least two dimensions according to thread partition description parameters corresponding to the at least two dimensions;

[0229] A data partitioning unit 203 is configured to perform thread partitioning processing on the data to be processed in the target dimension to obtain at least one block of data and a thread processing time constraint corresponding to each block of data;

[0230] The data prefetching unit 204 is configured to prefetch any block of data from the first storage space to the second storage space according to the thread processing time constraint corresponding to the block of data.

[0231] In one possible implementation, the preset conditions include: the number of thread tasks corresponding to the target dimension is not less than a preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the target dimension is not greater than a data amount threshold.

[0232] In a possible implementation manner, the preset condition further includes: the target dimension is located first among the at least two dimensions.

[0233] In a possible implementation manner, an arrangement position of a first dimension corresponding to the target dimension in the at least two dimensions is adjacent to an arrangement position of the target dimension in the at least two dimensions, and an arrangement position of the first dimension in the at least two dimensions is forward of an arrangement position of the target dimension in the at least two dimensions;

[0234] The preset conditions also include: the number of thread tasks corresponding to the first dimension is not less than a preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the first dimension is greater than a data amount threshold;

[0235] or,

[0236] The preset condition also includes: the number of thread tasks corresponding to the first dimension is less than a preset number threshold.

[0237] In a possible implementation manner, the at least two dimensions include N dimensions arranged sequentially, and the dimension screening unit 202 is specifically configured to:

[0238] Initialize m to 1;

[0239] Determine whether the number of thread tasks corresponding to the mth dimension is less than a preset number threshold;

[0240] If the number of thread tasks corresponding to the m-th dimension is not less than a preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the m-th dimension is not greater than a data amount threshold, then the m-th dimension is determined as the target dimension;

[0241] If the number of thread tasks corresponding to the m-th dimension is not less than the preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the m-th dimension is greater than the data amount threshold, then update m and continue to perform the step of determining whether the number of thread tasks corresponding to the m-th dimension is less than the preset number threshold;

[0242] If the number of thread tasks corresponding to the mth dimension is less than the preset number threshold, then update m and continue to execute the step of determining whether the number of thread tasks corresponding to the mth dimension is less than the preset number threshold.

[0243] In one possible implementation, the at least two dimensions include N dimensions arranged sequentially, the number of thread tasks corresponding to the (n+1)th dimension is not less than the number of thread tasks corresponding to the (n)th dimension, where n is a positive integer and n+1≤N.

[0244] The dimension screening unit 202 is specifically configured to:

[0245] Initialize m to 1;

[0246] Determine whether the number of thread tasks corresponding to the mth dimension is less than a preset number threshold;

[0247] If the number of thread tasks corresponding to the m-th dimension is less than the preset number threshold, update m and continue to perform the step of determining whether the number of thread tasks corresponding to the m-th dimension is less than the preset number threshold;

[0248] If the number of thread tasks corresponding to the m-th dimension is not less than a preset number threshold, determining whether the amount of pre-fetched data of the thread task corresponding to the m-th dimension is greater than a data amount threshold;

[0249] If the amount of pre-fetched data of the thread task corresponding to the m-th dimension is not greater than the data amount threshold, determining the m-th dimension as the target dimension;

[0250] If the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold, then update m and continue to execute the step of determining whether the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold.

[0251] In a possible implementation, the dimension screening unit 202 is specifically configured to: if m is less than N-1, update m; if m is equal to N-1, determine the Nth dimension as the target dimension.

[0252] In a possible implementation, the target processor includes a first storage space and a second storage space; and the data volume threshold is determined according to a prefetch instruction constraint of the target processor.

[0253] In one possible implementation, for any dimension, if the dimension is located first among the at least two dimensions, the number of thread tasks corresponding to the dimension is determined based on the ratio between the size representation value and the number of threads in the dimension;

[0254] If the dimension is not the first one in the at least two dimensions, the number of thread tasks corresponding to the dimension is determined based on the size representation value of at least one second dimension corresponding to the dimension, the size representation value of the dimension and the number of threads, and the arrangement position of the second dimension in the at least two dimensions is ahead of the arrangement position of the dimension in the at least two dimensions.

[0255] In one possible implementation, for any of the dimensions, if the dimension is located in a non-tail position in the at least two dimensions, the amount of data pre-fetched by the thread task corresponding to the dimension is determined based on the size representation value of at least one third dimension corresponding to the dimension, and the arrangement position of the third dimension in the at least two dimensions is later than the arrangement position of the dimension in the at least two dimensions.

[0256] In a possible implementation manner, the data to be processed is vector data; and / or the data to be processed is input data of a data processing model.

[0257] In a possible implementation manner, the data processing model is an image processing model; and the data to be processed is image data.

[0258] In one possible implementation, the data prefetching method is applied to a terminal device; and / or the first storage space is a memory in a target processor, and the second storage space is a cache in the target processor.

[0259] Based on the relevant content of the above-mentioned data prefetching device 200, it can be known that for the data prefetching device 200 provided in the embodiment of the present application, when the data to be processed (for example, the input data of a certain model, etc.) is stored in the first storage space (for example, the memory), and the data size of the data to be processed includes size representation values ​​under at least two dimensions (for example, batch, height, width, depth, etc.), first, based on the data size, the thread partitioning description parameters corresponding to each dimension are determined, so that the thread partitioning description parameters corresponding to the dimension can represent some parameter information involved in thread partitioning processing of the data to be processed on the dimension (for example, the number of thread tasks and the amount of data prefetched by the thread tasks); then, based on the thread partitioning description parameters corresponding to the at least two dimensions, a target dimension that meets the preset conditions is selected from the at least two dimensions, so that the target dimension can represent the dimension used by the optimal thread partitioning scheme; then, The data to be processed is thread-divided on the target dimension to obtain at least one block of data and a thread processing time constraint corresponding to each block of data (for example, constraint information such as the block of data will be processed by the second thread when executing the third task); finally, for any block of data, according to the thread processing time constraint corresponding to the block of data, the block of data is prefetched from the first storage space to the second storage space (for example, cache), so that the moment when the second storage space stores the block of data meets the thread processing time constraint corresponding to the block of data. This can effectively ensure that the thread corresponding to the block of data can smoothly read the block of data from the second storage space, thereby effectively avoiding the defect that when the data size of the data to be processed is too large, the hit rate of the second storage space decreases (for example, the cache hit rate decreases) due to the inability to prefetch the data to be processed to the second storage space in time, thereby helping to improve the prefetching effect.

[0260] In addition, an embodiment of the present application also provides an electronic device, which includes a processor and a memory: the memory is used to store instructions or computer programs; the processor is used to execute the instructions or computer programs in the memory, so that the electronic device executes any implementation of the data prefetching method provided in the embodiment of the present application.

[0261] See also Figure 3 , which shows a schematic structural diagram of an electronic device 300 suitable for implementing the embodiments of the present disclosure. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 3The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0262] like Figure 3 As shown, the electronic device 300 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. Various programs and data required for the operation of the electronic device 300 are also stored in the RAM 303. The processing device 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0263] Typically, the following devices may be connected to the I / O interface 305: an input device 306 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 307 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 308 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 309. The communication device 309 may allow the electronic device 300 to communicate with other devices wirelessly or by wire to exchange data. Figure 3 The electronic device 300 is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0264] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 309, or installed from the storage device 308, or installed from the ROM 302. When the computer program is executed by the processing device 301, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0265] The electronic device provided by the embodiment of the present disclosure and the method provided by the above embodiment belong to the same inventive concept. For technical details not fully described in this embodiment, please refer to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0266] An embodiment of the present application further provides a computer-readable medium, in which instructions or computer programs are stored. When the instructions or computer programs are executed on a device, the device executes any implementation of the data prefetching method provided in the embodiment of the present application.

[0267] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0268] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0269] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0270] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device can perform the method.

[0271] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0272] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0273] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit / module does not, in some cases, limit the unit itself.

[0274] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0275] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0276] It should be noted that the various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems or devices disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0277] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0278] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0279] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be implemented directly using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0280] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data pre-fetching method, characterized in that: The method comprises: When the first storage space stores data to be processed, and the data size of the data to be processed includes size representation values ​​in at least two dimensions, determining thread partition description parameters corresponding to each dimension according to the data size, the thread partition description parameters including the number of thread tasks and the amount of pre-fetched data of the thread tasks; Selecting a target dimension that meets a preset condition from the at least two dimensions according to the thread partition description parameters corresponding to the at least two dimensions; Perform thread division processing on the data to be processed on the target dimension to obtain at least one block of data and a thread processing time constraint corresponding to each block of data; For any block of data, the block of data is pre-fetched from the first storage space to the second storage space according to the thread processing time constraint corresponding to the block of data.

2. The method according to claim 1, characterized in that The preset conditions include: the number of thread tasks corresponding to the target dimension is not less than a preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the target dimension is not greater than a data amount threshold.

3. The method according to claim 2, characterized in that The preset condition also includes: the target dimension is located first among the at least two dimensions.

4. The method according to claim 2, characterized in that: The arrangement position of the first dimension corresponding to the target dimension in the at least two dimensions is adjacent to the arrangement position of the target dimension in the at least two dimensions, and the arrangement position of the first dimension in the at least two dimensions is ahead of the arrangement position of the target dimension in the at least two dimensions; The preset condition also includes: the number of thread tasks corresponding to the first dimension is not less than a preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the first dimension is greater than a data amount threshold; or, The preset condition also includes: the number of thread tasks corresponding to the first dimension is less than a preset number threshold.

5. The method according to claim 1, characterized in that The at least two dimensions include N dimensions arranged in sequence, and the process of determining the target dimension includes: Initialize m to 1; Determine whether the number of thread tasks corresponding to the mth dimension is less than a preset number threshold; If the number of thread tasks corresponding to the m-th dimension is not less than a preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the m-th dimension is not greater than a data amount threshold, the m-th dimension is determined as the target dimension; If the number of thread tasks corresponding to the m-th dimension is not less than the preset number threshold, and the amount of pre-fetched data of the thread tasks corresponding to the m-th dimension is greater than the data amount threshold, then update m, and continue to perform the step of determining whether the number of thread tasks corresponding to the m-th dimension is less than the preset number threshold; If the number of thread tasks corresponding to the mth dimension is less than the preset number threshold, then m is updated, and the step of determining whether the number of thread tasks corresponding to the mth dimension is less than the preset number threshold is continued.

6. The method according to claim 1, characterized in that The at least two dimensions include N dimensions arranged in sequence, the number of thread tasks corresponding to the n+1th dimension is not less than the number of thread tasks corresponding to the nth dimension, n is a positive integer, n+1≤N; The process of determining the target dimension includes: Initialize m to 1; Determine whether the number of thread tasks corresponding to the mth dimension is less than a preset number threshold; If the number of thread tasks corresponding to the m-th dimension is less than the preset number threshold, update m, and continue to perform the step of determining whether the number of thread tasks corresponding to the m-th dimension is less than the preset number threshold; If the number of thread tasks corresponding to the m-th dimension is not less than the preset number threshold, determining whether the amount of pre-fetched data of the thread tasks corresponding to the m-th dimension is greater than the data amount threshold; If the amount of pre-fetched data of the thread task corresponding to the m-th dimension is not greater than the data amount threshold, determining the m-th dimension as the target dimension; If the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold, then m is updated, and the step of determining whether the amount of pre-fetched data of the thread task corresponding to the mth dimension is greater than the data amount threshold is continued.

7. The method according to claim 5 or 6, characterized in that: The updating of m comprises: If the m is less than N-1, update the m; The method further comprises: If m is equal to N-1, the Nth dimension is determined as the target dimension.

8. The method according to claim 2, 5 or 6, characterized in that: The target processor includes a first storage space and a second storage space; The data volume threshold is determined according to the prefetch instruction constraint of the target processor.

9. The method according to claim 1, characterized in that: For any of the dimensions, if the dimension is located at the first place in the at least two dimensions, the number of thread tasks corresponding to the dimension is determined according to the ratio between the size representation value under the dimension and the number of threads; If the dimension is not in the first position among the at least two dimensions, the number of thread tasks corresponding to the dimension is determined based on the size representation value of at least one second dimension corresponding to the dimension, the size representation value of the dimension and the number of threads, and the arrangement position of the second dimension in the at least two dimensions is ahead of the arrangement position of the dimension in the at least two dimensions.

10. The method according to claim 1, characterized in that For any of the dimensions, if the dimension is located in a non-tail position in the at least two dimensions, the amount of data pre-fetched by the thread task corresponding to the dimension is determined based on the size representation value of at least one third dimension corresponding to the dimension, and the arrangement position of the third dimension in the at least two dimensions is later than the arrangement position of the dimension in the at least two dimensions.

11. The method according to claim 1, characterized in that: The data to be processed is vector data; and / or, The data to be processed is input data of the data processing model.

12. The method according to claim 11, characterized in that The data processing model is an image processing model; the data to be processed is image data.

13. The method according to claim 1, characterized in that The data pre-fetching method is applied to a terminal device; and / or, The first storage space is a memory in a target processor, and the second storage space is a cache in the target processor.

14. A data pre-fetching device, characterized in that: include: A first determining unit is used to determine, when the first storage space stores data to be processed and the data size of the data to be processed includes size representation values ​​in at least two dimensions, thread partition description parameters corresponding to each dimension according to the data size, wherein the thread partition description parameters include the number of thread tasks and the amount of data pre-fetched by the thread tasks; A dimension screening unit, configured to select a target dimension that meets a preset condition from the at least two dimensions according to thread partition description parameters corresponding to the at least two dimensions; A data partitioning unit, configured to perform thread partitioning processing on the data to be processed on the target dimension to obtain at least one block of data and a thread processing time constraint corresponding to each block of data; The data pre-fetching unit is used to pre-fetch any block of data from the first storage space to the second storage space according to the thread processing time constraint corresponding to the block of data.

15. An electronic device, characterized in that: The device comprises: a processor and a memory; The memory is used to store instructions or computer programs; The processor is used to execute the instructions or computer programs in the memory so that the electronic device executes the method according to any one of claims 1 to 13.

16. A computer readable medium, characterized in that The computer-readable medium stores instructions or computer programs, and when the instructions or computer programs are executed on a device, the device executes the method according to any one of claims 1 to 13.

Images