Data batch processing method, device and storage medium

Abstract

The present application relates to face recognition in artificial intelligence, and specifically to a data batch processing method, device and storage medium. The method includes obtaining a target task and determining an instruction stream corresponding to the target task; determining a complete set of indexes corresponding to the target task; each index element in the complete set of indexes is used to point to each content element in the matrix to be processed; dividing the complete set of indexes into multiple data index sets, and assigning each data index set to a corresponding processing channel; wherein the number of elements in the data index set assigned to each processing channel is greater than the number of caches corresponding to the cache unit; based on the data index sets assigned to each processing channel, at least one round of target operations corresponding to the instruction stream is executed in parallel through each processing channel until an operation result corresponding to the target task is obtained. The use of this method can improve the processing efficiency of data batch processing.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a data batch processing method, device and storage medium. Background Art

[0002] Learning models can improve model accuracy through extensive computation, and in practical applications, machine learning models can be used to deploy a wide range of online services. Machine learning models typically include multiple access-intensive operators, such as unary operators, binary operators, and reduction operators.

[0003] In traditional solutions, access-intensive operators are generally calculated in a pipeline manner. For example, for unary operators, the data to be processed is often processed sequentially. Because different data to be processed have dependencies during the data processing process, for example, the data processing of the second data to be processed depends on the data processing results of the first data to be processed. Therefore, the subsequent data processing process must wait until the previous data processing process is completed before it can be executed, which can easily cause data processing jams. As a result, the overall time consumption for processing access-intensive operators is very long, and there is a problem of low efficiency in batch processing of large amounts of data. Summary of the Invention

[0004] Based on this, it is necessary to provide a data batch processing method, device, computer equipment and storage medium that can improve the efficiency of data batch processing in order to address the above technical problems.

[0005] A data batch processing method, the method comprising:

[0006] Obtaining a target task and determining an instruction stream corresponding to the target task;

[0007] Determine a complete set of indexes corresponding to the target task; each index element in the complete set of indexes is used to point to each content element in the matrix to be processed;

[0008] Dividing the entire index set into a plurality of data index sets, and assigning each of the data index sets to a corresponding processing channel; wherein the number of elements in the data index set assigned to each processing channel is greater than the number of caches corresponding to the cache unit;

[0009] Based on the data index sets respectively allocated to the processing channels, respectively, executing at least one round of target operations corresponding to the instruction stream through each processing channel in parallel until an operation result corresponding to the target task is obtained;

[0010] Among them, when each processing channel executes the target operation of the current round, it searches for the corresponding content element from the cache unit as the operation object according to the current index element in the corresponding data index set. When the corresponding content element is not found from the cache unit, the cache unit is triggered to obtain the cached number of content elements according to the current index element for overwriting and updating.

[0011] In one embodiment, the target operation of each round includes a set of operations executed multiple times in a cycle, and each set of operations includes a first target operation and a second target operation;

[0012] When executing the target operation of the current round, each processing channel searches the cache unit for the corresponding content element as the operation object according to the current index element in the corresponding data index set, including:

[0013] When each processing channel executes the first target operation in the current loop of the current round, searching the cache unit for a corresponding content element according to the corresponding current index element as an operation object of the first target operation;

[0014] When each processing channel executes the second target operation in the current loop of the current round, the intermediate operation data obtained by executing the second target operation in the previous loop is obtained, and the intermediate operation data and the content element found through the first target operation are used as operation objects.

[0015] In one embodiment, when executing the target operation of the current round, each processing channel searches the cache unit for the corresponding content element as the operation object according to the current index element in the corresponding data index set, including:

[0016] Determine the maximum number of concurrent connections supported by local resources;

[0017] When executing the target operation of the current round, each processing channel searches for the content element with the maximum concurrent number from the cache unit according to the current index element in the corresponding data index set, and uses each found content element as an operation object in parallel to execute the target operation in parallel.

[0018] In one embodiment, the method further comprises:

[0019] Acquire a face image to be detected, and input the face image to be detected into the face detection model;

[0020] Acquire a target task through the face detection model, trigger and execute an instruction stream corresponding to the target task, and obtain an operation result corresponding to the target task;

[0021] A face detection result is determined according to the operation result.

[0022] In one embodiment, executing at least one round of target operations corresponding to the instruction stream according to the address table and each index element in each index subset until obtaining an operation subresult corresponding to the target subtask includes:

[0023] For each data index set, determining a to-be-processed index subset in the corresponding data index set, and determining a current to-be-processed index subset from the to-be-processed index subset;

[0024] Execute at least one round of target operations corresponding to the instruction stream according to the address table corresponding to the corresponding data index set and each index element in the current pending index subset, and update the current pending index subset to a completed index subset after completing the target operations;

[0025] Enter the process of processing the next index subset to be processed, and return to the step of determining the index subset to be processed in the corresponding data index set to continue execution until all index subsets in each data index set are based on all at least one round of target operations corresponding to the instruction stream, and obtain the operation sub-result corresponding to the target subtask.

[0026] A data batch processing device, comprising:

[0027] An index set acquisition module is used to acquire a target task and determine an instruction stream corresponding to the target task; determine an index set corresponding to the target task; each index element in the index set is used to point to each content element in the matrix to be processed;

[0028] a data index set acquisition module, configured to divide the entire index set into a plurality of data index sets and assign each of the data index sets to a corresponding processing channel; wherein the number of elements in the data index set assigned to each processing channel is greater than the number of cache elements corresponding to the cache unit;

[0029] A target operation execution module is used to execute at least one round of target operations corresponding to the instruction stream in parallel through each processing channel based on the data index set respectively assigned to each processing channel, until an operation result corresponding to the target task is obtained; wherein, when each processing channel executes the target operation of the current round, it searches the cache unit for the corresponding content element as the operation object according to the current index element in the corresponding data index set, and when the corresponding content element is not found in the cache unit, it triggers the cache unit to obtain the cached number of content elements according to the current index element for overwriting and updating.

[0030] In one embodiment, the data index set acquisition module is also used to determine the number of channels of the processing channel and the number of caches corresponding to the cache unit; according to the number of channels and the number of caches, the full index set is divided to obtain multiple data index sets, and each data index set is respectively assigned to the corresponding processing channel.

[0031] In one embodiment, the target operation execution module is also used to determine, for each processing channel in a plurality of processing channels, at least one set of to-be-processed data indexes corresponding to the corresponding processing channel; for each processing channel, determine the current set of to-be-processed data indexes from the corresponding set of to-be-processed data indexes; based on the current set of to-be-processed data indexes corresponding to each processing channel, execute at least one round of target operations corresponding to the instruction stream in parallel through each processing channel, and update the current set of to-be-processed data indexes to a completed data index set after completing the target operation; enter the process of processing the next set of to-be-processed data indexes, and return to the step of determining, for each processing channel in a plurality of processing channels, at least one set of to-be-processed data indexes corresponding to the corresponding processing channel, until at least one round of target operations corresponding to the instruction stream is executed based on each data index set in the entire index set.

[0032] In one embodiment, an index full set acquisition module is used to determine an instruction stream corresponding to the target task; the instruction stream includes more than one operation instruction determined by a triggering order; the more than one operation instructions include a first instruction and a second instruction, and the first instruction is triggered before the second instruction; the target operation corresponding to the instruction stream includes a first target operation corresponding to the first instruction, and a second target operation corresponding to the second instruction.

[0033] In one embodiment, the target operation of each round includes a group of operations that are executed in multiple cycles, and each group of operations includes a first target operation and a second target operation; the target operation execution module also includes a loop execution module, which is used to search for the corresponding content element from the cache unit as the operation object of the first target operation according to the corresponding current index element when each processing channel executes the first target operation in the current cycle of the current round; when each processing channel executes the second target operation in the current cycle of the current round, obtain the intermediate operation data obtained by executing the second target operation in the previous cycle, and use the intermediate operation data and the content element found by the first target operation as the operation objects.

[0034] In one embodiment, the target operation execution module also includes a concurrent processing module for determining the maximum concurrent number supported by local resources; when each processing channel executes the target operation of the current round, it searches for the content element with the maximum concurrent number from the cache unit according to the current index element in the corresponding data index set, and uses each found content element as an operation object in parallel to execute the target operation in parallel.

[0035] In one embodiment, the data batch processing device is also used to trigger the local kernel to obtain continuous content elements corresponding to the cache quantity from the local memory according to the current index element when the corresponding content element is not found from the cache unit, and trigger an overwrite update of the storage content in the cache unit based on the obtained content elements; the overwritten content elements stored in the cache unit are used for the execution of subsequent target operations.

[0036] In one embodiment, the data batch processing device is further used to determine the number of page tables of the address table corresponding to each data index set; divide the data index set according to the number of page tables to obtain at least one index subset; each target page table in the address table corresponds to at least one matrix row in the index subset, and the target page table is used to store address information of index elements that are continuous in the corresponding matrix row; according to the address table corresponding to each processing channel and the index subset in the data index set allocated to each processing channel, at least one round of target operation corresponding to the instruction stream is executed in parallel through each processing channel until an operation sub-result corresponding to the target subtask is obtained; and the operation sub-results are combined to obtain an operation result corresponding to the target task.

[0037] In one embodiment, the data batch processing device is also used for each of the processing channels to search for corresponding target address information according to the target page table corresponding to the current matrix row where the current index element is located when executing the target operation of the current round, so as to obtain the corresponding content element as the operation object based on the target address information. When the corresponding content element is not found in the target page table, the target page table is triggered to overwrite and update the address information according to the current index element.

[0038] In one embodiment, a data batch processing device is deployed with an inference engine for execution, and the data batch processing device is applied to a face detection model; wherein the target task is one of the memory-intensive tasks generated by the face detection model when performing face detection.

[0039] In one embodiment, the data batch processing device is also used to obtain a facial image to be detected and input the facial image to be detected into the face detection model; obtain a target task through the face detection model, and trigger and execute the instruction stream corresponding to the target task to obtain an operation result corresponding to the target task; determine the face detection result based on the operation result.

[0040] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0041] Obtaining a target task and determining an instruction stream corresponding to the target task;

[0042] Determine a complete set of indexes corresponding to the target task; each index element in the complete set of indexes is used to point to each content element in the matrix to be processed;

[0043] Dividing the entire index set into a plurality of data index sets, and assigning each of the data index sets to a corresponding processing channel; wherein the number of elements in the data index set assigned to each processing channel is greater than the number of caches corresponding to the cache unit;

[0044] Based on the data index sets respectively allocated to the processing channels, respectively, executing at least one round of target operations corresponding to the instruction stream through each processing channel in parallel until an operation result corresponding to the target task is obtained;

[0045] Among them, when each processing channel executes the target operation of the current round, it searches for the corresponding content element from the cache unit as the operation object according to the current index element in the corresponding data index set. When the corresponding content element is not found from the cache unit, the cache unit is triggered to obtain the cached number of content elements according to the current index element for overwriting and updating.

[0046] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0047] Obtaining a target task and determining an instruction stream corresponding to the target task;

[0048] Determine a complete set of indexes corresponding to the target task; each index element in the complete set of indexes is used to point to each content element in the matrix to be processed;

[0049] Dividing the entire index set into a plurality of data index sets, and assigning each of the data index sets to a corresponding processing channel; wherein the number of elements in the data index set assigned to each processing channel is greater than the number of caches corresponding to the cache unit;

[0050] Based on the data index sets respectively allocated to the processing channels, respectively, executing at least one round of target operations corresponding to the instruction stream through each processing channel in parallel until an operation result corresponding to the target task is obtained;

[0051] Among them, when each processing channel executes the target operation of the current round, it searches for the corresponding content element from the cache unit as the operation object according to the current index element in the corresponding data index set. When the corresponding content element is not found from the cache unit, the cache unit is triggered to obtain the cached number of content elements according to the current index element for overwriting and updating.

[0052] The above-mentioned data batch processing method, apparatus, computer equipment and storage medium can determine the mass flow and index set corresponding to the target task by obtaining the target task. By determining the index set, the index set can be divided to obtain multiple data index sets, and each data index set is assigned to a corresponding processing channel. By assigning each data index set to a corresponding processing channel, at least one round of target operation can be performed in parallel based on the assigned data index set based on each processing channel until the operation result corresponding to the target task is obtained. Since the index set is divided based on the number of channels and the number of caches, the number of elements in the divided data index set can be greater than the number of caches corresponding to the cache unit. Therefore, when multiple processing channels read data from the cache unit in parallel based on their corresponding index elements, cache miss events may occur simultaneously. Multi-channel memory allows memory reads to occur simultaneously, so cache miss events that should have occurred at multiple different time stages can be concentrated at the same time, thereby hiding the memory access delay of some cache miss events, thereby improving the processing efficiency of data batch processing.

[0053] A data batch processing method, the method comprising:

[0054] Obtaining a target subtask and determining an instruction stream corresponding to the target subtask;

[0055] Determining a data index set corresponding to the target subtask and the number of page tables of the address table in the local resource, wherein the data index set is used to search for content elements in the matrix to be processed;

[0056] Dividing the data index set according to the number of page tables to obtain at least one index subset; each target page table in the address table corresponds to at least one matrix row in the index subset, and the target page table is used to store address information of index elements that are continuous with the corresponding matrix row;

[0057] Executing at least one round of target operations corresponding to the instruction stream according to the address table and each index element in each index subset until an operation subresult corresponding to the target subtask is obtained;

[0058] Among them, when executing the target operation of the current round, the corresponding target address information is searched according to the target page table corresponding to the current matrix row where the current index element is located, so as to obtain the corresponding content element as the operation object based on the target address information. When the corresponding content element is not found in the target page table, the target page table is triggered to overwrite and update the address information according to the current index element.

[0059] A data batch processing device, comprising:

[0060] A subtask determination module, configured to obtain a target subtask and determine an instruction stream corresponding to the target subtask;

[0061] a data partitioning module, configured to determine a data index set corresponding to the target subtask and the number of page tables in the address table in the local resource, wherein the data index set is used to search for content elements in the matrix to be processed; partition the data index set according to the number of page tables to obtain at least one index subset; each target page table in the address table corresponds to at least one matrix row in the index subset, and the target page table is used to store address information of index elements that are continuous with the corresponding matrix row;

[0062] A sub-result determination module is used to execute at least one round of target operations corresponding to the instruction stream according to the address table and each index element in each index subset, until an operation sub-result corresponding to the target subtask is obtained; wherein, when executing the target operation of the current round, the corresponding target address information is searched according to the target page table corresponding to the current matrix row where the current index element is located, so as to obtain the corresponding content element as the operation object based on the target address information; when the corresponding content element is not found in the target page table, the target page table is triggered to overwrite and update the address information according to the current index element.

[0063] In one embodiment, the sub-result determination module is also used to determine, for each data index set, a subset of to-be-processed indexes in the corresponding data index set, and determine a current subset of to-be-processed indexes from the subset of to-be-processed indexes; execute at least one round of target operations corresponding to the instruction stream according to the address table corresponding to the corresponding data index set and each index element in the current subset of to-be-processed indexes, and update the current subset of to-be-processed indexes to a completed index subset after completing the target operation; enter the process of processing the next subset of to-be-processed indexes, and return to the step of determining the subset of to-be-processed indexes in the corresponding data index set to continue executing, until at least one round of target operations corresponding to the instruction stream are executed based on all index subsets in each data index set, and an operation sub-result corresponding to the target subtask is obtained.

[0064] In one embodiment, the sub-result determination module is also used to determine the maximum number of concurrencies supported by local resources; group the index subsets according to the maximum number of concurrencies to obtain at least one concurrent subset; and execute a round of target operations corresponding to a round of instruction streams based on each of the concurrent subsets in turn.

[0065] In one embodiment, the target operation of each round includes a set of operations that are executed in multiple cycles, and the number of cycles corresponds to the number of page tables; the sub-result determination module is also used to, for each concurrent subset, in the process of executing the target operation of each round, execute the set of operations in a cycle of the number of page tables based on each matrix row in the concurrent subset.

[0066] In one embodiment, a data batch processing device is deployed with an inference engine for execution, and the data batch processing device is applied to a face detection model; wherein, the target subtask is a task in the target task for processing part of the data in the matrix to be processed; the target task is one of the memory-intensive tasks generated by the face detection model when performing face detection.

[0067] A computer device includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0068] Obtaining a target subtask and determining an instruction stream corresponding to the target subtask;

[0069] Determining a data index set corresponding to the target subtask and the number of page tables of the address table in the local resource, wherein the data index set is used to search for content elements in the matrix to be processed;

[0070] Dividing the data index set according to the number of page tables to obtain at least one index subset; each target page table in the address table corresponds to at least one matrix row in the index subset, and the target page table is used to store address information of index elements that are continuous with the corresponding matrix row;

[0071] Executing at least one round of target operations corresponding to the instruction stream according to the address table and each index element in each index subset until an operation subresult corresponding to the target subtask is obtained;

[0072] Among them, when executing the target operation of the current round, the corresponding target address information is searched according to the target page table corresponding to the current matrix row where the current index element is located, so as to obtain the corresponding content element as the operation object based on the target address information. When the corresponding content element is not found in the target page table, the target page table is triggered to overwrite and update the address information according to the current index element.

[0073] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0074] Obtaining a target subtask and determining an instruction stream corresponding to the target subtask;

[0075] Determining a data index set corresponding to the target subtask and the number of page tables of the address table in the local resource, wherein the data index set is used to search for content elements in the matrix to be processed;

[0076] Dividing the data index set according to the number of page tables to obtain at least one index subset; each target page table in the address table corresponds to at least one matrix row in the index subset, and the target page table is used to store address information of index elements that are continuous with the corresponding matrix row;

[0077] Executing at least one round of target operations corresponding to the instruction stream according to the address table and each index element in each index subset until an operation subresult corresponding to the target subtask is obtained;

[0078] Among them, when executing the target operation of the current round, the corresponding target address information is searched according to the target page table corresponding to the current matrix row where the current index element is located, so as to obtain the corresponding content element as the operation object based on the target address information. When the corresponding content element is not found in the target page table, the target page table is triggered to overwrite and update the address information according to the current index element.

[0079] The aforementioned data batch processing method, apparatus, computer device, and storage medium can determine the instruction stream and data index set corresponding to the target subtask by obtaining the target subtask. By determining the data index set, the data index set can be divided based on the number of page tables in the address table to obtain at least one index subset. By determining the index subset, at least one round of target operations corresponding to the instruction stream can be executed based on the address table and each index element in each index subset. Since the data index set is divided based on the number of page tables, each index subset obtained by the division can contain matrix rows with the same number of page table rows. Since each target page table in the address table corresponds to at least one matrix row in the index subset, the target page table stores the address information of the continuous index elements in the corresponding matrix row. Therefore, when the target operation is performed based on the address table and the current index subset, the address information of all index elements in the current index subset can be completely stored in the same address table. Therefore, it is only necessary to perform a comprehensive overwrite update on the original address table that does not contain address information. Subsequently, the address information of all index elements in the current index subset can be found based on this address table without the need to overwrite and update the page tables in the address table. In this way, the number of overwrite updates to the address table is reduced, the clock cycles consumed by the overwrite update are saved, and the processing efficiency of batch data processing is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Figure 1 This is an application environment diagram of a data batch processing method in one embodiment;

[0081] Figure 2A Schematic diagram of a normal pipeline process in one embodiment;

[0082] Figure 2B 1 is a flow chart of an abnormal pipeline in one embodiment;

[0083] Figure 3 1 is a flow chart of a method for batch processing of data in one embodiment;

[0084] Figure 4 A schematic diagram of a multi-channel parallel processing flow in one embodiment;

[0085] Figure 5 1 is a schematic diagram of a face detection process in one embodiment;

[0086] Figure 6 1 is a flow chart of a method for batch processing of data in another embodiment;

[0087] Figure 7 1 is a flow chart of a method for batch processing of data in a specific embodiment;

[0088] Figure 8is a flow chart of a data batch processing method in another specific embodiment;

[0089] Figure 9 This is an application scenario diagram of a data batch processing method in another embodiment;

[0090] Figure 10 is a structural block diagram of a data batch processing device in one embodiment;

[0091] Figure 11 is a structural block diagram of a data batch processing device in one embodiment;

[0092] Figure 12 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0093] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0094] See also Figure 1 , Figure 1 This is an optional architectural diagram of a data batch processing system provided in an embodiment of the present application; Figure 1 As shown, to support a data batch processing application, in a data batch processing system 100, a user terminal 102 communicates with a computer device 104 via a network. The user terminal 102 includes a first user terminal 102-1 and a second user terminal 102-2. A user can initiate a service request through the first user terminal 102-1. For example, when it is determined that an image to be detected has been received, the first user terminal 102-1 can display a detection control 102-12 on the graphical interface 102-11 to initiate a face detection request. The computer device 104 generates more than one computing task based on the service request, wherein the more than one computing task includes a target task. The computer device 104 can obtain the target task and determine the instruction stream corresponding to the target task, and simultaneously execute at least one round of target operations corresponding to the instruction stream based on multiple processing channels until the operation result corresponding to the target task is obtained. The computer device 104 can determine the service processing result corresponding to the service request based on the results of each operation, and feed back the service processing result to the second user terminal 102-2. For example, the computer device 104 can return the face detection result to the second user terminal 102-2, so that the second user terminal 102-2 can display the face detection result based on the graphical interface 102-21. The first user terminal 102-1 and the second user terminal 102-2 can be the same terminal or different terminals.

[0095] Among them, the user terminal 102 can be but is not limited to various personal computers, laptops, smart phones, tablets and portable wearable devices, and the computer device 104 can specifically be a terminal or a server. The server can be implemented as an independent server or a server cluster composed of multiple servers.

[0096] It should be noted that the computer device 104 can be deployed with machine learning models with different functions, and different types of online services can be deployed through machine learning models with different functions. For example, the computer device can provide face detection services, face registration services, or identity recognition services through a machine learning model with a face detection function. The computer device can also provide image classification services, target object recognition services, or monitoring and analysis services through a machine learning model with an image processing function (such as a semantic segmentation model or an image classification model). Different machine learning models will involve the operation of access-intensive operators when providing services, such as unary operators, binary operators, InstNorm operators (a variance operator), and reduction operators. When the target task is to execute access-intensive operators, then at this time, the data batch processing method mentioned in the embodiment of the present application can be used to perform batch operations on a large amount of data to be processed, thereby greatly improving processing efficiency. Of course, the target task can also be other computing tasks that need to be processed in batches, which can be related to specific application scenarios, and the embodiment of the present application does not limit this.

[0097] It should also be noted that the data batch processing methods mentioned in the various embodiments of this application are mainly aimed at the machine learning model implemented by artificial intelligence technology, which needs to perform more than one computing task when performing calculations, and this includes the target tasks mentioned in the embodiments of this application. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.

[0098] Artificial intelligence (AI) technology is a comprehensive discipline encompassing a wide range of fields, encompassing both hardware and software technologies. Foundational AI technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0099] It can be understood that the data batch processing method in each embodiment of the present application specifically relates to the machine learning technology of artificial intelligence, wherein machine learning (ML) is a multi-disciplinary interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their own performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning generally include technologies such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and formula-based learning.

[0100] It should be noted that the terms "first", "second" and similar terms used in this application do not indicate any order, quantity or importance, but are only used to distinguish different components. Unless the context clearly indicates otherwise, the singular form "a", "an" or "the" and similar terms do not indicate a quantity limitation, but rather indicate the presence of at least one. The quantities of "multiple" or "multiple copies" mentioned in the various embodiments of this application refer to the quantity of "at least two", for example, "multiple" refers to "at least two" and "multiple copies" refers to "at least two copies".

[0101] To better understand this solution, the following further describes the traditional solution. For access-intensive unary operators, the normal working flow of the CPU pipeline is as follows during sequential calculations: Figure 2A As shown in the figure, a set of operations consisting of one data loading operation (Load) and one summing operation (Add) is performed in one loop. When performing the first loop, the first data to be processed can be summed after loading it, and the intermediate operand of the first loop operation can be obtained. After obtaining the intermediate operand of the first loop operation, the second loop operation can be entered. And so on, until the intermediate operand of the last loop operation is obtained. Figure 2AAssume that each instruction stage consumes one clock cycle, regard a Load and an Add operation as a group of operations, and need to execute n groups of operations in a loop. The first group of operations takes 7 clock cycles, and the end time of each group of operations is 3 clock cycles later than the previous loop. Therefore, the total time of looping n groups of operations is 7+(n-1)*3 clock cycles. When the number of loops is high enough, the average time taken to execute each group of operations is 3 clock cycles.

[0102] When performing data loading operations, if the data to be loaded is not found in the cache unit, it is necessary to read the data to be loaded from the memory, which will cause blocking and reduce the efficiency of the pipeline. Figure 2B As shown in the figure, when the data to be loaded corresponding to Load1 does not exist in the cache unit, the clock cycle required to execute MEM (read data from the register) will increase from 1 clock cycle to 4 clock cycles, and the next operation has a dependency on this operation. For example, the execution of Add1 depends on the data loaded by executing Load1. Therefore, the execution of the next operation needs to wait for the completion of this operation, which ultimately leads to a decrease in pipeline efficiency. Figure 2A A schematic diagram of a normal pipeline process in one embodiment is shown. Figure 2B A schematic diagram of a flow chart of an abnormal pipeline in one embodiment is shown.

[0103] In one embodiment, Figure 3 As shown, a data batch processing method is provided, which is applied to Figure 1 The computer device in the example is used to illustrate the process, including the following steps:

[0104] Step S302: Acquire a target task and determine an instruction stream corresponding to the target task.

[0105] Among them, the target task is a computing task to be executed, which can specifically be the execution of access-intensive operators, such as reduction tasks, InstNorm tasks, etc. An instruction stream is a sequence of instructions that a computer program needs to execute. The instruction stream mentioned in the embodiment of the present application includes more than one target instruction determined by the triggering order. After the target instruction is triggered, the computer device will execute the target operation corresponding to the target instruction. Different types of target instructions will correspond to different target operations. For example, when the target task is an addition reduction task, the corresponding instruction stream includes data loading instructions and summation instructions, and the target operation corresponding to the target instruction may specifically include data loading operations and summation operations.

[0106] It can be understood that each target instruction in the instruction stream is an instruction related to the target task, and different types of target instructions can realize the execution of the target task after being triggered and executed. For example, when the target task is an addition and reduction task, the target instructions in the corresponding instruction stream can specifically be data loading instructions and summing instructions. When the target task is a maximum value task, the corresponding target instructions can specifically include shuffling instructions and maximum value instructions. When the target task is a variance calculation task, the corresponding target instructions can specifically include shuffling instructions, multiplication instructions, summing instructions, and instructions for division, etc. It can be understood that the target instructions included in the instruction stream are related to the specific target task, and the embodiments of the present application do not limit this.

[0107] Specifically, after determining a target task, the computer device can determine the instruction stream required to execute the target task. The instruction stream may include multiple cycles of a single target instruction, or multiple cycles of more than one target instruction with a fixed triggering order. The triggering order determination means that the order in which each of the more than one target instructions is issued is fixed. For example, when the instruction stream includes a first instruction and a second instruction, the first instruction must be issued before the second instruction, meaning that the second instruction can only be issued after the first instruction has been issued.

[0108] In one embodiment, dependencies exist between different types of target instructions. For example, when the target task is an addition reduction task, the corresponding instruction stream includes a data load instruction (Load instruction) and a sum instruction (Add instruction). The operand of the Add instruction includes the data obtained by executing the Load instruction, so these two instructions have a dependency relationship. In other words, the computer device can only issue the corresponding Add instruction based on the target operand after issuing the Load instruction and executing it to obtain the target operand.

[0109] Step S304: determining a complete set of indexes corresponding to the target task; each index element in the complete set of indexes is used to point to each content element in the matrix to be processed.

[0110] Among them, the index set refers to a set composed of at least one index element, and each index element in the index set is used to point to each content element in the matrix to be processed. The matrix to be processed refers to a set composed of content elements that have not performed the target task. For example, when the content elements that have not performed the target task are data "0", "1", "2" and "3", the corresponding matrix to be processed can be a one-dimensional array [0,1,2,3] containing one matrix row, so that when the identifier (array name) of this one-dimensional array is src, src[i] (i is 0,1,2 or 3) can be used as the index set, and src[0] in the index set points to the value 0 in the one-dimensional array, src[1] points to the value 1 in the one-dimensional array, and so on. In a specific embodiment, the matrix to be processed can be a one-dimensional array containing only one matrix row, or a two-dimensional array containing multiple rows and columns of data, or a three-dimensional array with a spatial structure, etc. This embodiment is not limited here.

[0111] Specifically, when determining the target task, the computer device can determine the matrix to be processed corresponding to the target task, and determine the identifier of the matrix to be processed and the number of rows and columns contained in the matrix to be processed, and determine the full set of indexes corresponding to the target task based on the number of rows, columns and identifier. Among them, the identifier is information that uniquely identifies a matrix to be processed. For example, when it is determined that the matrix to be processed is a two-dimensional array containing 10 rows and 8 columns, and the identifier of this two-dimensional array is src, the computer device can determine the corresponding full set of indexes as src[i] (0≤i≤80, i is an integer).

[0112] In one embodiment, developers can pre-enter the matrix to be processed using the full set of indices, so that the computer device can store the matrix to be processed in memory for subsequent processing. For example, developers can enter the matrix to be processed using the code: int src[4] = {0, 1, 2, 3}, where src is an identifier, {0, 1, 2, 3} are the content elements in the matrix to be processed, and src[i] (0≤i≤4, i is an integer) is the full set of indices.

[0113] In one embodiment, a developer may directly specify an identifier, a number of rows, and a number of columns, so that a computer device may determine a corresponding full set of indexes based on the specified identifier, number of rows, and number of columns.

[0114] Step S306 , dividing the entire index set into multiple data index sets, and assigning each data index set to a corresponding processing channel; wherein the number of elements in the data index set assigned to each processing channel is greater than the cache quantity corresponding to the cache unit.

[0115] Among them, the processing channel refers to the memory channel. A computer device may be equipped with multiple memory controllers, each of which can work independently and in parallel, and respectively control a processing channel to address, read data, and process data, thereby increasing the memory bandwidth by times the number of channels, and the data processing speed also increases by times the number of channels accordingly. A cache unit, also known as a cache block, is a unit in a cache. A cache may include at least one cache unit, and each cache unit can store a cache number of data. The number of caches can be determined based on the storage space size of the cache unit and the size of the space occupied by the data to be stored. For example, when the size of a cache block is 16B and one data occupies 4B, one cache unit can store 4 data.

[0116] Specifically, the computer device may divide the entire index set according to the number of caches corresponding to the cache units to obtain multiple data index sets, and assign each data index set to a corresponding processing channel. It is easy to understand that until the operation result corresponding to the target task is obtained, when the number of divided data index sets is less than or equal to the number of processing channels, each processing channel may be assigned one or zero data index sets; when the number of divided data index sets is greater than the number of processing channels, each processing channel may be assigned one or more data index sets.

[0117] In one embodiment, the computer device may divide the data index set according to the number of processing channels to obtain data index sets of the same number of processing channels. The number of elements in each data index set is greater than the cache capacity corresponding to the cache unit. For example, when the matrix to be processed contains 80 content elements and has two processing channels, the computer device may use the index elements corresponding to the first 40 content elements as data index set 1, and the index elements corresponding to the last 40 content elements as data index set 2, and assign data index set 1 to processing channel 1, and assign data index set 2 to processing channel 2. That is, processing channel 1 is designated to extract the first 40 content elements in the matrix to be processed from the cache unit and process the extracted first 40 content elements; processing channel 2 is designated to extract the last 40 content elements in the matrix to be processed from the cache unit and process the extracted last 40 content elements.

[0118] In one embodiment, the entire set of indexes is divided into multiple data index sets, and each data index set is assigned to a corresponding processing channel, including: determining the number of channels of the processing channel and the number of caches corresponding to the cache unit; dividing the entire set of indexes according to the number of channels and the number of caches to obtain multiple data index sets, and assigning each data index set to a corresponding processing channel.

[0119] Specifically, the computer device determines the number of processing channels of the machine and the number of caches corresponding to the cache unit, and divides the full set of indexes according to the number of channels and the number of caches to obtain multiple data index sets. The number of elements contained in each data index set is greater than the number of caches corresponding to the cache unit. For example, when the matrix to be processed contains 80 content elements and the number of caches corresponding to the cache unit is 4, the computer device can use the index elements corresponding to every 10 content elements as a data index set, for example, data index set 1 is src[i] (0≤i≤9, i is an integer), data index set 2 is src[i] (10≤i≤19, i is an integer)...data index set 8 is src[i] (70≤i≤79, i is an integer).

[0120] Furthermore, the computer device allocates each data index set to a corresponding processing channel, so that the processing channel can search for corresponding content elements from the cache unit according to the index elements in the data index set.

[0121] In one embodiment, the computer device may sequentially assign the data index sets to the processing channels, or may randomly assign the data index sets to the processing channels. For example, when there are four data index sets, the computer device may assign data index sets 1 and 3 to processing channel 1, and data index sets 2 and 4 to processing channel 2. Alternatively, the computer device may randomly assign data index sets 3 and 4 to processing channel 1, and data index sets 1 and 2 to processing channel 2. This embodiment is not limited here.

[0122] In the above embodiment, since the entire index set is divided based on the number of channels and the number of caches, the number of elements in the divided data index set can be greater than the number of caches corresponding to the cache unit. Therefore, when multiple processing channels subsequently read data from the cache unit in parallel based on their respective corresponding index elements, cache miss events (cache miss) may occur at the same time, thereby improving the overall data batch processing efficiency.

[0123] Step S308: Based on the data index sets assigned to each processing channel, at least one round of target operations corresponding to the instruction stream is executed in parallel through each processing channel until an operation result corresponding to the target task is obtained; wherein, when each processing channel executes the target operation of the current round, it searches the cache unit for the corresponding content element as the operation object according to the current index element in the corresponding data index set. When the corresponding content element is not found in the cache unit, the cache unit is triggered to obtain the cached number of content elements according to the current index element for overwriting and updating.

[0124] Specifically, each processing channel executes at least one round of target operations corresponding to the instruction stream in parallel based on the data index set assigned to it, that is, each processing channel determines the corresponding current data index set to be processed, and executes at least one round of target operations in parallel based on the corresponding current data index set to be processed. For example, channel 1 processes data index set 1, and at the same time, channel 2 processes data index set 2. Among them, for each processing channel in the multiple processing channels, the index elements contained in the current data index set are traversed, and the corresponding target operations are executed based on the instructions in the instruction stream. Among them, the instruction stream includes at least one round of instruction stream, and one round of target operations can be executed based on one round of instruction stream. One round of target operations may include a set of operations with multiple cycles.

[0125] For example, when a group of operations includes a data loading operation and a summation operation, for each of the multiple processing channels, the index elements contained in the current data index set are traversed, and based on the index element currently traversed, the content element pointed to by the index element currently traversed is processed through the target operation. More specifically, when executing a group of operations in this loop, the processing channel determines the current index element in the current data index set, and performs a data loading operation, extracting the corresponding target content element from the cache based on the current index element; the processing channel performs a summation operation, adding the extracted target content element to the intermediate operand obtained by performing the summation operation in the previous loop, and obtaining the intermediate operand obtained by performing the summation operation in this loop. This cycle is repeated until each index element contained in the current data index set is traversed.

[0126] It's worth noting that when each processing channel performs the target operation of the current round, it searches the cache for the corresponding content element as the operation object according to the current index element in the corresponding data index set. If the corresponding content element is not found in the cache, the cache unit is triggered to obtain the cached content element based on the current index element for overwriting and updating. In other words, each processing channel may not find the corresponding content element in the cache at the same time, thereby simultaneously triggering the cache unit to obtain the cached content element based on the current index element for overwriting and updating.

[0127] In one embodiment, step S306 and step S308 can be performed alternately until an operation result corresponding to the target task is obtained. Specifically, for each processing channel in the multiple processing channels, an unprocessed data index set can be pre-selected and assigned, so that each processing channel can process the assigned data index set, and after the processing is completed, the assigned data index set is updated to a completed data index set. When performing the next round of data index set allocation process, for each processing channel in the multiple processing channels, the computer device can select a data index set from all the unprocessed data index sets and assign it to the corresponding processing channel, until at least one round of target operations corresponding to the instruction stream is executed based on each data index set in the entire index set.

[0128] For example, when the matrix to be processed contains 80 content elements, the computer device can use the index elements corresponding to every 10 content elements as a data index set. For example, data index set 1 is src[i] (0≤i≤9, i is an integer), data index set 2 is src[i] (10≤i≤19, i is an integer)...data index set 8 is src[i] (70≤i≤79, i is an integer). Furthermore, the computer device can assign data index set 1 to processing channel 1 and data index set 2 to processing channel 2, so that processing channel 1 can extract content elements in the matrix to be processed from the cache unit based on the index elements in data index set 1 and perform corresponding processing on the extracted content elements. Similarly, processing channel 2 can extract content elements in the matrix to be processed from the cache unit based on the index elements in data index set 2 and perform corresponding processing on the extracted content elements. Furthermore, when entering the next round of data index set allocation process, the computer device may allocate data index set 3 to processing channel 1, data index set 4 to processing channel 2, and so on, until each data index set in the entire index set is based on and executes at least one round of target operations corresponding to the instruction stream.

[0129] In one embodiment, reference Figure 4 Assuming there are four processing channels, when loading four content elements that are far apart in the matrix to be processed at the same time, that is, when the four processing channels process the corresponding data index sets in parallel, since the number of elements in the data index set allocated to each processing channel is greater than the number of cache elements corresponding to the cache unit, and the cache unit can store continuous and cached content elements, the four content elements loaded at the same time will not be in the same cache unit, resulting in four cache miss events at the same time. Among them, a cache miss event refers to when the corresponding content element is not found in the cache unit, triggering the reading of the corresponding content element from the memory, and triggering the cache unit to obtain the cached content elements according to the current index element for overwriting and updating.

[0130] For example, when there are four cache units and no content elements are stored in the cache units in the initial state, when processing channel 1 executes the MEM instruction of Load1 in the fourth clock cycle, the computer device determines that the first target content element that needs to be loaded and pointed to by the current index element in data index set 1 does not exist in any cache unit, the computer device can read the first target content element from the memory based on the current index element and store the cached number of content elements connected to the first target content element in cache unit 1. When processing channel 2 executes the MEM instruction of Load2 in the fifth clock cycle, because the number of index elements between data index set 1 and data index set 2 is greater than the number of caches in the cache unit, it can be determined that the second target content element that needs to be loaded and pointed to by the current index element in data index set 2 does not exist in any cache unit. At this time, the computer device reads the second target content element from the memory and stores the cached number of content elements connected to the second target content element in cache unit 2. This iterative process causes cache miss events to occur in parallel in all four processing channels.

[0131] Multi-channel memory allows memory reads to occur simultaneously, so cache misses that should occur at multiple different time stages can be concentrated at the same time, thereby hiding the memory access latency of some cache misses. When it is necessary to execute n Load and Add operations in a loop and there are 4 processing channels, the 4 Load operations and 4 Add operations can be regarded as a group of operations. The first group of operations takes 20 clock cycles, and each subsequent group of operations is only 16 clock cycles apart from the previous group. Therefore, the total time is approximately: 20+(n / 8-1)*16. When the number of loops is high enough, the average computational time for each group of operations is approximately 16 / 8=2 clock cycles, which is less than Figure 2A In this way, the overall efficiency of data batch processing is improved. Figure 4 A schematic diagram of a multi-channel parallel processing flow in one embodiment is shown.

[0132] In one embodiment, when the corresponding content element is not found from the cache unit, the cache unit is triggered to obtain the cached number of content elements according to the current index element for overwriting and updating, including: when the corresponding content element is not found from the cache unit, the local kernel is triggered to obtain continuous content elements corresponding to the cached number from the local memory according to the current index element, and triggering the overwriting and updating of the stored content in the cache unit based on the obtained content elements; the overwritten stored content elements in the cache unit are used for the execution of subsequent target operations.

[0133] Specifically, when the computer device fails to find the corresponding content element in each cache unit, the computer kernel is triggered to search the local memory for the content element pointed to by the current index element based on the current index element. Starting from the content element pointed to by the current index element, the computer kernel retrieves a continuous number of content elements corresponding to the cached quantity. The retrieved continuous number of content elements corresponding to the cached quantity is then overwritten and updated in one of the cache units. The overwritten content element stored in the cache unit is used for subsequent execution of the target operation.

[0134] In this embodiment, by caching the content elements in the cache unit, the content elements can be directly read from the cache unit later, thereby improving the processing efficiency of data batch processing.

[0135] In the above-mentioned data batch processing method, by obtaining the target task, the mass flow and the full set of indexes corresponding to the target task can be determined. By determining the full set of indexes, the full set of indexes can be divided to obtain multiple data index sets, and each data index set can be assigned to a corresponding processing channel. By assigning each data index set to a corresponding processing channel, at least one round of target operation can be performed in parallel based on the assigned data index set based on each processing channel until the operation result corresponding to the target task is obtained. Since the full set of indexes is divided based on the number of channels and the number of caches, the number of elements in the divided data index sets can be greater than the number of caches corresponding to the cache units. Therefore, when multiple processing channels read data from the cache units in parallel based on their corresponding index elements, cache miss events may occur simultaneously. Multi-channel memory allows memory reads to occur simultaneously, so cache miss events that should have occurred at multiple different time stages can be concentrated at the same time, thereby hiding the memory access delay of some cache miss events, thereby improving the processing efficiency of data batch processing.

[0136] In one embodiment, based on the data index sets respectively assigned to each processing channel, at least one round of target operations corresponding to the instruction stream is executed in parallel through each processing channel, including: for each processing channel in a plurality of processing channels, determining at least one to-be-processed data index set assigned to the corresponding processing channel; for each processing channel, determining the current to-be-processed data index set from the corresponding to-be-processed data index set; based on the current to-be-processed data index set corresponding to each processing channel, executing at least one round of target operations corresponding to the instruction stream through each processing channel in parallel, and updating the current to-be-processed data index set to the completed data index set after completing the target operation; entering the process of processing the next to-be-processed data index set, and returning to the step of determining at least one to-be-processed data index set assigned to the corresponding processing channel for each processing channel in a plurality of processing channels, until at least one round of target operations corresponding to the instruction stream is executed based on each data index set in the entire index set.

[0137] Specifically, for each of the multiple processing channels, at least one data index set to be processed is determined to be assigned to it, and a current index set to be processed is determined from the at least one data index set to be processed to be assigned to it, so that each processing channel can execute at least one round of target operations corresponding to the instruction stream in parallel based on its corresponding current data index set to be processed. For each of the multiple processing channels, when the processing channel completes the target operation, the current data index set to be processed can be updated to a completed data index set, and the next data index set to be processed can be determined from the data index set to be processed to be assigned to it, and the next data index set to be processed can be processed, until at least one round of target operations corresponding to the instruction stream is executed based on each data index set in the entire index set.

[0138] It is easy to understand that a single processing channel can process each data index set in the order in which the data index sets are assigned, or it can randomly process the data index sets to be processed that are matched to it. For example, when processing channel 1 is specified to process data index set 1, data index set 2, and data index set 3, processing channel 1 can process data index set 1, data index set 2, and data index set 3 in sequence, or it can randomly process data index set 3 first, and then process data index set 1 and data index set 2. This embodiment is not limited here.

[0139] In this embodiment, the processing efficiency of data batch processing can be improved by processing the data index set in parallel through multiple channels.

[0140] In one embodiment, the instruction stream includes more than one operation instruction determined by a triggering order; the more than one operation instruction includes a first instruction and a second instruction, and the first instruction is triggered before the second instruction; the target operation corresponding to the instruction stream includes a first target operation corresponding to the first instruction, and a second target operation corresponding to the second instruction.

[0141] Specifically, the instruction stream may include a first instruction and a second instruction, and the first instruction is triggered before the second instruction. The computer device can execute a first target operation based on the first instruction, and can execute a second target operation based on the second instruction. In one embodiment, the target task includes a specification task, the corresponding first instruction is a data loading instruction (Load instruction), and the second instruction is a summation instruction (Add instruction). During each cyclic transmission of the instruction stream, the computer device will first transmit the Load instruction, and then execute the corresponding data loading operation to obtain the corresponding target content element. The corresponding target content element is then used as the operation object of the Add instruction to transmit and execute the Add instruction.

[0142] In the above embodiment, the instruction stream includes a first instruction and a second instruction. The execution of the first and second instructions is dependent on each other. Specifically, the first instruction is triggered before the second instruction, and the triggering object of the second instruction also includes the target content element obtained by executing the first instruction. In this way, through the dependency relationship between different instructions, the entire instruction stream can jointly assist in the execution of the target task.

[0143] In one embodiment, the target operation of each round includes a group of operations executed in multiple cycles, and each group of operations includes a first target operation and a second target operation; when each processing channel executes the target operation of the current round, it searches for the corresponding content element from the cache unit according to the current index element in the corresponding data index set as the operation object, including: when each processing channel executes the first target operation in the current cycle of the current round, it searches for the corresponding content element from the cache unit according to the corresponding current index element as the operation object of the first target operation; when each processing channel executes the second target operation in the current cycle of the current round, it obtains the intermediate operation data obtained by executing the second target operation in the previous cycle, and uses the intermediate operation data and the content element found by the first target operation as the operation object.

[0144] The target operation of each round includes a group of operations that are executed in multiple cycles, and each group of operations includes a first target operation and a second target operation.

[0145] Specifically, each processing channel can execute a set of operations for multiple cycles based on each index element in the data index set. For each processing channel in the multiple processing channels, when executing the first target operation in the set of operations in the current cycle, the corresponding content element can be searched from the cache unit based on the current index element, and the searched content element can be used as the operation object of the first target operation. When executing the second target operation in the set of operations in the current cycle, the intermediate operation data obtained by executing the second target operation in the previous cycle can be obtained, and the intermediate operation data and the content element found by the first target operation can be used as the operation object. For example, when the current index element is src[5], the content element pointed to by the current index element is 8 (src[5]=8), the first target operation is a data load operation, and the second target operation is a sum operation, when the processing channel executes the first target operation in a set of operations in the current loop, it can find the corresponding content element 8 from the cache based on src[5]. When executing the second target operation in a set of operations in the current loop, it can sum the intermediate operation data obtained by executing the second target operation in the previous loop with the content element 8 to obtain the intermediate operation data obtained by executing the second target operation in the current loop. This iteration is repeated until at least one round of target operations corresponding to the instruction stream is completed.

[0146] In this embodiment, the first target operation and the second target operation are performed alternately so that the entire target operation can assist in achieving the execution of the target task.

[0147] In one embodiment, when executing the target operation of the current round, each processing channel searches for the corresponding content element from the cache unit according to the current index element in the corresponding data index set as the operation object, including: determining the maximum concurrency supported by the local resource; when executing the target operation of the current round, each processing channel searches for the content element with the maximum concurrency from the cache unit according to the current index element in the corresponding data index set, and uses each found content element in parallel as the operation object to execute the target operation in parallel.

[0148] The maximum number of concurrent operations supported by local resources refers to the number of data to be processed that can be processed simultaneously by a single processing channel. In one embodiment, multiple data to be processed can be processed simultaneously based on SIMD. SIMD (Single Instruction Multiple Data) is a technology that uses a controller to control multiple processors and simultaneously performs the same operation on each of a set of data (also known as a "data vector") to achieve spatial parallelism.

[0149] Specifically, for each of the multiple processing channels, the maximum number of concurrencies supported by the local resources is determined, and the content element pointed to by the current index element is determined from the cache unit. Based on the maximum number of concurrencies, the content element pointed to by the current index element is used as the starting point to obtain the content elements with the maximum number of concurrencies. The found content elements are used in parallel as operation objects to execute the target operation in parallel.

[0150] In one embodiment, when the processing channel obtains the current index element, it can determine the maximum concurrent number of index elements based on the maximum concurrent number supported by the local resource and the current index element, and search for corresponding content elements from the cache unit based on the maximum concurrent number of index elements. For example, when the maximum concurrent number supported by the local resource is 8 and the current index element is src[0], the processing channel determines that the maximum concurrent number of index elements is src[0] to src[7] based on src[0] and the maximum concurrent number of 8, so that the processing channel can search for the content elements corresponding to src[0] to src[7] from the cache unit based on src[0] to src[7].

[0151] In the above embodiment, by searching for multiple content elements according to the maximum concurrency supported by the local resources and using the multiple found content elements in parallel as operation objects to execute the target operation in parallel, the efficiency of executing the target operation can be greatly improved, thereby improving the processing efficiency of data batch processing.

[0152] In one embodiment, the target task includes at least one target subtask, and each target subtask corresponds to one of the data index sets in the multiple data index sets; the above-mentioned data batch processing method also includes: determining the number of page tables of the address table corresponding to each data index set; dividing the data index set according to the number of page tables to obtain at least one index subset; each target page table in the address table corresponds to at least one matrix row in the index subset, and the target page table is used to store the address information of the index elements with continuity in the corresponding matrix row; based on the data index sets respectively assigned to each processing channel, at least one round of target operations corresponding to the instruction stream are executed in parallel through each processing channel until the operation result corresponding to the target task is obtained, including: according to the address tables respectively corresponding to each processing channel and the index subsets in the data index sets respectively assigned to each processing channel, at least one round of target operations corresponding to the instruction stream are executed in parallel through each processing channel until the operation subresult corresponding to the target subtask is obtained; and the operation subresults are integrated to obtain the operation result corresponding to the target task.

[0153] The address table, also known as the TLB (Translation Lookaside Buffer), is a storage space used to cache the translation relationship between virtual addresses and physical addresses. Specifically, the address table may include at least one page table, each of which is used to store the translation relationship between multiple consecutive virtual addresses and corresponding physical addresses. Each target page table in the address table may correspond to at least one matrix row in the index subset, and the page table is used to store the address information of the consecutive index elements in the corresponding matrix row. A matrix row in the index subset points to a matrix row in the matrix to be processed.

[0154] The target task includes at least one target subtask, and each target subtask corresponds to one of the multiple data index sets.

[0155] Specifically, different processing channels may correspond to different address tables, that is, each processing channel may correspond to a TLB. Therefore, the computer device may determine the number of page tables contained in the address table corresponding to each processing channel, and based on the number of page tables contained in the address table corresponding to each processing channel, determine the number of page tables contained in the address tables corresponding to the data index sets assigned to different processing channels. For example, when data index set 1 is assigned to processing channel 1, and the number of page tables contained in address table 1 corresponding to processing channel 1 is 4, the computer device determines that the number of page tables in the address table corresponding to data index set 1 is 4. Furthermore, the computer device divides the data index sets corresponding to the number of page tables, and treats the index elements corresponding to the content elements of each page table row in the matrix to be processed as an index subset.

[0156] For example, in the above example, when the matrix to be processed is a two-dimensional element group with 100 rows and 8 columns, and data index set 1 points to the content elements of the first 8 rows in the matrix to be processed, the computer device divides data index set 1 according to the number of page tables in address table 1 to obtain two index subsets, wherein the first index subset points to the content elements of the first 4 rows in the matrix to be processed, and the second index subset points to the content elements of rows 5 to 8 in the matrix to be processed. It is easy to understand that the index subset may include at least one matrix row, so that each matrix row points to a matrix row in the matrix to be processed, for example, the first matrix row in the index subset points to the first matrix row in the matrix to be processed. For convenience of description, the matrix rows in the index subset are referred to as index matrix rows, the matrix rows in the matrix to be processed are referred to as matrix rows to be processed, and the matrix rows in the concurrent subset are referred to as concurrent matrix rows.

[0157] Furthermore, for each of the multiple processing channels, at least one round of target operations corresponding to the instruction stream is sequentially executed based on the index subset and the corresponding address table until an operation sub-result corresponding to the target subtask is obtained. Each processing channel may execute the target operations in parallel. Upon obtaining the operation sub-results corresponding to each target subtask, the computer device synthesizes the operation sub-results to obtain an operation result corresponding to the target task.

[0158] In one embodiment, for each processing channel in the multiple processing channels, each data index set assigned to it can be processed in turn; for each data index set in the multiple data index sets, each index subset contained in the current data index set can be processed in turn, that is, based on the index subsets and the corresponding address tables in turn, at least one round of target operations corresponding to the instruction stream is executed.

[0159] In one embodiment, a computer device obtains a target task and determines an instruction stream corresponding to the target task. The computer device determines a complete set of indexes corresponding to the target task, divides the complete set of indexes into multiple data index sets, and assigns each data index set to a corresponding processing channel. For each of the multiple processing channels, the number of page tables assigned to the address table corresponding to each data index set is determined, and the data index set is divided according to the number of page tables to obtain at least one index subset. According to the address table corresponding to each processing channel and the index subset in the data index set assigned to each processing channel, at least one round of target operations corresponding to the instruction stream is executed in parallel by each processing channel until the operation sub-result corresponding to the target sub-task is obtained, and the operation sub-results are combined to obtain the operation result corresponding to the target task.

[0160] In the above embodiment, since the data index set is divided based on the number of page tables, each index subset obtained by the division can contain matrix rows with the same number of page table rows, and since each target page table in the address table corresponds to at least one matrix row in the index subset, the target page table stores address information of continuous index elements in the corresponding matrix row. Therefore, when the target operation is performed based on the address table and the current index subset, the address information of all index elements in the current index subset can be completely stored in the same address table, so that only the original address table that does not contain address information needs to be fully updated once, and then the address information of all index elements in the current index subset can be found based on this address table, without the need to update the page table in the address table, thereby reducing the number of overwrite updates of the address table, saving clock cycles consumed by the overwrite update, and improving the processing efficiency of data batch processing.

[0161] In one embodiment, when each processing channel executes the target operation of the current round, it searches for the corresponding target address information according to the target page table corresponding to the current matrix row where the current index element is located, so as to obtain the corresponding content element as the operation object based on the target address information. When the corresponding content element is not found in the target page table, the target page table is triggered to overwrite and update the address information according to the current index element.

[0162] Specifically, when each processing channel executes the target operation of the current round, the computer device can determine the virtual address of the content element pointed to by the current index element, and search the corresponding physical address from the target page table corresponding to the current matrix row where the current index element is located based on the virtual address, that is, search for the corresponding target address information, and search for the corresponding content element from the cache unit based on the target address information. When the computer device fails to find the corresponding content element from the target page table, an address information miss event (TLB miss) is triggered, triggering the computer kernel to search the memory for the target address information corresponding to the content element pointed to by the current index element. The computer kernel determines the number of target addresses that can be stored in each page table in the address table, and uses the found target address information as a starting point to obtain the stored number of consecutive target address information, and overwrites and updates a page table in the address table with the obtained stored number of consecutive target address information.

[0163] In the above embodiment, by caching the address information in the address table, the target address information can be directly read from the address table later, thereby improving the processing efficiency of data batch processing.

[0164] In one embodiment, the above-mentioned data batch processing method is executed by an inference engine, and the data batch processing method is applied to a face detection model; wherein the target task is one of the memory-intensive tasks generated by the face detection model when performing face detection.

[0165] Specifically, the data batch processing method mentioned in each embodiment of the present application is executed by an inference engine deployed on a computer device, and the data batch processing method is applied to a face detection model. When performing business processing, the face detection model needs to perform multiple computing tasks, one of which is a memory-intensive task, that is, executing access-intensive operators.

[0166] For example, computer equipment can provide online services by deploying a face detection model, and additive reduction is an important operator in the face detection model. If this additive reduction operator runs too long on the GPU, the overall inference task efficiency of the face detection model will be low. However, the data batch processing method provided in each embodiment of the present application can efficiently implement the execution of memory-intensive tasks, thereby improving the computational efficiency of the additive reduction operator, thereby improving the business processing efficiency of face detection, thereby improving online response speed, and reducing service delays.

[0167] In a specific application scenario, by loading the face detection model through the deployed inference engine, a variety of online services can be provided, such as face detection services, face recognition services, face monitoring services, etc. Users can trigger a business request corresponding to a certain service through the user terminal, such as a face retrieval request corresponding to a face recognition service. Then the computer device can perform corresponding business processing and feedback through the face detection model. In the process of business processing, the face detection model needs to perform processing on access-intensive operators. For example, the face detection model needs to perform addition and reduction processing on the feature vectors extracted in the intermediate processing process. At this time, the feature vectors can be subjected to addition and reduction operations through the data batch processing method mentioned in the embodiments of this application, so as to continue the subsequent computing tasks. Among them, the inference engine refers to the module used to complete the reasoning function in the application system.

[0168] In the above-described embodiment, the inference engine can be used to parallelize computational tasks within the face detection model, including access-intensive tasks. This parallel processing of access-intensive tasks allows cache miss events that would otherwise occur at different time stages to occur simultaneously, effectively enabling the execution of access-intensive tasks. This in turn improves the business processing efficiency of the face detection model, increases the online response speed of online services provided by the face detection model, and reduces service latency.

[0169] In one embodiment, the above-mentioned data batch processing also includes: obtaining the face image to be detected, and inputting the face image to be detected into the face detection model; obtaining the target task through the face detection model, and triggering and executing the instruction stream corresponding to the target task to obtain the operation result corresponding to the target task; determining the face detection result based on the operation result.

[0170] Specifically, the computer device can obtain more than one frame of facial images to be detected, and input each facial image to be detected into the face detection model. In the process of performing face detection, the face detection model will generate a variety of computing tasks, and the access-intensive task is the target task among the multiple computing tasks. When the target task is obtained, the face detection model triggers the instruction stream corresponding to the target task and executes it, obtains the operation result corresponding to the target task, and determines the corresponding face detection result based on the operation result. For example, when the target task is an additive reduction task, the face detection model can obtain the operation result obtained by executing the additive reduction operator, and then normalize the operation result to obtain the normalized facial features, and obtain the face detection result based on the normalized facial features.

[0171] In one embodiment, reference Figure 5 When a facial image to be detected is obtained, the computer device can load the face detection model through the deployed inference engine and perform inference on the facial image to be detected based on the face detection model to obtain the operation result of the target task. The face detection model determines the corresponding face detection result 502 based on the operation result of the target task and displays the face detection result 502 accordingly. Figure 5 A schematic diagram of the process of face detection in one embodiment is shown.

[0172] In one embodiment, Figure 6 As shown, a data batch processing method is provided, which is applied to Figure 1 The computer device in the example is used to illustrate the process, including the following steps:

[0173] Step S602: Acquire the target subtask and determine the instruction stream corresponding to the target subtask.

[0174] Step S604 : determining a data index set corresponding to the target subtask and the number of page tables in the address table in the local resource, wherein the data index set is used to search for content elements in the matrix to be processed.

[0175] Specifically, a processing task for processing a content element pointed to by a data index set can be considered a target subtask, such that the target task includes at least one target subtask, and each target subtask corresponds to one of the data index sets in the multiple data index sets. Furthermore, after determining the target subtask, the computer device can determine the number of page tables in the address table in the local resource and the instruction stream required to execute the target subtask. The instruction stream can include multiple cycles of a single target instruction, or multiple cycles of more than one target instruction with a predetermined triggering order.

[0176] Step S606: Divide the data index set according to the number of page tables to obtain at least one index subset; each target page table in the address table corresponds to at least one matrix row in the index subset, and the target page table is used to store address information of index elements that are continuous with the corresponding matrix row.

[0177] Specifically, the computer device can divide the data index set according to the number of page tables, and use the index elements corresponding to the content elements of each page table row in the matrix to be processed as an index subset. For example, when the number of page tables is 4, the index elements pointing to the content elements of the first 4 rows in the matrix to be processed can be divided into one index subset, and the index elements pointing to the content elements of the 5th to 8th rows in the matrix to be processed can be divided into another index subset. The index subset may include multiple index matrix rows, and each index matrix row in the index subset points to the corresponding matrix row to be processed. For example, the first row of index elements in the index subset points to the first row of content elements in the matrix to be processed. Each target page table in the address table corresponds to at least one index matrix row in the index subset, and the target page table is used to store the address information of the index elements that are continuous with the corresponding index matrix row. For example, when the address table contains 4 page tables, index subset 1 contains 4 rows of index matrix rows, and the 4 rows of index matrix rows contained in index subset 1 are used to point to the content elements of the first to fourth rows in the matrix to be processed, page table 1 can be used to store the address information of the first row of content elements pointed to by the first row of index matrix rows in index subset 1, that is, to store the address information of the continuous index elements contained in the first row of index matrix rows in index subset 1; page table 2 can be used to store the address information of the continuous index elements contained in the second row of index matrix rows in index subset 1; and so on, until the address information corresponding to all index elements in index subset 1 is stored in the address table.

[0178] Step S608: Execute at least one round of target operations corresponding to the instruction stream according to the address table and each index element in each index subset, until an operation sub-result corresponding to the target subtask is obtained; wherein, when executing the target operation of the current round, the corresponding target address information is searched according to the target page table corresponding to the current matrix row where the current index element is located, so as to obtain the corresponding content element as the operation object based on the target address information. When the corresponding content element is not found in the target page table, the target page table is triggered to overwrite and update the address information according to the current index element.

[0179] Specifically, the computer device executes at least one round of target operations corresponding to the instruction stream based on the index subset and the corresponding address table until the operation sub-result corresponding to the target subtask is obtained. It is worth noting that when executing the target operation of the current round, the corresponding target address information is searched from the corresponding target page table according to the current index element in the corresponding index subset, so as to obtain the corresponding content element based on the target address information; when the corresponding target address information is not found from the address table, the computer kernel is triggered to obtain the address information of the storage number of content elements according to the current index element to overwrite and update the page table. Among them, the storage number refers to the number of address information that can be stored in the page table.

[0180] In the aforementioned data batch processing method, by obtaining a target subtask, an instruction stream and a data index set corresponding to the target subtask can be determined. By determining the data index set, the data index set can be divided based on the number of page tables in the address table to obtain at least one index subset. By determining the index subset, at least one round of target operations corresponding to the instruction stream can be executed based on the address table and each index element in each index subset. Since the data index set is divided based on the number of page tables, each index subset obtained by the division can contain matrix rows with the same number of page table rows. Since each target page table in the address table corresponds to at least one matrix row in the index subset, the target page table stores the address information of the continuous index elements in the corresponding matrix row. Therefore, when the target operation is performed based on the address table and the current index subset, the address information of all index elements in the current index subset can be completely stored in the same address table. Therefore, it is only necessary to perform a comprehensive overwrite update on the original address table that does not contain address information. Subsequently, the address information of all index elements in the current index subset can be found based on this address table without the need to overwrite and update the page tables in the address table. In this way, the number of overwrite updates to the address table is reduced, the clock cycles consumed by the overwrite update are saved, and the processing efficiency of batch data processing is improved.

[0181] In one embodiment, at least one round of target operations corresponding to the instruction stream is executed according to the address table and each index element in each index subset until an operation subresult corresponding to the target subtask is obtained, including: for each data index set, determining the to-be-processed index subset in the corresponding data index set, and determining the current to-be-processed index subset from the to-be-processed index subset; executing at least one round of target operations corresponding to the instruction stream according to the address table corresponding to the corresponding data index set and each index element in the current to-be-processed index subset, and updating the current to-be-processed index subset to the completed index subset after completing the target operation; entering the process of processing the next to-be-processed index subset, and returning to the step of determining the to-be-processed index subset in the corresponding data index set to continue executing, until at least one round of target operations corresponding to the instruction stream is executed based on all index subsets in each data index set, and an operation subresult corresponding to the target subtask is obtained.

[0182] Specifically, for each data index set, the computer device determines the to-be-processed index subset in the corresponding data index set, and processes the to-be-processed index subset in turn until the operation sub-result corresponding to the target sub-task is obtained. In order to better understand this embodiment, the following is an explanation taking the execution of a data index set as an example. When the current to-be-processed data index set needs to be processed, the computer device determines the current to-be-processed index subset in the current to-be-processed data index set, and based on the current to-be-processed index subset and the corresponding address table, executes at least one round of target operations corresponding to the instruction stream. When it is determined that the current to-be-processed index subset has been processed, the computer device updates the current to-be-processed index subset to a completed index subset, and enters the process of processing the next to-be-processed index subset, until each index subset in the current to-be-processed data index set has been processed, and the operation sub-result of the target sub-task corresponding to the current to-be-processed data index set is obtained.

[0183] For example, when index subset 1 points to the content elements in the first to fourth rows of the matrix to be processed, the computer device can obtain the content elements in the first to fourth rows of the matrix to be processed based on index subset 1 and the corresponding address table, and perform the target operation on the obtained content elements based on the instruction stream. When index subset 1 is processed, the computer device obtains index subset 2 pointing to the content elements in the fifth to eighth rows of the matrix to be processed, and obtains the content elements in the fourth to eighth rows of the matrix to be processed based on index subset 2 and the corresponding address table, and performs the target operation on the obtained content elements based on the instruction stream. This process continues in this way until each index subset in the current index set of data to be processed has been processed.

[0184] In this embodiment, by traversing the index subsets, each index subset can be processed in turn, thereby ultimately obtaining the operation result of the target subtask.

[0185] In one embodiment, at least one round of target operations corresponding to the instruction stream is executed based on the address table and each index element in each index subset, including: determining the maximum concurrency supported by local resources; grouping the index subsets according to the maximum concurrency to obtain at least one concurrent subset; and executing a round of target operations corresponding to a round of instruction stream based on each concurrent subset in turn.

[0186] Specifically, the computer device determines the maximum number of concurrencies supported by the local resources, and groups the index subsets by columns based on the maximum number of concurrencies to obtain at least one concurrent subset. Wherein, each concurrent matrix row in each concurrent subset contains index elements that do not exceed the maximum number of concurrencies. For example, when the maximum number of concurrencies is 4, the matrix to be processed is a two-dimensional array containing 100 rows and 8 columns, and the index subset 1 includes four index matrix rows, and each index matrix row includes 8 index elements, the computer device can divide the index subset 1 into two concurrent subsets according to the maximum number of concurrencies 4, wherein the first concurrent subset includes the index elements of the first index matrix column to the fourth index matrix column in the index subset 1, and the second concurrent subset includes the index elements of the fifth index matrix column to the eighth index matrix column in the index subset 1.

[0187] Furthermore, the computer device executes a round of target operations corresponding to a round of instruction stream based on each concurrent subset in turn. For example, the computer device preferentially executes a round of target operations based on concurrent subset 1, and then executes a round of target operations based on concurrent subset 2.

[0188] In this embodiment, the index subset is divided by the maximum concurrency number to obtain the concurrent subset, so that a matrix row in the concurrent subset can be executed in parallel, thereby improving the processing efficiency of data batch processing.

[0189] In one embodiment, the target operation of each round includes a set of operations that are executed in multiple cycles, and the number of cycles corresponds to the number of page tables; based on each concurrent subset, a round of target operations corresponding to a round of instruction stream is executed in sequence, including: for each concurrent subset, during the execution of the target operation of each round, a set of operations is executed based on each matrix row in the concurrent subset, and the number of page tables is cycled.

[0190] Specifically, a concurrent subset corresponds to a round of target operations, and each round of target operations includes a set of operations that are executed multiple times in a loop, where the number of loops is consistent with the number of page tables. Thus, when the computer device executes a round of target operations corresponding to a round of instruction streams, it can execute a set of operations based on each concurrent matrix row in the concurrent subset, looping the number of page tables. For example, when the computer device executes a round of target operations based on concurrent subset 1, it can preferentially execute a set of target operations for the first loop based on the first concurrent matrix row in concurrent subset 1, and then execute a set of target operations for the second loop based on the second concurrent matrix row in the concurrent subset, until a set of operations is executed for the number of page tables.

[0191] It is easy to understand that since the computer device executes concurrent subsets in sequence, and for each concurrent subset, a set of operations is performed based on the matrix rows in the concurrent subset, and the page table is looped for the number of times, therefore, when executing the first concurrent subset in the index subset, the computer device can store the address information corresponding to each index element in the index subset into the corresponding page table of the address table, so that when processing the remaining concurrent subsets in the index subset except the first concurrent subset, the computer device can directly find the corresponding target address information from the address table, and find the corresponding content element based on the corresponding target address information, without having to find the target address information from the memory. In this way, the number of times the target address information is found from the memory is greatly reduced (the number of TLB Misses is reduced), and the processing efficiency of batch data processing is improved.

[0192] For example, when the number of page tables is 4, and the current index subset includes two concurrent subsets, and each concurrent subset includes 4 concurrent matrix rows, the computer device may preferentially process the first concurrent matrix row in concurrent subset 1 . Since each target page table in the address table corresponds to at least one matrix row in the index subset, when the address information corresponding to the first concurrent matrix row in concurrent subset 1 does not exist in the address table, the computer device can simultaneously store the address information corresponding to the first concurrent matrix row in concurrent subset 1 and the address information corresponding to the first concurrent matrix row in concurrent subset 2 in page table 1 in the address table. When the computer device processes the second concurrent matrix row in concurrent subset 1, since each target page table in the address table corresponds to at least one matrix row in the index subset, when the address information corresponding to the second concurrent matrix row in concurrent subset 1 does not exist in the address table, the computer device can simultaneously store the address information corresponding to the second concurrent matrix row in concurrent subset 1 and the address information corresponding to the second concurrent matrix row in concurrent subset 2 in page table 2 in the address table. This iteration is repeated until concurrent subset 1 is completely processed.

[0193] When concurrent subset 1 is processed, the address information corresponding to each index element in the index subset is stored in the corresponding page table of the address table. Therefore, when processing concurrent subset 2, the corresponding target address information can be directly found from the address table without having to find the corresponding target address information from the memory.

[0194] Similarly, when the number of concurrent matrix rows contained in each concurrent subset is less than or equal to the number of cache units, and the number of cache units that can be stored in each cache unit is greater than the maximum concurrent number, when the computer device completes processing the first concurrent subset in the index subset, the content elements pointed to by each index unit in the index subset are stored in the corresponding cache unit. Therefore, when executing except the first concurrent subset in the index subset, the computer device can directly search for the corresponding content element from the corresponding cache unit without searching for the corresponding content element from the content. This reduces the probability of cache misses and improves the processing efficiency of data batch processing. In other words, after searching all the data in the cache, the cache can be overwritten and updated, thereby improving cache utilization, reducing the number of times content elements are searched from the memory, and improving the processing efficiency of data batch processing.

[0195] In this embodiment, since the concurrent subsets are processed in sequence, and for each concurrent subset, in the execution process of the target operation in each round, the set of operations are executed based on the matrix rows in the concurrent subset, the page table is looped and executed a number of times. Therefore, after searching the data in the page table once, the page table can be overwritten and updated, thereby improving the utilization rate of the page table, reducing the number of times the target address information is searched from the memory, and improving the processing efficiency of batch data processing.

[0196] In one embodiment, the data batch processing method is executed by an inference engine, and the data batch processing method is applied to a face detection model; wherein the target subtask is a task in the target task for processing part of the data in the matrix to be processed; the target task is one of the memory-intensive tasks generated by the face detection model when performing face detection.

[0197] Specifically, the aforementioned data batch processing method is executed by an inference engine and applied to a face detection model. The face detection model, when performing business processing, needs to execute multiple computing tasks, one of which is the target task. The target task includes at least one target subtask, which is a task within the target task that processes a portion of the data in the matrix to be processed.

[0198] In the above embodiment, through the inference engine, the target subtasks in the face detection model can be processed in sequence according to the above data batch processing method. In this way, the execution of access-intensive tasks can be efficiently realized, thereby improving the business processing efficiency of the face detection model, so as to improve the online response speed when providing online services through the face detection model and reduce service delays.

[0199] To better understand this embodiment, the following compares the traditional solution with this solution. In the traditional solution, the following code can be used to perform the addition and reduction operation on the d1 dimension:

[0200]

[0201] Where d1 represents the number of rows of the matrix to be processed, d2 represents the number of columns of the matrix to be processed, src represents the identifier of the matrix to be processed, and src[j*d2+i] represents the index element. When executing the above code, the computer device can find the corresponding target address information from the address table based on src[j*d2+i], and find the corresponding content element from the cache unit based on the target address information. For example, when src[3] = 8, the computer device can determine that the corresponding content element is 8 based on src[3].

[0202] When executing the above code, the index elements obtained in two adjacent loops need to span data of length d2. For example, in the first loop, when i=0, j=0, the corresponding index element is src[0]. In the second loop, when i=0, j=1, the corresponding index element is src[d2]. There are d2 data between src[0] and src[d2]. Generally speaking, when d2 exceeds the cache capacity of the cache unit (exceeds the cache block size), a cache miss occurs every time the content element corresponding to src[j*d2+i] is searched in each loop. When the span of d2 exceeds the page table size, a TLB miss occurs every time the target address information corresponding to src[j*d2+i] is searched in each loop. Therefore, when d1*d2 target address information and content elements are searched, the total number of cache misses is d1*d2, and the total number of TLB misses is d1*d2. TLB misses have a greater memory access overhead than cache misses, generally requiring at least five memory accesses. Therefore, under the traditional scheme, the total number of memory accesses per operation is approximately 6*d1*d2.

[0203] In the embodiment of the present application, still taking the addition reduction operator as an example, assuming that the address table (TLB) can store 4 page tables and the maximum number of concurrency supported by the local resources is 8 (SIMD processes 8 data at a time), the above code can be updated as follows:

[0204]

[0205] Wherein, d1 represents the number of rows of the matrix to be processed, d2 represents the number of columns of the matrix to be processed, src represents the identifier of the matrix to be processed, and src[j*d2+i:j*d2+i+7] represents a concurrent matrix row in the concurrent subset. Since under this technical solution, the cache unit can be overwritten and updated after all the content elements stored in the cache unit are searched once, the cache unit can be fully utilized. Therefore, in the embodiment of the present application, the number of cache miss events (cache miss) generated by executing the above code is d1*d2 / m, that is, a cache miss event (cache miss) is generated for every m content elements searched. Wherein, m is the number of caches corresponding to the cache unit (the size of the cache unit).

[0206] Similarly, after all the address information stored in the page table has been searched once, the page table can be overwritten and updated, thus fully utilizing the page table. Assuming that a page table (typically 4KB) can store the address information of p data elements (1KB for float), the total number of address information miss events (TLB miss) is d1*d2 / p, that is, an address information miss event (TLB miss) is generated for every address information corresponding to P content elements.

[0207] In this embodiment of the application, since a part of the results are temporarily stored in the dst array, a cache miss event will also occur when the content element corresponding to dst[i:i+7] is stored in the memory. Since m is the number of caches corresponding to the cache unit (the size of the cache unit), and d2 is the number of columns of the matrix to be executed, the total number of cache miss events corresponding to the dst array generated by executing the above code is d2 / m times.

[0208] When an address information miss event (TLB miss) requires 5 memory accesses and a cache miss event (cache miss) requires 1 memory access, the total number of memory accesses in the embodiment of the present application is d1*d2 / m+5*(d1*d2 / p)+d2 / m. Because the data that a page table can store is much larger than the cache unit, after the address information stored in the page table is searched once and then the page table is overwritten and updated, 5*(d1*d2 / p) can be ignored. Therefore, the final total number of memory accesses is (d1*d2 / m+d2 / m), which is less than 6*d1*d2 in the traditional solution.

[0209] Since cache units are frequently overwritten and updated in traditional solutions, resulting in a large number of repeated DRAM (Dynamic Random Access Memory) accesses, this embodiment effectively utilizes the data updated to the cache unit and the data updated to the TLB each time, thereby greatly reducing the number of memory accesses and significantly improving the processing efficiency of data batch processing.

[0210] In one embodiment, Figure 7 As shown, a flow chart of a data batch processing method in a specific embodiment is provided:

[0211] S702, obtain the target task and determine the instruction stream corresponding to the target task; the instruction stream includes more than one operation instruction determined by the triggering order; the more than one operation instruction includes a first instruction and a second instruction, and the first instruction is triggered before the second instruction; the target operation corresponding to the instruction stream includes a first target operation corresponding to the first instruction, and a second target operation corresponding to the second instruction.

[0212] S704 , determining the complete set of indexes corresponding to the target task, and determining the number of processing channels and the number of caches corresponding to the cache units; each index element in the complete set of indexes is used to point to each content element in the matrix to be processed.

[0213] S706, divide the entire index set according to the number of channels and the number of caches to obtain multiple data index sets, and assign each data index set to a corresponding processing channel; wherein the number of elements in the data index set assigned to each processing channel is greater than the number of caches corresponding to the cache unit.

[0214] S708, for each processing channel among the multiple processing channels, determine at least one data index set to be processed corresponding to the corresponding processing channel, and determine the number of page tables of the address table corresponding to each data index set, divide the data index set according to the number of page tables, and obtain at least one index subset.

[0215] S710 : For each processing channel, determine a current to-be-processed data index set from the corresponding to-be-processed data index set.

[0216] S712, based on the current data index set to be processed corresponding to each processing channel, execute at least one round of target operations corresponding to the instruction stream in parallel through each processing channel, and update the current data index set to be processed to the completed data index set after completing the target operation; wherein, for each index subset in the current data index set to be processed, according to the address table corresponding to each processing channel and the index subset allocated to each processing channel, execute at least one round of target operations corresponding to the instruction stream in parallel through each processing channel until the operation sub-result corresponding to the target subtask is obtained; and the operation sub-results are combined to obtain the operation result corresponding to the target task.

[0217] S714, enter the process of processing the next data index set to be processed, and return to the step of determining at least one data index set to be processed assigned to the corresponding processing channel for each processing channel in the multiple processing channels, and continue to execute until at least one round of target operations corresponding to the instruction stream is executed based on each data index set in the entire index set, until the operation result corresponding to the target task is obtained.

[0218] S716: The target operation of each round includes a set of operations executed multiple times in a loop, each set of operations including a first target operation and a second target operation. When each processing channel executes the first target operation in the current loop of the current round, it searches the cache for the corresponding content element based on the corresponding current index element to serve as the operation target for the first target operation. When each processing channel executes the second target operation in the current loop of the current round, it obtains the intermediate operation data obtained from executing the second target operation in the previous loop, and uses the intermediate operation data and the content element found by the first target operation as the operation targets.

[0219] S718, when the corresponding content element is not found from the cache unit, trigger the local kernel to obtain content elements that are continuous and correspond to the cache quantity from the local memory according to the current index element, and trigger the overwriting and updating of the stored content in the cache unit based on the obtained content elements; the overwritten stored content elements in the cache unit are used for the execution of subsequent target operations.

[0220] The above-mentioned data batch processing method divides the entire index set based on the number of channels and the number of caches, so that the number of elements in the divided data index set can be greater than the number of caches corresponding to the cache unit. Therefore, when multiple processing channels read data from the cache unit in parallel based on their respective corresponding index elements, cache miss events may occur at the same time. Multi-channel memory allows memory reads to occur simultaneously, so cache miss events that should have occurred in multiple different time stages can be concentrated at the same time, thereby hiding the memory access delay of some cache miss events, thereby improving the processing efficiency of data batch processing.

[0221] In one embodiment, Figure 8 As shown, a flow chart of a data batch processing method in another specific embodiment is provided:

[0222] S802: Acquire a target subtask and determine an instruction stream corresponding to the target subtask.

[0223] S804: Determine a data index set corresponding to the target subtask and the number of page tables in the address table in the local resource. The data index set is used to search for content elements in the matrix to be processed.

[0224] S806, divide the data index set according to the number of page tables to obtain at least one index subset; each target page table in the address table corresponds to at least one matrix row in the index subset, and the target page table is used to store address information of index elements that are continuous with the corresponding matrix row.

[0225] S808 : For each data index set, determine a to-be-processed index subset in the corresponding data index set, and determine a current to-be-processed index subset from the to-be-processed index subsets.

[0226] S810, determine the maximum number of concurrent operations supported by local resources; group the current pending index subsets according to the maximum number of concurrent operations to obtain at least one concurrent subset; and sequentially perform a round of target operations on the concurrent subsets in the current pending index subsets.

[0227] S812, the target operation of each round includes a set of operations that are executed in multiple loops, and the number of loops corresponds to the number of page tables; for each concurrent subset, during the execution of the target operation of each round, a set of operations is executed based on each matrix row in the concurrent subset, looping the page tables times, and after completing the target operation, the current pending index subset is updated to the completed index subset.

[0228] S814, enter the process of processing the next to-be-processed index subset, and return to the step of determining the to-be-processed index subset in the corresponding data index set to continue execution, until all index subsets in each data index set are based on at least one round of target operations corresponding to the instruction stream, and the operation sub-result corresponding to the target subtask is obtained, until the operation sub-result corresponding to the target subtask is obtained.

[0229] Among them, when executing the target operation of the current round, the corresponding target address information is searched according to the target page table corresponding to the current matrix row where the current index element is located, so as to obtain the corresponding content element as the operation object based on the target address information. When the corresponding content element is not found in the target page table, the target page table is triggered to overwrite and update the address information according to the current index element.

[0230] This application also provides an application scenario, which applies the above-mentioned data batch processing method. Specifically, the application of the data batch processing method in this application scenario is as follows:

[0231] When a facial image to be detected is obtained, the computer device may input the image to the face detection model, which then generates a corresponding target task and, based on the corresponding instruction stream, executes at least one round of target operations to obtain the target task's operational results. Furthermore, the computer device performs subsequent calculations based on the target task's operational results to obtain a face detection result.

[0232] This application also provides an application scenario, which applies the above-mentioned data batch processing method. Specifically, the application of the data batch processing method in this application scenario is as follows:

[0233] When the image to be detected is obtained, the computer device can load the corresponding machine learning model through the inference engine based on the above method, and then the machine learning model can recognize the image to be detected, for example, Figure 9 , the face can be recognized to obtain the face recognition result 902; the water cup on the table can be recognized to obtain the water cup recognition result 904; or the face outline can be recognized to obtain the face outline recognition result 906, etc. Figure 9 An application scenario diagram of a data batch processing method in one embodiment is shown.

[0234] It is understood that the above application scenarios are merely illustrative of the solutions of this application. The data batch processing method of this application can also be applied to other scenarios without limitation. For example, license plate images captured by an application can be recognized to determine the license plate number. Another example is audio recognition of media data, with subsequent processing based on the successfully recognized audio information, such as simulated conversations.

[0235] It should be understood that although Figure 3 、 6 The steps in the flowcharts of 7-8 are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 3 、 6 At least part of the steps in 7-8 may include multiple steps or multiple stages. These steps or stages do not necessarily have to be performed at the same time, but can be performed at different times. The order of execution of these steps or stages is not necessarily one by one, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0236] In one embodiment, Figure 10 As shown, a data batch processing device 1000 is provided. The device can be a software module or a hardware module, or a combination of the two to form a part of a computer device. The device specifically includes: an index full set acquisition module 1002, a data index set acquisition module 1004 and a target operation execution module 1006, wherein:

[0237] The index set acquisition module 1002 is used to acquire the target task and determine the instruction stream corresponding to the target task; determine the index set corresponding to the target task; and each index element in the index set is used to point to each content element in the matrix to be processed.

[0238] The data index set acquisition module 1004 is used to divide the entire index set into multiple data index sets and assign each data index set to a corresponding processing channel; wherein the number of elements in the data index set assigned to each processing channel is greater than the cache quantity corresponding to the cache unit.

[0239] The target operation execution module 1006 is used to execute at least one round of target operations corresponding to the instruction stream in parallel through each processing channel based on the data index set allocated to each processing channel, until the operation result corresponding to the target task is obtained; wherein, when each processing channel executes the target operation of the current round, it searches the cache unit for the corresponding content element as the operation object according to the current index element in the corresponding data index set. When the corresponding content element is not found in the cache unit, the cache unit is triggered to obtain the cached number of content elements according to the current index element for overwriting and updating.

[0240] In one embodiment, the data index set acquisition module 1004 is also used to determine the number of channels of the processing channel and the number of caches corresponding to the cache unit; according to the number of channels and the number of caches, the entire index set is divided to obtain multiple data index sets, and each data index set is assigned to the corresponding processing channel.

[0241] In one embodiment, the target operation execution module 1006 is also used to determine, for each processing channel in a plurality of processing channels, at least one set of to-be-processed data indexes corresponding to the corresponding processing channel; for each processing channel, determine the current set of to-be-processed data indexes from the corresponding set of to-be-processed data indexes; based on the current set of to-be-processed data indexes corresponding to each processing channel, execute at least one round of target operations corresponding to the instruction stream in parallel through each processing channel, and update the current set of to-be-processed data indexes to the completed data index set after completing the target operation; enter the process of processing the next set of to-be-processed data indexes, and return to the step of determining, for each processing channel in a plurality of processing channels, at least one set of to-be-processed data indexes corresponding to the corresponding processing channel, until at least one round of target operations corresponding to the instruction stream is executed based on each data index set in the entire index set.

[0242] In one embodiment, the index full set acquisition module 1002 is used to determine an instruction stream corresponding to a target task; the instruction stream includes more than one operation instruction determined by a triggering order; the more than one operation instruction includes a first instruction and a second instruction, and the first instruction is triggered before the second instruction; the target operation corresponding to the instruction stream includes a first target operation corresponding to the first instruction, and a second target operation corresponding to the second instruction.

[0243] In one embodiment, the target operation of each round includes a group of operations that are executed in multiple loops, and each group of operations includes a first target operation and a second target operation; the target operation execution module 1006 also includes a loop execution module 1061, which is used to search for the corresponding content element from the cache unit as the operation object of the first target operation according to the corresponding current index element when each processing channel executes the first target operation in the current loop of the current round; when each processing channel executes the second target operation in the current loop of the current round, obtain the intermediate operation data obtained by executing the second target operation in the previous loop, and use the intermediate operation data and the content element found by the first target operation together as the operation object.

[0244] In one embodiment, the target operation execution module 1006 also includes a concurrent processing module 1062, which is used to determine the maximum concurrent number supported by local resources; when each processing channel executes the target operation of the current round, it searches for the content element with the maximum concurrent number from the cache unit according to the current index element in the corresponding data index set, and uses each found content element as an operation object in parallel to execute the target operation in parallel.

[0245] In one embodiment, the data batch processing device 1000 is also used to trigger the local kernel to obtain continuous content elements corresponding to the cache quantity from the local memory according to the current index element when the corresponding content element is not found from the cache unit, and trigger the overwriting and updating of the storage content in the cache unit based on the obtained content elements; the overwritten content elements stored in the cache unit are used for the execution of subsequent target operations.

[0246] In one embodiment, the data batch processing device 1000 is further used to determine the number of page tables of the address table corresponding to each data index set; divide the data index set according to the number of page tables to obtain at least one index subset; each target page table in the address table corresponds to at least one matrix row in the index subset, and the target page table is used to store address information of index elements that are continuous in the corresponding matrix row; according to the address tables corresponding to each processing channel and the index subsets in the data index set allocated to each processing channel, at least one round of target operations corresponding to the instruction stream are executed in parallel through each processing channel until an operation sub-result corresponding to the target subtask is obtained; and the operation sub-results are combined to obtain an operation result corresponding to the target task.

[0247] In one embodiment, the data batch processing device 1000 is also used for each processing channel to search for corresponding target address information according to the target page table corresponding to the current matrix row where the current index element is located when executing the target operation of the current round, so as to obtain the corresponding content element as the operation object based on the target address information. When the corresponding content element is not found in the target page table, the target page table is triggered to overwrite and update the address information according to the current index element.

[0248] In one embodiment, the data batch processing device 1000 is deployed with an inference engine for execution, and the data batch processing device 1000 is applied to a face detection model; wherein the target task is one of the memory-intensive tasks generated by the face detection model when performing face detection.

[0249] In one embodiment, the data batch processing device 1000 is also used to obtain a facial image to be detected and input the facial image to be detected into a face detection model; obtain a target task through the face detection model, and trigger and execute the instruction stream corresponding to the target task to obtain an operation result corresponding to the target task; determine the face detection result based on the operation result.

[0250] In one embodiment, Figure 11 As shown, a data batch processing device 1100 is provided. The device can be a software module or a hardware module, or a combination of the two to form a part of a computer device. The device specifically includes: a subtask determination module 1102, a data partitioning module 1104 and a sub-result determination module 1106, wherein:

[0251] The subtask determining module 1102 is configured to obtain a target subtask and determine an instruction stream corresponding to the target subtask.

[0252] The data partitioning module 1104 is used to determine the data index set corresponding to the target subtask and the number of page tables in the address table in the local resource. The data index set is used to find the content elements in the matrix to be processed; the data index set is divided according to the number of page tables to obtain at least one index subset; each target page table in the address table corresponds to at least one matrix row in the index subset, and the target page table is used to store the address information of the index elements that are continuous with the corresponding matrix row.

[0253] The sub-result determination module 1106 is used to execute at least one round of target operations corresponding to the instruction stream according to the address table and each index element in each index subset until an operation sub-result corresponding to the target subtask is obtained; wherein, when executing the target operation of the current round, the corresponding target address information is searched according to the target page table corresponding to the current matrix row where the current index element is located, so as to obtain the corresponding content element as the operation object based on the target address information. When the corresponding content element is not found in the target page table, the target page table is triggered to overwrite and update the address information according to the current index element.

[0254] In one embodiment, the sub-result determination module 1106 is also used to determine, for each data index set, a subset of to-be-processed indexes in the corresponding data index set, and determine a current subset of to-be-processed indexes from the subset of to-be-processed indexes; execute at least one round of target operations corresponding to the instruction stream according to the address table corresponding to the corresponding data index set and each index element in the current subset of to-be-processed indexes, and update the current subset of to-be-processed indexes to a completed index subset after completing the target operation; enter the process of processing the next subset of to-be-processed indexes, and return to the step of determining the subset of to-be-processed indexes in the corresponding data index set to continue executing, until at least one round of target operations corresponding to the instruction stream are executed based on all index subsets in each data index set, and an operation sub-result corresponding to the target subtask is obtained.

[0255] In one embodiment, the sub-result determination module 1106 is also used to determine the maximum number of concurrencies supported by local resources; group the index subsets according to the maximum number of concurrencies to obtain at least one concurrent subset; and execute a round of target operations corresponding to a round of instruction streams based on each concurrent subset in turn.

[0256] In one embodiment, the target operation of each round includes a set of operations that are executed in multiple loops, and the number of loops corresponds to the number of page tables; the sub-result determination module 1106 is also used to, for each concurrent subset, in the process of executing the target operation of each round, execute a set of operations based on each matrix row in the concurrent subset, looping the number of page tables times.

[0257] In one embodiment, the data batch processing device 1100 is deployed with an inference engine for execution, and the data batch processing device 1100 is applied to a face detection model; wherein, the target subtask is a task in the target task for processing part of the data in the matrix to be processed; the target task is one of the memory-intensive tasks generated by the face detection model when performing face detection.

[0258] For the specific definition of the data batch processing device, please refer to the definition of the data batch processing method above, and will not be repeated here. Each module in the above-mentioned data batch processing device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0259] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 12As shown. The computer device includes a processor, a memory, and a network interface connected via a system bus. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store data for batch processing. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a method for batch processing of data is implemented.

[0260] Those skilled in the art will understand that Figure 12 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0261] In one embodiment, a computer device is further provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps in the above method embodiments when executing the computer program.

[0262] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above-mentioned method embodiments when executed by a processor.

[0263] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the steps of each of the above-described method embodiments.

[0264] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include at least one of non-volatile and volatile memory. Non-volatile memory may include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0265] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0266] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. A data batch processing method, characterized in that: The method comprises: Obtaining a target task and determining an instruction stream corresponding to the target task; Determine a complete set of indexes corresponding to the target task; each index element in the complete set of indexes is used to point to each content element in the matrix to be processed; Dividing the entire index set into a plurality of data index sets, and assigning each of the data index sets to a corresponding processing channel; wherein the number of elements in the data index set assigned to each processing channel is greater than the number of caches corresponding to the cache unit, the processing channel being a memory channel controlled by a memory controller for addressing, reading data, and processing data, and each processing channel having a corresponding cache unit; Based on the data index sets respectively allocated to the processing channels, respectively, executing at least one round of target operations corresponding to the instruction stream through each processing channel in parallel until an operation result corresponding to the target task is obtained; Among them, when each processing channel executes the target operation of the current round, it searches for the corresponding content element from the cache unit as the operation object according to the current index element in the corresponding data index set. When the corresponding content element is not found from the cache unit, the cache unit is triggered to obtain the cached number of content elements according to the current index element for overwriting and updating.

2. The method according to claim 1, characterized in that The step of dividing the entire index set into a plurality of data index sets and assigning each of the data index sets to a corresponding processing channel includes: Determine the number of channels of the processing channel and the number of caches corresponding to the cache unit; The entire index set is divided according to the number of channels and the number of caches to obtain a plurality of data index sets, and each of the data index sets is respectively allocated to the corresponding processing channel.

3. The method according to claim 1, characterized in that The step of executing at least one round of target operations corresponding to the instruction stream in parallel through each processing channel based on the data index sets respectively allocated to each processing channel includes: For each processing channel among the plurality of processing channels, determining at least one to-be-processed data index set allocated to the corresponding processing channel; For each processing channel, determine the current data index set to be processed from the corresponding data index set to be processed; Based on a current to-be-processed data index set corresponding to each processing channel, respectively, executing at least one round of target operations corresponding to the instruction stream through each processing channel in parallel, and updating the current to-be-processed data index set to a completed data index set after completing the target operations; Enter the process of processing the next data index set to be processed, and return to the step of determining at least one data index set to be processed assigned to the corresponding processing channel for each processing channel in the multiple processing channels, and continue to execute until at least one round of target operations corresponding to the instruction stream is executed based on each data index set in the entire index set.

4. The method according to claim 1, wherein The instruction stream includes more than one operation instruction determined by a triggering order; the more than one operation instruction includes a first instruction and a second instruction, and the first instruction is triggered before the second instruction; the target operation corresponding to the instruction stream includes a first target operation corresponding to the first instruction, and a second target operation corresponding to the second instruction.

5. The method according to claim 1, wherein When the corresponding content element is not found in the cache unit, triggering the cache unit to obtain the cached content elements according to the current index element for overwriting and updating includes: When the corresponding content element is not found in the cache unit, the local kernel is triggered to obtain continuous content elements corresponding to the cache quantity from the local memory according to the current index element, and triggers the overwriting and updating of the storage content in the cache unit based on the obtained content elements; the overwritten content elements stored in the cache unit are used for the execution of subsequent target operations.

6. The method according to claim 1, characterized in that The target task includes at least one target subtask, and each target subtask corresponds to one of the multiple data index sets; the method further includes: Determine the number of page tables of the address table corresponding to each data index set; Dividing the data index set according to the number of page tables to obtain at least one index subset; each target page table in the address table corresponds to at least one matrix row in the index subset, and the target page table is used to store address information of index elements that are continuous with the corresponding matrix row; The step of executing at least one round of target operations corresponding to the instruction stream in parallel through each processing channel based on the data index sets respectively allocated to each processing channel until an operation result corresponding to the target task is obtained includes: executing, in parallel, at least one round of target operations corresponding to the instruction stream through each processing channel according to the address table corresponding to each processing channel and the index subset in the data index set allocated to each processing channel, until an operation subresult corresponding to the target subtask is obtained; The results of each operation are combined to obtain the operation result corresponding to the target task.

7. The method according to claim 6, characterized in that When executing the target operation of the current round, each processing channel searches for the corresponding target address information according to the target page table corresponding to the current matrix row where the current index element is located, so as to obtain the corresponding content element as the operation object based on the target address information. When the corresponding content element is not found in the target page table, the target page table is triggered to overwrite and update the address information according to the current index element.

8. The method according to any one of claims 1 to 7, characterized in that The method is executed by an inference engine and is applied to a face detection model; wherein the target task is one of the memory-intensive tasks generated by the face detection model when performing face detection.

9. A data batch processing method, characterized in that: The method further comprises: Obtaining a target subtask and determining an instruction stream corresponding to the target subtask; Determining a data index set corresponding to the target subtask and the number of page tables of the address table in the local resource, wherein the data index set is used to search for content elements in the matrix to be processed; Dividing the data index set according to the number of page tables to obtain at least one index subset; each target page table in the address table corresponds to at least one matrix row in the index subset, and the target page table is used to store address information of index elements that are continuous with the corresponding matrix row; Determining a maximum number of concurrent tasks supported by local resources; grouping the index subsets according to the maximum number of concurrent tasks to obtain at least one concurrent subset; and sequentially executing a round of target operations corresponding to a round of instruction streams based on each of the concurrent subsets until an operation subresult corresponding to the target subtask is obtained; Among them, when executing the target operation of the current round, the corresponding target address information is searched according to the target page table corresponding to the current matrix row where the current index element is located, so as to obtain the corresponding content element from the cache unit as the operation object based on the target address information. When the corresponding content element is not found from the cache unit based on the target address information in the target page table, the target page table is triggered to overwrite and update the address information according to the current index element, so as to update the cache unit.

10. The method according to claim 9, characterized in that The grouping of the index subsets according to the maximum concurrent number to obtain at least one concurrent subset includes: The index subsets are grouped by columns based on the maximum concurrency number to obtain at least one concurrent subset; wherein each concurrent matrix row in each concurrent subset contains index elements not exceeding the maximum concurrency number.

11. The method according to claim 9, characterized in that The target operation of each round includes a set of operations executed in multiple cycles, where the number of the multiple cycles corresponds to the number of page tables; The step of sequentially executing a round of target operations corresponding to a round of instruction streams based on each of the concurrent subsets includes: For each concurrent subset, during the execution of the target operation in each round, the set of operations is executed based on each matrix row in the concurrent subset, looping the number of page tables for the number of times.

12. The method according to any one of claims 9 to 11, characterized in that The method is executed by an inference engine and is applied to a face detection model; wherein the target subtask is a task in the target task for processing part of the data in the matrix to be processed; and the target task is one of the memory-intensive tasks generated by the face detection model when performing face detection.

13. A data batch processing device, characterized in that: The device comprises: An index set acquisition module is used to acquire a target task and determine an instruction stream corresponding to the target task; determine an index set corresponding to the target task; each index element in the index set is used to point to each content element in the matrix to be processed; a data index set acquisition module, configured to divide the entire index set into a plurality of data index sets and assign each of the data index sets to a corresponding processing channel; wherein the number of elements in the data index set assigned to each processing channel is greater than the number of caches corresponding to the cache unit; the processing channel is a memory channel controlled by a memory controller for addressing, reading data, and processing data, and each processing channel has a corresponding cache unit; A target operation execution module is used to execute at least one round of target operations corresponding to the instruction stream in parallel through each processing channel based on the data index set respectively assigned to each processing channel, until an operation result corresponding to the target task is obtained; wherein, when each processing channel executes the target operation of the current round, it searches the cache unit for the corresponding content element as the operation object according to the current index element in the corresponding data index set, and when the corresponding content element is not found in the cache unit, it triggers the cache unit to obtain the cached number of content elements according to the current index element for overwriting and updating.

14. The device according to claim 13, characterized in that The data index set acquisition module is also used to determine the number of channels of the processing channel and the number of caches corresponding to the cache unit; according to the number of channels and the number of caches, the full index set is divided to obtain multiple data index sets, and each data index set is respectively assigned to the corresponding processing channel.

15. The device according to claim 13, characterized in that The target operation execution module is further configured to determine, for each processing channel among the plurality of processing channels, at least one to-be-processed data index set assigned to the corresponding processing channel; determine, for each processing channel, a current to-be-processed data index set from the corresponding to-be-processed data index set; execute, in parallel, at least one round of target operations corresponding to the instruction stream through each processing channel based on the current to-be-processed data index set corresponding to each processing channel; and update the current to-be-processed data index set to a completed data index set after completing the target operation; Enter the process of processing the next data index set to be processed, and return to the step of determining at least one data index set to be processed assigned to the corresponding processing channel for each processing channel in the multiple processing channels, and continue to execute until at least one round of target operations corresponding to the instruction stream is executed based on each data index set in the entire index set.

16. The device according to claim 13, characterized in that The instruction stream includes more than one operation instruction determined by a triggering order; the more than one operation instruction includes a first instruction and a second instruction, and the first instruction is triggered before the second instruction; the target operation corresponding to the instruction stream includes a first target operation corresponding to the first instruction, and a second target operation corresponding to the second instruction.

17. The device according to claim 13, characterized in that The data batch processing device is also used to trigger the local kernel to obtain continuous content elements corresponding to the cache quantity from the local memory according to the current index element when the corresponding content element is not found in the cache unit, and trigger the overwriting and updating of the storage content in the cache unit based on the obtained content elements; the overwritten content elements stored in the cache unit are used for the execution of subsequent target operations.

18. The device according to claim 13, characterized in that The target task includes at least one target subtask, and each target subtask corresponds to one of the multiple data index sets; The data batch processing device is further configured to determine the number of page tables of the address table corresponding to each data index set; divide the data index set according to the number of page tables to obtain at least one index subset; each target page table in the address table corresponds to at least one matrix row in the index subset, and the target page table is configured to store address information of index elements that are continuous with the corresponding matrix row; According to the address tables corresponding to the respective processing channels and the index subsets in the data index sets allocated to the respective processing channels, at least one round of target operations corresponding to the instruction stream is executed in parallel through the respective processing channels until an operation sub-result corresponding to the target sub-task is obtained; and the operation sub-results are synthesized to obtain an operation result corresponding to the target task.

19. The device according to claim 18, characterized in that When executing the target operation of the current round, each processing channel searches for the corresponding target address information according to the target page table corresponding to the current matrix row where the current index element is located, so as to obtain the corresponding content element as the operation object based on the target address information. When the corresponding content element is not found in the target page table, the target page table is triggered to overwrite and update the address information according to the current index element.

20. The device according to any one of claims 13 to 19, characterized in that The device is executed by an inference engine and is applied to a face detection model; wherein the target task is one of the memory-intensive tasks generated by the face detection model when performing face detection.

21. A data batch processing device, characterized in that: The device comprises: A subtask determination module, configured to obtain a target subtask and determine an instruction stream corresponding to the target subtask; a data partitioning module, configured to determine a data index set corresponding to the target subtask and the number of page tables in the address table in the local resource, wherein the data index set is used to search for content elements in the matrix to be processed; partition the data index set according to the number of page tables to obtain at least one index subset; each target page table in the address table corresponds to at least one matrix row in the index subset, and the target page table is used to store address information of index elements that are continuous with the corresponding matrix row; A sub-result determination module is used to determine the maximum number of concurrencies supported by local resources; group the index subsets according to the maximum number of concurrencies to obtain at least one concurrent subset; execute a round of target operations corresponding to a round of instruction streams based on each of the concurrent subsets in turn until an operation sub-result corresponding to the target subtask is obtained; wherein, when executing the target operation of the current round, the corresponding target address information is searched according to the target page table corresponding to the current matrix row where the current index element is located, so as to obtain the corresponding content element from the cache unit as the operation object based on the target address information; when the corresponding content element is not found from the cache unit based on the target address information in the target page table, the target page table is triggered to overwrite and update the address information according to the current index element, so as to update the cache unit.

22. The device according to claim 21, characterized in that The sub-result determination module is further used to group the index subsets by columns based on the maximum concurrent number to obtain at least one concurrent subset; wherein each concurrent matrix row in each concurrent subset contains index elements not exceeding the maximum concurrent number.

23. The device according to claim 21, characterized in that The target operation of each round includes a set of operations executed in multiple cycles, where the number of the multiple cycles corresponds to the number of page tables; The sub-result determination module is further configured to, for each concurrent subset, in the execution process of each round of target operation, execute the set of operations based on each matrix row in the concurrent subset, looping the page table number of times.

24. The device according to any one of claims 21 to 23, characterized in that The device is executed by an inference engine, and the device is applied to a face detection model; wherein, the target subtask is a task in the target task for processing part of the data in the matrix to be processed; the target task is one of the memory-intensive tasks generated by the face detection model when performing face detection.

25. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

26. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 12 are implemented.

27. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images