Data transfer method based on inference embedding acceleration chip
Through the data handling method based on inference embedding to accelerate chips, the data handling tasks are analyzed and processed, and the data handling tasks are optimized by using embedding table to optimize data handling, the problems of low efficiency and low bandwidth utilization in inference business scenarios are solved, and the overall inference efficiency and bandwidth utilization are improved .
Patent Information
- Application Number
- CN202411354836.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-27
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-09-27
AI Technical Summary
In the inference business scenario, the multi-embedding graphics processor search solution corresponding to Sparse is inefficient, resulting in waste of costs and low utilization of memory bandwidth, affecting the overall inference efficiency and utilization of artificial intelligence accelerator.
Using the data handling method based on inference embedding acceleration chip, the data handling task group is obtained to obtain the to-process data handling task group, analyze the feature ID, data reading and writing address and task status in the task group, and use the pre-configured embedding table to perform data handling and improve bandwidth Utilization and reasoning efficiency.
The bandwidth utilization and inference efficiency are improved, the problems of low efficiency and low bandwidth utilization in the prior art are solved, and more efficient data handling and computing performance are achieved.
Smart Images

Figure CN118860949B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing technology, and in particular to a data transport method based on an inference embedding acceleration chip. Background Art
[0002] The embedding model is a large language model and a key technology in the field of search promotion. In particular, it contributes to business scenarios such as e-commerce, short videos, social media, search, and knowledge graphs. Embedding can well convert text or video into a high-dimensional semantic space representation through feature conversion.
[0003] Due to the huge number of materials that need to be embedded and represented, the scale of the embedding table in actual business is extremely large. The model in the field of search promotion is mainly divided into two parts: Sparse (i.e. a combination of multiple embeddings) and Dense (dense network structure).
[0004] In the process of implementing the present invention, the inventors found that the prior art has the following defects: At present, for the inference business scenario, the multi-embedding GPU search solution corresponding to Sparse, whether executed serially or in parallel, greatly affects the inference efficiency and generates cost waste. In addition, the bottleneck of embedding search is mainly the bandwidth of the video memory, so the theoretical execution time of each instruction seriously affects the bandwidth utilization and the overall efficiency of the inference link and the utilization of the artificial intelligence accelerator. Summary of the invention
[0005] The present invention provides a data handling method based on an inference embedding acceleration chip to improve bandwidth utilization and inference efficiency.
[0006] According to one aspect of the present invention, a data handling method based on an inference embedding acceleration chip is provided, which includes:
[0007] Acquire a currently pending data handling task group, and parse at least one currently pending data handling task corresponding to the currently pending data handling task group to obtain parsing results of each current data handling task;
[0008] The analysis results of each current data transfer task include a feature ID, a data read transfer start address, a data write transfer start address, a data transfer length, and a task transfer status;
[0009] By using at least one pre-configured embedding table, the current data handling tasks to be processed are processed according to the analysis results of each current data handling task, to obtain the completion result of the data handling task;
[0010] Wherein, each embedding table is configured based on an inference embedding acceleration chip;
[0011] The completion results of each data transfer task are fed back to the user.
[0012] According to another aspect of the present invention, a data handling device based on an inference embedding acceleration chip is provided, comprising:
[0013] A current data handling task analysis result determination module is used to obtain a current pending data handling task group, and analyze at least one current pending data handling task corresponding to the current pending data handling task group to obtain analysis results of each current data handling task;
[0014] The analysis results of each current data transfer task include a feature ID, a data read transfer start address, a data write transfer start address, a data transfer length, and a task transfer status;
[0015] The data handling task completion result determination module is used to process the current data handling tasks to be processed according to the analysis results of each current data handling task through at least one pre-configured embeddingtable to obtain the data handling task completion result;
[0016] Wherein, each embedding table is configured based on an inference embedding acceleration chip;
[0017] The data handling task completion result feedback module is used to feedback the completion results of each data handling task to the user.
[0018] According to another aspect of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the data handling method based on the inference embedding acceleration chip described in any embodiment of the present invention is implemented.
[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data handling method based on an inference embedding acceleration chip described in any embodiment of the present invention when executed.
[0020] The technical solution of the embodiment of the present invention obtains the current data handling task group to be processed, and parses at least one current data handling task to be processed corresponding to the current data handling task group to obtain the parsing results of each current data handling task; wherein each of the parsing results of the current data handling task includes a feature ID, a data read handling start address, a data write handling start address, a data handling length and a task handling status; through at least one pre-configured embedding table, the current data handling task to be processed is processed according to each of the current data handling task parsing results to obtain the data handling task completion result; wherein each of the embedding tables is configured based on an inference embedding acceleration chip; and the completion results of each of the data handling tasks are fed back to the user. The problems of low efficiency and low bandwidth utilization in the existing data handling technology are solved, and the bandwidth utilization and inference efficiency are improved.
[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0023] Figure 1 This is a flow chart of a data handling method based on an inference embedding acceleration chip provided in accordance with the first embodiment of the present invention;
[0024] Figure 2 is a flowchart of another data handling method based on an inference embedding acceleration chip provided in accordance with the second embodiment of the present invention;
[0025] Figure 3 It is a structural schematic diagram of a data handling device based on an inference embedding acceleration chip provided according to the third embodiment of the present invention;
[0026] Figure 4 It is a schematic diagram of the structure of an electronic device provided according to Embodiment 4 of the present invention. DETAILED DESCRIPTION
[0027] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.
[0028] It should be noted that the terms "target", "current", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0029] Embodiment 1
[0030] Figure 1 A flowchart of a data transfer method based on an inference embedding acceleration chip is provided for the first embodiment of the present invention. This embodiment is applicable to the case where data is transferred through an inference embedding acceleration chip. The method can be executed by a data transfer device based on an inference embedding acceleration chip. The data transfer device based on an inference embedding acceleration chip can be implemented in the form of hardware and / or software.
[0031] Correspondingly, such as Figure 1 As shown, the method includes:
[0032] S110, obtaining a currently pending data handling task group, and parsing at least one currently pending data handling task corresponding to the currently pending data handling task group to obtain parsing results of each current data handling task.
[0033] The analysis results of each current data transfer task include a feature ID, a data read transfer start address, a data write transfer start address, a data transfer length and a task transfer status.
[0034] The currently-to-be-processed data handling task group may include one or more currently-to-be-processed data handling tasks.
[0035] In this embodiment, it is necessary to perform data transfer processing on each currently pending data transfer task respectively. It is necessary to first perform analysis processing on each currently pending data transfer task to obtain the current data transfer task analysis result. Specifically, the current data transfer task analysis result may include feature ID, data read transfer start address, data write transfer start address, data transfer length and task transfer status.
[0036] Specifically, feature IDs are unique and have hot and cold effects. One feature ID corresponds to one read transfer and multiple write transfers, that is, after completing the read transfer process for one feature ID, multiple write transfer tasks can be processed. Feature ID has the characteristics of high precision and low precision compatibility, and can calculate the specific transfer address based on parameters such as feature ID precision.
[0037] In detail, the task transfer status may include a task creation status, a read transfer start status, a read transfer completion status, a write transfer start status, and a write transfer completion status.
[0038] For example, different states can be assigned values, for example, the task creation state can be represented by 0; the start read and transfer state can be represented by 1; the read and transfer completion state can be represented by 2; the start write and transfer state can be represented by 3; and the write and transfer completion state can be represented by 4. In addition, assuming that the current state is the start read and transfer state, the state is 1, and after the read and transfer task is completed, the state is modified to 2, that is, the read and transfer completion state.
[0039] The data read transfer start address may be a specific start address of the data read transfer corresponding to the current data transfer task to be processed. The data write transfer start address may be a specific start address of the data write transfer corresponding to the current data transfer task to be processed. The data transfer length may be the length of the data to be transferred.
[0040] In this embodiment, data transfer can be implemented according to a data read transfer start address or a data write transfer start address and a data transfer length.
[0041] S120. Process the current data handling tasks to be processed according to the analysis results of each current data handling task through at least one pre-configured embedding table to obtain a data handling task completion result.
[0042] Among them, each embedding table is configured based on an inference embedding acceleration chip.
[0043] Specifically, the configuration of each embedding table includes feature ID accuracy, feature ID scale, the dimension size of the high-dimensional semantic space corresponding to each feature ID, and the element accuracy of the high-dimensional semantic space.
[0044] Among them, embedding is an embedding table, which is a set of configuration lists configured based on the inference embedding acceleration chip. Specifically, the embedding table may include feature ID accuracy, feature ID scale, high-dimensional semantic space dimension size corresponding to each feature ID, and high-dimensional semantic space element accuracy. The data read transfer start address and data write transfer start address corresponding to different feature IDs can be calculated and processed based on the above parameters, so that effective data transfer and processing operations can be performed better based on the parameters in the embedding table.
[0045] The data transfer task completion result may be a data transfer completion result obtained after performing data transfer processing on the current data transfer task to be processed.
[0046] Optionally, at least one pre-configured embedding table is used to process the current data transfer tasks according to the analysis results of each current data transfer task to obtain the completion result of the data transfer task, including: if the task transfer status is the start read transfer status, then the feature ID, data read transfer start address, data transfer length and buffer address corresponding to each current data transfer task to be processed are obtained, and the data corresponding to each current data transfer task to be processed is transferred from the data transfer start address to the buffer address through at least one embedding table; if the task transfer status is the start write transfer status, then the feature ID, data write transfer start address, data transfer length, buffer address and output address list corresponding to each current data transfer task to be processed are obtained, and the data corresponding to each current data transfer task to be processed is transferred from the buffer address to each output address corresponding to the output address list in parallel through at least one embedding table.
[0047] In this embodiment, it is necessary to perform different data transfer task processing operations according to different task transfer states, so as to better perform data transfer processing and complete different data transfer tasks.
[0048] Specifically, if the task transfer status is the start read transfer status, the feature ID, data read transfer start address, data transfer length and buffer address corresponding to each currently pending data transfer task are obtained, and through at least one embedding table, the data corresponding to each currently pending data transfer task is transferred from the data transfer start address to the buffer address.
[0049] In detail, the method obtains the feature ID, data read and transfer start address, data transfer length and buffer address corresponding to each currently pending data transfer task, and transfers the data corresponding to each currently pending data transfer task from the data transfer start address to the buffer address through at least one embedding table, including: obtaining the feature ID corresponding to each currently pending data transfer task, deduplicating each feature ID, and obtaining the current data read and transfer merge processing task; obtaining the feature ID accuracy, feature ID scale, high-dimensional semantic space dimension size and high-dimensional semantic space element accuracy corresponding to each embedding table, calculating and processing the feature ID and the data read and transfer start address, and calculating the standard data read and transfer start address; according to the data transfer length, the data corresponding to the current data read and transfer merge processing task is transferred from the standard data read and transfer start address to the buffer address.
[0050] In this embodiment, according to the hot and cold effects of the feature ID, it can be known that a feature ID contains a read transfer. It can be understood that when the task transfer state is the start read transfer state, the current task is a read transfer task. For the read transfer task, it is necessary to first obtain the feature IDs corresponding to each currently processed data transfer task, and perform deduplication processing on each feature ID.
[0051] For example, there are 10 data transfer tasks currently to be processed. Among the feature IDs corresponding to these 10 data transfer tasks, 5 are feature ID1 and 5 are feature ID2. Therefore, it is necessary to deduplicate the feature ID1 and feature ID2 and retain one feature ID1 and one feature ID2.
[0052] Furthermore, the five currently pending data transport tasks corresponding to feature ID1 may be merged and processed; the five currently pending data transport tasks corresponding to feature ID2 may also be merged and processed, thereby obtaining the current data read transport merge processing task.
[0053] Furthermore, it is necessary to calculate and process the feature ID and the data read and transfer start address according to the feature ID precision, feature ID scale, high-dimensional semantic space dimension size and high-dimensional semantic space element precision corresponding to each embedding table obtained, and calculate and obtain the standard data read and transfer start address. Since the feature ID has the characteristics of high precision and compatibility with low precision, the corresponding feature ID can be matched and obtained according to the above feature ID precision, feature ID scale, high-dimensional semantic space dimension size and high-dimensional semantic space element precision corresponding to each feature ID, and the data read and transfer start address can be calculated according to the obtained feature ID, and the standard data read and transfer start address after processing can be obtained.
[0054] Continuing with the previous example, assuming that feature ID1 can be matched based on feature ID precision, feature ID scale, the size of the high-dimensional semantic space dimension corresponding to each feature ID, and the precision of the high-dimensional semantic space elements, it is necessary to calculate the standard data read and transfer start address corresponding to feature ID1.
[0055] Optionally, if the read data transfer task corresponding to the currently pending data transfer task is completed, the write data transfer task corresponding to the currently pending data transfer task is obtained and started through the scheduler according to the current task configuration; wherein the current task configuration includes a single data write transfer task independently completing multiple data corresponding to the same feature ID, or includes executing multiple output address lists of the same feature ID in parallel evenly based on the idle rate of parallel write transfer tasks, and transferring the data from the buffer address to each output address corresponding to the output address list.
[0056] In this embodiment, after completing the read data transfer task, the scheduler will automatically start the write transfer task of the task. For the write transfer task, the write transfer task can be processed individually or in parallel, and both can be transferred from the buffer address to the output addresses corresponding to the output address list.
[0057] Specifically, if the task transfer status is the start write transfer status, obtain the feature ID, data write transfer start address, data transfer length, buffer address and output address list corresponding to each currently pending data transfer task, and through at least one embedding table, transfer the data corresponding to each currently pending data transfer task from the buffer address in parallel to each output address corresponding to the output address list.
[0058] In detail, if the task transfer state is the start write transfer state, the feature ID, data write transfer start address, data transfer length, buffer address and output address list corresponding to each currently pending data transfer task are obtained, and through at least one embedding table, the data corresponding to each currently pending data transfer task is transferred in parallel from the buffer address to each output address corresponding to the output address list, including: obtaining and calculating the input offset address corresponding to the feature ID according to the standard data read transfer start address and the feature ID accuracy corresponding to each embedding table; calculating the standard data write transfer start address according to the data write transfer start address and the input offset address; according to the standard data write transfer start address, the data corresponding to each currently pending data transfer task is transferred in parallel from the buffer address to each output address corresponding to the output address list.
[0059] In this embodiment, since the write transfer task is executed after the read transfer task is completed, it is necessary to obtain the standard data read transfer start address, and then determine the matching feature ID in combination with the feature ID accuracy to calculate the current input offset address.
[0060] Furthermore, the standard data write transfer start address is calculated based on the calculated input offset address in combination with the data write transfer start address, so that data transfer processing can be performed based on the calculated standard data write transfer start address.
[0061] Specifically, if the task transfer status is a create task status, a read transfer completion status, or a write transfer completion status, the status corresponding to each currently pending data transfer task is updated through the scheduler.
[0062] Continuing with the previous example, if the task transfer status is the create task status, the status needs to be changed to 0 through the scheduler; if the task transfer status is the read transfer completion status, the status needs to be changed to 2 through the scheduler; if the task transfer status is the write transfer completion status, the status needs to be changed to 4 through the scheduler.
[0063] In addition, after a data handling task is completed, the scheduler can modify the status of the data handling task to the completed data handling status, otherwise it can be modified to the uncompleted data handling status. This can avoid multiple processing of the same data handling task. The scheduler can also be used to perform statistical processing on the number of idle tasks, and can also perform reasonable scheduling processing by statistically analyzing the idle rate of read handling tasks or write handling tasks.
[0064] S130: Feedback the completion results of each data transfer task to the user.
[0065] The technical solution of the embodiment of the present invention obtains the current data handling task group to be processed, and parses at least one current data handling task to be processed corresponding to the current data handling task group to obtain the parsing results of each current data handling task; wherein each of the parsing results of the current data handling task includes a feature ID, a data read handling start address, a data write handling start address, a data handling length and a task handling status; through at least one pre-configured embedding table, the current data handling task to be processed is processed according to each of the current data handling task parsing results to obtain the data handling task completion result; wherein each of the embedding tables is configured based on an inference embedding acceleration chip; and the completion results of each of the data handling tasks are fed back to the user. The problems of low efficiency and low bandwidth utilization in the existing data handling technology are solved, and the bandwidth utilization and inference efficiency are improved.
[0066] Embodiment 2
[0067] Figure 2 A flowchart of another data handling method based on an inference embedding acceleration chip provided according to Embodiment 2 of the present invention. This embodiment is optimized based on the above embodiments. In this embodiment, after obtaining the current data handling task group to be processed, data handling processing can also be performed by task splitting the data handling tasks.
[0068] Correspondingly, such as Figure 2 As shown, the method includes:
[0069] S210, obtaining the data handling task group currently to be processed.
[0070] S220: Acquire at least one currently-to-be-processed data transport task corresponding to the currently-to-be-processed data transport task group, and sequentially acquire a target currently-to-be-processed data transport task from each of the currently-to-be-processed data transport tasks.
[0071] S230, splitting the target data handling task currently to be processed to obtain a plurality of data handling subtasks currently to be processed and sub-feature IDs corresponding to the data handling subtasks.
[0072] In this embodiment, the target data handling task currently to be processed can also be split into tasks. The specific application scenario may be that the data volume of the target data handling task currently to be processed is relatively large and needs to be divided into multiple subtasks for separate processing operations.
[0073] Exemplarily, the target data handling task currently to be processed can be split into five data handling subtasks currently to be processed, and the subtasks can be matched with corresponding sub-feature IDs (for example, sub-feature ID1, sub-feature ID2, sub-feature ID3, sub-feature ID4 and sub-feature ID5).
[0074] S240: Create multiple subtask processing units, subtask allocation list tables and subtask indexes corresponding to the data handling subtasks respectively.
[0075] The subtask allocation list table includes a mapping relationship between sub-feature IDs and subtask indexes.
[0076] The subtask index can be matched to the subtask processing unit in one step according to the subfeature ID, and then the data handling and processing operation of the corresponding subfeature ID can be performed in the subtask processing unit.
[0077] S250. According to at least one embedding table, sub-feature ID, sub-task allocation list table, and sub-task index, in each sub-task processing unit, each data handling sub-task is processed separately, the completion result of each data handling sub-task is obtained and fed back, and the status corresponding to each data handling sub-task is updated through the scheduler.
[0078] In this embodiment, after determining the target data handling subtask to be processed, it is necessary to obtain the sub-feature ID corresponding to the data handling subtask, and then determine the subtask index corresponding to the sub-feature ID according to the task allocation list table, and then determine the subtask processing unit corresponding to the target data handling subtask to be processed according to the subtask index.
[0079] Furthermore, in the determined subtask processing unit, data transfer processing can be performed on the target data transfer subtask to be processed, the target data transfer subtask completion result can be obtained and the processing operation can be fed back, and the subtask status can be updated through the scheduler.
[0080] It can be understood that the data transfer tasks here may include data reading transfer tasks and data writing transfer tasks. The specific processing methods of the data reading transfer tasks and the data writing transfer tasks are consistent with the processing methods of the above embodiments and will not be repeated here.
[0081] S260, returning to execute the operation of sequentially acquiring a target currently-to-be-processed data transport task from each of the currently-to-be-processed data transport tasks, until all of the currently-to-be-processed data transport tasks are traversed and completed.
[0082] The technical solution of the embodiment of the present invention is as follows: after obtaining the currently-processed data handling task group, obtaining at least one currently-processed data handling task corresponding to the currently-processed data handling task group, and sequentially obtaining a target currently-processed data handling task in each of the currently-processed data handling tasks; splitting the target currently-processed data handling task to obtain a plurality of currently-processed data handling subtasks and a sub-feature ID corresponding to the data handling subtask; creating a plurality of subtask processing units, a subtask allocation list table and a subtask index corresponding to each of the data handling subtasks; processing each of the data handling subtasks in each of the subtask processing units according to at least one embedding table, a sub-feature ID, a subtask allocation list table and a subtask index, obtaining a completion result of each data handling subtask and feeding it back, and updating the status corresponding to each of the data handling subtasks through a scheduler; returning to execute the operation of sequentially obtaining a target currently-processed data handling task in each of the currently-processed data handling tasks until the traversal of each of the currently-processed data handling tasks is completed. Data reading and writing can be better processed by task splitting, which can improve bandwidth utilization and reasoning efficiency.
[0083] Embodiment 3
[0084] Figure 3 The structural diagram of a data handling device based on an inference embedding acceleration chip provided in the third embodiment of the present invention. The data handling device based on an inference embedding acceleration chip provided in this embodiment can be implemented by software and / or hardware, and can be configured in a terminal device or a server to implement a data handling method based on an inference embedding acceleration chip in the embodiment of the present invention. Figure 3 As shown, the device includes: a current data handling task analysis result determination module 310, a data handling task completion result determination module 320 and a data handling task completion result feedback module 330.
[0085] The current data handling task analysis result determination module 310 is used to obtain a current data handling task group to be processed, and analyze at least one current data handling task to be processed corresponding to the current data handling task group to be processed, to obtain analysis results of each current data handling task;
[0086] The analysis results of each current data transfer task include a feature ID, a data read transfer start address, a data write transfer start address, a data transfer length, and a task transfer status;
[0087] The data handling task completion result determination module 320 is used to process the current data handling tasks to be processed according to the analysis results of each current data handling task through at least one pre-configured embedding table to obtain the data handling task completion result;
[0088] Wherein, each embedding table is configured based on an inference embedding acceleration chip;
[0089] The data handling task completion result feedback module 330 is used to feedback the completion results of each data handling task to the user.
[0090] The technical solution of the embodiment of the present invention obtains the current data handling task group to be processed, and parses at least one current data handling task to be processed corresponding to the current data handling task group to obtain the parsing results of each current data handling task; wherein each of the parsing results of the current data handling task includes a feature ID, a data read handling start address, a data write handling start address, a data handling length and a task handling status; through at least one pre-configured embedding table, the current data handling task to be processed is processed according to each of the current data handling task parsing results to obtain the data handling task completion result; wherein each of the embedding tables is configured based on an inference embedding acceleration chip; and the completion results of each of the data handling tasks are fed back to the user. The problems of low efficiency and low bandwidth utilization in the existing data handling technology are solved, and the bandwidth utilization and inference efficiency are improved.
[0091] Based on the above embodiments, the task transfer status includes creating task status, starting read transfer status, reading transfer completion status, starting write transfer status and writing transfer completion status; one feature ID corresponds to one read transfer and multiple write transfers;
[0092] The configuration of each embedding table includes feature ID accuracy, feature ID scale, the dimension size of the high-dimensional semantic space corresponding to each feature ID, and the element accuracy of the high-dimensional semantic space.
[0093] On the basis of the above embodiments, the data transfer task completion result determination module 320 may specifically include: a start read transfer state processing unit, which may be specifically used for: if the task transfer state is the start read transfer state, then obtain the feature ID, data read transfer start address, data transfer length and buffer address corresponding to each currently processed data transfer task, and through at least one embedding table, transfer the data corresponding to each currently processed data transfer task from the data transfer start address to the buffer address; a start write transfer state processing unit, which may be specifically used for: if the task transfer state is the start write transfer state, then obtain the feature ID, data write transfer start address, data transfer length, buffer address and output address list corresponding to each currently processed data transfer task, and through at least one embedding table, transfer the data corresponding to each currently processed data transfer task from the buffer address to each output address corresponding to the output address list in parallel.
[0094] On the basis of the above embodiments, it can also be specifically used for: if the task transfer status creates the task status, the read transfer completion status or the write transfer completion status, then the status corresponding to each currently pending data transfer task is updated and processed through the scheduler; wherein, if the read data transfer task corresponding to the currently pending data transfer task is completed, obtain and start the write data transfer task corresponding to the currently pending data transfer task through the scheduler according to the current task configuration; wherein, the current task configuration includes a single data write transfer task independently completing multiple data corresponding to the same feature ID, or includes executing multiple output address lists of the same feature ID uniformly and in parallel based on the idle rate of parallel write transfer tasks, and the data is transferred from the buffer address to each output address corresponding to the output address list.
[0095] On the basis of the above embodiments, the start read and transfer state processing unit can be specifically used to: obtain the feature IDs corresponding to each currently processed data transfer task, deduplicate each feature ID, and obtain the current data read and transfer merge processing task; obtain the feature ID accuracy, feature ID scale, high-dimensional semantic space dimension size and high-dimensional semantic space element accuracy corresponding to each embedding table, calculate the feature ID and the data read and transfer start address, and calculate the standard data read and transfer start address; according to the data transfer length, move the data corresponding to the current data read and transfer merge processing task from the standard data read and transfer start address to the buffer address.
[0096] On the basis of the above embodiments, the start write transfer state processing unit can be specifically used to: obtain and calculate the input offset address corresponding to the feature ID according to the standard data read transfer start address and the feature ID precision corresponding to each embedding table; calculate the standard data write transfer start address according to the data write transfer start address and the input offset address; according to the standard data write transfer start address, transfer the data corresponding to each currently processed data transfer task from the buffer address to each output address corresponding to the output address list in parallel.
[0097] On the basis of the above embodiments, it also includes a task splitting processing module, which can be specifically used to, after obtaining the currently pending data handling task group, obtain at least one currently pending data handling task corresponding to the currently pending data handling task group, and obtain a target currently pending data handling task in each of the currently pending data handling tasks in turn; split the target currently pending data handling task to obtain multiple currently pending data handling subtasks and sub-feature IDs corresponding to the data handling subtasks; create multiple subtask processing units, subtask allocation list tables and subtask indexes corresponding to each of the data handling subtasks; wherein the subtask allocation list table includes a mapping relationship between the sub-feature ID and the subtask index; according to at least one embedding table, sub-feature ID, sub-task allocation list table, sub-task index, in each of the sub-task processing units, respectively process the data handling sub-task, obtain the completion result of each data handling sub-task and feedback it, and update the status corresponding to each data handling sub-task through the scheduler; return to execute the operation of obtaining a target current data handling task to be processed in each of the current data handling tasks to be processed in turn, until the traversal of each of the current data handling tasks to be processed is completed.
[0098] The data handling device based on the inference embedding acceleration chip provided in the embodiment of the present invention can execute the data handling method based on the inference embedding acceleration chip provided in any embodiment of the present invention, and has the functional modules and beneficial effects corresponding to the execution method.
[0099] Embodiment 4
[0100] Figure 4A schematic diagram of the structure of an electronic device 10 that can be used to implement the fourth embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.
[0101] like Figure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.
[0102] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0103] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a data handling method based on an inference embedding acceleration chip.
[0104] In some embodiments, the data handling method based on the inference embedding acceleration chip can be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the data handling method based on the inference embedding acceleration chip described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to execute the data handling method based on the inference embedding acceleration chip in any other appropriate manner (for example, by means of firmware).
[0105] The method includes: obtaining a currently pending data handling task group, and parsing at least one currently pending data handling task corresponding to the currently pending data handling task group to obtain parsing results of each current data handling task; wherein each of the currently pending data handling task parsing results includes a feature ID, a data read handling start address, a data write handling start address, a data handling length and a task handling status; processing the currently pending data handling tasks according to each of the currently pending data handling task parsing results respectively through at least one pre-configured embedding table to obtain a data handling task completion result; wherein each of the embedding tables is configured based on an inference embedding acceleration chip; and feeding back the completion results of each of the data handling tasks to a user.
[0106] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0107] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0108] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0109] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).
[0110] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.
[0111] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.
[0112] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.
[0113] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
[0114] Embodiment 5
[0115] Embodiment 5 of the present invention also provides a computer-readable storage medium, wherein the computer-readable instructions are used to execute a data handling method based on an inference embedding acceleration chip when executed by a computer processor, the method comprising: obtaining a current data handling task group to be processed, and parsing at least one current data handling task to be processed corresponding to the current data handling task group to be processed, to obtain parsing results of each current data handling task; wherein each parsing result of the current data handling task includes a feature ID, a data read handling start address, a data write handling start address, a data handling length and a task handling status; through at least one pre-configured embedding table, the current data handling task to be processed is processed according to each current data handling task parsing result to obtain a data handling task completion result; wherein each embedding table is configured based on the inference embedding acceleration chip; and the completion result of each data handling task is fed back to the user.
[0116] Of course, the computer-readable storage medium provided in the embodiment of the present invention has computer executable instructions that are not limited to the method operations described above, and can also execute related operations in data transfer based on the inference embedding acceleration chip provided in any embodiment of the present invention.
[0117] Through the above description of the implementation methods, the technicians in the relevant field can clearly understand that the present invention can be implemented by means of software and necessary general hardware, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute the methods described in each embodiment of the present invention.
[0118] It is worth noting that in the above-mentioned embodiment of data transfer based on inference embedding acceleration chip, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.
[0119] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.
Claims
1. A data handling method based on an inference embedding acceleration chip, characterized in that: include: Acquire a currently pending data handling task group, and parse at least one currently pending data handling task corresponding to the currently pending data handling task group to obtain parsing results of each current data handling task; The analysis results of each current data transfer task include a feature ID, a data read transfer start address, a data write transfer start address, a data transfer length, and a task transfer status; By using at least one pre-configured embedding table, the current data handling tasks to be processed are processed according to the analysis results of each current data handling task, to obtain the completion result of the data handling task; Wherein, each embedding table is configured based on an inference embedding acceleration chip; Feedback the completion results of each data transfer task to the user; The task transfer status includes the task creation status, the start read transfer status, the read transfer completion status, the start write transfer status and the write transfer completion status; one feature ID corresponds to one read transfer and multiple write transfers; The configuration of each embedding table includes feature ID accuracy, feature ID scale, the dimension size of the high-dimensional semantic space corresponding to each feature ID, and the element accuracy of the high-dimensional semantic space.
2. The method according to claim 1, characterized in that The method of processing the current data handling tasks to be processed according to the analysis results of each current data handling task by using at least one pre-configured embeddingtable to obtain the data handling task completion result includes: If the task transfer state is the start read transfer state, then obtain the feature ID, data read transfer start address, data transfer length and buffer address corresponding to each currently pending data transfer task, and transfer the data corresponding to each currently pending data transfer task from the data transfer start address to the buffer address through at least one embeddingtable; If the task transfer status is the start write transfer status, obtain the feature ID, data write transfer start address, data transfer length, buffer address and output address list corresponding to each currently pending data transfer task, and through at least one embedding table, transfer the data corresponding to each currently pending data transfer task from the buffer address in parallel to each output address corresponding to the output address list.
3. The method according to claim 2, characterized in that Also includes: If the task transfer status is the creation task status, the read transfer completion status or the write transfer completion status, the status corresponding to each currently pending data transfer task is updated through the scheduler; If the read data transport task corresponding to the currently pending data transport task is completed, the write data transport task corresponding to the currently pending data transport task is acquired and started through the scheduler according to the current task configuration; Among them, the current task configuration includes a single data write transfer task independently completing multiple data corresponding to the same feature ID, or includes uniformly and parallelly executing multiple output address lists of the same feature ID based on the idle rate of parallel write transfer tasks, and transferring the data from the buffer address to the output addresses corresponding to the output address list.
4. The method according to claim 3, characterized in that The step of obtaining the characteristic ID, the data read transfer start address, the data transfer length and the buffer address respectively corresponding to each currently pending data transfer task, and transferring the data corresponding to each currently pending data transfer task from the data transfer start address to the buffer address through at least one embedding table includes: Obtain the feature IDs corresponding to the currently pending data transfer tasks, perform deduplication processing on the feature IDs, and obtain the current data read transfer merge processing task; Obtain the feature ID precision, feature ID scale, high-dimensional semantic space dimension size and high-dimensional semantic space element precision corresponding to each embedding table, calculate the feature ID and the data read and transfer start address, and calculate the standard data read and transfer start address; According to the data transfer length, the data corresponding to the current data read transfer merging processing task is transferred from the standard data read transfer starting address to the buffer address.
5. The method according to claim 4, characterized in that If the task transfer state is the start write transfer state, then the characteristic ID, data write transfer start address, data transfer length, buffer address and output address list corresponding to each currently pending data transfer task are obtained, and the data corresponding to each currently pending data transfer task is transferred from the buffer address to each output address corresponding to the output address list in parallel through at least one embedding table, including: Obtain and calculate the input offset address corresponding to the feature ID according to the standard data read transfer start address and the feature ID precisions corresponding to each embedding table; Calculating a standard data write transfer start address according to the data write transfer start address and the input offset address; According to the standard data write transfer start address, the data corresponding to each currently-to-be-processed data transfer task is transferred in parallel from the buffer address to each output address corresponding to the output address list.
6. The method according to claim 5, characterized in that After obtaining the current data handling task group to be processed, the method further includes: Acquire at least one currently-to-be-processed data handling task corresponding to the currently-to-be-processed data handling task group, and sequentially acquire a target currently-to-be-processed data handling task from each of the currently-to-be-processed data handling tasks; Splitting the target data handling task currently to be processed to obtain a plurality of data handling subtasks currently to be processed and sub-feature IDs corresponding to the data handling subtasks; Creating a plurality of subtask processing units, a subtask allocation list table and a subtask index corresponding to each of the data handling subtasks; Wherein, the subtask allocation list table includes a mapping relationship between sub-feature IDs and subtask indexes; According to at least one embedding table, sub-feature ID, sub-task allocation list table, and sub-task index, in each sub-task processing unit, each data handling sub-task is processed respectively, a completion result of each data handling sub-task is obtained and fed back, and a status corresponding to each data handling sub-task is updated through a scheduler; Return to execute the operation of sequentially acquiring a target currently-to-be-processed data handling task from each of the currently-to-be-processed data handling tasks until all of the currently-to-be-processed data handling tasks are traversed and completed.
7. A data handling device based on an inference embedding acceleration chip, characterized in that: include: A current data handling task analysis result determination module is used to obtain a current pending data handling task group, and analyze at least one current pending data handling task corresponding to the current pending data handling task group to obtain analysis results of each current data handling task; The analysis results of each current data transfer task include a feature ID, a data read transfer start address, a data write transfer start address, a data transfer length, and a task transfer status; A data handling task completion result determination module is used to process the current data handling tasks to be processed according to the analysis results of each current data handling task through at least one pre-configured embedding table to obtain the data handling task completion result; Wherein, each embedding table is configured based on an inference embedding acceleration chip; A data handling task completion result feedback module is used to feedback the completion results of each data handling task to the user; Among them, the task transfer status includes task creation status, start read transfer status, read transfer completion status, start write transfer status and write transfer completion status; one feature ID corresponds to one read transfer and multiple write transfers; among them, the configuration of each embedding table includes feature ID accuracy, feature ID scale, high-dimensional semantic space dimension size corresponding to each feature ID and high-dimensional semantic space element accuracy.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for data handling based on an inference embedding acceleration chip as described in any one of claims 1-6 is implemented.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement a data handling method based on an inference embedding acceleration chip as described in any one of claims 1-6 when executed.
