Data processing method and related device

Abstract

A data processing method, comprising: dividing input data of an artificial intelligence model into multiple batches for parallel processing, performing a first hash operation on first data in a first batch among the multiple batches, and on the basis of the result of the first hash operation, determining a first hash group to which the first data belongs, wherein the first hash group corresponds to a first memory space in a memory; performing a second hash operation on the first data, and on the basis of the result of the second hash operation, determining a first hash bucket to which the first data belongs, wherein the first hash bucket belongs to the first hash group, and the first hash bucket is located in a first area in the first memory space; and locking the first hash bucket when the first data is written to the first area. In the method, multi-layer hash is incorporated, a finer-grained hash bucket is further configured on the basis of a hash group, and when a concurrent conflict occurs, resource loss caused by the concurrent conflict can be reduced by only locking the hash bucket, thereby improving the data processing efficiency, ensuring the overall throughput rate of the system, and improving system performance.

Description

Data processing method and related device

[0001] The present application claims priority to the Chinese Patent Application No. 202410636839.8, filed on May 17, 2024, and entitled "A data processing method and related device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0002] The present application relates to the field of artificial intelligence (AI) technology, and in particular to a data processing method, a data processing apparatus, a chip, a computing device, a computer readable storage medium, and a computer program product. BACKGROUND

[0003] With the development of AI technology, especially the continuous evolution of machine learning (ML) and deep learning, various AI models have been generated. For example, an AI model can include an item recommendation model applied to an e-commerce platform, or a video recommendation model applied to a short video platform.

[0004] In order to process a large amount of data or requests, the parameters of an AI model can be divided into multiple accelerators using a distributed system to achieve distributed computing and parallel processing, thereby improving the performance and scalability of the system. At the same time, using a distributed system for storage can ensure the reliability and high availability of data, reducing the risk of single point failure.

[0005] Among them, the parameters of the AI model can include parameters represented by sparse features. For example, in a recommendation system, user identification, user interest labels, and item attributes can be represented by sparse features. Generally, different batches of sparse features usually have a certain degree of repetition, and the number of sparse features in a batch is relatively large. During model training, the sparse features can be preprocessed, such as deduplication, to effectively reduce the communication overhead of sparse features.

[0006] However, the current preprocessing method is to deduplicate sparse features and other data using a hash table. When using multi-thread parallel execution for hash table-based deduplication, if a conflict occurs, the entire hash table will be locked, which affects the deduplication of data by other threads, resulting in a long time-consuming in the preprocessing stage, thereby reducing the overall throughput and performance of the distributed system. SUMMARY

[0007] The application provides a data processing method. The method introduces multi-layer hashing, and sets a more fine-grained hash bucket on the basis of a hash group. When a concurrent conflict occurs, the resource loss caused by the concurrent conflict can be reduced by locking only the hash bucket, the processor resource can be fully utilized, the data processing time can be shortened, the data processing efficiency can be improved, the overall throughput of the system can be ensured, and the system performance can be improved. The application also provides a data processing device, a chip, a computing device, a computer readable storage medium and a computer program product corresponding to the above method.

[0008] In a first aspect, the application provides a data processing method. The method can be applied to a distributed system, which can be a heterogeneous computing device or a heterogeneous computing device cluster formed by multiple heterogeneous computing devices. The distributed system can include a processor and an accelerator. The processor is located on the host side and can be a central processing unit (CPU), and the accelerator is located on the device side and can be a neural network processing unit (NPU), a graphics processing unit (GPU), or a tensor processing unit (TPU). The accelerator is used to cooperate with the processor to complete the training of an AI model.

[0009] Specifically, the processor can process input data of an artificial intelligence (AI) model in multiple batches in parallel. The data in different batches has a certain degree of repetition. For example, in a recommendation scenario or a search scenario, the sparse features such as personal information, hobbies, and historical behavior of adjacent batches have a certain degree of repetition, or the item labels and item categories of adjacent batches have a certain degree of repetition. For a first data in a first batch in the multiple batches, the processor can perform a first hash operation, and determine a first hash group to which the first data belongs according to a result of the first hash operation. The first hash group corresponds to a first memory space in memory. Then the processor performs a second hash operation on the first data, and determines a hash bucket to which the first data belongs according to a result of the second hash operation. The first hash bucket belongs to the first hash group, and the first hash bucket is located in a first region in the first memory space. When writing the first data into the first region, the processor can lock the first hash bucket.

[0010] The method introduces multi-layer hashing, specifically, a more fine-grained hash bucket is set on the basis of a hash group, when a parallel mechanism is used for data processing to improve processing efficiency, if a conflict occurs, only the corresponding hash bucket can be locked, other hash buckets in the hash group or other hash groups still support data insertion, thus reducing resource loss caused by concurrent conflicts, fully utilizing processor resources, shortening data processing time, improving data processing efficiency, guaranteeing the overall throughput of the system, and improving system performance.

[0011] In some possible implementation manners, the first hash group corresponds to a first memory space in a memory of a processor, or the first hash group corresponds to a first memory space in a memory of a first accelerator, and the first accelerator is used for processing the first data.

[0012] In the case that the hash group corresponds to a memory space in a memory of a processor, the hash group can correspond to a continuous memory space in the memory, and sequential reading can be implemented to improve reading performance. In the case that the hash group corresponds to a memory space in a memory of an accelerator, data can be written into the accelerator, and the overhead from the memory of the processor to the memory of the accelerator can be reduced.

[0013] In some possible implementation manners, the size of the first hash bucket can be equal to n times of the cache line size of a cache of a processor, or the size of the first hash bucket is equal to n times of the cache line size of a cache in the first accelerator, and n is a natural number greater than or equal to 1. In this way, the size of the first hash bucket can be aligned with the cache line, and when reading data in the hash bucket, the number of memory accesses can be reduced, and in some cases, only one memory access is needed, greatly improving the reading performance.

[0014] In some possible implementation manners, the first hash group further includes an overflow area. The processor can further determine whether the number of data written in the first hash bucket reaches a threshold. If yes, the processor can write the first data into the overflow area.

[0015] In the method, to ensure cache line alignment and fine-grained concurrent locking, the size of the hash bucket can be fixed, and considering the data access efficiency, the number of data that can be written into the hash bucket is limited. When the number of data written in the hash bucket reaches a threshold, the data can be written into an overflow area, so that the concurrent performance and memory access efficiency are considered.

[0016] In some possible implementation manners, the processor can further compare the first data with the data written in the first hash group. If the first data is not repeated in the data written in the first hash group, the processor determines to write the first data; if the first data is repeated in the data written in the first hash group, the processor records an address of the first data, and the address points to the data written in the first hash group and repeated with the first data.

[0017] The method can realize deduplication by comparing the first data with the data already written in the first hash group, and not performing writing on the repeated data, thereby reducing the overhead of data interaction between the processor and the accelerator.

[0018] In some possible implementation manners, when the input data are all written into the hash group, the processor can construct deduplication data according to the data already written in the hash group to which the input data belong. Specifically, the processor can splice the data already written in the hash group, for example, splice the data already written in the hash bucket and the data already written in the overflow area, to obtain the deduplication data. The processor can also construct recovery data according to the start address of at least one hash bucket in the hash group to which the input data belong and the offset address of the input data in the hash bucket, and the recovery data is used to recover the input data.

[0019] In some possible implementation manners, when the total length of the data already written in at least one hash group in the hash group to which the input data belong is less than the target length, the processor can write padding data in the at least one hash group until the total length of the data already written in the hash group reaches the target length. The processor can construct deduplication data according to the data already written in the hash group after padding.

[0020] The method can adapt to the parameter requirements of the collective communication, for example, adapt to the parameter requirements of the static all to all communication, by supplementing when the total length of the data already written in the hash group is less than the target length.

[0021] In some possible implementation manners, the processor can receive a user-configured supplement mode. Accordingly, when the supplement mode is turned on, the processor can write padding data in at least one hash group. The method supports the user to decide whether to supplement according to the business requirements, and has high flexibility.

[0022] In some possible implementation manners, when the input data are all written into the hash group, the processor can determine the accelerator corresponding to the hash group according to the result of the first hash operation on the data in the hash group to which the input data belong, and then the processor can send the data in the hash group to the accelerator. In this way, the data can be allocated to the corresponding accelerator in the granularity of the hash group, and the efficiency of data allocation is improved.

[0023] In a second aspect, the present application provides a data processing apparatus. The apparatus comprises:

[0024] The batch module is configured to divide the input data of the artificial intelligence model into multiple batches for parallel processing.

[0025] a grouping module configured to, for first data in a first batch of the plurality of batches, perform a first hash operation, and determine a first hash group to which the first data belongs according to a result of the first hash operation, the first hash group corresponding to a first memory space in a memory;

[0026] a bucketing module configured to, for the first data, perform a second hash operation, and determine a first hash bucket to which the first data belongs according to a result of the second hash operation, the first hash bucket belonging to the first hash group, and the first hash bucket being located in a first region in the first memory space;

[0027] a write control module configured to, when writing the first data into the first region, lock the first hash bucket.

[0028] In some possible implementation manners, the first hash group corresponds to a first memory space in a memory of a processor, or the first hash group corresponds to a first memory space in a memory of a first accelerator, the first accelerator being configured to process the first data.

[0029] In some possible implementation manners, a size of the first hash bucket is equal to n times of a cache line size of a cache of a processor, or a size of the first hash bucket is equal to n times of a cache line size of a cache in the first accelerator, the n being a natural number greater than or equal to 1.

[0030] In some possible implementation manners, the first hash group further includes an overflow area, and the write control module is specifically configured to:

[0031] determine whether a number of data already written in the first hash bucket reaches a threshold value;

[0032] if yes, write the first data into the overflow area.

[0033] In some possible implementation manners, the write control module is further configured to:

[0034] compare the first data with data already written in the first hash group;

[0035] if the first data is not duplicated in the data already written in the first hash group, determine to write the first data, or if the first data is duplicated in the data already written in the first hash group, record an address of the first data, the address pointing to the data already written in the first hash group and duplicated with the first data.

[0036] In some possible implementation manners, the apparatus further includes:

[0037] The constructing module is configured to construct deduplication data according to the data written in the hash group to which the input data belongs.

[0038] The constructing module is further configured to construct recovery data according to a start address of at least one hash bucket in the hash group to which the input data belongs and an offset address of the input data in the hash bucket to which the input data belongs, the recovery data being used to recover the input data.

[0039] In some possible implementation manners, the constructing module is specifically configured to:

[0040] When a total length of the data written in at least one hash group to which the input data belongs is less than a target length, the at least one hash group is written with padding data until the total length of the data written in the hash group reaches the target length, and the deduplication data is constructed according to the data written in the hash group after padding.

[0041] In some possible implementation manners, the apparatus further includes:

[0042] The configuring module is configured to receive a user-configured completion mode.

[0043] The constructing module is specifically configured to:

[0044] When the completion mode is turned on, the at least one hash group is written with padding data.

[0045] In some possible implementation manners, the apparatus further includes:

[0046] The assigning module is configured to determine, when the input data is written in the hash group, an accelerator corresponding to the hash group according to a result of first hash operation on the data in the hash group to which the input data belongs.

[0047] The sending module is configured to send the data in the hash group to the accelerator.

[0048] In a third aspect, the present application provides a chip. The chip includes a processor and a communication interface, the communication interface is configured to obtain input data of an artificial intelligence model, and the processor is configured to execute computer readable instructions to perform the data processing method in the first aspect of the present application or any implementation manner of the first aspect.

[0049] In a fourth aspect, the present application provides a computing device. The computing device includes at least one processor and a plurality of accelerators. The at least one processor and the plurality of accelerators communicate with each other. The at least one processor is configured to execute instructions stored in at least one memory to perform the data processing method in the first aspect or any implementation manner of the first aspect.

[0050] In a fifth aspect, the present application provides a computer readable storage medium, having stored therein instructions which instruct a computing device to perform the data processing method according to the first aspect or any implementation manner of the first aspect.

[0051] In a sixth aspect, the present application provides a computer program product comprising instructions which, when executed on a computing device, cause the computing device to perform the data processing method according to the first aspect or any implementation manner of the first aspect.

[0052] On the basis of the implementation manners of the above aspects, the present application can be further combined to provide more implementation manners. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical method of the embodiments of the present application, the drawings required to be used in the embodiments will be briefly introduced as follows.

[0054] Fig. 1 is a hardware structure diagram of a distributed system provided by the present application;

[0055] Fig. 2 is a flowchart of a data processing method provided by the present application;

[0056] Fig. 3 is a flowchart of processing concurrent conflicts by using a spin lock provided by the present application;

[0057] Fig. 4 is a schematic diagram of storing data by using a hash bucket and a data overflow area provided by the present application;

[0058] Fig. 5 is a schematic diagram of supplementing deduplicated data provided by the present application;

[0059] Fig. 6 is a flowchart of a data processing method provided by the present application;

[0060] Fig. 7 is a flowchart of data preprocessing in a training recommendation model scenario provided by the present application;

[0061] Fig. 8 is a structural schematic diagram of a data processing apparatus provided by the present application. DETAILED DESCRIPTION

[0062] The terms “first” and “second” in the embodiments of the present application are only for the purpose of description, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features limited by “first” and “second” can explicitly or implicitly include one or more of the features.

[0063] Firstly, some technical terms involved in the embodiments of the present application are introduced.

[0064] Artificial intelligence (abbreviated AI), also known as machine intelligence, specifically refers to a technology that correctly interprets external data, learns knowledge from the data, and flexibly applies the learned knowledge from the data to achieve specific goals and tasks. Among them, the data-driven AI technology can include but is not limited to machine learning (ML) and deep learning. Machine learning is a class of algorithms that automatically analyze rules from data and use the rules to predict unknown data. Deep learning is an algorithm that uses artificial neural networks as an architecture to learn representations of data.

[0065] An AI model refers to a mathematical model constructed based on an AI technology, for example, a mathematical model constructed by a machine learning algorithm or a deep learning algorithm. The AI model can be applied to various scenarios, including but not limited to search scenarios and recommendation scenarios. Among them, the search scenario is that a user provides a description of the user's intention to obtain content, and a system (such as a search system) returns results matching the description to the user, and the recommendation scenario is that a system (such as a recommendation system) actively recommends content to a user. Because there is no restriction of user-provided information, the recommendation scenario is more diverse, for example, the recommendation scenario can include personalized recommendation of the home page of each application (APP) and associated recommendation of a selection page, wherein the associated recommendation includes recommendation based on associated items and recommendation based on associated users. Accordingly, the AI model can be a search model or a recommendation model.

[0066] When training an AI model, the selection of features has a great impact on the effect or performance of the AI model. Features can be divided into sparse features and dense features. Sparse features are discrete features with a large number of possible values, which can usually be represented by a high-dimensional feature vector, most of the elements of which take the value 0. Taking search and recommendation scenarios as examples, sparse features can include user identifiers (user IDs), user attributes, item IDs, and item attributes. Among them, users and items can be represented as sparse features. For users, sparse features can include personal information, interests, and historical behavior. For items, sparse features can include attributes, tags, and categories of items. The above sparse features can be used to capture the relationship and similarity between users and items, thereby performing personalized recommendation and ranking.

[0067] Considering that the AI model needs to process a large amount of data or query requests, a distributed system can be used for model training and model inference. The distributed system can split the parameters of the AI model to multiple accelerators, for example, the distributed system can split the parameters of the AI model into multiple parts, and each part is stored in a different accelerator. Among them, the accelerator can be different accelerators of the same computing device, or accelerators of different computing devices, so as to realize distributed computing and parallel processing, improve the performance and scalability of the system, and fully utilize resources to improve resource utilization. At the same time, the distributed system uses distributed storage to ensure the reliability and high availability of data, reducing the risk of single point failure. Further, the distributed system can also realize parallel processing and distributed training of data. For example, the processes on each accelerator can process a part of the data in parallel, thereby improving the training speed and performance of the distributed system.

[0068] When training the AI model, the distributed system can issue training data, such as sparse features, to different accelerators through inter-card communication. Considering that the size of the sparse features is relatively large, and there is a certain degree of repetition in each batch of sparse features, the distributed system can preprocess the sparse features, such as deduplication, to effectively reduce the communication overhead of the sparse features.

[0069] Currently, the industry has proposed a deduplication method based on a hash table. A hash table, also known as a hash table, is a data structure that stores data in the form of key-value. Storing data in the form of key-value means that any key can uniquely correspond to a certain location in memory, and only the key to be searched needs to be input to quickly find the value corresponding to the key. In the present application, the host side of the distributed system can use the non-repetition of keys in the hash table to deduplicate the sparse features, wherein the value in the hash table is used to record the location of the current sparse feature. It should be noted that the processor of the host side of the distributed system, such as a central processing unit (CPU), can also construct a recovery vector based on the hash table, and the recovery vector can be combined with the deduplicated feature vector to recover the feature vector before deduplication.

[0070] However, when the above method uses a parallel mechanism to deduplicate data with a certain degree of repetition, such as sparse features, if a concurrent conflict occurs, the process or thread that triggers the insertion operation first will lock the entire hash table, and during the period when the hash table is locked, other processes or threads cannot insert data into the hash table. In the case of a batch of sparse features including a large amount of data, the processor resources are difficult to be fully utilized, resulting in long deduplication time, which affects the overall throughput of the system.

[0071] Therefore, the present application provides a data processing method. The method can be applied to a distributed system, which can be a heterogeneous computing device or a heterogeneous computing device cluster formed by multiple heterogeneous computing devices. The distributed system can include a processor and an accelerator. The processor, which can be a CPU, is located on the host side, and the accelerator, which can be a neural network processing unit (NPU), a graphics processing unit (GPU), or a tensor processing unit (TPU), is located on the device side. The accelerator is used in conjunction with the processor to complete the training of an AI model.

[0072] Specifically, the processor can process multiple batches of input data of the AI model in parallel. The data in different batches has a certain degree of repetition. For example, in a recommendation scenario or a search scenario, the sparse features such as personal information, hobbies, and historical behavior of adjacent batches have a certain degree of repetition, or the item labels and item categories of adjacent batches have a certain degree of repetition. For first data in a first batch of the multiple batches, the processor can perform a first hash operation, and determine a first hash group to which the first data belongs according to a result of the first hash operation. The first hash group corresponds to a first memory space in the memory. Then the processor performs a second hash operation on the first data, and determines a hash bucket to which the first data belongs according to a result of the second hash operation. The first hash bucket belongs to the first hash group, and the first hash bucket is located in a first region in the first memory space. When writing the first data into the first region, the processor can lock the first hash bucket.

[0073] The method introduces multiple layers of hash, specifically, on the basis of the hash group, a hash bucket with a finer granularity is also set. When the parallel mechanism is used for data processing to improve the processing efficiency, if a conflict occurs, only the corresponding hash bucket can be locked, and other hash buckets in the hash group or other hash groups still support data insertion, reducing the resource loss caused by concurrent conflicts, fully utilizing the processor resources, shortening the data processing time, improving the data processing efficiency, guaranteeing the overall throughput of the system, and improving the system performance.

[0074] In order to make the technical solutions of the present application clearer and easier to understand, the architecture of the distributed system of the present application is introduced below in combination with the drawings. The data processing method of the present application can be executed by the distributed system. The distributed system includes a processor and a plurality of accelerators. The processor can include but is not limited to a CPU, and the accelerator can be a GPU, NPU or TPU, etc. a computing card for accelerating calculation, also known as an acceleration card. In some examples, the distributed system can be a heterogeneous computing device including a plurality of computing cards, in which case the distributed system is a single-machine multi-card architecture. In other examples, the distributed system can be a computing device cluster including a plurality of heterogeneous computing devices. Among them, the host side of the computing device can include one CPU, or a plurality of CPUs. In this case, the distributed system can be a multi-machine multi-card architecture. Single-machine multi-card or multi-machine multi-card can be represented as x-machine x-card. It should be noted that "machine" in the above x-machine x-card represents host, and "card" represents computing card including NPU, GPU, etc.

[0075] In order to make the technical solutions of the present application clearer and easier to understand, the architecture of the distributed system of the present application is introduced below in combination with the drawings. The data processing method of the present application can be executed by the distributed system. The distributed system includes a processor and a plurality of accelerators. The processor can include but is not limited to a CPU, and the accelerator can be a GPU, NPU or TPU, etc. a computing card for accelerating calculation, also known as an acceleration card. In some examples, the distributed system can be a heterogeneous computing device including a plurality of computing cards, in which case the distributed system is a single-machine multi-card architecture. In other examples, the distributed system can be a computing device cluster including a plurality of heterogeneous computing devices. Among them, the host side of the computing device can include one CPU, or a plurality of CPUs. In this case, the distributed system can be a multi-machine multi-card architecture. Single-machine multi-card or multi-machine multi-card can be represented as x-machine x-card. It should be noted that "machine" in the above x-machine x-card represents host, and "card" represents computing card including NPU, GPU, etc.

[0076] Referring to a hardware structure diagram of a distributed system shown in FIG. 1, the distributed system can be a single-machine multi-card architecture server 10. The server 10 includes a processor on the host side and a plurality of accelerators on the device side. Among them, the processor on the host side is also connected with a memory (or called host memory), and the memory can be a dual in-line memory module (Dual In-line Memory Module, DIMM). Specifically, the DIMM can be a double data rate (double data rate, DDR) type, for example, the memory can be a DDR4 DIMM. In the example of FIG. 1, the host side can include 4 processors and 4 DDR4 DIMM groups, each processor is connected with 1 DDR4 DIMM group, and each DDR4 DIMM group includes 8 DDR4 DIMMs. Among them, the plurality of processors on the host side can be connected to form a hydra mesh.

[0077] Optionally, the host side further includes an interface, which can include a storage interface or a network interface. The storage interface can include, but is not limited to, a serial advanced technology attachment (SATA) interface, a new generation non-volatile memory (NVMe) interface, and the network interface can include, but is not limited to, a gigabit Ethernet (GE) interface. The host side can further include a memory. The memory can include a SATA-supported memory or an NVMe-supported memory, such as a SATA-supported mechanical hard disk or an NVMe-supported solid state drive (SSD).

[0078] The device side includes a plurality of accelerators, such as accelerator 1, accelerator 2, …, and accelerator m, where m can be an integer greater than or equal to 1. Further, the accelerators can also include a memory. In order to distinguish from the host memory, the memory in the accelerator can be referred to as a video memory. The video memory can be used to store input data of an AI model to facilitate AI calculation. The video memory can be a high bandwidth memory (HBM).

[0079] The processor of the host side and the m accelerators of the device side can be connected through a bus. In some examples, the logical address space of the video memory (such as HBM) in the accelerator can be mapped to the host side and managed by the host side. For example, in some cases, the processor of the host side can access the video memory of the accelerator based on the mapped logical address.

[0080] It should be noted that FIG. 1 is only used as an example of a single-machine multi-card architecture and does not constitute a limitation on the technical solutions of the present application. In other possible implementations of the present application, the distributed system can also be a multi-machine multi-card architecture, for example, a computing device cluster formed by a plurality of single-machine multi-card heterogeneous computing devices.

[0081] Based on the foregoing distributed system, the present application provides a data processing method. The following will be described in detail with reference to the accompanying drawings.

[0082] Referring to the flowchart of a data processing method shown in FIG. 2, the method can be executed by the host side of the distributed system, for example, by the processor of the host side. The method can include the following steps:

[0083] S202, the processor divides a plurality of input data of an AI model into a plurality of batches for parallel processing.

[0084] The input data of the AI model can be a data set including a large amount of data. In the AI model training scenario, the input data can be sparse features describing a user or an item, which can include ID type features such as user ID, item ID, video ID, and article ID. The sparse features can also include personal information, interests, and historical behaviors of the user, or the sparse features can include labels and categories of the item.

[0085] The processor can divide the input data into multiple batches according to a batch size set by the user and process the batches in parallel to improve data processing efficiency. The batch size can be set according to experience, for example, 64 or 1024. According to the size of the data set and the computing scale of the distributed system, the user can set different batch sizes.

[0086] In S204, the processor performs a first hash operation on the first data in the first batch, and determines a first hash group to which the first data belongs according to a result of the first hash operation.

[0087] Hash operation is a process of outputting a hash value through a hash function according to an input of an arbitrary length. The hash function can include but is not limited to a direct modulo function and a remainder function. The direct modulo function can be Max) mod size, where Max can be a large number and size can be the number of hash groups. The remainder function can be H(k) = k mod p, where k can be a key value and p can be the number of hash groups.

[0088] In this application, the number of hash groups can be determined according to the number of accelerators participating in data processing. In some examples, the number of hash groups can be equal to the number of accelerators in the distributed system, or the number of hash groups can be equal to k times the number of accelerators, where k is greater than 1. When the number of hash groups is k times the number of accelerators, each accelerator can process data in multiple hash groups.

[0089] For the first data in the first batch, the processor can perform a first hash operation with the number of hash groups as a parameter to obtain a result of the first hash operation. The result of the first hash operation can be a remainder output by the direct modulo function or the remainder function. The processor can determine the first hash group to which the first data belongs based on a mapping relationship between the result of the first hash operation and the hash groups. For example, the number of hash groups is 8, the result of the first hash operation is 0 to 7, and the hash groups numbered group0 to group7 are respectively corresponding. If the result of the first hash operation on the first data is 1, the first hash group to which the first data belongs is group1.

[0090] The first hash group corresponds to a first memory space in memory. In some examples, the first memory space can be a first memory space in memory (host memory) of a processor. In other examples, the first memory space can also be a first memory space in memory of the first accelerator. Further, considering read-write performance, a plurality of hash groups can correspond to a continuous memory space in memory, for example, a continuous memory space in memory of the processor.

[0091] In S206, the processor performs a second hash operation on the first data, and determines a first hash bucket to which the first data belongs according to a result of the second hash operation.

[0092] A hash bucket (also referred to as a bucket for short) is a basic unit for storing data in a hash table, also referred to as a hash slot or a storage bucket. In a hash table, a hash function maps a key to a corresponding hash bucket. Accordingly, when searching for data, the hash bucket can be determined based on the hash function, and then the hash bucket is searched. In this way, the hash table uses the structure of the hash function and the hash bucket to achieve efficient data storage and retrieval.

[0093] In the present application, a plurality of hash buckets are provided in each hash group. In specific implementation, the number of hash buckets in different hash groups can be the same, for example, the number of hash buckets in a plurality of hash groups can be uniformly set to 256. Considering the possibility of uneven data distribution, the number of hash buckets in different hash groups can also be different. For example, a hash group to which data with a high distribution ratio belongs can be provided with more hash buckets, and a hash group to which data with a low distribution ratio belongs can be provided with fewer hash buckets.

[0094] In order to improve read-write performance, the present application designs hash buckets aligned with cache lines (CacheLine). CacheLine is a cache organization method in a computing device. Cache is a hardware mechanism for improving computer access speed, which copies recently accessed data to a faster cache according to the principle of locality, so that the data can be obtained faster the next time it is accessed. Cache is composed of a plurality of CacheLines, and each CacheLine can store one or more data blocks.

[0095] Take the Cache Line of CPU as an example. The cache line of CPU is a part of memory hierarchy, which is used to transfer data between CPU cache and main memory. Cache line is the minimum unit of cache and also the minimum data block that the processor reads data from main memory to cache or writes data from cache to main memory. The cache line size can be determined by hardware design. For ease of calculation, the cache line size can usually be a power of 2 size, such as 32 bytes, 64 bytes or 128 bytes. CacheLine usually consists of two parts: Tag and Data. The tag is used to identify the address range of the data stored in the cache line, while the data is the actual data content stored. When a computing device needs to read or write a certain memory address, the computing device (for example, the CPU in the computing device) first checks whether there is a cache line corresponding to the memory address in the cache. If it exists, it is called cache hit (Cache Hit), and the data can be read or written directly from the cache, because the cache access speed is faster than the main memory access speed, which can improve the read-write performance. If it does not exist, it is called cache miss (Cache Miss), and the data needs to be read or written from the main memory and copied to the cache.

[0096] The cache line size is an important parameter, usually in bytes. Larger cache line can store more data, reduce the number of cache misses, and improve the efficiency of data access. Especially for access patterns with spatial locality and temporal locality, the efficiency of data access can be greatly improved through cache line. In mainstream computer architecture, CPU reads data from memory not in single byte or single word, but in a cache line (usually 64 bytes or 128 bytes). If the data can be completely stored in a cache line, the CPU only needs to access the cache once when reading this data, without multiple accesses. Based on this, the present application adopts the cache line alignment way in the design of the data structure of the hash bucket. For example, the size of the hash bucket can be set to equal to the cache line size, so that the CPU can read the data of a hash bucket by accessing the cache once, improving the efficiency of data access.

[0097] For ease of understanding, an example of a first hash bucket to which the first data belongs is illustrated. The size of the first hash bucket can be n times the cache line size of the cache of the processor (such as a CPU), or the size of the first hash bucket can be n times the cache line size of the cache in the first accelerator. Wherein n is a natural number greater than or equal to 1. The first hash bucket is located in a first region of the first memory space. Other hash buckets in the first hash group are located in other regions of the first memory space. It should be noted that when different hash groups correspond to the memory spaces of different accelerators, the sizes of the hash buckets in different hash groups can be n times the cache line size of the cache in the corresponding accelerator. For example, the size of the hash bucket in the second hash group can be n times the cache line size of the cache in the second accelerator.

[0098] Similar to determining the hash group to which the first data belongs, the processor can perform a second hash operation on the first data, and according to the mapping relationship between the result of the second hash operation and the hash bucket, determine the first hash bucket to which the first data belongs. Wherein the second hash operation can be a hash operation with the number of hash buckets as a parameter. The result of the second hash operation can be the remainder output by the direct modulo function or the remainder function. For example, the number of hash groups is 256, the result of the second hash operation is 0 to 255, which respectively corresponds to the hash buckets numbered bucket0 to bucket255, and if the result of the second hash operation on the first data is 5, then the first hash bucket to which the first data belongs is bucket5.

[0099] S208, when writing the first data into the first region, the processor locks the first hash bucket.

[0100] Considering efficiency, the processor can perform parallel processing on the input data, for example, the processor can start multiple threads to concurrently deduplicate the input data of the AI model. When multiple threads concurrently deduplicate the input data, there can be a case where multiple threads insert data with the same second hash operation result into the same hash bucket, thereby causing a concurrent conflict. Therefore, when writing the first data into the first region, the processor can lock the first hash bucket to solve the concurrent conflict. During the period when the first hash bucket is locked, when new data needs to be written into the first region (inserted into the first hash bucket), the new data can be written when the processor successfully writes the first data and unlocks the first hash bucket.

[0101] The processor locking the first hash bucket can update a lock variable of the first hash bucket to a first value, and the first value indicates that the lock state is in a locked state. The lock of the first hash bucket can be a spin lock based on compare and swap (CAS). The spin lock is a kind of lock for multi-thread synchronization, and the CAS-based spin lock is an optimistic lock, which optimistically considers that concurrent conditions are not serious, and a thread can repeatedly check whether a lock variable is available. Since the thread remains in execution in this process, it is a kind of busy waiting. Once the spin lock is obtained, the thread will keep the lock until the spin lock is explicitly released. The spin lock avoids scheduling overhead of process context, so the thread blocking time is very short.

[0102] For ease of understanding, the application also provides an example for illustration. As shown in FIG. 3, each hash bucket is provided with a spin lock. When two data with the same second hash operation result are inserted into the same hash bucket, for example, the data with the second hash operation result of 0 are simultaneously inserted into the hash bucket 0 (such as bucket 0), the thread repeatedly checks whether the lock variable 0 (such as the spin lock 0, denoted as spin lock 0) of the bucket 0 is available. If the thread 1 obtains the spin lock 0, the hash bucket can be locked, and after the storage is completed, the thread 1 can release the spin lock 0. Compared with the traditional read-write lock, the application uses a fine-grained lock for data storage, only needs a short waiting time, shortens the blocking time, and reduces the performance loss caused by concurrent conflicts.

[0103] Based on the above description, the data processing method of the application can determine the hash group to which the data belongs by performing a first hash operation on the data, and then perform a second hash operation to determine the first hash bucket to which the data belongs. In this way, when the data is written into the memory area where the hash bucket is located, the hash bucket can be locked by using a fine-grained lock, without using a traditional read-write lock to lock the entire hash table, thereby reducing the performance loss caused by concurrent conflicts and improving system performance.

[0104] In order to ensure cache line alignment and fine-grained concurrent lock, the size of the hash bucket usually needs to be fixed. Considering the efficiency of data access, the number of data stored in the hash bucket should not be too much, for example, the number of data stored in the hash bucket should not exceed a threshold (also referred to as the width of the hash bucket, denoted as width). In some cases, the data distribution of the input data is not uniform, and there can be more data belonging to the same hash bucket. Therefore, the application also designs an overflow area (overflow map) to balance the concurrent performance and memory access efficiency.

[0105] With the first hash group as an example, the first hash group further includes an overflow area. The processor can compare the first data with the data already written in the first hash group. If the first data is not duplicated with the data already written in the first hash group, the processor can determine to write the first data. If the first data is duplicated with the data already written in the first hash group, the processor can not write the first data, and record an address of the first data, where the address of the first data points to the data already written in the first hash group and duplicated with the first data. In other words, in the case that the first data is duplicated with the data already written in the first hash group, the processor can establish a mapping relationship between the first data and the duplicated data already written in the first hash group by recording the address.

[0106] In the case that it is determined to write the first data, the processor can determine whether the number of the data already written in the first hash bucket reaches a threshold. If yes, the processor can write the first data into the overflow area. If no, the processor can write the first data into the first hash bucket. Considering that the data with the same second hash operation result can be written into the hash bucket or the overflow area, based on this, the processor can compare the first data with the data already written in the first hash bucket or the overflow area, so as to reduce the deduplication calculation amount, shorten the deduplication time, and improve the deduplication efficiency.

[0107] For ease of understanding, the application further provides an example for illustration.

[0108] Referring to a flowchart of a data processing method shown in FIG. 4, the processor can set the number of hash groups according to the number of processors participating in parallel calculation, for example, set eight hash groups numbered group 0 to group 7, and set 256 hash buckets including hash bucket 0 to hash bucket 255 in each hash group. In order to facilitate data query, the width of the hash bucket can be set to 6, that is, the hash bucket can store up to 6 data. Each hash group further includes an overflow area. The overflow area can store overflow data of the hash buckets in the hash group. It should be noted that the overflow area can store the overflow data in a mapping manner, for example, the overflow area can store the overflow data and the offset address of the overflow data in the overflow area, so that the overflow data of a specified hash bucket can be quickly located subsequently. In the example of FIG. 4, group 0 to group 7 can correspond to a continuous memory space in the memory of the processor. For example, group 0 can correspond to a first memory space, and group 1 can correspond to a second memory space. Each hash bucket in the hash group can correspond to an area in the memory space, for example, the first hash bucket can correspond to a first area of the first memory space.

[0109] Specifically, the processor can divide the input data of the AI model into multiple batches for parallel processing. For the first batch of first data, the processor can perform a first hash operation on the first data, and determine a first hash group to which the first data belongs according to the result of the first hash operation. For example, the result of the first hash operation on a data is 0, and it can be determined that the hash group to which the data belongs is group 0. The result of the first hash operation on another data is 1, and it can be determined that the hash group to which the data belongs is group 1. The processor can perform a second hash operation on the first data, and determine a first hash bucket to which the first data belongs according to the result of the second hash operation. Taking the data belonging to group 0 as an example, the result of the second hash operation on a data is 0, and it is determined that the hash bucket to which the data belongs is hash bucket 0. The result of the second hash operation on another data is 1, and it is determined that the hash bucket to which the data belongs is hash bucket 1.

[0110] The processor can compare the first data with the data already written in the first hash group. If it is not repeated, the first data is written. If it is repeated, the first data is not written, and the address of the first data is recorded, for example, in the restored data. When the first data is written, the processor can determine whether the number of data already written in the first hash bucket reaches 6. If yes, the first data is written to the overflow area. If no, the first data is written to the first hash bucket. In the example of FIG. 4, the processor writes the data with the result of the second hash operation as 0 to hash bucket 0. When the number of data already written in hash bucket 0 reaches 6, the remaining data with the result of the second hash operation as 0 is written to the overflow area.

[0111] Further, when the input data has been written to the hash group, the processor can construct deduplicated data according to the data already written in the hash group to which the input data belongs. The processor can splice the data already written in each hash bucket in the hash group, the data already written in the overflow area, to obtain the deduplicated data. As shown in FIG. 4, for hash group 0, the processor can splice the data already written in hash bucket 0 (such as data 0 to data 5 in hash bucket 0), the data already written in hash bucket 1 (such as data 0 to data 3 in hash bucket 1), the data already written in hash bucket 255, and the data already written in the overflow area (such as overflow data of hash bucket 0, etc.), to obtain the deduplicated data. When splicing, the processor can take the spliced data as an element of a vector to obtain the deduplicated data in the form of a vector, also known as a deduplicated vector, denoted as unique vectors.

[0112] Next, the process of constructing the restore data is described. The processor can also construct the restore data from the start address of at least one hash bucket in the hash group to which the input data belongs and the offset address of the input data in the hash bucket to which the input data belongs, the restore data being used to restore the input data. The start address can be the start offset address, denoted as replace offset, and the offset address of the input data in the hash bucket to which the input data belongs can be represented by the index of the input in the hash bucket, denoted as data in bucket index. For the data in the hash bucket, the processor can obtain the address of each data in the hash bucket, such as the offset address of each data, from the start address and the offset address of the hash bucket. For example, for data i (i is an integer from 0 to 5) in hash bucket 0, the address of data i in hash bucket 0, denoted as bucket 0data i offset, can be obtained from the replace offset of hash bucket 0 and the data in bucket index of data i, where bucket 0data i offset = replace offset + data in bucket index. Similarly, the processor can obtain the address of the overflow data from the start address of the overflow area and the offset address of the overflow data in the overflow area. In this way, the processor can obtain the address of each data in the deduplicated data, which can be the offset address, or the offset, and the processor can splice the offset addresses. Then the processor can insert the offset address of the duplicate data into the spliced offset address according to the input data to obtain the restore data. Similar to the deduplicated data, the processor can splice or insert the offset address as an element of a vector to obtain the restore data in the form of a vector, also known as a restore vector.

[0113] In some possible implementations, the processor can construct the restore data in a parallel manner, specifically, in a multi-threaded manner to construct the restore vector. Specifically, each thread obtains a segment of the input data, then queries the offset of each segment of data in parallel, and then the processor splices the offset of each data queried by each thread in a block of continuous addresses to form the restore vector.

[0114] When all the input data are written into the hash groups, the processor can further determine the accelerator corresponding to the hash group according to the result of the first hash operation on the data in the hash group to which the input data belongs. Specifically, when the first hash operation result and the accelerator are in one-to-one correspondence, the processor can obtain the accelerator corresponding to the hash group according to the result of the first hash operation on the data in the hash group and the mapping relationship between the first hash operation result and the accelerator. For example, if the result of the first hash operation on the data in a hash group is 1, the accelerator corresponding to the hash group can be accelerator 1. The processor can send the data in the hash group to the accelerator corresponding to the hash group. For example, the processor can send the deduplicated data constructed by the data already written in the hash group to the accelerator corresponding to the hash group.

[0115] The data distribution in different hash groups can be different, for example, the difference in data repetition rate can be large, so that the lengths of the deduplicated data obtained after deduplication of different hash groups can be different. For example, the repetition rate of hash group 1 is 80%, and the repetition rate of hash group 2 is 60%. If the number of data belonging to hash group 1 and hash group 2 in the input data is 100, respectively, the length of the deduplicated data obtained after deduplication of data group 1 can be 100-100*80%=20, and the length of the deduplicated data obtained after deduplication of data group 2 can be 100-100*60%=40.

[0116] In the model training scenario, the accelerators can interact with each other through collective communication, and in order to better adapt to the parameter requirements of the collective communication, such as the parameter requirements of static all to all communication, the CPU can complete the deduplicated data (such as a deduplicated vector). Specifically, when the total length of the data already written in at least one hash group in the hash group to which the input data belongs is less than the target length, the processor can write padding data in the at least one hash group until the total length of the hash group reaches the target length. The padding data can be set according to experience, for example, it can be all-zero data. Then the processor can construct the deduplicated data according to the data already written in the hash group after padding.

[0117] Further, considering that the collective communication usually requires continuous addresses, the processor can also splice the data already written in each hash group to obtain the deduplicated data.

[0118] For ease of understanding, the following is illustrated in conjunction with the accompanying drawings. In this example, as shown in FIG. 5, the deduplicated data can include data that has been written in a plurality of hash groups, such as data that has been written in hash group 0, data that has been written in hash group 1, …, and data that has been written in hash group 7. Among them, the data that has been written in hash group i can include data in the input data of the AI model that has been written in hash group i, and when the total length of the data in the input data of the AI model that has been written in hash group i is less than the target length, padding data such as padding value is also written in hash group i, denoted as padding val. For ease of calculation, the above-mentioned padding value can take a value of several 0s.

[0119] Among them, the data processing method of the present application can support the user to decide whether to complete according to the business demand. Specifically, the processor can receive the user configured completion mode, and the completion mode can be turned on or off (not turned on). Correspondingly, the processor can detect whether the length of the data that has been written in the hash group is less than the target length when the user configured completion mode is turned on, and if so, the processor can write padding data in the hash group until the length of the padded data is equal to the target length.

[0120] Further, considering that the data distribution in different data groups can be different, the present application also supports configuring the initialization parameters of the processor for data processing to achieve ideal execution efficiency and running effect. Specifically, the processor can support the user to configure at least one of the thread number constraint, the completion mode, the memory pre-estimated usage, the initial allocated memory, or the number of hash buckets. Among them, the thread number constraint can include the maximum thread number and / or the minimum thread number. The processor can perform model data processing according to the above-mentioned initialization parameters configured by the user.

[0121] Specifically, the processor supports processing the data in the data group through multiple threads. Among them, the processor can receive the user configured thread number constraint, which includes the minimum thread number and / or the maximum thread number. When the processor obtains the input data of the AI model, the processor can determine the number of threads for processing the input data according to the batch size, wherein the number of threads satisfies the thread number constraint. For example, the number of threads can be greater than or equal to the minimum thread number and less than or equal to the maximum thread number.

[0122] Further, the present application also supports adaptive adjustment of parameters according to user configuration, data size, etc. to achieve the best running effect. Among them, the adjustment of the number of threads can refer to the related content description of determining the number of threads in the initialization phase.

[0123] The adaptive adjustment of the size of the hash bucket and the adaptive adjustment of the memory are described in detail below.

[0124] Specifically, the processor can receive a memory expected usage configured by a user, and then pre-allocate memory according to the memory expected usage. Wherein, the specification of the pre-allocated memory can be equal to the memory expected usage, or slightly larger than the memory expected usage. The processor can apply for allocating new memory if it detects insufficient memory during processing of data in a data group. Wherein, the processor can apply for allocating new memory in the case of suspension or failure of the current data group processing.

[0125] The processor can also determine at least one of the number of input data (data size) or the repetition rate, and then adjust the size of the hash bucket according to at least one of the data size or the repetition rate. For example, when the repetition rate is low, the processor can increase the size of the hash bucket to store more data.

[0126] The above describes the data processing method of the application from the perspective of the processor. Next, the data processing method of the application is described from the perspective of the accelerator.

[0127] Referring to a flowchart of a data processing method shown in FIG. 6, the method comprises the following steps:

[0128] S602, the first accelerator receives the deduplication data and the recovery data sent by the processor.

[0129] Wherein, the first accelerator can correspond to the first hash group, and the deduplication data sent by the processor to the first accelerator can be constructed according to the data already written in the first hash group. The data already written in the first hash group is the non-duplicate data belonging to the first hash group in the input data of the AI model. The recovery data can include the addresses of each data in the input data of the AI model, used to recover the input data of the AI model.

[0130] S604, the first accelerator receives the deduplication data sent by the second accelerator through the collective communication.

[0131] Similar to the first accelerator, the second accelerator can receive the deduplication data sent by the processor, and the deduplication data sent by the processor to the second accelerator can be constructed according to the data already written in the second hash group. The accelerators can transmit their respective deduplication data through the collective communication, based on which the first accelerator can receive the deduplication data sent by the second accelerator through the collective communication.

[0132] Wherein, S602 and S604 can be executed in parallel or in sequence, and the present embodiment does not limit the order of execution.

[0133] S606, the first accelerator recovers the input data of the AI model through the recovery data according to the deduplication data sent by the processor and the deduplication data sent by the second accelerator through the collective communication.

[0134] The first accelerator can splice the deduplicated data received from the processor and the deduplicated data received from the second accelerator to obtain full-amount deduplicated data. Then, the first accelerator can obtain data pointed by each address from the full-amount deduplicated data according to the full-amount deduplicated data and the addresses of the input data in the recovery data, thereby recovering the input data of the AI model.

[0135] It should be noted that in some possible implementations, the processor can also construct the full-amount deduplicated data, and the first accelerator can receive the full-amount deduplicated data and perform a recovery operation in combination with the recovery data, thereby recovering the input data of the AI model.

[0136] Further, the first accelerator can perform model training according to the recovered input data of the AI model. The first accelerator and the second accelerator can perform parallel training of the AI model through a parallel mechanism. For example, the first accelerator and the second accelerator can perform parallel training of the AI model in a model parallel manner.

[0137] Based on the foregoing description, the present application provides a data processing method. In the method, the first accelerator can receive deduplicated data and recovery data sent by the processor, and receive deduplicated data sent by the second accelerator through set communication. The full-amount deduplicated data can be obtained by aggregating the deduplicated data of different accelerators. The complete input data can be recovered from the full-amount deduplicated data and the recovery data. The method can reduce the communication overhead between the processor and the accelerator by introducing a hash group for data deduplication and combining the recovery data for recovery, while ensuring recovery. The accelerators can interact with each other based on set communication, thereby improving efficiency.

[0138] Next, the data processing method of the present application will be described in detail in combination with a specific application scenario. The method is used in the data preprocessing stage of a recommendation system. The recommendation system uses a recommendation model to perform content recommendation. The recommendation model can be trained using a large number of sparse features. Before the sparse features are subjected to set communication, feature deduplication and recovery vector construction and other preprocessing need to be performed. The data processing method of the present application can efficiently complete related tasks.

[0139] Referring to a scene diagram of data preprocessing shown in FIG. 7, considering that the parameter scale of the recommendation model is relatively large, a computing device cluster can be used to train the recommendation model. The computing device cluster includes at least one computing device. In the example of FIG. 7, the computing device cluster includes a CPU on the host side and a plurality of NPUs on the device side. The plurality of NPUs includes an NPU numbered rank 0 and an NPU numbered rank 1. When training a large-scale recommendation model, data preprocessing is usually required. The data preprocessing can be performed by the host side of the computing device, for example, by the CPU on the host side.

[0140] The CPU on the host side obtains input data of the AI model, and performs parallel processing on the input data in batches. The input data can include ID type features, also known as feature IDs. The input data can include data processed by the NPU numbered rank 0 and data processed by the NPU numbered rank 1. The CPU on the host side can perform the aforementioned data processing method on the data in multiple batches to construct deduplicated data (for example, unique IDs) and recovery data (for example, recover IDs offset).

[0141] It should be noted that when the computing device cluster adopts a multi-machine multi-card architecture, the CPU on the host side can also perform collective communication, for example, IDs all to all communication, to obtain the full amount of deduplicated data and the corresponding recovery data processed by the NPU numbered rank 0, and the right amount of deduplicated data and the corresponding recovery data processed by the NPU numbered rank 1.

[0142] The unique IDs are usually sparse data, and in order to facilitate the calculation and optimization of the AI model, the sparse features can usually be converted into dense features. The dense features can be represented by a low-dimensional feature vector, most of the elements of which are non-zero values. The conversion of sparse features into dense features can be achieved by feature encoding or feature embedding, such as word embedding technology. The CPU can perform a gather operation to convert the sparse deduplicated data into dense deduplicated data, simply referred to as dense data. The dense data can be embedding data obtained by word embedding, also known as embedding weight. The gather is an operator used to return the elements of a tensor at a given index. Then, the first CPU and the second CPU can distribute the embedding weight to the NPU, and the NPU trains the recommendation model based on the embedding weight.

[0143] The NPU can also perform set communication between NPU, for example, all to all communication, to obtain the embedding weight corresponding to the deduplicated feature ID. Then, each NPU can also obtain the embedding weight corresponding to the restored feature ID through gather (such as tensorflow gather) according to the restored feature ID. The NPU can perform model training based on the embedding weight corresponding to the restored feature ID.

[0144] The above embodiments are exemplarily illustrated in the search or recommendation scenario. In other possible implementation manners of the embodiments of the present application, the data processing method of the present application can also be used in other scenarios requiring data deduplication.

[0145] Based on the foregoing data processing method, the present application provides a data processing apparatus. The data processing apparatus of the embodiments of the present application is introduced from the perspective of functional modularization in combination with the accompanying drawings.

[0146] Referring to a structural schematic diagram of a data processing apparatus shown in FIG. 8, the data processing apparatus 800 includes:

[0147] The batch module 802 is configured to divide the input data of the artificial intelligence model into multiple batches for parallel processing.

[0148] The grouping module 804 is configured to perform a first hash operation on first data in a first batch of the multiple batches, and determine a first hash group to which the first data belongs according to a result of the first hash operation, the first hash group corresponding to a first memory space in the memory.

[0149] The bucketing module 806 is configured to perform a second hash operation on the first data, and determine a first hash bucket to which the first data belongs according to a result of the second hash operation, the first hash bucket belonging to the first hash group, and the first hash bucket being located in a first region in the first memory space.

[0150] The write control module 808 is configured to lock the first hash bucket when the first data is written into the first region.

[0151] The batch module 802, the grouping module 804, the bucketing module 806 and the write control module 808 can be implemented by software or by hardware.

[0152] When implemented by software, the batching module 802, the grouping module 804, the bucketing module 806, and the write control module 808 can be an application running on a computing device (e.g., a CPU on a host side of the computing device). Taking the bucketing module 806 as an example, the application can be a computing engine. The application can also be provided to users in the form of a virtualization service. The virtualization service can include a virtual machine (VM) service, a bare metal server (BMS) service, or a container service. The VM service can be a service that provides a VM for users to use on demand by virtualizing a VM resource pool on multiple physical hosts through a virtualization technology. The BMS service is a service that provides a BMS for users to use on demand by virtualizing a BMS resource pool on multiple physical hosts. The container service is a service that provides a container for users to use on demand by virtualizing a container resource pool on multiple physical hosts. A VM is a virtual computer, i.e., a logical computer. A BMS is a high-performance computing service that can be scaled elastically, and has the same computing performance as a traditional physical machine and the feature of secure physical isolation. A container is a kernel virtualization technology that can provide lightweight virtualization to achieve the purpose of isolating user spaces, processes, and resources. It should be understood that the VM service, the BMS service, and the container service in the above virtualization service are only specific examples. In actual applications, the virtualization service can also be other lightweight or heavyweight virtualization services, which are not limited here.

[0153] When implemented by hardware, the batching module 802, the grouping module 804, the bucketing module 806, and the write control module 808 can include at least one processor or the like. Alternatively, the batching module 802, the grouping module 804, the bucketing module 806, and the write control module 808 can also be devices implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), and the like. The PLD can be implemented by a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0154] The batch module 802 can be configured to perform S202 in the foregoing method embodiments. The specific implementation can be described with reference to the related content of S202 in the embodiment shown in FIG. 2.

[0155] In some possible implementation manners, the first hash group corresponds to a first memory space in a memory of a processor, or the first hash group corresponds to a first memory space in a memory of a first accelerator used for processing the first data.

[0156] In some possible implementation manners, the size of the first hash bucket is equal to n times of a cache line size of a cache of a processor, or the size of the first hash bucket is equal to n times of a cache line size of a cache in the first accelerator, where n is a natural number greater than or equal to 1.

[0157] In some possible implementation manners, the first hash group further includes an overflow area, and the write control module 808 is specifically configured to:

[0158] determine whether a quantity of data written in the first hash bucket reaches a threshold value;

[0159] if yes, write the first data into the overflow area.

[0160] In some possible implementation manners, the write control module 808 is further configured to:

[0161] compare the first data with data written in the first hash group;

[0162] if the first data is not duplicated in the first hash group, determine to write the first data; or if the first data is duplicated in the first hash group, record an address of the first data, where the address points to data written in the first hash group and duplicated with the first data.

[0163] In some possible implementation manners, the apparatus 800 further includes:

[0164] a construction module 810 configured to, when the input data are all written into hash groups, construct deduplicated data according to data written in a hash group to which the input data belong;

[0165] The construction module 810 is further configured to construct recovery data according to a start address of at least one hash bucket in a hash group to which the input data belong and an offset address of the input data in the hash bucket, where the recovery data is used to recover the input data.

[0166] Similar to the bucketing module 806, the construction module 810 can be implemented by software or by hardware.

[0167] When implemented by software, the constructing module 810 can be an application program running on a computing device. The application program can also be provided to users in the form of a virtualization service, for example, provided to users in the form of a VM service, a BMS service, or a container service. When implemented by hardware, the constructing module 810 can include at least one processor, for example, at least one CPU, etc. Alternatively, the constructing module 810 can also be a device implemented by an ASIC, or a PLD, etc.

[0168] In some possible implementation manners, the constructing module 810 is specifically configured to:

[0169] When the total length of the data written in at least one hash group in the hash group to which the input data belongs is less than the target length, write padding data in the at least one hash group until the total length of the data written in the hash group reaches the target length; and construct deduplication data according to the data written in the hash group after padding.

[0170] In some possible implementation manners, the apparatus 800 further includes:

[0171] The configuring module 812 is configured to receive a user-configured completion mode.

[0172] The constructing module 810 is specifically configured to:

[0173] When the completion mode is turned on, write padding data in the at least one hash group.

[0174] Similar to the bucketing module 806, the configuring module 812 can be implemented by software or by hardware.

[0175] When implemented by software, the configuring module 812 can be an application program running on a computing device. The application program can be an interactive component providing a configuration interface. When implemented by hardware, the configuring module 812 can include at least one processor, for example, at least one CPU, etc. Alternatively, the configuring module 812 can also be a device implemented by an ASIC, or a PLD, etc.

[0176] In some possible implementation manners, the apparatus 800 further includes:

[0177] The allocating module 814 is configured to, when the input data has all been written in hash groups, determine the accelerator corresponding to a hash group according to a result of first hash operation on data in the hash group to which the input data belongs.

[0178] The sending module 816 is configured to send the data in the hash group to the accelerator.

[0179] The allocation module 814 and the sending module 816 can be implemented by software or by hardware.

[0180] When implemented by software, the allocation module 814 and the sending module 816 can be an application program running on a computing device. When implemented by hardware, the allocation module 814 can include at least one processor, such as at least one CPU, etc. Alternatively, the allocation module 814 can also be a device implemented by an ASIC or a PLD, etc., and the sending module 816 can be a bus device.

[0181] Based on the foregoing data processing method and data processing apparatus, the present application further provides a chip. The chip includes a processor and a communication interface, the communication interface is configured to obtain input data of an AI model, and the processor is configured to execute computer readable instructions to execute the foregoing data processing method and realize the functions of the foregoing data processing apparatus.

[0182] Further, the present application further provides a computing device. The computing device can be a heterogeneous computing device, and specifically can include at least one processor and a plurality of accelerators. The at least one processor can execute computer readable instructions to execute the foregoing data processing method and realize the functions of the foregoing data processing apparatus.

[0183] The embodiments of the present application further provide a computer readable storage medium. The computer readable storage medium can be any available medium or data storage device including one or more available media that a computing device can store. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk), etc. The computer readable storage medium includes instructions that instruct the computing device to execute the foregoing application applied to the data processing apparatus 800 for executing the data processing method.

[0184] The embodiments of the present application further provide a computer program product containing instructions. The computer program product can be software or a program product containing instructions that can run on a computing device or be stored in any available medium. When the computer program product runs on at least one computing device, the at least one computing device executes the foregoing model data processing method. The embodiments of the present application further provide a computer program product containing instructions. When the computer program product runs on at least one computing device, the at least one computing device executes the foregoing data processing method.

[0185] Finally, it should be noted that the above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the same; although the present application has been described in detail with reference to the foregoing examples, those of ordinary skill in the art should understand that the technical solutions recorded in the foregoing examples can still be modified, or some of the technical features can be replaced by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A data processing method, characterized by, The method comprises: The input data of the artificial intelligence model is divided into multiple batches and processed in parallel; For first data in a first batch of the multiple batches, a first hash operation is performed, and a first hash group to which the first data belongs is determined according to a result of the first hash operation, the first hash group corresponding to a first memory space in memory; A second hash operation is performed on the first data, and a first hash bucket to which the first data belongs is determined according to a result of the second hash operation, the first hash bucket belonging to the first hash group, and the first hash bucket being located in a first region in the first memory space; When the first data is written into the first region, the first hash bucket is locked.

2. The method of claim 1, wherein, The first hash group corresponds to a first memory space in memory of a processor, or the first hash group corresponds to a first memory space in memory of a first accelerator used for processing the first data.

3. The method according to claim 1 or 2, characterized in that, The size of the first hash bucket is equal to n times the cache line size of the cache of the processor, or the size of the first hash bucket is equal to n times the cache line size of the cache in the first accelerator, and the n is a natural number greater than or equal to 1.

4. The method according to any one of claims 1 to 3, characterized in that, The first hash group further comprises an overflow area, and the method further comprises: determining whether the number of data written in the first hash bucket reaches a threshold value; if yes, the first data is written into the overflow area.

5. The method according to any one of claims 1 to 4, characterized in that, The method further comprises: comparing the first data with data already written in the first hash group; if the first data does not repeat the data already written in the first hash group, it is determined to write the first data; if the first data repeats the data already written in the first hash group, the address of the first data is recorded, and the address points to the data already written in the first hash group which repeats the first data.

6. The method according to any one of claims 1 to 5, characterized in that, The method further comprises: when the input data has been written into the hash group, constructing deduplication data according to the data already written in the hash group to which the input data belongs; constructing recovery data according to the start address of at least one hash bucket in the hash group to which the input data belongs and the offset address of the input data in the hash bucket, the recovery data being used to recover the input data.

7. The method of claim 6, wherein, The constructing deduplication data according to the data already written in the hash group to which the input data belongs comprises: when the total length of the data already written in at least one hash group in the hash group to which the input data belongs is less than a target length, writing padding data in the at least one hash group until the total length of the data already written in the hash group reaches the target length; and constructing deduplication data according to the data already written in the hash group after padding.

8. The method of claim 7, wherein, The method further comprises: receiving a user-configured completion mode; the writing padding data in the at least one hash group comprises: when the completion mode is turned on, writing padding data in the at least one hash group.

9. The method according to any one of claims 1 to 8, characterized in that, The method further comprises: when the input data has been written into the hash group, determining an accelerator corresponding to the hash group according to a result of a first hash operation on data in the hash group to which the input data belongs; sending the data in the hash group to the accelerator.

10. A data processing apparatus, characterized by, The device comprises: a batch module for processing input data of an artificial intelligence model in batches; a grouping module for performing a first hash operation on first data in a first batch of the batches, and determining a first hash group to which the first data belongs according to a result of the first hash operation, the first hash group corresponding to a first memory space in a memory; a bucketing module for performing a second hash operation on the first data, and determining a first hash bucket to which the first data belongs according to a result of the second hash operation, the first hash bucket belonging to the first hash group, and the first hash bucket being located in a first area in the first memory space; a write control module for locking the first hash bucket when the first data is written into the first area.

11. The apparatus of claim 10, wherein, The first hash group corresponds to a first memory space in a memory of a processor, or the first hash group corresponds to a first memory space in a memory of a first accelerator used for processing the first data.

12. The apparatus of claim 10 or 11, wherein, The size of the first hash bucket is equal to n times of a cache line size of a cache of a processor, or the size of the first hash bucket is equal to n times of a cache line size of a cache in the first accelerator, the n being a natural number greater than or equal to 1.

13. The apparatus of any one of claims 10 to 12, wherein, The first hash group further comprises an overflow area, and the write control module is specifically configured to: determine whether a quantity of data written into the first hash bucket reaches a threshold value; if yes, write the first data into the overflow area.

14. The apparatus of any one of claims 10 to 13, wherein, The write control module is further configured to: compare the first data with data written into the first hash group; if the first data is not duplicated in the data written into the first hash group, determine to write the first data; if the first data is duplicated in the data written into the first hash group, record an address of the first data, the address pointing to data written into the first hash group and duplicated with the first data.

15. The apparatus of any one of claims 10 to 14, wherein, The device further comprises: a construction module for constructing deduplication data according to data written into a hash group to which the input data belongs when the input data are all written into the hash group; the construction module is further configured to construct recovery data according to a start address of at least one hash bucket in the hash group to which the input data belongs and an offset address of the input data in the hash bucket, the recovery data being used for recovering the input data.

16. The apparatus of claim 15, wherein, The construction module is specifically configured to: when a total length of data written into at least one hash group to which the input data belongs is less than a target length, write padding data into the at least one hash group until the total length of the data written into the hash group reaches the target length; construct the deduplication data according to the data written into the padding hash group.

17. The apparatus of claim 16, wherein, The device further comprises: a configuration module for receiving a completion mode configured by a user; the construction module is specifically configured to: when the completion mode is turned on, write padding data into the at least one hash group.

18. The apparatus of any one of claims 10 to 17, wherein, The device further comprises: An allocation module is configured to determine an accelerator corresponding to a hash group according to a result of a first hash operation on data in the hash group to which the input data belongs when the input data are all written into the hash group. A sending module is configured to send the data in the hash group to the accelerator.

19. A chip, characterized by The chip comprises a processor and a communication interface, the communication interface is configured to obtain input data of an artificial intelligence model, and the processor is configured to execute computer readable instructions to perform the data processing method in any one of claims 1 to 9.

20. A computing device, comprising: The computing device comprises at least one processor and a plurality of accelerators; the at least one processor executes the computer readable instructions to perform the method in any one of claims 1 to 9.

21. A computer-readable storage medium, characterized in that, The computer readable instructions are used to implement the method in any one of claims 1 to 9.

22. A computer program product, characterised in that, The computer readable instructions are used to implement the method in any one of claims 1 to 9.

Images