Data processing method and related equipment

Abstract

The data processing method comprises the following steps: dividing input data of an artificial intelligence model into a plurality of batches for parallel processing, performing first hash operation on first data in a first batch in the plurality of batches, and determining a first hash group to which the first data belongs according to a result of the first hash operation, the first hash group corresponds to a first memory space in the memory, second hash operation is performed on the first data, a first hash bucket to which the first data belongs is determined according to a result of the second hash operation, and the first hash bucket belongs to the first hash group and is located in a first area in the first memory space; and when the first data is written into the first region, locking the first hash bucket. According to the method, multiple layers of Hash are introduced, the Hash buckets with finer granularity are further arranged on the basis of the Hash groups, and when the concurrent conflicts occur, only the Hash buckets are locked, so that resource loss caused by the concurrent conflicts is reduced, the data processing efficiency is improved, the overall throughput rate of the system is guaranteed, and the system performance is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence (AI) technology, and more particularly to a data processing method, data processing device, chip, computing device, computer-readable storage medium, and computer program product. Background Technology

[0002] With the development of AI technology, especially the continuous evolution of machine learning (ML) and deep learning, various AI models have emerged. For example, AI models can include product recommendation models applied to e-commerce platforms or video recommendation models applied to short video platforms.

[0003] To handle large amounts of data or requests, distributed systems can be used to distribute the parameters of AI models across multiple accelerators, enabling distributed computing and parallel processing, thus improving system performance and scalability. Simultaneously, using distributed systems for storage ensures data reliability and high availability, reducing the risk of single points of failure.

[0004] The parameters of an AI model can include those represented by sparse features. For example, in a recommender system, user identifiers, user interest tags, and item attributes can be represented using sparse features. Typically, sparse features from different batches often have some overlap, and the number of sparse features in a batch can be quite large. During model training, preprocessing such as deduplication can be performed on sparse features to effectively reduce the communication overhead of sparse features.

[0005] However, current preprocessing methods use hash tables to deduplicate data with sparse features. When using multi-threaded parallel execution of hash table-based deduplication, if a collision occurs, the entire hash table will be locked, thus affecting other threads' deduplication of data. This results in excessively long preprocessing time, leading to a decrease in the overall throughput and performance of the distributed system. Summary of the Invention

[0006] This application provides a data processing method that introduces multi-layer hashing, setting up finer-grained hash buckets on top of hash groups. When concurrent collisions occur, resource consumption caused by concurrent collisions can be reduced by locking only the hash buckets, fully utilizing processor resources, shortening data processing time, improving data processing efficiency, ensuring overall system throughput, and enhancing system performance. This application also provides data processing apparatus, chips, computing devices, computer-readable storage media, and computer program products corresponding to the above method.

[0007] Firstly, this application provides a data processing method. This method can be applied to a distributed system, which can be a heterogeneous computing device or a cluster of heterogeneous computing devices. The distributed system may include processors and accelerators. The processor is located on the host side and can be a central processing unit (CPU), while the accelerator is located on the device side and can be a neural network processing unit (NPU), a graphics processing unit (GPU), or a tensor processing unit (TPU). The accelerator works in conjunction with the processor to train an AI model.

[0008] Specifically, the processor can divide the input data of the artificial intelligence (AI) model into multiple batches for parallel processing. The data in different batches has a certain degree of overlap; for example, in recommendation or search scenarios, sparse features such as personal information, interests, and historical behavior in adjacent batches may have some overlap, or item tags and item categories in adjacent batches may have some overlap. For the first data in the first batch, the processor can perform a first hash operation to determine the first hash group to which the first data belongs. The first hash group corresponds to the first memory space in memory. Then, the processor performs a second hash operation on the first data to determine the hash bucket to which the first data belongs. The first hash bucket belongs to the first hash group and is located in the first region of the first memory space. When writing the first data to the first region, the processor can lock the first hash bucket.

[0009] This method introduces multi-layered hashing, specifically setting up finer-grained hash buckets on top of hash groups. When using parallel mechanisms for data processing to improve processing efficiency, if a conflict occurs, only the corresponding hash bucket can be locked, while other hash buckets within the hash group or other hash groups can still support data insertion. This reduces resource consumption caused by concurrent conflicts, fully utilizes processor resources, shortens data processing time, improves data processing efficiency, ensures overall system throughput, and enhances system performance.

[0010] In some possible implementations, the first hash group corresponds to a first memory space in the processor's memory, or the first hash group corresponds to a first memory space in the memory of a first accelerator, which is used to process the first data.

[0011] In this context, hash groups can correspond to contiguous memory spaces within the processor's memory, enabling sequential reads and improving read performance. When hash groups correspond to memory spaces within the accelerator's memory, data can be written to the accelerator, reducing the overhead of transferring data from the processor's memory to the accelerator's memory.

[0012] In some possible implementations, the size of the first hash bucket can be equal to n times the size of the processor's cache lines, or the size of the first hash bucket can be equal to n times the size of the cache lines in the first accelerator, where n is a natural number greater than or equal to 1. This aligns the size of the first hash bucket with the cache lines, reducing the number of memory accesses when reading data from the hash bucket. In some cases, it may even require only one memory access, significantly improving read performance.

[0013] In some possible implementations, the first hash group also includes an overflow area. The processor can also determine whether the amount of data written to the first hash bucket has reached a threshold. If so, the processor can write the first data to the overflow area.

[0014] In this method, in order to ensure cache line alignment and fine-grained concurrent locking, the size of the hash bucket can be fixed, and considering the efficiency of data access, the amount of data that can be written in the hash bucket is limited. When the amount of data written in the hash bucket reaches the threshold, the data can be written to the overflow area, thereby achieving a balance between concurrency performance and memory access efficiency.

[0015] In some possible implementations, the processor can also compare the first data with the data already written in the first hash group. If the first data does not duplicate the data already written in the first hash group, the processor determines to write the first data; if the first data duplicates the data already written in the first hash group, the processor records the address of the first data, which points to the data already written in the first hash group that duplicates the first data.

[0016] This method compares the first data with the data already written in the first hash group, performs writing on non-duplicate data, and does not perform writing on duplicate data, thus achieving deduplication and reducing the overhead of data interaction between the processor and accelerator.

[0017] In some possible implementations, once all input data has been written to the hash group, the processor can construct deduplicated data based on the data already written in the hash group to which the input data belongs. Specifically, the processor can concatenate the data already written in the hash group, for example, concatenating the data already written in the hash bucket and the data already written in the overflow area, thereby obtaining deduplicated data. The processor can also construct recovery data based on the starting address of at least one hash bucket in the hash group to which the input data belongs and the offset address of the input data in the hash bucket to which it belongs. This recovery data is used to recover the input data.

[0018] In some possible implementations, if the total length of the data already written in at least one hash group to which the input data belongs is less than the target length, the processor can write padding data in at least one hash group until the total length of the data already written in the hash group reaches the target length. The processor can then construct deduplicated data based on the data already written in the padded hash groups.

[0019] This method pads the data when the total length of the data already written in the hash group is less than the target length, which can adapt to the parameter requirements of set communication, such as the parameter requirements of static all-to-all communication.

[0020] In some possible implementations, the processor can receive a user-configured completion mode. Accordingly, when the completion mode is enabled, the processor can write padding data into at least one hash group. This method allows users to decide whether to perform completion based on business needs, offering high flexibility.

[0021] In some possible implementations, once all input data has been written to a hash group, the processor can determine the accelerator corresponding to the hash group based on the result of a first hash operation on the data in that hash group. The processor can then send the data from the hash group to the accelerator. This allows data to be allocated to the corresponding accelerator at the hash group level, improving the efficiency of data allocation.

[0022] Secondly, this application provides a data processing apparatus. The apparatus includes:

[0023] The batching module is used to divide the input data of the artificial intelligence model into multiple batches for parallel processing;

[0024] The grouping module is used to perform a first hash operation on the first data in the first batch of the multiple batches, and determine the first hash group to which the first data belongs based on the result of the first hash operation. The first hash group corresponds to a first memory space in memory.

[0025] The bucketing module is used to perform a second hash operation on the first data, and determine the first hash bucket to which the first data belongs based on the result of the second hash operation. The first hash bucket belongs to the first hash group and is located in a first region within the first memory space.

[0026] The write control module is used to lock the first hash bucket when writing the first data to the first area.

[0027] In some possible implementations, the first hash group corresponds to a first memory space in the processor's memory, or the first hash group corresponds to a first memory space in the memory of a first accelerator, which is used to process the first data.

[0028] In some possible implementations, the size of the first hash bucket is equal to n times the cache line size of the processor's cache, or the size of the first hash bucket is equal to n times the cache line size of the cache in the first accelerator, where n is a natural number greater than or equal to 1.

[0029] In some possible implementations, the first hash group further includes an overflow area, and the write control module is specifically used for:

[0030] Determine whether the amount of data written to the first hash bucket has reached the threshold;

[0031] If so, write the first data into the overflow area.

[0032] In some possible implementations, the write control module is also used for:

[0033] Compare the first data with the data already written to the first hash group;

[0034] If the first data is not duplicated with the data already written in the first hash group, determine to write the first data; if the first data is duplicated with the data already written in the first hash group, record the address of the first data, the address pointing to the data already written in the first hash group that is duplicated with the first data.

[0035] In some possible implementations, the device further includes:

[0036] The construction module is used to construct deduplicated data based on the data already written in the hash group to which the input data belongs, when all the input data has been written to the hash group.

[0037] The construction module is further configured to construct recovery data based on the starting address of at least one hash bucket in the hash group to which the input data belongs and the offset address of the input data in the hash bucket to which it belongs, and the recovery data is used to recover the input data.

[0038] In some possible implementations, the building module is specifically used for:

[0039] If the total length of the data already written in at least one hash group to which the input data belongs is less than the target length, padding data is written in the at least one hash group until the total length of the data already written in the hash group reaches the target length; based on the data already written in the padding hash group, deduplicated data is constructed.

[0040] In some possible implementations, the device further includes:

[0041] The configuration module is used to receive the auto-completion mode configured by the user.

[0042] The building module is specifically used for:

[0043] When the completion mode is enabled, fill data is written to the at least one hash group.

[0044] In some possible implementations, the device further includes:

[0045] The allocation module is used to determine the accelerator corresponding to the hash group based on the result of the first hash operation performed on the data in the hash group to which the input data belongs when all the input data has been written into the hash group.

[0046] A sending module is used to send the data in the hash group to the accelerator.

[0047] Thirdly, this application provides a chip. The chip includes a processor and a communication interface, the communication interface being used to acquire input data from an artificial intelligence model, and the processor being used to execute computer-readable instructions to perform the data processing method as described in the first aspect of this application or any implementation thereof.

[0048] Fourthly, this application provides a computing device. The computing device includes at least one processor and a plurality of accelerators. The at least one processor and the plurality of accelerators communicate with each other. The at least one processor is configured to execute instructions stored in at least one memory to perform a data processing method as described in the first aspect or any implementation thereof.

[0049] Fifthly, this application provides a computer-readable storage medium storing instructions that instruct a computing device to perform the data processing method as described in the first aspect or any implementation thereof.

[0050] In a sixth aspect, this application provides a computer program product containing instructions that, when run on a computing device, cause the computing device to perform the data processing method as described in the first aspect or any implementation thereof.

[0051] Based on the implementation methods provided in the above aspects, this application can be further combined to provide more implementation methods. Attached Figure Description

[0052] To more clearly illustrate the technical methods of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below.

[0053] Figure 1 A hardware architecture diagram of a distributed system provided in this application;

[0054] Figure 2 A flowchart of a data processing method provided in this application;

[0055] Figure 3 This application provides a schematic diagram of a concurrent conflict handling process using spinlocks.

[0056] Figure 4 This application provides a schematic diagram of storing data using hash buckets and data overflow areas;

[0057] Figure 5 A schematic diagram illustrating the process of completing deduplicated data as provided in this application;

[0058] Figure 6 A flowchart of a data processing method provided in this application;

[0059] Figure 7 This application provides a schematic diagram of the data preprocessing process in a training recommendation model scenario;

[0060] Figure 8 This is a schematic diagram of the structure of a data processing device provided in this application. Detailed Implementation

[0061] The terms "first" and "second" used in the embodiments of this application are for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first" and "second" may explicitly or implicitly include one or more of that feature.

[0062] First, some technical terms involved in the embodiments of this application will be introduced.

[0063] Artificial intelligence (AI), also known as machine intelligence, is the technology of correctly interpreting external data, learning knowledge from that data, and flexibly applying that knowledge to achieve specific goals and tasks. Data-driven AI technologies include, but are not limited to, machine learning (ML) and deep learning. Machine learning is a class of algorithms that automatically analyze data to obtain patterns and use those patterns to predict unknown data. Deep learning is an algorithm that uses artificial neural networks as its architecture to learn representations of data.

[0064] AI models refer to mathematical models built using AI technology, such as those constructed through machine learning or deep learning algorithms. AI models can be applied to various scenarios, including but not limited to search and recommendation scenarios. In search scenarios, users provide a description of the content they intend to obtain, and the system (such as a search system) returns results matching that description. In recommendation scenarios, the system (such as a recommendation system) proactively recommends content to users. Because there are no restrictions based on user-provided information, recommendation scenarios are more diverse. For example, recommendation scenarios can include personalized recommendations on the homepages of various applications (APPs), and related recommendations on selection pages. Related recommendations include recommendations based on related items and recommendations based on related users. Accordingly, the AI ​​model can be a search model or a recommendation model.

[0065] In training AI models, the choice of features has a significant impact on the model's effectiveness and performance. Features can be categorized into sparse features and dense features. Sparse features are discrete features with a large number of possible values, typically represented by a high-dimensional feature vector where most elements are zero. Taking search and recommendation scenarios as examples, sparse features can include user identifiers (user IDs), user attributes, item IDs, and item attributes. Both users and items can be represented as sparse features. For users, sparse features can include personal information, interests, and historical behavior. For items, sparse features can include item attributes, tags, and categories. These sparse features can be used to capture the relationships and similarities between users and items, thereby enabling personalized recommendations and rankings.

[0066] Considering that AI models need to process large amounts of data or query requests, distributed systems can be used for model training and inference. Distributed systems can distribute the parameters of an AI model across multiple accelerators. For example, a distributed system can divide the parameters of an AI model into multiple parts, each stored on a different accelerator. These accelerators can be different accelerators on the same computing device or accelerators on different computing devices. This enables distributed computing and parallel processing, improving system performance and scalability, and making full use of resources to increase resource utilization. Simultaneously, distributed systems using distributed storage can ensure data reliability and high availability, reducing the risk of single points of failure. Furthermore, distributed systems can also achieve parallel data processing and distributed training. For example, processes on each accelerator can process a portion of the data in parallel, thereby improving the training speed and performance of the distributed system.

[0067] When training AI models, distributed systems can distribute training data, such as sparse features, to different accelerators via inter-card communication. Considering the large scale of sparse features and the degree of repetition in each batch, distributed systems can perform preprocessing such as deduplication on the sparse features to effectively reduce the communication overhead.

[0068] Currently, the industry has proposed a deduplication method based on hash tables. A hash table, also known as a hash table, is a data structure that stores data in key-value pairs. Storing data in key-value pairs means that any key can uniquely correspond to a specific location in memory; by simply inputting the search key, the corresponding value can be quickly found. Specifically, in this application, the host side of the distributed system can utilize the unique key characteristic of a hash table to deduplicate sparse features. The value in the hash table records the current position of the sparse feature. It should be noted that the processor on the host side of the distributed system, such as a central processing unit (CPU), can also construct a recovery vector based on the hash table. This recovery vector can be combined with the deduplicated feature vector to recover the feature vector before deduplication.

[0069] However, when the above method uses a parallel mechanism to deduplicate data with a certain degree of repetition, such as sparse features, if a concurrent conflict occurs, the process or thread that triggers the insertion operation first will lock the entire hash table. During the period when the hash table is locked, other processes or threads cannot insert data into the hash table. When faced with a batch of sparse features including a large amount of data, processor resources are difficult to fully utilize, resulting in long deduplication time and affecting the overall throughput of the system.

[0070] In view of this, this application provides a data processing method. This method can be applied to a distributed system, which can be a heterogeneous computing device or a cluster of heterogeneous computing devices. The distributed system may include processors and accelerators. The processor is located on the host side and can be a CPU, while the accelerator is located on the device side and can be a neural network processing unit (NPU), graphics processing unit (GPU), or tensor processing unit (TPU). The accelerator works in conjunction with the processor to train an AI model.

[0071] Specifically, the processor can divide the input data of the AI ​​model into multiple batches for parallel processing. Data from different batches has a certain degree of overlap; for example, in recommendation or search scenarios, sparse features such as personal information, interests, and historical behavior in adjacent batches may have some overlap, or item tags and categories in adjacent batches may have some overlap. For the first data in the first batch, the processor can perform a first hash operation to determine the first hash group to which the first data belongs. The first hash group corresponds to the first memory space in memory. Then, the processor performs a second hash operation on the first data to determine the hash bucket to which the first data belongs. The first hash bucket belongs to the first hash group and is located in the first region of the first memory space. When writing the first data to the first region, the processor can lock the first hash bucket.

[0072] This method introduces multi-layered hashing, specifically setting up finer-grained hash buckets on top of hash groups. When using parallel mechanisms for data processing to improve processing efficiency, if a conflict occurs, only the corresponding hash bucket can be locked, while other hash buckets within the hash group or other hash groups can still support data insertion. This reduces resource consumption caused by concurrent conflicts, fully utilizes processor resources, shortens data processing time, improves data processing efficiency, ensures overall system throughput, and enhances system performance.

[0073] To make the technical solution of this application clearer and easier to understand, the architecture of the distributed system of this application is described below with reference to the accompanying drawings. The data processing method of this application can be executed by a distributed system. The distributed system includes a processor and multiple accelerators. The processor may include, but is not limited to, a CPU, and the accelerator may be a computing card used to accelerate computing, such as a GPU, NPU, or TPU, also called an accelerator card. In some examples, the distributed system may be a heterogeneous computing device including multiple computing cards. In this case, the distributed system is a single-machine multi-card architecture. In other examples, the distributed system may be a cluster of computing devices including multiple heterogeneous computing devices. The host side of the computing device may include one CPU or multiple CPUs. In this case, the distributed system can be a multi-machine multi-card architecture. Single-machine multi-card or multi-machine multi-card can be represented as x machines x cards. It should be noted that in the above x machines x cards, "machine" refers to the host, and "card" refers to a computing card including NPU, GPU, etc.

[0074] To make the technical solution of this application clearer and easier to understand, the following example uses a distributed system with a single-machine multi-card architecture.

[0075] See Figure 1 The diagram illustrates a hardware architecture of a distributed system, which can be a server 10 with a single-machine, multi-GPU architecture. Server 10 includes a host-side processor and multiple accelerators on the device side. The host-side processor is also connected to memory (or host memory), which can be a Dual In-line Memory Module (DIMM). Specifically, the DIMM can be of double data rate (DDR) type, such as DDR4 DIMM. Figure 1 In the example, the host side can include four processors and four DDR4 DIMM groups, with each processor connected to one DDR4 DIMM group, and each DDR4 DIMM group including eight DDR4 DIMMs. Multiple processors on the host side can be connected to form a hydra mesh.

[0076] Optionally, the host side may also include interfaces, which may include storage interfaces or network interfaces. The storage interface may include, but is not limited to, a Serial Advanced Technology Attachment (SATA) interface or a Next-Generation Non-Volatile Memory Express (NVMe) interface. The network interface may include, but is not limited to, a Gigabit Ethernet (GE) interface. The host side may also include memory. This memory may include SATA-enabled memory or NVMe-enabled memory, such as a SATA-enabled hard disk drive (HDD) or an NVMe-enabled solid-state drive (SSD).

[0077] The device side includes multiple accelerators, such as accelerator 1, accelerator 2, ..., accelerator m, where m can be an integer greater than or equal to 1. Furthermore, the accelerators may also include memory; to distinguish it from host memory, the memory in the accelerator can be called video memory. Video memory can be used to store the input data of the AI ​​model for AI computation. This video memory can be high-bandwidth memory (HBM).

[0078] The host-side processor and the m accelerators on the device side can be connected via a bus. In some examples, the logical address space of the video memory (such as HBM) in the accelerators can be mapped to the host side and managed by the host. For example, in some cases, the host-side processor can access the accelerator's video memory based on the mapped logical address.

[0079] It should be noted that, Figure 1 The example of a single-machine multi-GPU architecture is provided for illustration only and does not constitute a limitation on the technical solution of this application. In other possible implementations of this application, the distributed system can also be a multi-machine multi-GPU architecture, such as a computing device cluster formed by multiple heterogeneous computing devices with a single-machine multi-GPU architecture.

[0080] Based on the aforementioned distributed system, this application provides a data processing method. A detailed description is provided below with reference to the accompanying drawings.

[0081] See Figure 2 The flowchart illustrates a data processing method that can be executed by the host side of a distributed system, such as by a host-side processor. The method may include the following steps:

[0082] S202, The processor divides the multiple input data of the AI ​​model into multiple batches for parallel processing.

[0083] The input data for AI models can be datasets containing massive amounts of data. In AI model training scenarios, input data can be sparse features describing users or items. These sparse features can include identity (ID) features, such as user ID, item ID, video ID, and article ID. Sparse features can also include users' personal information, interests, and historical behavior, or they can include item tags and categories.

[0084] The processor can divide the input data into multiple batches for parallel processing based on the user-defined batch size, thereby improving data processing efficiency. The batch size can be set empirically, for example, to 64 or 1024. Users can set different batch sizes depending on the dataset size and the computational scale of the distributed system.

[0085] S204. For the first data in the first batch of multiple batches, the processor performs a first hash operation and determines the first hash group to which the first data belongs based on the result of the first hash operation.

[0086] Hash operations are a method of processing an input of arbitrary length through a hash function, outputting a hash value. Hash functions can include, but are not limited to, the modulo function and the division function. The modulo function can be f(x) = (x + Max) mod size, where Max can be a large number and size can be the number of hash groups. The division function can be H(k) = k mod p, where k can be the key and p can be the number of hash groups.

[0087] In this application, the number of hash groups can be determined based on the number of accelerators participating in data processing. In some examples, the number of hash groups can be equal to the number of accelerators in the distributed system, or the number of hash groups can be equal to k times the number of accelerators, where k is greater than 1. When the number of hash groups is k times the number of accelerators, each accelerator can process data from multiple hash groups.

[0088] For the first piece of data in the first batch, the processor can perform a first hash operation with the number of hash groups as a parameter, and obtain the result of the first hash operation. The result of the first hash operation can be the remainder output by a direct modulo function or a division function. Based on the mapping relationship between the result of the first hash operation and the hash groups, the processor can determine the first hash group to which the first piece of data belongs. For example, if the number of hash groups is 8, and the result of the first hash operation is 0 to 7, corresponding to hash groups numbered group0 to group7 respectively, if the result of the first hash operation on the first piece of data is 1, then the first hash group to which the first piece of data belongs is group1.

[0089] The first hash group corresponds to the first memory space in memory. In some examples, the first memory space can be the first memory space in the processor's memory (host memory). In other examples, the first memory space can also be the first memory space in the memory of the first accelerator. Furthermore, considering read and write performance, multiple hash groups can correspond to a contiguous block of memory, such as a contiguous block of memory in the processor's memory.

[0090] S206. The processor performs a second hash operation on the first data and determines the first hash bucket to which the first data belongs based on the result of the second hash operation.

[0091] A hash bucket (or simply bucket) is the basic unit used to store data in a hash table; it is also called a hash slot or storage bucket. In a hash table, a hash function maps keys to corresponding hash buckets. Accordingly, when searching for data, the hash bucket can be determined first based on the hash function, and then that hash bucket can be searched. In this way, hash tables achieve efficient data storage and retrieval by utilizing the structure of hash functions and hash buckets.

[0092] In this application, each hash group is configured with multiple hash buckets. In practice, the number of hash buckets in different hash groups can be the same; for example, the number of hash buckets in multiple hash groups can be uniformly set to 256. Considering the possibility of uneven data distribution, the number of hash buckets in different hash groups can also be different. For example, hash groups containing data with a higher distribution ratio can have more hash buckets, while hash groups containing data with a lower distribution ratio can have fewer hash buckets.

[0093] To improve read and write performance, this application designs hash buckets aligned with cache lines. A cache line is a cache organization method in computing devices. A cache is a hardware mechanism used to improve computer access speed; based on the principle of locality, it copies recently accessed data to a faster cache so that the data can be retrieved faster on subsequent accesses. A cache consists of multiple cache lines, and each cache line can store one or more data blocks.

[0094] Let's take a CPU cache line as an example. A CPU cache line is part of the memory hierarchy used to transfer data between the CPU cache and main memory. A cache line is the smallest unit of cache and the smallest block of data the processor reads from main memory into the cache or writes data from the cache back to main memory. The size of a cache line can be determined by the hardware design. For ease of calculation, cache line sizes are typically powers of 2, such as 32 bytes, 64 bytes, or 128 bytes. A cache line typically consists of two parts: a tag and data. The tag identifies the address range of data stored in the cache line, while the data is the actual stored data content. When a computing device needs to read or write to a memory address, the computing device (e.g., the CPU) first checks if a cache line corresponding to that memory address exists in the cache. If it exists, it's called a cache hit, and data can be read or written directly from the cache. Since cache access is faster than main memory access, it improves read and write performance. If the cache does not exist, it is called a cache miss, and data needs to be read from or written to main memory and copied to the cache.

[0095] Cache line size is an important parameter, typically measured in bytes. Larger cache lines can store more data, reduce cache misses, and improve data access efficiency. This is especially true for access patterns exhibiting spatial and temporal locality, where cache lines can significantly improve data access efficiency. In mainstream computer architectures, the CPU reads data from memory not byte by byte or word by word, but in units of cache lines (typically 64 or 128 bytes). If data can be stored entirely in a cache line, the CPU only needs to access the cache once to read that data, instead of multiple accesses. Based on this, this application employs cache line alignment in the design of the hash bucket data structure. For example, the size of the hash bucket can be set equal to the cache line size, allowing the CPU to read the data of an entire hash bucket with a single cache access, thus improving data access efficiency.

[0096] For ease of understanding, let's take the first hash bucket containing the first data as an example. The size of the first hash bucket can be n times the size of a cache line in the processor's (e.g., CPU) cache, or n times the size of a cache line in the first accelerator's cache. Here, n is a natural number greater than or equal to 1. The first hash bucket is located in the first region of the first memory space. The other hash buckets in the first hash group are located in other regions of the first memory space. It should be noted that when different hash groups correspond to the memory spaces of different accelerators, the size of the hash buckets in different hash groups can be n times the size of a cache line in the corresponding accelerator's cache. For example, the size of the hash buckets in the second hash group can be n times the size of a cache line in the second accelerator's cache.

[0097] Similar to determining the hash group to which the first data belongs, the processor can perform a second hash operation on the first data. Based on the mapping between the result of the second hash operation and the hash buckets, the processor determines the first hash bucket to which the first data belongs. The second hash operation can be performed with the number of hash buckets as a parameter. The result of the second hash operation can be the remainder output by a direct modulo function or a division function. For example, if the number of hash groups is 256, and the result of the second hash operation is 0 to 255, corresponding to hash buckets numbered bucket0 to bucket255 respectively, and if the result of the second hash operation on the first data is 5, then the first hash bucket to which the first data belongs is bucket5.

[0098] S208. When writing the first data to the first area, the processor locks the first hash bucket.

[0099] To address efficiency concerns, processors can process input data in parallel. For example, a processor can launch multiple threads to concurrently deduplicate the input data of an AI model. However, during concurrent deduplication, multiple threads might insert data with the same second hash calculation result into the same hash bucket, leading to concurrency conflicts. To resolve this, the processor can lock the first hash bucket when writing the first data to the first region, thus mitigating concurrency conflicts. While the first hash bucket is locked, new data that needs to be written to the first region (inserted into the first hash bucket) can be written after the processor successfully writes the first data and releases the lock on the first hash bucket.

[0100] In this process, locking the first hash bucket can be achieved by updating the lock variable of the first hash bucket to a first value, indicating that the lock is in a locked state. The lock of the first hash bucket can be a spin lock based on Compare-And-Swap (CAS). Spin locks are a type of lock used for multi-threaded synchronization. CAS-based spin locks are optimistic locks, assuming that concurrency is not severe, allowing threads to repeatedly check the availability of the lock variable. Because the thread continues execution during this process, it is a busy-waiting process. Once a thread acquires a spin lock, it will hold the lock until it is explicitly released. Spin locks avoid the scheduling overhead of process context, resulting in very short thread blocking times.

[0101] For ease of understanding, this application also provides an example. For example... Figure 3 As shown, each hash bucket is equipped with a spin lock. When two data items with the same second hash operation result are inserted into the same hash bucket, for example... Figure 3 When data whose second hash operation result is 0 is simultaneously inserted into hash bucket 0 (e.g., bucket 0), the thread repeatedly checks whether the lock variable 0 of bucket 0 (e.g., spin lock 0) is available. If thread 1 acquires spin lock 0, it can lock the hash bucket. After storage is complete, thread 1 can release spin lock 0. Compared to traditional read-write locks, this application uses fine-grained locks for data storage, requiring only a short waiting time, thus reducing blocking time and performance loss caused by concurrency conflicts.

[0102] Based on the above description, the data processing method of this application first performs a hash operation on the data, determines the hash group to which the data belongs based on the result of this hash operation, and then performs another hash operation to determine the first hash bucket to which the data belongs based on the result of this hash operation. In this way, when writing data to the memory area where the hash bucket is located, the hash bucket can be locked by a fine-grained lock, without the need to use a traditional read-write lock to lock the entire hash table, reducing the performance loss caused by concurrency conflicts and improving system performance.

[0103] To ensure cache line alignment and fine-grained concurrent locking, the size of the hash bucket is typically fixed. Considering data access efficiency, the amount of data stored in a hash bucket should not be excessive; for example, the amount of data stored in a hash bucket should not exceed a threshold (also known as the hash bucket width). In some cases, the distribution of input data is not uniform, and a large number of data items may belong to the same hash bucket. Therefore, this application also designs an overflow map to balance concurrency performance and memory access efficiency.

[0104] Taking the first hash group as an example, the first hash group also includes an overflow area. The processor can compare the first data with the data already written in the first hash group. If the first data is not a duplicate of the data already written in the first hash group, the processor can determine to write the first data. If the first data is a duplicate of the data already written in the first hash group, the processor can choose not to write the first data, but record the address of the first data, where the address of the first data points to the data already written in the first hash group that is a duplicate of the first data. In other words, when the first data is a duplicate of the data already written in the first hash group, the processor can establish a mapping relationship between the first data and the duplicate data already written in the first hash group by recording the address.

[0105] Upon determining that the first data should be written, the processor can check whether the amount of data already written in the first hash bucket has reached a threshold. If so, the processor can write the first data to the overflow area. If not, the processor can write the first data to the first hash bucket. Considering that data with the same second hash operation result can be written to either the hash bucket or the overflow area, the processor can compare the first data with the data already written to the first hash bucket or the overflow area to reduce the amount of deduplication calculation, shorten the deduplication time, and improve deduplication efficiency.

[0106] For ease of understanding, this application also provides an example.

[0107] See Figure 4 The diagram illustrates a data processing method where the number of hash groups can be set according to the number of processors participating in parallel computing. For example, eight hash groups can be set, numbered from group 0 to group 7. Each hash group contains 256 hash buckets, including hash buckets 0 to hash bucket 255. To facilitate data retrieval, the width of a hash bucket can be set to 6, meaning a hash bucket can store a maximum of 6 data items. Each hash group also includes an overflow area. The overflow area stores overflow data from hash buckets within the hash group. It should be noted that the overflow area can use a mapping method to store overflow data; for example, the overflow area can store the overflow data along with its offset address within the overflow area, facilitating quick location of the overflow data in a specific hash bucket later. Figure 4 In the example, groups 0 through 7 can correspond to a contiguous memory space in the processor's memory. For example, group 0 can correspond to the first memory space, and group 1 can correspond to the second memory space. Each hash bucket in a hash group can correspond to a region in the memory space; for example, the first hash bucket can correspond to the first region of the first memory space.

[0108] Specifically, the processor can divide the input data of the AI ​​model into multiple batches for parallel processing. For the first data in the first batch, the processor can perform a first hash operation on the first data, and determine the first hash group to which the first data belongs based on the result of the first hash operation. For example, if the result of the first hash operation on a piece of data is 0, then the hash group to which the data belongs is group 0; if the result of the first hash operation on another piece of data is 1, then the hash group to which the data belongs is group 1. The processor can perform a second hash operation on the first data, and determine the first hash bucket to which the first data belongs based on the result of the second hash operation. Taking data belonging to group 0 as an example, if the result of the second hash operation on a piece of data is 0, then the hash bucket to which the data belongs is hash bucket 0; if the result of the second hash operation on another piece of data is 1, then the hash bucket to which the data belongs is hash bucket 1.

[0109] The processor can compare the first data with the data already written in the first hash group. If they are not duplicates, the first data is written; otherwise, it is not written, and the address of the first data is recorded, for example, in the recovered data. When writing the first data, the processor can determine if the number of data already written in the first hash bucket has reached 6. If yes, the first data is written to the overflow area; otherwise, it is written to the first hash bucket. Figure 4 In the example, the processor writes data whose second hash operation result is 0 to hash bucket 0. When the number of data written to hash bucket 0 reaches 6, the remaining data whose second hash operation result is 0 is written to the overflow area.

[0110] Furthermore, once all input data has been written to the hash group, the processor can construct deduplicated data based on the data already written to the hash group to which the input data belongs. The processor can concatenate the data already written to each hash bucket in the hash group and the data already written to the overflow area to obtain deduplicated data. For example... Figure 4 As shown, for hash group 0, the processor can concatenate the data already written in hash bucket 0 (such as data 0 to data 5 in hash bucket 0), the data already written in hash bucket 1 (such as data 0 to data 3 in hash bucket 1)...the data already written in hash bucket 255, and the data already written in the overflow area (such as overflow data from hash bucket 0), to obtain deduplicated data. During concatenation, the processor can use the concatenated data as elements of a vector to obtain deduplicated data in vector form, also known as deduplicated vectors, denoted as unique vectors.

[0111] Next, the process of constructing the recovery data will be explained. The processor can also construct recovery data based on the starting address of at least one hash bucket in the hash group to which the input data belongs and the offset address of the input data within that hash bucket. This recovery data is used to recover the input data. The starting address can be the starting offset address, denoted as `replaceoffset`, and the offset address of the input data within its hash bucket can be represented by the index of the input data in the hash bucket, denoted as `data in bucket index`. For data within a hash bucket, the processor can obtain the address of each piece of data within the hash bucket based on the starting address and offset address of the hash bucket; for example, it can obtain the offset address of each piece of data. For instance, for data `i` (where `i` is an integer from 0 to 5) in hash bucket 0, the address of data `i` in hash bucket 0 can be obtained based on the `replace offset` of hash bucket 0 and the `data in bucket index` of data `i`, denoted as `bucket 0data i offset`, where `bucket 0data i offset` = `replace offset` + `data in bucket index`. Similarly, the processor can obtain the address of the overflow data based on the starting address of the overflow area and the offset address of the overflow data within the overflow area. In this way, the processor can obtain the address of each data point in the deduplicated data. This address can be an offset address, or offset, which the processor can concatenate. Then, based on the input data, the processor can insert the offset address of the duplicate data into the concatenated offset address to obtain the restored data. Similar to the deduplicated data, when concatenating or inserting offset addresses, the processor can use the offset address as an element of a vector to obtain the restored data in vector form, also known as a restore vector.

[0112] In some possible implementations, the processor can construct the recovery data in parallel, specifically by using multiple threads to construct the recovery vector. Specifically, each thread obtains a segment of the input data, then queries the offset of each segment in parallel. The processor then concatenates the offsets retrieved by each thread into a contiguous block of addresses to form the recovery vector.

[0113] Once all input data has been written to the hash group, the processor can determine the accelerator corresponding to the hash group based on the result of the first hash operation performed on the data in that hash group. Specifically, when there is a one-to-one correspondence between the first hash operation result and the accelerator, the processor can obtain the accelerator corresponding to the hash group based on the result of the first hash operation performed on the data in the hash group and the mapping relationship between the first hash operation result and the accelerator. For example, if the result of the first hash operation on the data in a hash group is 1, then the accelerator corresponding to that hash group can be accelerator 1. The processor can send the data in the hash group to the accelerator corresponding to the hash group. For example, the processor can send deduplicated data constructed from the data already written in the hash group to the accelerator corresponding to the hash group.

[0114] The data distribution in different hash groups can be different; for example, the data repetition rate may vary significantly. This can lead to different lengths of the deduplicated data obtained after deduplication in different hash groups. For instance, if hash group 1 has a repetition rate of 80% and hash group 2 has a repetition rate of 60%, and the number of data belonging to hash group 1 and hash group 2 in the input data is 100 each, then the length of the deduplicated data obtained after deduplication in hash group 1 can be 100 - 100 * 80 = 20, and the length of the deduplicated data obtained after deduplication in hash group 2 can be 100 - 100 * 60% = 40.

[0115] In model training scenarios, accelerators can interact with each other via ensemble communication. To better adapt to the parameter requirements of ensemble communication, such as static all-to-all communication, the CPU can pad deduplicated data (e.g., deduplicated vectors). Specifically, when the total length of the data already written in at least one hash group to which the input data belongs is less than the target length, the processor can write padding data in at least one hash group until the total length of the hash groups reaches the target length. The padding data can be set empirically, for example, it can be all zeros. Then, the processor can construct deduplicated data based on the data already written in the padded hash groups.

[0116] Furthermore, considering that aggregate communication typically requires contiguous addresses, the processor can also concatenate the data already written in each hash group to obtain deduplicated data.

[0117] To facilitate understanding, an example is provided below with reference to the accompanying diagram. In this example, as shown... Figure 5As shown, the deduplicated data can include data already written in multiple hash groups, such as data already written in hash group 0, hash group 1, ..., hash group 7. The data already written in hash group i can include data from the AI ​​model's input data that has already been written to hash group i. When the total length of the data already written to hash group i from the AI ​​model's input data is less than the target length, padding data, such as a padding value, is also written to hash group i. For ease of calculation, the padding value can take the value of several zeros.

[0118] The data processing method of this application allows users to decide whether to perform completion based on business needs. Specifically, the processor can receive a completion mode configured by the user, which can be enabled or disabled (disabled). Correspondingly, when the completion mode configured by the user is enabled, the processor can detect whether the length of the data already written in the hash group is less than the target length. If so, the processor can write padding data into the hash group until the length of the padded data is equal to the target length.

[0119] Furthermore, considering that the data distribution may differ across different data groups, this application also supports configuring the processor's initialization parameters for data processing to achieve ideal execution efficiency and performance. Specifically, the processor can support user configuration of at least one of the following: thread count constraints, completion mode, estimated memory usage, and the number of initially allocated memory or hash buckets. The thread count constraints may include a maximum and / or minimum number of threads. The processor can then process the model data according to the user-configured initialization parameters.

[0120] Specifically, the processor supports processing data in a data set using multiple threads. The processor can receive user-configured thread count constraints, including a minimum and / or maximum number of threads. When the processor receives input data from the AI ​​model, it can determine the number of threads to process the input data based on the batch size, ensuring that the number of threads satisfies the thread count constraints. For example, the number of threads can be greater than or equal to the minimum number of threads and less than or equal to the maximum number of threads.

[0121] Furthermore, this application also supports adaptive parameter adjustment based on user configuration, data scale, and other information to achieve optimal performance. For details on adjusting the number of threads, please refer to the description of determining the number of threads during the initialization phase.

[0122] The following section provides a detailed explanation of adaptive adjustment of hash bucket size and adaptive adjustment of memory.

[0123] Specifically, the processor can receive the user-configured estimated memory usage and then pre-allocate memory based on that estimated usage. The pre-allocated memory can be equal to or slightly larger than the estimated memory usage. If the processor detects insufficient memory while processing data in a data group, it can request the allocation of new memory. The processor can also request new memory if processing of the current data group is aborted or fails.

[0124] The processor can also determine at least one of the following: the amount of input data (data size) or the repetition rate, and then adjust the size of the hash bucket based on either the data size or the repetition rate. For example, when the repetition rate is low, the processor can increase the size of the hash bucket to store more data.

[0125] The above describes the data processing method of this application from the perspective of the processor. The following describes the data processing method of this application from the perspective of the accelerator.

[0126] See Figure 6 The flowchart shown illustrates a data processing method, which includes the following steps:

[0127] S602, the first accelerator receives the deduplication data and recovery data sent by the processor.

[0128] The first accelerator can correspond to the first hash group, and the deduplicated data sent by the processor to the first accelerator can be constructed based on the data already written in the first hash group. The data already written in the first hash group consists of unique data belonging to the first hash group from the input data of the AI ​​model. The recovered data can include the addresses of each data point in the input data of the AI ​​model, used to recover the input data of the AI ​​model.

[0129] S604, The first accelerator receives deduplicated data sent by the second accelerator through aggregated communication.

[0130] Similar to the first accelerator, the second accelerator can receive deduplicated data sent by the processor. The deduplicated data sent by the processor to the second accelerator can be constructed based on the data already written in the second hash group. Accelerators can transmit their respective deduplicated data through set communication. Based on this, the first accelerator can receive the deduplicated data sent by the second accelerator through set communication.

[0131] S602 and S604 can be executed in parallel or sequentially; this embodiment does not impose any restrictions on this.

[0132] S606, the first accelerator recovers the input data of the AI ​​model by using the deduplication data sent by the processor and the deduplication data sent by the second accelerator through aggregate communication.

[0133] The first accelerator can concatenate the deduplicated data received from the processor and the second accelerator to obtain the full deduplicated data. Then, based on the full deduplicated data and the addresses of each input data in the recovered data, the first accelerator can retrieve the data pointed to by each address from the full deduplicated data, thereby recovering the input data of the AI ​​model.

[0134] It should be noted that in some possible implementations, the processor can also construct the full amount of deduplicated data. The first accelerator can receive the full amount of deduplicated data and combine it with the recovered data to perform a recovery operation, thereby recovering the input data of the AI ​​model.

[0135] Furthermore, the first accelerator can train the model based on the input data of the recovered AI model. The first accelerator can train the AI ​​model in parallel with the second accelerator through a parallel mechanism. For example, the first and second accelerators can use model parallelism or other methods to train the AI ​​model in parallel.

[0136] Based on the foregoing description, this application provides a data processing method. In this method, a first accelerator can receive deduplicated and restored data sent by a processor, and also receive deduplicated data sent by a second accelerator via aggregated communication. By aggregating the deduplicated data from different accelerators, full deduplicated data can be obtained. The complete input data can be recovered from the full deduplicated and restored data. This method introduces hash groups for data deduplication and combines this with the recovered data for recovery, reducing communication overhead between the processor and accelerators while ensuring recoverability. Accelerators can exchange deduplicated data based on aggregated communication, improving efficiency.

[0137] Next, the data processing method of this application will be described in detail with a specific application scenario. This method is used in the data preprocessing stage of a recommendation system. The recommendation system uses a recommendation model for content recommendation. The recommendation model can be trained using a large number of sparse features. Before the sparse features are aggregated, preprocessing such as feature deduplication and vector reconstruction is usually required. The data processing method of this application can efficiently complete the relevant tasks.

[0138] See Figure 7 The diagram illustrates a data preprocessing scenario. Considering the large parameter size of the recommendation model, a computing device cluster can be used to train the model. This computing device cluster includes at least one computing device. Figure 7In the example, the computing device cluster includes a CPU on the host side and multiple NPUs on the device side. The multiple NPUs include an NPU numbered rank 0 and an NPU numbered rank 1. When training large-scale recommendation models, data preprocessing is typically required. Data preprocessing can be performed on the host side of the computing device, for example, by the host-side CPU.

[0139] The host-side CPU acquires input data from the AI ​​model and processes it in batches in parallel. This input data may include ID-type features, also known as feature IDs. The input data may include data processed by the NPU numbered rank 0 and data processed by the NPU numbered rank 1. The host-side CPU can execute the aforementioned data processing methods on multiple batches of data to construct deduplicated data (e.g., unique IDs) and recovered data (e.g., recover IDsoffset).

[0140] It should be noted that when the computing device cluster adopts a multi-machine, multi-card architecture, the host-side CPUs can also perform aggregated communication, such as IDs all-to-all communication, to obtain the full deduplicated data and corresponding recovery data processed by the NPU with the ID number 0, and the weighted deduplicated data and corresponding recovery data processed by the NPU with the ID number 1.

[0141] Unique IDs are typically sparse data. To facilitate computation and optimization in AI models, sparse features are often transformed into dense features. Dense features can be represented by low-dimensional feature vectors, where most elements are non-zero. This transformation can be achieved through feature encoding or feature embedding, such as word embedding. The CPU can perform a `gather` operation to convert sparse deduplicated data into dense deduplicated data, simply referred to as dense data. This dense data can be embedding data obtained through word embedding, also known as embedding weight. `gather` is an operator that returns the tensor element at a given index / subscript. Then, the first and second CPUs can distribute the embedding weights to the NPU, which trains the recommendation model based on these embedding weights.

[0142] Furthermore, NPUs can perform aggregated communication, such as all-to-all communication, to obtain the embedding weights corresponding to the deduplicated feature IDs. Then, each NPU can obtain the corresponding embedding weights for the recovered feature IDs through a gather (e.g., TensorFlow gather). The NPUs can then train the model based on these recovered feature IDs and their corresponding embedding weights.

[0143] The above embodiments are illustrated using search or recommendation scenarios. In other possible implementations of the embodiments of this application, the data processing method of this application can also be used in other scenarios that require data deduplication.

[0144] Based on the aforementioned data processing method, this application provides a data processing apparatus. The data processing apparatus of this application, according to embodiments thereof, will now be described from a functional modular perspective with reference to the accompanying drawings.

[0145] See Figure 8 The diagram shows the structure of a data processing device 800, which includes:

[0146] The batching module 802 is used to divide the input data of the artificial intelligence model into multiple batches for parallel processing;

[0147] Grouping module 804 is used to perform a first hash operation on the first data in the first batch of the plurality of batches, and determine the first hash group to which the first data belongs based on the result of the first hash operation, wherein the first hash group corresponds to a first memory space in memory;

[0148] Bucketing module 806 is used to perform a second hash operation on the first data, determine the first hash bucket to which the first data belongs based on the result of the second hash operation, the first hash bucket belongs to the first hash group, and the first hash bucket is located in a first region within the first memory space.

[0149] The write control module 808 is used to lock the first hash bucket when writing the first data to the first area.

[0150] Among them, the batching module 802, grouping module 804, bucketing module 806 and write control module 808 can be implemented by software or by hardware.

[0151] When implemented in software, the batching module 802, grouping module 804, bucketing module 806, and write control module 808 can be applications running on computing devices (e.g., the CPU on the host side of the computing device). Taking bucketing module 806 as an example, the application can be a computing engine. The application can also be provided to users in the form of virtualization services. Virtualization services can include virtual machine (VM) services, bare metal server (BMS) services, or container services. VM services can be services that use virtualization technology to create virtual machine (VM) resource pools on multiple physical hosts to provide VMs for users to use on demand. BMS services are services that create virtual BMS resource pools on multiple physical hosts to provide BMS for users to use on demand. Container services are services that create virtual container resource pools on multiple physical hosts to provide containers for users to use on demand. A VM is a simulated virtual computer, that is, a logical computer. A BMS is a scalable, high-performance computing service with computing performance indistinguishable from traditional physical machines and features secure physical isolation. Containers are a kernel virtualization technology that provides lightweight virtualization to isolate user space, processes, and resources. It should be understood that the VM service, BMS service, and container service mentioned above are merely specific examples. In practical applications, virtualization services can also include other lightweight or heavyweight virtualization services, which are not specifically limited here.

[0152] When implemented in hardware, the batching module 802, grouping module 804, bucketing module 806, and write control module 808 may include at least one processor. Alternatively, the batching module 802, grouping module 804, bucketing module 806, and write control module 808 may also be devices implemented using application-specific integrated circuits (ASICs) or programmable logic devices (PLDs). The PLD can be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL), or any combination thereof.

[0153] The batching module 802 can be used to execute S202 in the aforementioned method embodiment, and its specific implementation can be as follows: Figure 2Description of S202 in the illustrated embodiment.

[0154] In some possible implementations, the first hash group corresponds to a first memory space in the processor's memory, or the first hash group corresponds to a first memory space in the memory of a first accelerator, which is used to process the first data.

[0155] In some possible implementations, the size of the first hash bucket is equal to n times the cache line size of the processor's cache, or the size of the first hash bucket is equal to n times the cache line size of the cache in the first accelerator, where n is a natural number greater than or equal to 1.

[0156] In some possible implementations, the first hash group further includes an overflow area, and the write control module 808 is specifically used for:

[0157] Determine whether the amount of data written to the first hash bucket has reached the threshold;

[0158] If so, write the first data into the overflow area.

[0159] In some possible implementations, the write control module 808 is further configured to:

[0160] Compare the first data with the data already written to the first hash group;

[0161] If the first data is not duplicated with the data already written in the first hash group, determine to write the first data; if the first data is duplicated with the data already written in the first hash group, record the address of the first data, the address pointing to the data already written in the first hash group that is duplicated with the first data.

[0162] In some possible implementations, the device 800 further includes:

[0163] The construction module 810 is used to construct deduplicated data based on the data already written in the hash group to which the input data belongs when all the input data has been written to the hash group.

[0164] The construction module 810 is further configured to construct recovery data based on the starting address of at least one hash bucket in the hash group to which the input data belongs and the offset address of the input data in the hash bucket to which it belongs, the recovery data being used to recover the input data.

[0165] Similar to the bucketing module 806, the building module 810 can be implemented in software or in hardware.

[0166] When implemented in software, building module 810 can be an application running on a computing device. This application can also be provided to users as a virtualization service, such as a VM service, BMS service, or container service. When implemented in hardware, building module 810 can include at least one processor, such as at least one CPU. Alternatively, building module 810 can also be a device implemented using an ASIC or PLD.

[0167] In some possible implementations, the building module 810 is specifically used for:

[0168] If the total length of the data already written in at least one hash group to which the input data belongs is less than the target length, padding data is written in the at least one hash group until the total length of the data already written in the hash group reaches the target length; based on the data already written in the padding hash group, deduplicated data is constructed.

[0169] In some possible implementations, the device 800 further includes:

[0170] Configuration module 812 is used to receive the auto-completion mode configured by the user;

[0171] The construction module 810 is specifically used for:

[0172] When the completion mode is enabled, fill data is written to the at least one hash group.

[0173] Similar to the bucketing module 806, the configuration module 812 can be implemented in software or in hardware.

[0174] When implemented in software, the configuration module 812 can be an application running on a computing device. This application can be an interactive component that provides a configuration interface. When implemented in hardware, the configuration module 812 can include at least one processor, such as at least one CPU. Alternatively, the configuration module 812 can also be a device implemented using an ASIC or a PLD.

[0175] In some possible implementations, the device 800 further includes:

[0176] The allocation module 814 is used to determine the accelerator corresponding to the hash group based on the result of the first hash operation performed on the data in the hash group to which the input data belongs when all the input data has been written into the hash group.

[0177] The sending module 816 is used to send the data in the hash group to the accelerator.

[0178] The allocation module 814 and the sending module 816 can be implemented in software or in hardware.

[0179] When implemented in software, the allocation module 814 and the sending module 816 can be applications running on a computing device. When implemented in hardware, the allocation module 814 can include at least one processor, such as at least one CPU. Alternatively, the allocation module 814 can be a device implemented using an ASIC or PLD, and the sending module 816 can be a bus device.

[0180] Based on the aforementioned data processing method and data processing apparatus, this application also provides a chip. The chip includes a processor and a communication interface. The communication interface is used to acquire input data from an AI model, and the processor is used to execute computer-readable instructions to perform the aforementioned data processing method and realize the functions of the aforementioned data processing apparatus.

[0181] Furthermore, this application also provides a computing device. This computing device can be a heterogeneous computing device, specifically including at least one processor and multiple accelerators. The at least one processor can execute computer-readable instructions to perform the aforementioned data processing method and realize the functions of the aforementioned data processing apparatus.

[0182] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to perform the data processing method described above applied to the data processing apparatus 800.

[0183] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product runs on at least one computing device, it causes the at least one computing device to perform the aforementioned model data processing method. This application also provides a computer program product containing instructions. When the computer program product runs on at least one computing device, it causes the at least one computing device to perform the aforementioned data processing method.

[0184] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present invention.

Claims

1. A data processing method, characterized in that, The method includes: The input data for the artificial intelligence model is divided into multiple batches and processed in parallel. For the first data in the first batch of the plurality of batches, a first hash operation is performed, and the first hash group to which the first data belongs is determined based on the result of the first hash operation. The first hash group corresponds to the first memory space in memory. A second hash operation is performed on the first data, and the first hash bucket to which the first data belongs is determined based on the result of the second hash operation. The first hash bucket belongs to the first hash group and is located in the first region within the first memory space. When writing the first data into the first region, the first hash bucket is locked.

2. The method according to claim 1, characterized in that, The first hash group corresponds to a first memory space in the processor's memory, or the first hash group corresponds to a first memory space in the memory of a first accelerator, and the first accelerator is used to process the first data.

3. The method according to claim 1 or 2, characterized in that, The size of the first hash bucket is equal to n times the size of the cache line of the processor's cache, or the size of the first hash bucket is equal to n times the size of the cache line of the cache in the first accelerator, where n is a natural number greater than or equal to 1.

4. The method according to any one of claims 1 to 3, characterized in that, The first hash group further includes an overflow area, and the method further includes: Determine whether the amount of data written to the first hash bucket has reached the threshold; If so, write the first data into the overflow area.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: Compare the first data with the data already written to the first hash group; If the first data is not duplicated with the data already written in the first hash group, determine to write the first data; if the first data is duplicated with the data already written in the first hash group, record the address of the first data, the address pointing to the data already written in the first hash group that is duplicated with the first data.

6. The method according to any one of claims 1 to 5, characterized in that, The method further includes: Once all the input data has been written to the hash group, deduplicated data is constructed based on the data already written to the hash group to which the input data belongs. Recovery data is constructed based on the starting address of at least one hash bucket in the hash group to which the input data belongs and the offset address of the input data in the hash bucket to which it belongs. The recovery data is used to recover the input data.

7. The method according to claim 6, characterized in that, The step of constructing deduplicated data based on the data already written in the hash group to which the input data belongs includes: If the total length of the data already written in at least one hash group to which the input data belongs is less than the target length, padding data is written in the at least one hash group until the total length of the data already written in the hash group reaches the target length; based on the data already written in the padding hash group, deduplicated data is constructed.

8. The method according to claim 7, characterized in that, The method further includes: Receive the autocomplete mode configured by the user; The step of writing padding data to the at least one hash group includes: When the completion mode is enabled, fill data is written to the at least one hash group.

9. The method according to any one of claims 1 to 8, characterized in that, The method further includes: Once all the input data has been written into the hash group, the accelerator corresponding to the hash group is determined based on the result of the first hash operation performed on the data in the hash group to which the input data belongs. Send the data from the hash group to the accelerator.

10. A data processing apparatus, characterized in that, The device includes: The batching module is used to divide the input data of the artificial intelligence model into multiple batches for parallel processing; The grouping module is used to perform a first hash operation on the first data in the first batch of the multiple batches, and determine the first hash group to which the first data belongs based on the result of the first hash operation. The first hash group corresponds to a first memory space in memory. The bucketing module is used to perform a second hash operation on the first data, and determine the first hash bucket to which the first data belongs based on the result of the second hash operation. The first hash bucket belongs to the first hash group and is located in a first region within the first memory space. The write control module is used to lock the first hash bucket when the first data is written to the first area.

11. The apparatus according to claim 10, characterized in that, The first hash group corresponds to a first memory space in the processor's memory, or the first hash group corresponds to a first memory space in the memory of a first accelerator, and the first accelerator is used to process the first data.

12. The apparatus according to claim 10 or 11, characterized in that, The size of the first hash bucket is equal to n times the size of the cache line of the processor's cache, or the size of the first hash bucket is equal to n times the size of the cache line of the cache in the first accelerator, where n is a natural number greater than or equal to 1.

13. The apparatus according to any one of claims 10 to 12, characterized in that, The first hash group also includes an overflow area, and the write control module is specifically used for: Determine whether the amount of data written to the first hash bucket has reached the threshold; If so, write the first data into the overflow area.

14. The apparatus according to any one of claims 10 to 13, characterized in that, The write control module is also used for: Compare the first data with the data already written to the first hash group; If the first data does not overlap with the data already written in the first hash group, then determine to write the first data; If the first data is duplicated with data already written in the first hash group, record the address of the first data, where the address points to data already written in the first hash group that is duplicated with the first data.

15. The apparatus according to any one of claims 10 to 14, characterized in that, The device further includes: The construction module is used to construct deduplicated data based on the data already written in the hash group to which the input data belongs, when all the input data has been written to the hash group. The construction module is further configured to construct recovery data based on the starting address of at least one hash bucket in the hash group to which the input data belongs and the offset address of the input data in the hash bucket to which it belongs, and the recovery data is used to recover the input data.

16. The apparatus according to claim 15, characterized in that, The building module is specifically used for: If the total length of the data already written in at least one hash group to which the input data belongs is less than the target length, padding data is written in the at least one hash group until the total length of the data already written in the hash group reaches the target length. Based on the data already written in the filled hash group, construct deduplicated data.

17. The apparatus according to claim 16, characterized in that, The device further includes: The configuration module is used to receive the auto-completion mode configured by the user. The building module is specifically used for: When the completion mode is enabled, fill data is written to the at least one hash group.

18. The apparatus according to any one of claims 10 to 17, characterized in that, The device further includes: The allocation module is used to determine the accelerator corresponding to the hash group based on the result of the first hash operation performed on the data in the hash group to which the input data belongs when all the input data has been written into the hash group. A sending module is used to send the data in the hash group to the accelerator.

19. A chip, characterized in that, The chip includes a processor and a communication interface, the communication interface being used to acquire input data from an artificial intelligence model, and the processor being used to execute computer-readable instructions to perform the data processing method as described in any one of claims 1 to 9.

20. A computing device, characterized in that, The computing device includes at least one processor and a plurality of accelerators; the at least one processor executes the computer-readable instructions to perform the method as claimed in any one of claims 1 to 9.

21. A computer-readable storage medium, characterized in that, Includes computer-readable instructions; the computer-readable instructions are used to implement the method according to any one of claims 1 to 9.

22. A computer program product, characterized in that, Includes computer-readable instructions; the computer-readable instructions are used to implement the method according to any one of claims 1 to 9.

Images