Data processing method and related equipment

Abstract

The invention provides a data processing method, which is applied to a service script based on an artificial intelligence (AI) model and a scene of training the AI model by using sparse features in a data set, and comprises the following steps: receiving a sparse table initialization request, calling a memory allocation interface to apply for a memory block for storing a sparse table in response to the sparse table initialization request, the number of the memory blocks is determined according to the scale of the sparse features in the data set, and then storing the corresponding relation between the feature keys of the sparse features and the address of the sparse table. According to the method, when a sparse table is initialized, a memory allocation interface is called to apply for a memory block, the capacity of the sparse table does not need to be set in a table building interface of the sparse table, and automatic capacity expansion is achieved through the scale of feature keys in a data set. For different models, data sets or hardware forms, the table establishment interface of the sparse table does not need a user to adjust parameters for initializing the sparse table, the generalization is good, the training efficiency is improved, and the user experience is improved.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence (AI) technology, and in particular to a data processing method, apparatus, chip, computing device, computing device cluster, computer-readable storage medium, and computer program product. Background Art

[0002] With the continuous development of AI technology, more and more industries and fields are adopting AI models (sometimes referred to simply as models for ease of description) to achieve intelligent and automated business operations. For example, in the e-commerce industry, e-commerce platforms use trained AI models to recommend items of interest to users. Another example is in the social networking field, where social platforms use trained AI models to recommend interesting images and text to users, achieving personalized recommendations.

[0003] An AI model is a mathematical model built using AI technology and used to make predictions about unknown data. A typical application of AI models is recommendation models, including but not limited to item recommendation models, video recommendation models, text recommendation models, and image recommendation models. These AI models can be data-driven and typically require training using large amounts of data (e.g., sample data).

[0004] The sample data scale of AI models represented by recommendation models can usually reach tens of terabytes (TB) or even hundreds of TB. When training the above AI models, large-scale sparse features are usually used, including ID-type features such as item identifiers (ID) and user IDs. To this end, it is usually necessary to vector map the above sparse features, for example, by embedding them into dense vectors and storing them in a sparse table to facilitate training using a deep learning framework. In specific implementation, the business script of the AI ​​model can specify the parameters of the created sparse table, and then the sparse table can be initialized according to the above parameters when it is initialized.

[0005] However, when the dataset is changed, the above solution needs to adjust the parameters of the initialization sparse table, which has poor generalization and low training efficiency, making it difficult to meet business needs. Summary of the Invention

[0006] The present application provides a data processing method that does not require setting the capacity of a sparse table, but instead automatically expands the capacity based on the size of the feature keys in the dataset. The sparse table creation interface does not require the user to adjust the parameters for initializing the sparse table for different models, datasets, or hardware forms, and has good generalization, improving training efficiency, and enhancing user experience. The present application also provides devices, chips, computing devices, computing device clusters, computer-readable storage media, and computer program products corresponding to the above-mentioned data processing method.

[0007] In a first aspect, the present application provides a data processing method. The method is applied to a first chip on the host side of a computing device. The first chip may be, for example, a central processing unit (CPU). The first chip is used for a business script based on an AI model, and the AI ​​model is trained using sparse features in a data set.

[0008] Specifically, the first chip can receive a sparse table initialization request, and then in response to the sparse table initialization request, call a memory allocation interface to apply for a memory block, which is used to store the sparse table, wherein the number of memory blocks is determined according to the scale of the sparse features in the data set, and then the first chip stores the correspondence between the feature key of the sparse feature and the address of the sparse table.

[0009] When initializing a sparse table, this method calls a memory allocation API to request memory blocks. The number of memory blocks is determined based on the size of the sparse features in the dataset. This eliminates the need to set the sparse table's capacity in the sparse table creation API; instead, the capacity is automatically expanded based on the size of the feature keys in the dataset. This API eliminates the need to adjust initialization parameters for different models, datasets, or hardware configurations, resulting in improved generalization, improved training efficiency, and a better user experience.

[0010] In some possible implementations, the first chip may first send a first call request to the memory allocation interface to apply for a first memory block, which is used to store the vector corresponding to the feature key of the first sparse feature in the data set. When the remaining capacity of the first memory block meets the set conditions, for example, the remaining capacity of the first memory block is 0 (the first memory block is full), or the remaining capacity of the first memory block is less than the set value or the set ratio, the first chip may send a second call request to the memory allocation interface to apply for a second memory block, which is used to store the vector corresponding to the feature key of the second sparse feature in the data set.

[0011] This allows on-demand memory block requests. When a dataset changes, causing a change in the capacity of the sparse table, or when other factors change, causing a change in the capacity of the sparse table, there is no need to adjust the parameters for initializing the sparse table. This allows for seamless dataset switching, which has good generalization.

[0012] In some possible implementations, a first chip receives an expansion prompt sent by a second chip on the device side of a computing device, where the expansion prompt is used to prompt the expansion of a sparse table. The expansion prompt may be generated when the second chip detects that the remaining capacity of the first memory block meets a set condition, and then the first chip generates a second call request for a memory allocation interface based on the expansion prompt, and then the first chip sends the second call request for the memory allocation interface.

[0013] This allows for dynamic application of memory blocks to facilitate expansion of sparse tables to meet the needs of model training at different stages.

[0014] In some possible implementations, when a change in the allocatable memory space of the sparse table is detected, for example, the change in the allocatable memory space of the sparse table is caused by a change in the data set, an AI model, or a change in the hardware form, the first chip can also send change information of the allocatable memory space, which is used to indicate the allocation of memory blocks according to the changed allocatable memory space.

[0015] When allocating memory, the memory space can be divided into the allocatable memory space for the sparse table and the allocatable memory space for the model graph. When the allocatable memory space of the sparse table changes, the first chip can send a change message to instruct the allocation of memory blocks according to the changed allocatable memory space, thereby achieving adaptiveness when the dataset, model, or hardware form changes.

[0016] In some possible implementations, the sparse table initialization request does not include the capacity of the sparse table. The sparse table creation interface does not require the user to adjust parameters for different models, datasets, and hardware forms, and has good generalization, improving the user experience.

[0017] In some possible implementations, the first chip may further receive a query request including a target feature key. Accordingly, the first chip may determine a target address corresponding to the target feature key based on a correspondence between the feature key of the sparse feature and the address of the sparse table, and then access the sparse table based on the target address to obtain a vector corresponding to the target feature key.

[0018] This allows the address of the vector corresponding to the sparse feature to be obtained from the hash map. Based on this address, the corresponding vector can be obtained and used for model training. Because the capacity of the sparse table can be dynamically adjusted based on demand, this solves the problem in traditional methods where the sparse table capacity is too small, causing the gather operator to overflow, leading to training failure and significant resource waste.

[0019] In some possible implementations, the memory block is further used to store optimizer parameters, and the query request is further used to query the optimizer parameters. Accordingly, the first chip can access the sparse table based on the target address to obtain the vector corresponding to the target feature key and the optimizer parameters.

[0020] In this method, the sparse table and optimizer parameters are coupled, and a single query can return the vector of sparse features in the sparse table and the optimizer parameters, reducing the number of operator operations and optimizing query performance.

[0021] In some possible implementations, the computing device may be a heterogeneous device, and the device side of the computing device includes a second chip. Accordingly, when the first chip calls a memory allocation interface to request a memory block, it may call a memory allocation interface of the second chip to request a memory block of the second chip.

[0022] This can fully utilize the computing resources on the device side and improve the efficiency of model training.

[0023] In a second aspect, the present application provides a data processing method. The method can be applied to a second chip on the device side of a computing device. The second chip can be an acceleration card (or accelerator), including but not limited to a graphics processing unit (GPU) and a neural network processing unit (NPU) processor. The second chip is used to train an AI model using sparse features in a data set.

[0024] Specifically, the second chip receives a call request from a memory allocation interface, which is used to apply for a memory block, which is used to store a sparse table. Then, the second chip responds to the call request from the memory allocation interface, allocates a memory block from the video memory on the device side, initializes the sparse table in the memory block, and then returns the address of the sparse table.

[0025] Since the second chip can dynamically allocate memory blocks based on the scale of sparse features, the capacity of the sparse table can be dynamically adjusted according to demand, thereby solving the problem in traditional methods where the capacity of the sparse table is set too small, resulting in gather operator overflow, training failure, and a large amount of resource waste.

[0026] In some possible implementations, the second chip may allocate a first memory block from the device-side video memory in response to the first call request of the memory allocation interface. When detecting that the remaining capacity of the first memory block meets a set condition, the second chip allocates a second memory block from the device-side video memory.

[0027] This allows for dynamic allocation of memory blocks based on demand, meeting the need to store sparse tables during model training while avoiding waste of memory resources.

[0028] In some possible implementations, the second chip can also send a capacity expansion prompt to the first chip on the host side of the computing device. Accordingly, the second chip can allocate a second memory block from the device's video memory in response to a second call request to the memory allocation interface generated by the first chip based on the capacity expansion prompt. This allows for dynamic allocation of memory blocks to meet service needs.

[0029] In some possible implementations, the memory block is also used to store optimizer parameters, and the second chip can also return the vector corresponding to the target feature key in the query request and the optimizer parameters in response to the query request.

[0030] In this way, the vector of sparse features in the sparse table and the optimizer parameters can be returned through a single query, reducing the number of operator operations and optimizing query performance.

[0031] In a third aspect, the present application provides a data processing method. This method is applied to a computing device. The computing device includes a first chip on the host side and a second chip on the device side. The first chip may be a CPU, and the second chip may be a GPU or an NPU. The first and second chips are used in business scripts based on AI models, using sparse features in a dataset to collaboratively train the AI ​​model.

[0032] Specifically, the first chip receives a sparse table initialization request, and then responds to the sparse table initialization request, calls a memory allocation interface to apply for a memory block, which is used to store the sparse table. The number of memory blocks is determined according to the scale of the sparse features in the data set. Accordingly, the second chip responds to the request of the first chip, allocates the memory block from the video memory on the device side, initializes the sparse table in the memory block, and returns the address of the sparse table. The first chip stores the correspondence between the feature key of the sparse feature and the address of the sparse table.

[0033] When initializing a sparse table, this method calls a memory allocation API to request memory blocks. The number of memory blocks is determined based on the size of the sparse features in the dataset. This eliminates the need to set the sparse table's capacity in the sparse table creation API; instead, the capacity is automatically expanded based on the size of the feature keys in the dataset. This API eliminates the need to adjust initialization parameters for different models, datasets, or hardware configurations, resulting in improved generalization, improved training efficiency, and a better user experience.

[0034] In some possible implementations, the first chip calls a memory allocation interface to apply for a memory block, including:

[0035] The first chip sends a first call request of a memory allocation interface to apply for a first memory block, where the first memory block is used to store a vector corresponding to a feature key of a first sparse feature in the data set;

[0036] When the remaining capacity of the first memory block meets the set condition, a second call request of the memory allocation interface is sent to apply for a second memory block, where the second memory block is used to store the vector corresponding to the second sparse feature in the data set.

[0037] In some possible implementations, the method further includes:

[0038] When detecting a change in the allocatable memory space of the sparse table, the first chip sends change information of the allocatable memory space, where the change information is used to instruct allocation of memory blocks according to the changed allocatable memory space.

[0039] In some possible implementations, the method further includes:

[0040] The first chip receives a query request, where the query request includes a target feature key;

[0041] The first chip determines the target address corresponding to the target feature key according to the correspondence between the feature key of the sparse feature and the address of the sparse table;

[0042] The first chip accesses the sparse table according to the target address to obtain a vector corresponding to the target feature key.

[0043] In some possible implementations, the memory block is further used to store optimizer parameters, and the query request is further used to query the optimizer parameters;

[0044] The first chip accesses the sparse table according to the target address to obtain a vector corresponding to the target feature key, including:

[0045] The first chip accesses the sparse table according to the target address to obtain the vector corresponding to the target feature key and the parameters of the optimizer.

[0046] By coupling the optimizer parameters with the sparse table, this method can return the vector of sparse features in the sparse table and the optimizer parameters in one query, reducing the number of operator operations and optimizing query performance.

[0047] In a fourth aspect, the present application provides a data processing device. The device is deployed on a first chip on the host side of a computing device, the first chip is used to train the AI ​​model based on a business script of the AI ​​model using sparse features in a data set, and the device includes:

[0048] A communication module, configured to receive a sparse table initialization request;

[0049] an interface calling module, configured to call a memory allocation interface to apply for memory blocks in response to the sparse table initialization request, wherein the memory blocks are used to store the sparse table, and the number of the memory blocks is determined according to the scale of the sparse features in the data set;

[0050] A storage module is used to store the correspondence between the feature key of the sparse feature and the address of the sparse table.

[0051] In some possible implementations, the interface calling module is specifically configured to:

[0052] Sending a first call request of a memory allocation interface to apply for a first memory block, where the first memory block is used to store a vector corresponding to a feature key of a first sparse feature in the data set;

[0053] When the remaining capacity of the first memory block meets the set conditions, a second call request of the memory allocation interface is sent to apply for a second memory block, and the second memory block is used to store the vector corresponding to the feature key of the second sparse feature in the data set.

[0054] In some possible implementations, the interface calling module is specifically configured to:

[0055] receiving a capacity expansion prompt sent by a second chip on a device side of the computing device;

[0056] A second call request of the memory allocation interface is generated according to the capacity expansion prompt, and the second call request of the memory allocation interface is sent.

[0057] In some possible implementations, the communication module is further configured to:

[0058] When a change in the allocatable memory space of the sparse table is detected, change information of the allocatable memory space is sent, where the change information is used to instruct allocation of memory blocks according to the changed allocatable memory space.

[0059] In some possible implementations, the sparse table initialization request does not include the capacity of the sparse table.

[0060] In some possible implementations, the communication module is further configured to:

[0061] receiving a query request, wherein the query request includes a target feature key;

[0062] The device further comprises:

[0063] The query module is used to determine the target address corresponding to the target feature key according to the correspondence between the feature key of the sparse feature and the address of the sparse table, access the sparse table according to the target address, and obtain the vector corresponding to the target feature key.

[0064] In some possible implementations, the memory block is further used to store optimizer parameters, and the query request is further used to query the optimizer parameters;

[0065] The query module is specifically used for:

[0066] The sparse table is accessed according to the target address to obtain the vector corresponding to the target feature key and the parameters of the optimizer.

[0067] In some possible implementations, the device side of the computing device includes a second chip, and the interface calling module is specifically configured to:

[0068] The memory allocation interface of the second chip is called to apply for the memory block of the second chip.

[0069] In a fifth aspect, the present application provides a data processing device. The device can be deployed on a second chip on the device side of a computing device. The device includes:

[0070] A communication module is used to receive a call request of a memory allocation interface, wherein the call request of the memory allocation interface is used to apply for a memory block, and the memory block is used to store a sparse table;

[0071] an allocation module, configured to allocate a memory block from a display memory on a device side in response to a call request of the memory allocation interface;

[0072] An initialization module, configured to initialize a sparse table in the memory block;

[0073] The communication module is further configured to return the address of the sparse table.

[0074] In some possible implementations, the allocation module is specifically configured to:

[0075] In response to a first call request of the memory allocation interface, allocating a first memory block from a video memory on a device side;

[0076] When it is detected that the remaining capacity of the first memory block meets a set condition, a second memory block is allocated from the video memory on the device side.

[0077] In some possible implementations, the communication module is further configured to:

[0078] Sending a capacity expansion prompt to a first chip on a host side of a computing device;

[0079] The allocation module is specifically used to:

[0080] In response to a second call request of the memory allocation interface generated by the first chip according to the capacity expansion prompt, a second memory block is allocated from the video memory on the device side.

[0081] In some possible implementations, the memory block is also used to store optimizer parameters. The communication module is also used to:

[0082] In response to a query request, a vector corresponding to the target feature key in the query request and the optimizer parameters are returned.

[0083] In a sixth aspect, the present application provides a data processing apparatus. The apparatus can be deployed on a computing device. The computing device includes a first chip on a host side and a second chip on a device side. The apparatus can be deployed in a distributed manner across the first and second chips. The data processing apparatus can include a first communication module, an interface call module, a storage module, a second communication module, an allocation module, and an initialization module.

[0084] Specifically, the first communication module is used to receive a sparse table initialization request, and the interface call module is used to respond to the sparse table initialization request and call the memory allocation interface to apply for a memory block, which is used to store the sparse table. The number of memory blocks is determined according to the scale of the sparse features in the data set. The allocation module is used to allocate the memory block from the video memory on the device side in response to the application of the first chip. The initialization module is used to initialize the sparse table in the memory block. The second communication module is also used to return the address of the sparse table, and the storage module is used to store the correspondence between the feature key of the sparse feature and the address of the sparse table.

[0085] In some possible implementations, the interface calling module is specifically configured to:

[0086] Sending a first call request of a memory allocation interface to apply for a first memory block, where the first memory block is used to store a vector corresponding to a feature key of a first sparse feature in the data set;

[0087] When the remaining capacity of the first memory block meets the set condition, a second call request of the memory allocation interface is sent to apply for a second memory block, where the second memory block is used to store the vector corresponding to the second sparse feature in the data set.

[0088] In some possible implementations, the first communication module is further configured to:

[0089] When detecting a change in the allocatable memory space of the sparse table, the first chip sends change information of the allocatable memory space, where the change information is used to instruct allocation of memory blocks according to the changed allocatable memory space.

[0090] In some possible implementations, the first communication module is further configured to:

[0091] receiving a query request, wherein the query request includes a target feature key;

[0092] The device further comprises:

[0093] A query module is used to access the sparse table according to the target address to obtain the vector corresponding to the target feature key.

[0094] In some possible implementations, the memory block is further used to store optimizer parameters, and the query request is further used to query the optimizer parameters; accordingly, the query module is specifically used to:

[0095] The sparse table is accessed according to the target address to obtain the vector corresponding to the target feature key and the parameters of the optimizer.

[0096] In a seventh aspect, the present application provides a chip. This chip may be, for example, a first chip, comprising a processor and a communication interface, the communication interface being configured to receive computer-readable instructions, and the processor being configured to execute the computer-readable instructions, thereby performing the data processing method described in any implementation of the first aspect of the present application.

[0097] In an eighth aspect, the present application provides a chip. This chip may be, for example, the second chip, comprising a processor and a communication interface, the communication interface being configured to receive computer-readable instructions, and the processor being configured to execute the computer-readable instructions, thereby performing the data processing method as described in any implementation of the second aspect of the present application.

[0098] In a ninth aspect, the present application provides a computing device comprising a first chip on a host side and a second chip on a device side, wherein the first chip is configured to execute the method described in any implementation of the second aspect of the present application.

[0099] In a tenth aspect, the present application provides a computing device cluster. The computing device cluster includes at least one computing device, wherein the at least one computing device includes a first chip on a host side and a second chip on a device side, wherein the first chip is configured to execute the data processing method described in any implementation of the first aspect of the present application.

[0100] In an eleventh aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores instructions for executing the data processing method described in any one of the implementations of the first, second or third aspects above.

[0101] In the twelfth aspect, the present application provides a computer program product containing instructions, which, when running on a chip, a computing device or a computing device cluster, enables the chip, the computing device or the computing device cluster to execute the data processing method described in any one of the implementation methods of the first aspect, the second aspect or the third aspect above.

[0102] Based on the implementation methods provided in the above aspects, this application can also be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0103] In order to more clearly illustrate the technical methods of the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments.

[0104] Figure 1 A schematic diagram of a sparse table initialization provided in an embodiment of the present application;

[0105] Figure 2 A hardware structure diagram of a server provided in an embodiment of the present application;

[0106] Figure 3 A schematic diagram of a software framework deployed on a server provided in an embodiment of the present application;

[0107] Figure 4 An interactive flow chart of a data processing method provided in an embodiment of the present application;

[0108] Figure 5 A schematic diagram of initializing a sparse table provided in an embodiment of the present application;

[0109] Figure 6 A flowchart of a data processing method provided in an embodiment of the present application;

[0110] Figure 7 A flowchart of another data processing method provided in an embodiment of the present application;

[0111] Figure 8 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0112] Figure 9 A schematic diagram of the structure of another data processing device provided in an embodiment of the present application;

[0113] Figure 10 A schematic structural diagram of another data processing device provided in an embodiment of the present application;

[0114] Figure 11 A schematic diagram of the structure of a computing device provided in an embodiment of the present application;

[0115] Figure 12A schematic diagram of the structure of a computing device cluster provided in an embodiment of the present application. DETAILED DESCRIPTION

[0116] The terms "first" and "second" in the embodiments of this application are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Therefore, features specified as "first" or "second" may explicitly or implicitly include one or more of the features.

[0117] First, some technical terms involved in the embodiments of this application are introduced.

[0118] In machine learning, a feature is a measurable and quantifiable property or characteristic of an object, person, or phenomenon. Features can be roughly divided into two categories: sparse features and dense features.

[0119] Sparse features are features that appear discontinuously in a dataset. When sparse features are represented as multidimensional vectors, most of the values ​​in the multidimensional vector are zero. For example, a sample in a dataset can be represented as a 10,000-dimensional vector. The vector for a particular sample can be represented as follows: 0 123:0.1262496448831 2187:0.02032214969320 represents the class label, 123 and 2187 are the feature indices, and the data after the feature indices is the feature representation. In this example, the vector has data in dimensions 123 and 2097, and the remaining dimensions are zero. Originally represented in 10,000 dimensions, the sample is now represented in two dimensions, making it a sparse feature. Sparse features are common in natural language processing (NLP) and recommender systems. For example, item identifiers (IDs) and user IDs in recommender systems are sparse features. Sparse features can be represented using sparse matrices.

[0120] Dense features, also known as dense features, are features that appear frequently or regularly in a dataset. When represented as a multidimensional vector, the majority of the values ​​in the vector are nonzero. Dense features are common in image and speech recognition. Dense features are typically represented as dense vectors. Dense features are generally easier to process due to their higher density of nonzero values.

[0121] The difference between sparse features and dense features lies in the difference in the distribution of the values ​​of sparse features and dense features in the dataset. Sparse features have very few non-zero values, while dense features have many non-zero values. This distribution difference causes machine learning algorithms to perform differently on sparse features and dense features. For example, most machine learning algorithms or frameworks are designed to process dense vectors, and processing units such as graphical processing units (GPUs) also have significant advantages in dense computing. Therefore, in scenarios where sparse features are used for model training, sparse features can be represented as dense vectors.

[0122] Embedding is a form of dense vector representation. Sparse features can be represented as dense vectors using the embedding layer in a neural network. This layer typically includes a large number of embedding parameters. Essentially, the embedding layer constructs a query matrix, or a dictionary (dict) data structure. Embeddings can store data using key-value pairs (KV). For example, in a text recommendation scenario, the embedding layer can represent words in a text as word vectors. Sentence vectors can be derived from the word vectors of multiple words in a sentence, and the recommendation system can make text recommendations based on the similarity of sentence vectors. For word vectors, the dictionary constructed by the embedding layer has a key that represents the index of a word in the vocabulary, and a value that represents the n-dimensional vector representation of the word. The embedding layer can be a fully connected layer that takes a one-hot encoding of the feature as input. The parameter of this fully connected layer is a "word vector table." The query process of the embedding layer is implemented through matrix multiplication using the one-hot input.

[0123] Based on this, when using sparse features for model training, sparse tables can be used to embed sparse features. Specifically, developers can specify the parameters of the created sparse table in the business script of the AI ​​model, such as the capacity of the sparse table, which is recorded as vocabulary size. When a heterogeneous computing device including an accelerator is used for model training, the sparse table may include a sparse table on the device side and a sparse table on the host side. Accordingly, the parameters of the sparse table may include the capacity of the sparse table on the device side (recorded as device vocabulary size) and the capacity of the sparse table on the host side (recorded as host vocabulary size). Then, see Figure 1, the sparse table begins to initialize and creates a vector corresponding to the sparse feature. This vector can be a variable and can be updated iteratively. Next, the HashMap is initialized, specifically to determine the relationship between the feature key of the sparse feature (such as user ID, item ID) and the offset address of the variable corresponding to the sparse feature in the sparse table, and store them. After the communication from the host to the device is initialized, the training process is started. The training process can read the sparse features from the sample set, obtain the offset of the variable corresponding to the sparse feature in the sparse table based on the feature ID of the sparse feature, and access the sparse table based on the offset to obtain the variable corresponding to the sparse feature, thereby realizing model training.

[0124] However, during model training, the dataset can be changed based on business needs. Adjusting the dataset often requires adjusting the parameters used to initialize the sparse table, such as the vocabulary size, which significantly impacts generalization.

[0125] In view of this, the present application provides a data processing method. This method is applied to business scripts based on AI models, and is used in scenarios where sparse features in a data set are used to train AI models. When the sparse table is initialized, the memory allocation interface is called to apply for memory blocks, wherein the number of memory blocks can be determined based on the scale of the sparse features in the data set. In this way, the capacity of the sparse table does not need to be set in the table creation interface of the sparse table, but is automatically expanded by the scale of the feature key in the data set. For different models, data sets or hardware forms, the table creation interface of the sparse table does not require the user to adjust the parameters for initializing the sparse table, has good generalization, improves training efficiency, and enhances user experience.

[0126] The data processing method of the present application can be applied to a computing device or a computing device cluster. The computing device or computing device cluster can execute the data processing method of the present application when training a model. Among them, the computing device can be a server or a terminal. The server can be a cloud server in a cloud environment, or a self-owned physical server, and the terminal can include but is not limited to a desktop computer, a laptop computer, and a smart phone. The computing device can adopt a single-machine single-card architecture, or a single-machine multi-card architecture, and the computing device cluster can be a multi-machine multi-card architecture. It should be noted that the "machine" in the above x machine x card represents the host, and the "card" represents an accelerator card and other devices. Among them, the accelerator card can include but is not limited to a GPU or a neural network processing unit (NPU). The following is an example of a server with a single-machine multi-card architecture as the computing device.

[0127] See also Figure 2As shown in FIG. 1 , a hardware structure diagram of a server is shown, wherein the server 20 includes a host 22 and at least one device 24. The host 22 and the at least one device 24 are connected.

[0128] Host 22 includes a processor and a memory. The processor may be a central processing unit (CPU) and the memory may be a dual in-line memory module (DIMM). The DIMM may be a double data rate (DDR) type, for example, the memory may be a DDR4 DIMM. Figure 2 In the example, host 22 includes four CPUs and four DDR4 DIMM groups. Each CPU is connected to one DDR4 DIMM group, and each DDR4 DIMM group contains eight DDR4 DIMMs. Multiple CPUs in host 22 can be connected to form a hydra mesh.

[0129] Optionally, host 22 further includes an interface, such as one or more of a Serial Advanced Technology Attachment (SATA) interface, a next-generation non-volatile memory express (NVMe) interface, and a Gigabit Ethernet (GE) interface. Host 22 may also include a memory. The memory may include a SATA-supported memory or an NVMe-supported memory, such as a SATA-supported mechanical hard disk or an NVMe-supported solid-state drive (SSD).

[0130] Device 24 includes an accelerator. Figure 2 In the example, the accelerator included in device24 can be a neural network processor NPU. Figure 2 The example of device 24 including eight NPUs is used for illustration. In other possible implementations of the embodiment of the present application, device 24 may also include more accelerators, such as a greater number of accelerators or more types of accelerators.

[0131] Then, see Figure 3The framework diagram of the software deployed on the server shown in the figure shows that the user can install firmware 302 and driver 304 on the server 20. Among them, firmware 302 is usually a program written in read-only memory that can directly control the hardware, interact with the hardware, and check whether the hardware has any errors. Driver 304 is specifically a small piece of code added to the operating system that contains information about the hardware. When a computer program requests to interact with a piece of hardware, driver 304 can act as a converter of instructions between the hardware and the program that uses it. For example, firmware 302 can control device 24, interact with device 24, and check whether device 24 has any errors. Driver 304 can act as a converter of instructions between device 24 and the program that uses it.

[0132] Furthermore, when the hardware architecture of the server 20 adopts a heterogeneous computing architecture (including a computing architecture of computing units using different types of instruction sets), the user can also install a heterogeneous computing framework 306 on the server 20. In a distributed training scenario, the heterogeneous computing framework 306 can be a heterogeneous computing framework for neural networks (Compute Architecture for Neuro Net, CANN). CANN can support users to quickly build AI applications by providing a multi-level programming interface. Among them, AI applications refer to applications built based on trained AI models. It should be noted that the heterogeneous computing framework 306 is an optional framework. The above framework is not installed on the server 20, and the data processing method of the embodiment of the present application can also be executed. The role of the above framework is to improve the efficiency of building AI applications.

[0133] Then, the user can install the deep learning framework 308 on the server 20. The deep learning framework 308 is used to build a large-scale computational graph by compiling the method for implementing the model, and automatically implement the gradient calculation in the computational graph. The computational graph is also called the graph compilation result. In this way, when performing distributed training, the graph compilation result can be executed to perform related calculations for distributed training. Depending on the compilation method, deep learning frameworks can be divided into frameworks that support static compilation and frameworks that support dynamic compilation. Users can choose to install one or more deep learning frameworks 308 on the server 20 according to business needs. In some embodiments, the deep learning framework 308 may not be installed on the server 20. In this case, the server 20 can implement the model from scratch using a programming language such as Python.

[0134] Users can also install an AI software development kit (SDK) 310 on the server 20. SDK 310 provides an extremely easy-to-use application programming interface (API) to accelerate the development of AI applications. SDKs are typically tools that are compatible with accelerators and can achieve acceleration.

[0135] In the embodiment of the present application, the installation package of the data processing device 312 can be encapsulated in the installation package of the SDK 310. When the user installs the SDK 310 on the server 20, the data processing device 312 can also be installed on the server 20 along with the SDK 310. Accordingly, the SDK 310 can be called by a third-party client's training script or customized AI framework, and the third-party client can launch a training task on a training node (e.g., a computing device).

[0136] Furthermore, users can also deploy a model library 314 on the server 20. This model library 314 includes AI models implemented using a unified framework, with standardized parameters and APIs. The AI ​​models include reusable configuration items defined in the unified framework. This reduces the configuration work for the AI ​​models.

[0137] Based on the aforementioned computing device, the present application further provides a data processing method, which will be described below in conjunction with embodiments.

[0138] See also Figure 4 The flowchart of a data processing method shown is applied to a computing device, which includes a first chip located on a host side and a second chip located on a device side. The first chip can be a CPU, and the second chip can be an accelerator card such as a GPU or NPU. The first chip and the second chip are used for business scripts based on an AI model, and sparse features in a data set are used to collaboratively train the AI ​​model. The method includes the following steps:

[0139] S402: The first chip receives a sparse table initialization request.

[0140] The sparse table initialization request is used to initialize the sparse table to provide data support for model training. During specific implementation, the sparse table initialization request can be generated based on the table building interface of the sparse table. In an embodiment of the present application, the table building interface of the sparse table does not include the capacity of the sparse table. In other words, the user does not need to set the capacity of the sparse table, including the capacity of the sparse table on the device side (device vocabulary size) and the capacity of the sparse table on the host side (host vocabulary size).

[0141] S404: The first chip calls a memory allocation interface to apply for a memory block in response to the sparse table initialization request.

[0142] Specifically, the first chip can initialize the sparse table in response to the sparse table initialization request. When initializing the sparse table, the first chip can call the memory allocation interface to apply for a memory block, thereby storing the sparse table through the memory block. The memory allocation interface includes but is not limited to aclmalloc. Among them, the size of the memory block allocated using the memory allocation interface is usually fixed. For example, the memory block allocated by a memory allocation operation can be 200 megabytes (Megabyte, MB). The first chip can call the memory allocation interface multiple times according to the scale of the sparse features in the data set, thereby applying for multiple memory blocks. The number of memory blocks can be determined according to the scale of the sparse features in the data set. Among them, the scale of the sparse features can be characterized by the number of sparse features, such as the number of non-repeated sparse features.

[0143] Taking into account that the sparse features in the data set can be read in batches, the first chip can first apply for a small number of memory blocks, and then gradually expand the capacity. Specifically, the first chip can send a first call request of the memory allocation interface to apply for the first memory block. The first memory block is used to store the vector corresponding to the feature key of the first sparse feature in the data set. Among them, the first sparse feature can be the sparse feature of the first batch. When the remaining capacity of the first memory block meets the set conditions, the first chip can send a second call request of the memory allocation interface to apply for the second memory block. The second memory block is used to store the vector corresponding to the second sparse feature in the data set. Among them, the second sparse feature can be the sparse feature of the second batch. It should be noted that the first batch and the second batch can be one batch or multiple batches.

[0144] exist Figure 4 In the example, the first chip can send a call request (e.g., a first call request, a second call request) to the second chip based on the memory allocation interface of the second chip to request a memory block. In this case, the memory block requested by the first chip is a memory block in the device-side video memory (also called a video memory block).

[0145] In some possible implementations, the first chip may also generate a memory allocation interface call request based on the memory allocation interface of the first chip to apply for a memory block. In this case, the memory block applied for by the first chip may be a memory block in the host memory.

[0146] S406 : In response to the request of the first chip, the second chip allocates the memory block from the video memory on the device side and initializes the sparse table in the memory block.

[0147] Device memory is also called video memory. The second chip can allocate a memory block from the device's video memory in response to a request from the first chip, such as a call request to a memory allocation interface sent by the first chip, to store the sparse table. The video memory of the second chip can be divided into allocatable memory space for the sparse table and allocatable memory space for the model graph. The second chip can allocate a memory block from the allocatable memory space for the sparse table divided in the video memory.

[0148] The memory blocks allocated by the second chip using the memory allocation interface are typically fixed in size. The second chip can determine the number of memory blocks to allocate based on the scale of sparse features in the dataset, such as the scale of feature keys. Since sparse features in a dataset are typically read and trained in batches, the second chip can allocate the required number of memory blocks multiple times.

[0149] In some possible implementations, the second chip may allocate the first memory block from the device-side video memory in response to the first call request of the memory allocation interface. When it is detected that the remaining capacity of the first memory block meets a set condition, such as the remaining capacity being less than a set value or a set ratio, where the set value may be 0 or 5%, the second chip may allocate the second memory block from the device-side video memory.

[0150] The second chip can also send a capacity expansion prompt to the first chip on the host side of the computing device when it detects that the remaining capacity of the first memory block meets a set condition. The capacity expansion prompt is used to prompt the first chip to call the memory allocation interface again to apply for a new memory block to expand the sparse table. Accordingly, the first chip can generate a second call request to the memory allocation interface based on the capacity expansion prompt and send the second call request to the second chip. The second chip can then allocate a second memory block from the device-side video memory.

[0151] Furthermore, when the available memory allocation space of the sparse table in the device-side video memory is insufficient, the first chip may allocate a new memory block (such as a second memory block) from the host-side memory. In some possible implementations, when the available memory allocation space of the sparse table in the device-side video memory is insufficient, the second chip may also use an elimination mechanism to eliminate vectors corresponding to feature keys with usage rates less than a preset value from the sparse table in order to write vectors corresponding to new feature keys.

[0152] The second chip may initialize the sparse table in the memory block by creating a variable corresponding to the feature key, thereby storing the vector corresponding to the feature key in the sparse table in the form of a variable. Figure 5As shown, the sparse features in the data set are usually read in batches. When the feature keys of a batch of sparse features arrive at the first chip, the second chip can read the feature keys one by one. When the vector corresponding to the feature key is not stored in the sparse table, the feature key can be mapped and the vector corresponding to the feature key can be stored in the memory block. When the remaining capacity of the memory block meets the set conditions, the second chip can send a capacity expansion prompt. In response to the capacity expansion prompt, the first chip can send a call request to the memory allocation interface to apply for a new memory block. The second chip can allocate a new memory block in response to the call request of the memory allocation interface. Accordingly, when the second chip reads a new feature key, it can store the vector corresponding to the new feature key in the new memory block.

[0153] During the model training process, the data set may change, the AI ​​model trained using the data set may also change, or the hardware form (such as the NPU form) that performs the training task may also change. The video memory capacity of different NPUs may be different, resulting in different allocatable memory space for the sparse table. The model graphs of different AI models are different, and the allocatable memory space of the model graphs may also be different. When the video memory capacity is fixed, the allocatable memory space of the sparse table may be different. The first chip can sense whether the allocatable memory space of the sparse table has changed. When a change in the allocatable memory space of the sparse table is detected, the first chip can send change information of the allocatable memory space, which is used to indicate the allocation of memory blocks according to the changed allocatable memory space.

[0154] S408: The second chip returns the address of the sparse table.

[0155] The address of the sparse table can be an address (addr) obtained by allocating memory space during runtime. Different from an offset address (offset), the address addr of the sparse table can be used to directly access a memory block to obtain the sparse table.

[0156] It should be noted that the sparse table of the present application can be adjusted dynamically. For example, the second chip can dynamically adjust the capacity of the sparse table according to the scale of the sparse features in the data set (for example, the scale of the feature keys of the sparse features). When the second chip allocates a new memory block to expand the capacity of the sparse table, the address of the new sparse table can be returned. This can avoid the problem that when a new feature key arrives, the capacity of the sparse table is too small to store the corresponding offset, which in turn causes the gather operator used for querying during training to overflow and the training to fail.

[0157] S410: The first chip stores the correspondence between the feature key of the sparse feature and the address of the sparse table.

[0158] In this embodiment, since the return value of allocating the memory block is the address addr of the sparse table instead of offset, the first chip can store the correspondence between the feature key of the sparse feature and the address of the sparse table when the hash map is initialized.

[0159] During model training, the first chip receives a query request that includes a target feature key (e.g., target user ID, target item ID). Accordingly, the first chip can determine the target address corresponding to the target feature key based on the correspondence between the feature key of the sparse feature and the address of the sparse table. The first chip then accesses the sparse table based on the target address to obtain the variable corresponding to the target feature key.

[0160] Furthermore, model training not only requires embedding layers to vectorize sparse features, but also typically involves the use of optimizers for parameter optimization. During the backpropagation (BP) process in deep learning, optimizers guide the parameters of the loss function (also known as the objective function) to the correct values. The updated parameters ensure that the loss function (or objective function) approaches the global minimum.

[0161] Some parameters of the optimizer are related to the feature key, for example, the momentum and acceleration of the optimizer are related to the feature key. Therefore, the first chip can store the above parameters of the optimizer in the sparse table when applying for video memory to initialize the sparse table. That is, the memory block is also used to store the parameters of the optimizer. Accordingly, the query request can also be used to query the parameters of the optimizer. The first chip can access the sparse table according to the target address to obtain the variables corresponding to the target feature key and the parameters of the optimizer. Among them, the first chip can obtain the table items in the sparse table through the target address, and by splitting the content of the table items, it can obtain the variables corresponding to the target feature key and the parameters of the optimizer. In this way, the contents of the sparse table and the optimizer can be returned in one query, which reduces the number of operator operations and optimizes performance.

[0162] Based on the above description, it can be seen that the data processing method of the present application calls the memory allocation interface to apply for memory blocks when the sparse table is initialized, wherein the number of memory blocks is determined according to the scale of the sparse features in the data set. In this way, there is no need to set the capacity of the sparse table in the table creation interface of the sparse table, but the capacity is automatically expanded by the scale of the feature key in the data set. For different models, data sets or hardware forms, the table creation interface of the sparse table does not require the user to adjust the parameters of the initialized sparse table, has good generalization, improves training efficiency, and enhances user experience.

[0163] The above embodiment describes the sparse table initialization process from the perspective of collaboration between the first chip on the host side and the second chip on the device side. In actual use, the first chip on the host side can also request host-side memory to initialize the sparse table. The following describes a data processing method provided by this application from the perspective of the first chip, combined with an embodiment.

[0164] See also Figure 6 The flowchart of a data processing method shown in FIG. 1 is a flowchart of a data processing method that can be executed by a first chip. The first chip is used for a business script based on an AI model, and the sparse features in a data set are used to train the AI ​​model. The method includes the following steps:

[0165] S602: The first chip receives a sparse table initialization request.

[0166] A sparse table initialization request is used to initialize a sparse table. This request can be generated by the sparse table creation API. The sparse table creation API does not accept the sparse table capacity as input parameters; accordingly, the sparse table initialization request does not include the sparse table capacity.

[0167] S604: The first chip calls a memory allocation interface to apply for a memory block in response to the sparse table initialization request.

[0168] Memory blocks are used to store sparse tables. The number of memory blocks can be determined based on the size of the sparse features in the dataset. Different memory blocks can store vectors corresponding to different feature keys of sparse features.

[0169] In a specific implementation, the first chip can generate a first call request for the memory allocation interface according to the memory allocation interface, and apply for the first memory block based on the first call request. When the first chip applies for memory on the host side, the first chip can generate a first call request according to the memory allocation interface on the host side. When the first chip applies for memory on the device side, the first chip can generate a first call request according to the memory allocation interface on the device side and send the first call request of the memory allocation interface to the second chip. The first memory block is used to store the vector corresponding to the feature key of the first sparse feature in the data set.

[0170] Furthermore, when the remaining capacity of the first memory block meets the set conditions, the first chip can send a second call request of the memory allocation interface to apply for a second memory block. The second memory block is used to store the vector corresponding to the feature key of the second sparse feature in the data set. It should be noted that the first chip can receive the expansion prompt sent by the second chip to determine that the remaining capacity of the first memory block meets the set conditions (for example, the remaining capacity is insufficient). In other words, the first chip can receive the expansion prompt sent by the second chip on the device side of the computing device, generate a second call request of the memory allocation interface according to the expansion prompt, and then send the second call request of the memory allocation interface.

[0171] Taking into account that the dataset may change during the model training process, the AI ​​model trained using the dataset may change, and the hardware form for executing the training task may change, the above changes may cause the allocatable memory space of the sparse table to change. When the first chip detects a change in the allocatable memory space of the sparse table, it may send information about the change in the allocatable memory space. In this way, when the first chip applies for a new memory block, it may allocate the memory block according to the changed allocatable memory space. Among them, when the first chip applies for a new memory block from the second chip, the second chip may allocate the memory block according to the changed allocatable memory space.

[0172] S606: The first chip stores the correspondence between the feature key of the sparse feature and the address of the sparse table.

[0173] The first chip can store the correspondence between the feature key of the sparse feature and the address of the sparse table according to the address addr of the sparse table returned by the allocated memory block when the hash map is initialized. The first chip can use a KV storage method to store the correspondence between the feature key of the sparse feature and the address of the sparse table. For example, the key can be the feature key of the sparse feature, and the value can be the address addr of the sparse table.

[0174] During model training, the first chip receives a query request that includes a target feature key (e.g., target user ID, target item ID). Accordingly, the first chip can determine the target address corresponding to the target feature key based on the correspondence between the feature key of the sparse feature and the address of the sparse table. The first chip then accesses the sparse table based on the target address to obtain the variable corresponding to the target feature key.

[0175] The memory block requested by the first chip is also used to store optimizer parameters. The query request is also used to query the optimizer parameters. Accordingly, the first chip can obtain the variables corresponding to the target feature key and the optimizer parameters based on the target address.

[0176] Since the accelerator card on the device side generally has higher processing efficiency and better performance for dense vectors, a sparse table can be initialized on the device side to improve model training efficiency. The second chip on the device side can collaborate with the first chip on the host side to implement the data processing method of this application. The following describes a data processing method provided by this application from the perspective of the second chip and in combination with an embodiment.

[0177] See also Figure 7 A flowchart of a data processing method is shown, which can be performed by a second chip. The second chip is used to train an AI model using sparse features in a data set. The method includes the following steps:

[0178] S702: The second chip receives a call request of a memory allocation interface.

[0179] The memory allocation interface may be a memory allocation interface provided by the device, such as aclmalloc. The first chip may call the memory allocation interface to generate a call request for the memory allocation interface. The call request for the memory allocation interface is used to apply for a memory block, which is used to store the sparse table.

[0180] The second chip can receive one or more call requests depending on the size of sparse features in the dataset, such as the size of feature keys. Given that sparse features in a dataset are typically read in batches, the second chip can receive a call request from the next memory allocation interface (e.g., a second call request) only if the remaining capacity of the memory block requested by the previous memory allocation interface call request (e.g., the first call request) meets a set condition.

[0181] S704 : The second chip allocates a memory block from the video memory on the device side in response to the call request of the memory allocation interface, and initializes a sparse table in the memory block.

[0182] Specifically, the second chip can allocate a first memory block from the video memory on the device side in response to the first call request of the memory allocation interface, and when it detects that the remaining capacity of the first memory block meets the set conditions, the second chip allocates a second memory block from the video memory on the device side. Furthermore, the second chip can also send a capacity expansion prompt to the first chip on the host side of the computing device when it detects that the remaining capacity of the first memory block meets the set conditions. Accordingly, the first chip can generate a second call request of the memory allocation interface based on the capacity expansion prompt, and send the second call request of the memory allocation interface to the second chip. The second chip can allocate a second memory block from the video memory on the device side in response to the second call request of the memory allocation interface.

[0183] S704: The second chip returns the address of the sparse table.

[0184] The address of the sparse table may be an address (addr) obtained by allocating memory space during runtime. The address addr of the sparse table may be used to directly access the memory block, thereby obtaining the sparse table.

[0185] It should be noted that the sparse table of this application can be dynamically adjusted. For example, the second chip can dynamically adjust the capacity of the sparse table based on the scale of sparse features in the dataset (e.g., the scale of the feature keys of the sparse features). This can avoid the problem that when a new feature key arrives, the sparse table capacity is too small to store the corresponding offset, which in turn causes the gather operator used for querying during training to overflow and training failure.

[0186] In some possible implementations, the memory block is also used to store optimizer parameters. The second chip can also respond to a query request and return the variables and optimizer parameters corresponding to the target feature key in the query request, thereby returning the sparse table and optimizer contents in one query, reducing the number of operator operations and optimizing performance.

[0187] Based on the aforementioned method embodiment, the present application further provides a data processing device 312. The data processing device 312 of the embodiment of the present application is introduced below from the perspective of functional modularization.

[0188] See also Figure 8 The data processing device 312 is a schematic structural diagram of a data processing device 312 deployed on a first chip on the host side of a computing device. The first chip is used for training the AI ​​model using sparse features in a data set based on a business script of the AI ​​model. The device 312 includes:

[0189] Communication module 802, configured to receive a sparse table initialization request;

[0190] An interface calling module 804 is configured to call a memory allocation interface to apply for memory blocks in response to the sparse table initialization request. The memory blocks are used to store the sparse table, and the number of the memory blocks is determined according to the size of the sparse features in the data set.

[0191] The storage module 806 is configured to store the correspondence between the feature key of the sparse feature and the address of the sparse table.

[0192] Exemplarily, the communication module 802 , the interface calling module 804 , and the storage module 806 may be implemented by software.

[0193] When implemented via software, the communication module 802, interface call module 804, and storage module 806 can be applications running on a computing device. These applications can be provided to users as virtualized services. Virtualized services can include virtual machine (VM) services, bare metal server (BMS) services, and container services. VM services can use virtualization technology to create a virtual machine (VM) resource pool on multiple physical hosts (such as computing devices) to provide users with VMs on demand. BMS services create a virtual BMS resource pool on multiple physical hosts to provide users with BMSs on demand. Container services create a virtual container resource pool on multiple physical hosts to provide users with containers on demand. A VM is a simulated virtual computer, or logically a single computer. BMS is a scalable, high-performance computing service with computing performance comparable to traditional physical machines and secure physical isolation. Containers are a kernel virtualization technology that provides lightweight virtualization to isolate user space, processes, and resources. It should be understood that the VM service, BMS service and container service in the above-mentioned virtualization services are merely specific examples. In actual applications, the virtualization service may also be other lightweight or heavyweight virtualization services, which are not specifically limited here.

[0194] When the communication module 802 and the storage module 806 can also be implemented by hardware, the communication module 802 can be a communication interface, and the storage module 806 can be a memory.

[0195] In some possible implementations, the interface calling module 804 is specifically configured to:

[0196] Sending a first call request of a memory allocation interface to apply for a first memory block, where the first memory block is used to store a vector corresponding to a feature key of a first sparse feature in the data set;

[0197] When the remaining capacity of the first memory block meets the set conditions, a second call request of the memory allocation interface is sent to apply for a second memory block, and the second memory block is used to store the vector corresponding to the feature key of the second sparse feature in the data set.

[0198] In some possible implementations, the interface calling module 804 is specifically configured to:

[0199] receiving a capacity expansion prompt sent by a second chip on a device side of the computing device;

[0200] A second call request of the memory allocation interface is generated according to the capacity expansion prompt, and the second call request of the memory allocation interface is sent.

[0201] In some possible implementations, the communication module 802 is further configured to:

[0202] When a change in the allocatable memory space of the sparse table is detected, change information of the allocatable memory space is sent, where the change information is used to instruct allocation of memory blocks according to the changed allocatable memory space.

[0203] In some possible implementations, the sparse table initialization request does not include the capacity of the sparse table.

[0204] In some possible implementations, the communication module 802 is further configured to:

[0205] receiving a query request, wherein the query request includes a target feature key;

[0206] The device 312 further includes:

[0207] The query module 808 is used to determine the target address corresponding to the target feature key according to the correspondence between the feature key of the sparse feature and the address of the sparse table, access the sparse table according to the target address, and obtain the vector corresponding to the target feature key.

[0208] Similar to the interface calling module 804, the query module 808 can be implemented by software. When implemented by software, the query module 808 can be an application running on a computing device, such as a VM service or a container service.

[0209] In some possible implementations, the memory block is further used to store optimizer parameters, and the query request is further used to query the optimizer parameters;

[0210] The query module 808 is specifically used to:

[0211] The sparse table is accessed according to the target address to obtain the vector corresponding to the target feature key and the parameters of the optimizer.

[0212] In some possible implementations, the device side of the computing device includes a second chip, and the interface calling module 804 is specifically configured to:

[0213] The memory allocation interface of the second chip is called to apply for the memory block of the second chip.

[0214] See also Figure 9The data processing device 312 is a schematic structural diagram of another data processing device 312. The data processing device 312 can be deployed in a second chip on the device side of a computing device. The second chip is used to train an AI model using sparse features in a data set. The device 312 includes:

[0215] The communication module 902 is used to receive a call request of a memory allocation interface, where the call request of the memory allocation interface is used to apply for a memory block, where the memory block is used to store a sparse table;

[0216] an allocation module 904 for allocating a memory block from a display memory on a device side in response to a call request of the memory allocation interface;

[0217] Initialization module 906, configured to initialize a sparse table in the memory block;

[0218] The communication module 902 is further configured to return the address of the sparse table.

[0219] Exemplarily, the communication module 902 , allocation module 904 , and initialization module 906 may be implemented by software.

[0220] When implemented in software, the communication module 902, the allocation module 904, and the initialization module 906 may be applications running on a computing device. The applications may be provided to users as virtualized services. The virtualized services may include VM services, BMS services, and container services.

[0221] The communication module 802 may also be implemented by hardware, for example, the communication module 802 may be a communication interface.

[0222] In some possible implementations, the allocation module 904 is specifically configured to:

[0223] In response to a first call request of the memory allocation interface, allocating a first memory block from a video memory on a device side;

[0224] When it is detected that the remaining capacity of the first memory block meets a set condition, a second memory block is allocated from the video memory on the device side.

[0225] In some possible implementations, the communication module 902 is further configured to:

[0226] Sending a capacity expansion prompt to a first chip on a host side of a computing device;

[0227] The allocation module 904 is specifically configured to:

[0228] In response to a second call request of the memory allocation interface generated by the first chip according to the capacity expansion prompt, a second memory block is allocated from the video memory on the device side.

[0229] In some possible implementations, the memory block is also used to store optimizer parameters. The communication module 902 is also used to:

[0230] In response to a query request, a vector corresponding to the target feature key in the query request and the optimizer parameters are returned.

[0231] See also Figure 10 The following is a schematic diagram of the structure of another data processing device 312, which can be deployed in a computing device. The computing device includes a first chip on the host side and a second chip on the device side. The device 312 can be deployed in a distributed manner on the first chip and the second chip. The data processing device may include a first communication module 1002, an interface call module 1004, a storage module 1006, a second communication module 1003, an allocation module 1005, and an initialization module 1007.

[0232] Specifically, the first communication module 1002 is used to receive a sparse table initialization request, and the interface call module 1004 is used to respond to the sparse table initialization request and call the memory allocation interface to apply for a memory block, which is used to store the sparse table. The number of memory blocks is determined according to the scale of the sparse features in the data set. The allocation module 1005 is used to allocate the memory block from the video memory on the device side in response to the application of the first chip. The initialization module 1007 is used to initialize the sparse table in the memory block. The second communication module 1003 is also used to return the address of the sparse table, and the storage module 1006 is used to store the feature key of the sparse feature and the correspondence between the address of the sparse table.

[0233] For example, the first communication module 1002, the interface calling module 1004, the storage module 1006, and the second communication module 1003, the allocation module 1005, and the initialization module 1007 can be implemented by software. Specifically, the first communication module 1002, the interface calling module 1004, and the storage module 1006 can be software deployed on the host side, and the second communication module 1003, the allocation module 1005, and the initialization module 1007 can be software deployed on the device side.

[0234] When implemented via software, the first communication module 1002, the interface calling module 1004, the storage module 1006, the second communication module 1003, the allocation module 1005, and the initialization module 1007 may be applications running on a computing device. The applications may be provided to users as virtualized services. Virtualized services may include VM services, BMS services, and container services.

[0235] The first communication module 1002 and the second communication module 1003 may also be implemented by hardware, for example, the first communication module 1002 and the second communication module 1003 may be communication interfaces. The storage module 1006 may also be implemented by hardware, for example, it may be a memory.

[0236] In some possible implementations, the interface calling module 1004 is specifically configured to:

[0237] Sending a first call request of a memory allocation interface to apply for a first memory block, where the first memory block is used to store a vector corresponding to a feature key of a first sparse feature in the data set;

[0238] When the remaining capacity of the first memory block meets the set condition, a second call request of the memory allocation interface is sent to apply for a second memory block, where the second memory block is used to store the vector corresponding to the second sparse feature in the data set.

[0239] In some possible implementations, the first communication module 1002 is further configured to:

[0240] When detecting a change in the allocatable memory space of the sparse table, the first chip sends change information of the allocatable memory space, where the change information is used to instruct allocation of memory blocks according to the changed allocatable memory space.

[0241] In some possible implementations, the first communication module 1002 is further configured to:

[0242] receiving a query request, wherein the query request includes a target feature key;

[0243] The device 312 further includes:

[0244] The query module 1008 is configured to access the sparse table according to the target address to obtain a vector corresponding to the target feature key.

[0245] In some possible implementations, the memory block is further used to store optimizer parameters, and the query request is further used to query the optimizer parameters; accordingly, the query module 1008 is specifically configured to:

[0246] The sparse table is accessed according to the target address to obtain the vector corresponding to the target feature key and the parameters of the optimizer.

[0247] The present application also provides a chip. The chip includes a processor and a communication interface. The communication interface is used to receive computer-readable instructions, and the processor is used to execute the computer-readable instructions, thereby executing the data processing method of the aforementioned embodiment. In some embodiments, the chip can be a first chip, such as a CPU on the host side, the communication interface is used to receive computer-readable instructions, and the processor is used to execute the computer-readable instructions to execute the method or steps performed by the first chip in the aforementioned embodiment. In other embodiments, the chip can be a second chip, such as an NPU on the device side, the communication interface is used to receive computer-readable instructions, and the processor is used to execute computer-readable instructions to execute the method or steps performed by the second chip in the aforementioned embodiment.

[0248] The present application also provides a computing device 1100. Figure 11 As shown, computing device 1100 includes a bus 1102, a processor 1104, an accelerator 1105, a memory 1106, and a communication interface 1108. Processor 1104, accelerator 1105, memory 1106, and communication interface 1108 communicate with each other via bus 1102. Computing device 1100 can be a server or a terminal device. It should be understood that this application does not limit the number of processors, accelerators, and memories in computing device 1100.

[0249] The bus 1102 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 11 The bus 1102 may include a path for transmitting information between various components of the computing device 1100 (eg, memory 1106, processor 1104, accelerator 1105, communication interface 1108).

[0250] The processor 1104 may include a central processing unit (CPU), and the accelerator may include any one or more processors such as a graphics processing unit (GPU) and a neural network processing unit (NPU).

[0251] The memory 1106 may include a volatile memory, such as a random access memory (RAM). The memory 1106 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD). The memory 1106 stores executable program code, and the processor 1104 executes the executable program code to implement the aforementioned data processing method. Specifically, the memory 1106 stores instructions for executing the data processing method.

[0252] The communication interface 1108 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1100 and other devices or a communication network.

[0253] Embodiments of the present application also provide a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smartphone.

[0254] like Figure 12 As shown, the computing device cluster includes at least one computing device 100. The memory 106 in one or more computing devices 100 in the computing device cluster may store the same data processing apparatus for executing instructions of the data processing method.

[0255] In some possible implementations, one or more computing devices 100 in the computing device cluster may also be used to execute part of the instructions of the data processing method. In other words, a combination of one or more computing devices 100 may jointly execute the instructions of the data processing method.

[0256] It should be noted that the memories 106 in different computing devices 100 in the computing device cluster may store different instructions for executing part of the functions of the data processing apparatus.

[0257] The embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that can be stored by a computing device or a data storage device such as a data center that contains one or more available media. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to execute the above-mentioned data processing method applied to the data processing device.

[0258] The present application also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored on any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes the aforementioned data processing method.

[0259] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the protection scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A data processing method, characterized in that: A first chip applied to a host side of a computing device, the first chip being used for a business script based on an artificial intelligence (AI) model, using sparse features in a data set to train the AI ​​model, the method comprising: Receive a sparse table initialization request; In response to the sparse table initialization request, calling a memory allocation interface to apply for a memory block, the memory block is used to store the sparse table, and the number of the memory blocks is determined according to the scale of the sparse features in the data set; The correspondence between the feature key of the sparse feature and the address of the sparse table is stored.

2. The method according to claim 1, characterized in that The calling of the memory allocation interface to apply for a memory block includes: Sending a first call request of a memory allocation interface to apply for a first memory block, where the first memory block is used to store a vector corresponding to a feature key of a first sparse feature in the data set; When the remaining capacity of the first memory block meets the set conditions, a second call request of the memory allocation interface is sent to apply for a second memory block, and the second memory block is used to store the vector corresponding to the feature key of the second sparse feature in the data set.

3. The method according to claim 2, characterized in that When the remaining capacity of the first memory block meets the set condition, sending the second call request of the memory allocation interface includes: receiving a capacity expansion prompt sent by a second chip on a device side in the computing device; A second call request of the memory allocation interface is generated according to the capacity expansion prompt, and the second call request of the memory allocation interface is sent.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: When a change in the allocatable memory space of the sparse table is detected, change information of the allocatable memory space is sent, where the change information is used to indicate allocation of memory blocks according to the changed allocatable memory space.

5. The method according to any one of claims 1 to 4, characterized in that: The sparse table initialization request does not include the capacity of the sparse table.

6. The method according to any one of claims 1 to 5, characterized in that: The method further comprises: receiving a query request, wherein the query request includes a target feature key; Determine the target address corresponding to the target feature key according to the correspondence between the feature key of the sparse feature and the address of the sparse table; The sparse table is accessed according to the target address to obtain a vector corresponding to the target feature key.

7. The method according to claim 6, characterized in that The memory block is also used to store the parameters of the optimizer, and the query request is also used to query the parameters of the optimizer; The step of accessing the sparse table according to the target address to obtain a vector corresponding to the target feature key includes: The sparse table is accessed according to the target address to obtain the vector corresponding to the target feature key and the parameters of the optimizer.

8. The method according to any one of claims 1 to 7, characterized in that: The device side of the computing device includes a second chip, and calling the memory allocation interface to apply for a memory block includes: The memory allocation interface of the second chip is called to apply for the memory block of the second chip.

9. A data processing device, characterized in that: The device is deployed on a first chip on the host side of a computing device, the first chip is used for training the AI ​​model based on a business script of an artificial intelligence AI model using sparse features in a data set, and the device includes: A communication module, used for receiving a sparse table initialization request; An interface calling module, configured to call a memory allocation interface to apply for a memory block in response to the sparse table initialization request, wherein the memory block is used to store the sparse table, and the number of the memory blocks is determined according to the scale of the sparse features in the data set; A storage module is used to store the correspondence between the feature key of the sparse feature and the address of the sparse table.

10. The device according to claim 9, characterized in that The interface calling module is specifically used for: Sending a first call request of a memory allocation interface to apply for a first memory block, where the first memory block is used to store a vector corresponding to a feature key of a first sparse feature in the data set; When the remaining capacity of the first memory block meets the set conditions, a second call request of the memory allocation interface is sent to apply for a second memory block, and the second memory block is used to store the vector corresponding to the feature key of the second sparse feature in the data set.

11. The device according to claim 10, characterized in that The interface calling module is specifically used for: receiving a capacity expansion prompt sent by a second chip on a device side in the computing device; A second call request of the memory allocation interface is generated according to the capacity expansion prompt, and the second call request of the memory allocation interface is sent.

12. The device according to any one of claims 9 to 11, characterized in that The communication module is also used for: When a change in the allocatable memory space of the sparse table is detected, change information of the allocatable memory space is sent, where the change information is used to indicate allocation of memory blocks according to the changed allocatable memory space.

13. The device according to any one of claims 9 to 12, characterized in that The sparse table initialization request does not include the capacity of the sparse table.

14. The device according to any one of claims 9 to 13, characterized in that The communication module is also used for: receiving a query request, wherein the query request includes a target feature key; The device also includes: A query module is used to determine the target address corresponding to the target feature key according to the correspondence between the feature key of the sparse feature and the address of the sparse table, access the sparse table according to the target address, and obtain the vector corresponding to the target feature key.

15. The method according to claim 14, characterized in that The memory block is also used to store the parameters of the optimizer, and the query request is also used to query the parameters of the optimizer; The query module is specifically used for: The sparse table is accessed according to the target address to obtain the vector corresponding to the target feature key and the parameters of the optimizer.

16. The device according to any one of claims 9 to 15, characterized in that The device side of the computing device includes a second chip, and the interface calling module is specifically used for: The memory allocation interface of the second chip is called to apply for the memory block of the second chip.

17. A chip, characterized in that: The chip comprises a processor and a communication interface, wherein the communication interface is used to receive computer-readable instructions, and the processor is used to execute the computer-readable instructions, thereby executing the method according to any one of claims 1 to 8.

18. A computing device, characterized in that The computing device comprises a first chip on a host side and a second chip on a device side, wherein the first chip is used to execute the method according to any one of claims 1 to 8.

19. A computing device cluster, characterized in that: The computing device cluster includes at least one computing device, and the at least one computing device includes a first chip on a host side and a second chip on a device side, and the first chip is used to execute the method according to any one of claims 1 to 8.

20. A computer-readable storage medium, characterized in that: The method comprises computer-readable instructions; the computer-readable instructions are used to implement the method according to any one of claims 1 to 8.

21. A computer program product, characterized in that The method comprises computer-readable instructions; the computer-readable instructions are used to implement the method according to any one of claims 1 to 8.

Images