A data filtering method, device and storage medium

Abstract

This application provides a data filtering method, device, and storage medium, relating to the field of artificial intelligence chip technology. The method includes: performing multiple rounds of iterative filtering on a target dataset using a thread bundle and a target queue, and obtaining the target filtering result based on the K data items retained in the target queue at the time of termination; each round of iterative filtering includes: reading data from the target dataset in parallel using multiple threads in the thread bundle; for the first thread whose read data meets the enqueueing condition, writing the corresponding read data into the target queue in parallel according to the write offset of each first thread to achieve conflict-free parallel enqueueing; when the target queue is full or the target dataset has been read, sorting the data in the target queue and removing data other than the first K data items from the target queue; and updating the enqueueing condition based on the Kth data item in the target queue; this greatly reduces the amount of data involved in sorting, thereby reducing chip resource consumption and improving the efficiency of data filtering.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence chip technology, and in particular to a data filtering method, device and storage medium. Background Technology

[0002] With the development of artificial intelligence technology, the application of AI chips is becoming increasingly widespread. Handling large-scale Top-K selection problems is a common scenario for AI chips. The Top-K selection problem refers to selecting K data points from N data points according to preset rules. This problem is a type of problem between sorting problems (when K=N) and finding the maximum value (when K=1), where N is greater than 0, K is greater than 0, and K<=N. For example, in the inference process of large language models, the goal is to quickly select the K tokens with the highest probabilities from a very large vocabulary (tens or hundreds of thousands of words). In this case, K is typically in the tens to hundreds.

[0003] In related technologies, the N data points are first fully sorted to obtain the sorted result; then, the data points that are among the top K in the sorted result are selected from the N data points. However, in scenarios involving large-scale data (i.e., N is much larger than K), fully sorting the N data points requires a large amount of chip resources and has low processing efficiency. Summary of the Invention

[0004] This application provides a data filtering method, device, and storage medium to reduce chip resource consumption in processing Top-K selection problems while improving processing efficiency.

[0005] On the one hand, embodiments of this application provide a data filtering method, the method comprising: The target dataset is subjected to multiple rounds of iterative filtering using thread bundles and target queues until the iteration stops when the stopping condition is met. The target filtering result is obtained based on the K data points retained in the target queue at the time of termination, where K is greater than 0. Each round of iterative screening includes the following steps: Data is read from the target dataset in parallel by multiple threads in the thread bundle; Data that meets the enqueue conditions of the target queue from the data read by each of the multiple threads is written into the target queue in parallel. When the target queue is full or the thread bundle has read the target dataset, sort the multiple data in the target queue and remove the data in the target queue except for the first K data; Update the enqueue condition based on the Kth data in the target queue.

[0006] On one hand, embodiments of this application provide a data filtering device, which includes: The processing module is used to perform multiple rounds of iterative filtering on the target dataset using thread bundles and a target queue, terminating when the iteration stopping condition is met. Each round of iterative filtering includes the following steps: Data is read from the target dataset in parallel by multiple threads in the thread bundle; Data that meets the enqueue conditions of the target queue from the data read by each of the multiple threads is written into the target queue in parallel. When the target queue is full or the thread bundle has read the target dataset, sort the multiple data in the target queue and remove the data in the target queue except for the first K data; Update the enqueue condition based on the Kth data in the target queue.

[0007] The output module is used to obtain the target filtering result based on the K data points retained in the target queue at the time of termination, where K is greater than 0.

[0008] Optionally, the processing module is specifically used for: When the enqueue condition is not less than the decision threshold of the target queue, the multiple data in the target queue are sorted in descending order.

[0009] Optionally, the processing module is specifically used for: When the enqueue condition is not greater than the decision threshold of the target queue, the multiple data in the target queue are sorted in ascending order.

[0010] Optionally, the processing module is specifically used for: Update the decision threshold of the target queue to the Kth data.

[0011] Optionally, the target dataset is a subset of data in the original dataset, and the K data items retained by the target queue at the time of termination are the first partial results of the target dataset; The output module is specifically used for: Obtain the second local result corresponding to each other data subset in the original dataset; Based on the first local result and the obtained second local results, the target screening result is generated.

[0012] Optionally, the output module is specifically used for: The first local result and the obtained second local results are merged to obtain a merged dataset; The data in the merged dataset is sorted to obtain the target sorting result; Based on the target sorting result, K data points are obtained from the merged dataset as the target filtering result.

[0013] Optionally, the processing module is specifically used for: The multiple threads are divided into a first thread whose read data meets the enqueue condition, and a second thread whose read data does not meet the enqueue condition; For each first thread, the write offset of the first thread is determined based on the thread number of the first thread and the thread numbers of each second thread; According to the write offset of each first thread, the data read by each first thread is written to the target queue in parallel.

[0014] Optionally, the processing module is specifically used for: For each first thread, obtain the original write position of the first thread in the target queue, and based on the original write position and the write offset of the first thread, obtain the target write position of the first thread in the target queue. According to the target write position of each first thread, the data read by each first thread is written to the target queue in parallel.

[0015] On one hand, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the data filtering method described above.

[0016] On one hand, embodiments of this application provide a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer device, cause the computer device to perform the steps of the above-described data filtering method.

[0017] In this embodiment, firstly, during the process of multiple rounds of iterative filtering of the target dataset, the queuing conditions are continuously updated and data in the target dataset that does not meet the queuing conditions are removed. In this way, the thread bundle can eliminate a large amount of non-Top-K data as early as possible. In each round of iterative filtering, only multiple data that have entered the target queue need to be sorted, which greatly reduces the amount of data involved in sorting, thereby reducing chip resource consumption and improving the efficiency of data filtering.

[0018] Secondly, multiple threads within a thread bundle read data from the target dataset in parallel and write data that meets the enqueue criteria into the same target queue in parallel. This parallel enqueue mechanism ensures efficient data filtering, thereby improving the efficiency of handling the Top-K selection problem.

[0019] In addition, a target queue is allocated in shared memory for the thread bundle. This allows the thread bundle to communicate using shared memory and native instructions, avoiding global thread synchronization and greatly reducing communication overhead, while also achieving efficient collaboration within the thread bundle. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the structure of an artificial intelligence chip provided in an embodiment of this application; Figure 2 A flowchart illustrating a data filtering method provided in an embodiment of this application; Figure 3 A schematic diagram of queue writing provided for related technologies; Figure 4 This is a schematic diagram of queue writing provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of a data filtering device provided in an embodiment of this application; Figure 6 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0022] To make the objectives, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0023] refer to Figure 1 This is a structural diagram of an artificial intelligence chip applicable to the embodiments of this application. The artificial intelligence chip 100 includes at least: video memory 101 and multiple computing units 102. The computing unit 102 may be a streaming processing cluster (SPC). Each computing unit 102 includes multiple execution units 103.

[0024] Each execution unit 103 includes: shared memory and registers; the execution unit 103 supports one or more thread bundles, each thread bundle including: multiple threads, each thread corresponding to a thread number.

[0025] Video memory 101 can be high-bandwidth memory (HBM) or other types of memory. Shared memory is a temporary memory with a smaller capacity than video memory 101, but a faster data exchange speed. Compared to shared memory, register 105 has a smaller capacity but a faster data exchange speed.

[0026] In this embodiment of the application, a fixed-size circular queue is allocated in shared memory for each thread bundle, and the enqueue conditions for each circular queue are initialized.

[0027] Each thread bundle can independently handle one Top-K selection problem, and multiple thread bundles can process multiple Top-K selection problems in parallel. For example, if it is necessary to process Top-K selection problems for multiple batches, multiple thread bundles can be used to compute and process these multiple Top-K selection problems in parallel.

[0028] Of course, a Top-K selection problem can also be processed by multiple thread bundles, and this application does not impose specific limitations on this. For example, if a Top-K selection problem needs to be processed for a batch, and N is much greater than K, then a Top-K selection problem can be processed by multiple thread bundles.

[0029] The following discussion will focus on a Top-K selection problem handled independently by a single thread bundle: The target dataset is subjected to multiple rounds of iterative filtering using a single thread and a corresponding target queue (i.e., a circular queue) until the iteration stops, at which point the K data points remaining in the target queue are taken as the target filtering result. Each round of iterative filtering includes the following steps: Data is read from the target dataset in parallel by multiple threads within a thread bundle. The target dataset can be pre-stored in the central processing unit (CPU) outside of the video memory 101 or the AI ​​chip 100. For each thread, if the data read by the thread meets the enqueue condition of the target queue, the data read by the thread is written to the target queue. When the target queue is full or the thread bundle has finished reading the target dataset, the multiple data in the target queue are sorted, and data other than the first K data in the target queue are removed. Then, based on the Kth data in the target queue, the enqueue condition is updated.

[0030] In this embodiment, firstly, during the multiple rounds of iterative filtering of the target dataset, the queuing conditions are continuously updated and data in the target dataset that does not meet the queuing conditions are removed. In this way, the thread bundle can eliminate a large amount of non-Top-K data as early as possible. Only in each round of iterative filtering, multiple data entering the target queue need to be sorted, which greatly reduces the amount of data involved in sorting, thereby reducing chip resource consumption and improving the efficiency of data filtering.

[0031] Secondly, multiple threads within a thread bundle read data from the target dataset in parallel and write data that meets the enqueue criteria into the same target queue in parallel. This parallel enqueue mechanism ensures efficient data filtering, thereby improving the efficiency of handling the Top-K selection problem.

[0032] In addition, a target queue is allocated in shared memory for the thread bundle. This allows the thread bundle to communicate using shared memory and native instructions, avoiding global thread synchronization. This not only greatly reduces communication overhead but also enables efficient collaboration within the thread bundle.

[0033] In some embodiments, when a Top-K selection problem is processed jointly by multiple thread bundles, each thread bundle obtains a partial result (i.e., the K data points retained by the target queue at the time of termination) in the same way as described above. Then, the multiple partial results are merged to obtain a merged dataset. Then, the data in the merged dataset are sorted to obtain the target sorting result. Finally, K data points are obtained from the merged dataset according to the target sorting result as the target selection result.

[0034] At this point, multiple thread bundles can perform data filtering in parallel to obtain corresponding local results, achieving thread grid-level parallelism. This not only makes full use of the resources of the artificial intelligence chip, but also greatly improves the efficiency of processing Top-K selection problems, especially suitable for scenarios where N is much greater than K.

[0035] In addition to the structure described above, the artificial intelligence chip 100 in this application may also include other structures, and this application does not specifically limit such structures.

[0036] Artificial intelligence chips 100 can be: Graphics Processing Unit (GPU), General-Purpose Graphics Processing Units (GPGPU), Domain Specific Architecture (DSA), etc.

[0037] The following is based on Figure 1The diagram shown illustrates the architecture of an AI chip and details the process of a data filtering method. (See attached image.) Figure 2 This method is executed by an artificial intelligence chip and includes the following steps: Step 201: Perform multiple rounds of iterative filtering on the target dataset using thread bundles and target queues until the iteration stops when the stopping condition is met.

[0038] Specifically, the data filtering method of this application can be applied to various scenarios, such as image processing, speech processing, and text processing. In different application scenarios, the physical meaning of the data in the target dataset can be different.

[0039] For example, in a text processing scenario, the data in the target dataset can be text data used in tasks such as text generation and text recognition.

[0040] For example, in a speech processing scenario, the data in the target dataset can be speech data used in tasks such as speech enhancement, speech recognition, and speech synthesis.

[0041] For example, in an image processing scenario, the data in the target dataset can be image data used in tasks such as image preprocessing, image segmentation, and object detection.

[0042] In this embodiment, a thread bundle includes multiple threads, each corresponding to a thread number; for example, a thread bundle includes 16 threads, with corresponding thread numbers from L0 to L15. Each thread bundle corresponds to a target queue, which can be a circular queue or other types of queues, wherein the target queue is located in the shared memory of the artificial intelligence chip.

[0043] In this embodiment, the iteration stopping condition can be that the thread bundle has finished reading the target dataset, that is, the entire target dataset has been filtered. Since the process of each round of iteration filtering is similar, the following description uses one round of iteration filtering as an example.

[0044] In each round of iterative screening, the following steps 2011-2015 are performed: Step 2011: Read data from the target dataset in parallel using multiple threads in a thread bundle.

[0045] Specifically, multiple threads in a thread bundle collaborate to read data from the target dataset they are responsible for. The target dataset can be pre-stored in the AI ​​chip, such as in video memory; the target dataset can also be pre-stored in the CPU outside the AI ​​chip; of course, the target dataset can also be stored in other locations, which this application does not specifically limit.

[0046] Step 2012: Data that meets the enqueue conditions of the target queue from the data read by each of the multiple threads is written into the target queue in parallel.

[0047] Specifically, the sorting rules used are different for different Top-K selection problems. For example, when the Top-K selection problem is to select the K largest data points from N data points, the sorting rule is usually to sort from largest to smallest and then select the first K data points.

[0048] When the Top-K selection problem involves choosing the K smallest data points from N data points, the sorting rule is typically: sort the data in ascending order, and then select the first K data points. Therefore, different enqueue conditions can be set for different Top-K selection problems.

[0049] In some embodiments, when the Top-K selection problem involves selecting the K largest data points from N data points, the enqueue condition is set to be no less than the decision threshold of the target queue; the initial value of the decision threshold can be set to negative infinity. Thus, if the data read by the thread is no less than the decision threshold of the target queue, it indicates that the data is likely Top-K data, and therefore, the data is first written into the target queue; if the data read by the thread is less than the decision threshold of the target queue, it indicates that the data is not Top-K data, and therefore, the data is directly discarded.

[0050] In some embodiments, when the Top-K selection problem involves selecting the K smallest data points from N data points, the enqueue condition is set to be no greater than the decision threshold of the target queue; the initial value of the decision threshold can be set to positive infinity. Thus, if the data read by a thread is no greater than the decision threshold of the target queue, it indicates that the data may be Top-K data, and therefore, the data is first written into the target queue; if the data read by a thread is greater than the decision threshold of the target queue, it indicates that the data is not Top-K data, and therefore, the data is directly discarded.

[0051] In this embodiment, multiple threads employ a parallel enqueue mechanism. To avoid data conflicts, consecutive write positions are pre-allocated to the multiple threads in the target queue according to their respective thread numbers; that is, the original write position corresponding to each thread. For example, see... Figure 3 The thread bundle is set to include 16 threads, with corresponding thread numbers from L0 to L15. Starting from the tail of the target queue, 16 write positions are allocated to the 16 threads, namely positions X0 to X15. Among them, thread number L0 corresponds to position X0, thread number L1 corresponds to position X1, and so on.

[0052] However, when multiple threads in a thread bundle read data from the target dataset in parallel, some threads may read data that meets the enqueue criteria, while others may read data that does not. In this case, only the data that meets the enqueue criteria needs to be written to the target queue in parallel, without writing the data that does not meet the enqueue criteria. For example, if thread L0 reads data 0 that meets the enqueue criteria, then data 0 is written to position X0 in the target queue; if thread L1 reads data 1 that does not meet the enqueue criteria, then data 1 does not need to be written to the target queue; if thread L2 reads data 2 that meets the enqueue criteria, then data 2 is written to position X2 in the target queue, and so on. This results in empty positions in the target queue, such as position X1, which not only wastes resources but also affects the efficiency of data filtering.

[0053] Therefore, in this embodiment, multiple threads are divided into a first thread whose read data meets the enqueue condition and a second thread whose read data does not meet the enqueue condition; for each first thread, the write offset of the first thread is determined based on the thread number of the first thread and the thread numbers of each second thread; and then, according to the write offset of each first thread, the data read by each first thread is written to the target queue in parallel.

[0054] Specifically, a thread-beam voting function is used to mark threads whose read data meets the enqueue criteria (i.e., the first thread), and threads whose read data does not meet the enqueue criteria are designated as the second thread. For each first thread, second threads with thread numbers less than the first thread are selected. Then, based on the number of selected second threads, the write offset is determined, where the write offset refers to the number of write positions offset towards the starting position (e.g., the tail of the queue).

[0055] After obtaining the write offset, for each first thread, the original write position of the first thread in the target queue is obtained, and based on the original write position and the write offset of the first thread, the target write position of the first thread in the target queue is obtained; then, according to the target write position of each first thread, the data read by each first thread is written to the target queue in parallel.

[0056] For example, see Figure 4If thread L0 reads data 0 that satisfies the enqueue condition, then position X0 in the target queue is directly used as the target write position for data 0. If thread L1 reads data 1 that does not satisfy the enqueue condition, then there is no need to write data 1 to the target queue. If thread L2 reads data 2 that satisfies the enqueue condition, then there is an unmarked thread (i.e., thread L1) before thread L2. Therefore, the write offset for thread L2 is determined to be one write position. The original write position for thread L2 is position X2. Therefore, it moves one write position towards the starting position (i.e., position X0) to obtain the target write position, i.e., position X1. At this point, thread L2 is associated with position X1 in the target queue, and so on.

[0057] In this way, by writing the data read by each first thread into the target queue in parallel according to the target write position of each first thread, there will be no blank positions in the target queue. This not only achieves conflict-free parallel enqueueing, but also makes full use of queue resources, thereby ensuring the efficiency of data filtering.

[0058] Step 2013: When the target queue is full or the thread bundle has finished reading the target dataset, sort the multiple data in the target queue and remove the data in the target queue except for the first K data.

[0059] Specifically, multiple threads within a thread bundle continuously read data and write it into a target queue. When the target queue is full, data reading can no longer continue. At this point, the thread bundle internally calls a sorting function based on a sorting network to sort the multiple data items in the target queue.

[0060] When the Top-K selection problem involves selecting the K largest data points from N data points, the data in the target queue are sorted in descending order. Then, the top K data points in the target queue are retained, and all other data points in the target queue are removed.

[0061] When the Top-K selection problem involves selecting the K smallest data points from N data points, the data in the target queue are sorted in ascending order. Then, the top K data points in the target queue are retained, and all other data points in the target queue are removed.

[0062] After removing all data except the first K data from the target queue, the target queue is adjusted from a full state to a partially full state, and then the partially full target queue is used to enter the next round of iterative filtering.

[0063] In some embodiments, when the thread bundle has finished reading the target dataset, it indicates that this round of iteration filtering is the last round of iteration filtering, and at this time, the thread bundle will no longer continue reading data. Internally, the thread bundle calls a sorting function based on a sorting network to sort multiple data in the target queue and remove data from the target queue except for the first K data; the sorting method used has been described above and will not be repeated here.

[0064] Step 2014: Determine whether the iteration stopping condition is met. If yes, proceed to step 202; otherwise, proceed to step 2015.

[0065] Step 2015: Update the enqueue conditions based on the Kth data in the target queue.

[0066] Using the target queue that is not full and the updated enqueue conditions, proceed to the next round of iterative filtering, i.e., return to execution step 2011.

[0067] In some embodiments, the decision threshold of the target queue is updated to the Kth data.

[0068] When the Top-K selection problem involves selecting the K largest data points from N data points, the enqueue condition is that the data must be no less than the decision threshold of the target queue. During each round of iterative selection, the decision threshold is updated to the Kth data point, which means the decision threshold is updated to the smallest data point currently retained in the target queue.

[0069] In this way, in the next round of iterative filtering, the data read by the thread must be greater than or equal to the minimum data currently retained in the target queue in order to enter the target queue. This can quickly remove a large amount of non-topk data. Furthermore, as the number of iterative filtering rounds increases, the decision threshold will also be updated to a larger value, which will eliminate more non-topk data, thereby greatly improving the data filtering efficiency.

[0070] When the Top-K selection problem involves selecting the K smallest data points from N data points, the enqueue condition is that the value is not greater than the decision threshold of the target queue. In each round of iterative selection, the decision threshold is updated to the Kth data point, which means the decision threshold is updated to the maximum value currently retained in the target queue.

[0071] In this way, in the next round of iterative filtering, the data read by the thread must be less than or equal to the minimum data currently retained in the target queue in order to enter the target queue. This can quickly remove a large amount of non-topk data. Furthermore, as the number of iterative filtering rounds increases, the decision threshold will also be updated to a smaller value, which will eliminate more non-topk data, thereby greatly improving the data filtering efficiency.

[0072] Step 202: Based on the K data points retained in the target queue at the time of termination, obtain the target screening results.

[0073] Specifically, in a scenario where a thread bundle independently processes a Top-K selection problem, the K data items retained in the target queue at the time of termination can be directly used as the target selection result.

[0074] Multiple thread bundles can also process multiple Top-K selection problems in parallel. In the parallel processing, the processing procedure of each thread bundle is the same as the process described in steps 201-202 above, which will not be repeated here.

[0075] In this embodiment, firstly, during the process of multiple rounds of iterative filtering of the target dataset, the queuing conditions are continuously updated and data in the target dataset that does not meet the queuing conditions are removed. In this way, the thread bundle can eliminate a large amount of non-Top-K data as early as possible. In each round of iterative filtering, only multiple data that have entered the target queue need to be sorted, which greatly reduces the amount of data involved in sorting, thereby reducing chip resource consumption and improving the efficiency of data filtering.

[0076] Secondly, multiple threads within a thread bundle read data from the target dataset in parallel and write data that meets the enqueue criteria into the same target queue in parallel. This parallel enqueue mechanism ensures efficient data filtering, thereby improving the efficiency of handling the Top-K selection problem.

[0077] In addition, a target queue is allocated in shared memory for the thread bundle. This allows the thread bundle to communicate using shared memory and native instructions, avoiding global thread synchronization and greatly reducing communication overhead, while also achieving efficient collaboration within the thread bundle.

[0078] In some embodiments, in a scenario where multiple thread bundles jointly process a Top-K selection problem, the target dataset is a subset of data in the original dataset, and the K data retained by the target queue at termination is the first partial result of a subset of data.

[0079] Obtain the second local result corresponding to each other data subset in the original dataset; then generate the target filtering result based on the first local result and the obtained second local results.

[0080] In practice, multiple data subsets in the original dataset are distributed to multiple thread bundles for parallel processing. Each thread bundle processes the distributed data subset according to steps 201-202 described above to obtain the corresponding local results. Then, the target filtering results are obtained by combining multiple local results.

[0081] In some embodiments, the first partial result and the obtained second partial results are merged to obtain a merged dataset; then the data in the merged dataset are sorted to obtain a sorting result; and then, according to the sorting result, K data points are obtained from the merged dataset as target filtering results.

[0082] Specifically, each of the first local results and each of the second local results includes K data points. The first local results and each of the second local results are merged to obtain a merged dataset. The amount of data in the merged dataset is the product of the total number of thread bundles and K. Its size is relatively small. Therefore, the sorting overhead is small when performing global sorting on the merged dataset.

[0083] In this embodiment, the method for sorting the merged dataset can be radix sort, bitonic sort, etc. The radix sort method will be described in detail below: The process involves bucket sorting the elements digit by digit until all elements have been bucket sorted, and then using the bucket sort result of the last element as the target sort result. Here, "all elements" includes the ones place, tens place, hundreds place, etc.

[0084] Bucket sorting for each person includes the following steps: Obtain the value of each data point in the current position in the merged dataset, and then perform histogram statistics on the obtained values ​​of the current position to obtain the target histogram. The horizontal axis of the target histogram is the index (e.g., 0-9); the vertical axis of the target histogram is the number of occurrences. For example, if the current position is the units digit, and there are 10 data points in the merged dataset with the units digit being index 0, then the number of occurrences of index 0 is 10.

[0085] For each occurrence count on the target histogram, calculate the prefix sum of that occurrence count, and determine the new positions of the multiple data associated with that occurrence count based on the prefix sum; where the prefix sum refers to the sum of the occurrence counts corresponding to all the preceding indices.

[0086] The data is reordered according to its new position in the merged dataset, and the reordered merged dataset is used to proceed to the next bucket sort.

[0087] In this embodiment of the application, when the Top-K selection problem is to select the K largest data points from N data points, the data in the merged dataset are sorted in descending order, and then the top K data points are selected as the target filtering results.

[0088] When the Top-K selection problem involves selecting the K smallest data points from N data points, the data in the merged dataset are sorted in ascending order, and then the top K data points are selected as the target selection results.

[0089] In this embodiment, multiple thread bundles perform data filtering in parallel to obtain corresponding local results, achieving thread grid-level parallelism. This not only makes full use of the resources of the artificial intelligence chip, but also greatly improves the efficiency of processing Top-K selection problems, especially suitable for scenarios where N is much greater than K.

[0090] Based on the same technical concept, this application provides a schematic diagram of the structure of a data filtering device, such as... Figure 5 As shown, the data filtering device 500 includes: Processing module 501 is used to perform multiple rounds of iterative filtering on the target dataset using thread bundles and target queues, terminating when the iteration stopping condition is met. Each round of iterative filtering includes the following steps: Data is read from the target dataset in parallel by multiple threads in the thread bundle; Data that meets the enqueue conditions of the target queue from the data read by each of the multiple threads is written into the target queue in parallel. When the target queue is full or the thread bundle has read the target dataset, sort the multiple data in the target queue and remove the data in the target queue except for the first K data; Update the enqueue condition based on the Kth data in the target queue.

[0091] Output module 502 is used to obtain target filtering results based on the K data points retained in the target queue at the time of termination, where K is greater than 0.

[0092] Optionally, the processing module 501 is specifically used for: When the enqueue condition is not less than the decision threshold of the target queue, the multiple data in the target queue are sorted in descending order.

[0093] Optionally, the processing module 501 is specifically used for: When the enqueue condition is not greater than the decision threshold of the target queue, the multiple data in the target queue are sorted in ascending order.

[0094] Optionally, the processing module 501 is specifically used for: Update the decision threshold of the target queue to the Kth data.

[0095] Optionally, the target dataset is a subset of data in the original dataset, and the K data items retained by the target queue at the time of termination are the first partial results of the target dataset; The output module 502 is specifically used for: Obtain the second local result corresponding to each other data subset in the original dataset; Based on the first local result and the obtained second local results, the target screening result is generated.

[0096] Optionally, the output module 502 is specifically used for: The first local result and the obtained second local results are merged to obtain a merged dataset; The data in the merged dataset is sorted to obtain the target sorting result; Based on the target sorting result, K data points are obtained from the merged dataset as the target filtering result.

[0097] Optionally, the processing module 501 is specifically used for: The multiple threads are divided into a first thread whose read data meets the enqueue condition, and a second thread whose read data does not meet the enqueue condition; For each first thread, the write offset of the first thread is determined based on the thread number of the first thread and the thread numbers of each second thread; According to the write offset of each first thread, the data read by each first thread is written to the target queue in parallel.

[0098] Optionally, the processing module 501 is specifically used for: For each first thread, obtain the original write position of the first thread in the target queue, and based on the original write position and the write offset of the first thread, obtain the target write position of the first thread in the target queue. According to the target write position of each first thread, the data read by each first thread is written to the target queue in parallel.

[0099] In this embodiment, firstly, during the process of multiple rounds of iterative filtering of the target dataset, the queuing conditions are continuously updated and data in the target dataset that does not meet the queuing conditions are removed. In this way, the thread bundle can eliminate a large amount of non-Top-K data as early as possible. In each round of iterative filtering, only multiple data that have entered the target queue need to be sorted, which greatly reduces the amount of data involved in sorting, thereby reducing chip resource consumption and improving the efficiency of data filtering.

[0100] Secondly, multiple threads within a thread bundle read data from the target dataset in parallel and write data that meets the enqueue criteria into the same target queue in parallel. This parallel enqueue mechanism ensures efficient data filtering, thereby improving the efficiency of handling the Top-K selection problem.

[0101] In addition, a target queue is allocated in shared memory for the thread bundle. This allows the thread bundle to communicate using shared memory and native instructions, avoiding global thread synchronization and greatly reducing communication overhead, while also achieving efficient collaboration within the thread bundle.

[0102] In the embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0103] Based on the same technical concept, embodiments of this application provide a computer device, such as... Figure 6 As shown, it includes at least one artificial intelligence chip 100 and a memory 601 connected to at least one artificial intelligence chip 100. In this embodiment, the specific connection medium between the artificial intelligence chip 100 and the memory 601 is not limited. Figure 6 Taking the connection between the AI ​​chip 100 and the memory 601 via a bus as an example, the bus can be divided into address bus, data bus, control bus, etc.

[0104] In this embodiment of the application, the memory 601 stores instructions that can be executed by at least one artificial intelligence chip 100. By executing the instructions stored in the memory 601, at least one artificial intelligence chip 100 can perform the steps of the above-described data filtering method.

[0105] The artificial intelligence chip 100 serves as the control center of the computer device. It connects to various parts of the computer device via various interfaces and lines, and performs data filtering by running or executing instructions stored in the memory 601 and accessing data stored in the memory 601. Optionally, the artificial intelligence chip 100 may include one or more processing units. The artificial intelligence chip 100 may integrate an application processor and a modem processor. The application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may not be integrated into the artificial intelligence chip 100. In some embodiments, the artificial intelligence chip 100 and the memory 601 may be implemented on the same chip; in other embodiments, they may be implemented on separate chips.

[0106] The artificial intelligence chip 100 can be a general-purpose processor, such as a central processing unit (CPU), digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0107] Memory 601, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory 601 may include at least one type of storage medium, such as flash memory, hard disk, multimedia card, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), magnetic memory, magnetic disk, optical disk, etc. Memory 601 can be any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer device, but is not limited thereto. Memory 601 in the embodiments of this application may also be a circuit or any other device capable of implementing storage functions for storing program instructions and / or data.

[0108] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program executable by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the above-described data filtering method.

[0109] Based on the same inventive concept, this application provides a computer program product, which includes a computer program stored on a computer-readable storage medium. The computer program includes program instructions, which, when executed by a computer device, cause the computer device to perform the steps of the above-described data filtering method.

[0110] Those skilled in the art will understand that embodiments of the present invention can be provided as methods or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0111] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer apparatus or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0112] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer device or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0113] These computer program instructions may also be loaded onto a computer device or other programmable data processing equipment to cause a series of operational steps to be performed on the computer device or other programmable equipment to produce a process implemented by the computer device, thereby providing instructions that execute on the computer device or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0114] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0115] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A data filtering method, characterized in that, include: The target dataset is subjected to multiple rounds of iterative filtering using thread bundles and target queues until the iteration stops when the stopping condition is met. The target filtering result is obtained based on the K data points retained in the target queue at the time of termination, where K is greater than 0. Each round of iterative screening includes the following steps: Data is read from the target dataset in parallel by multiple threads in the thread bundle; Data that meets the enqueue conditions of the target queue from the data read by each of the multiple threads is written into the target queue in parallel. When the target queue is full or the thread bundle has read the target dataset, sort the multiple data in the target queue and remove the data in the target queue except for the first K data; Update the enqueue condition based on the Kth data in the target queue.

2. The method as described in claim 1, characterized in that, The sorting of multiple data items in the target queue includes: When the enqueue condition is not less than the decision threshold of the target queue, the multiple data in the target queue are sorted in descending order.

3. The method as described in claim 1, characterized in that, The sorting of multiple data items in the target queue includes: When the enqueue condition is not greater than the decision threshold of the target queue, the multiple data in the target queue are sorted in ascending order.

4. The method as described in claim 2 or 3, characterized in that, The step of updating the enqueue condition based on the Kth data in the target queue includes: Update the decision threshold of the target queue to the Kth data.

5. The method as described in claim 1, characterized in that, The target dataset is a subset of data in the original dataset, and the K data items retained by the target queue at the time of termination are the first partial results of the target dataset; The process of obtaining target screening results based on the K data points retained in the target queue at the time of termination includes: Obtain the second local result corresponding to each other data subset in the original dataset; Based on the first local result and the obtained second local results, the target screening result is generated.

6. The method as described in claim 5, characterized in that, The step of generating the target screening result based on the first local result and each of the obtained second local results includes: The first local result and the obtained second local results are merged to obtain a merged dataset; The data in the merged dataset is sorted to obtain the target sorting result; Based on the target sorting result, K data points are obtained from the merged dataset as the target filtering result.

7. The method as described in claim 1, characterized in that, The step of writing data that meets the enqueue conditions of the target queue from the data read by each of the multiple threads into the target queue in parallel includes: The multiple threads are divided into a first thread whose read data meets the enqueue condition, and a second thread whose read data does not meet the enqueue condition; For each first thread, the write offset of the first thread is determined based on the thread number of the first thread and the thread numbers of each second thread; According to the write offset of each first thread, the data read by each first thread is written to the target queue in parallel.

8. The method as described in claim 7, characterized in that, The step of writing the data read by each first thread into the target queue in parallel according to the write offset of each first thread includes: For each first thread, obtain the original write position of the first thread in the target queue, and based on the original write position and the write offset of the first thread, obtain the target write position of the first thread in the target queue. According to the target write position of each first thread, the data read by each first thread is written to the target queue in parallel.

9. A computer device comprising a memory, an artificial intelligence chip, and a computer program stored in the memory and running on the artificial intelligence chip, characterized in that, When the artificial intelligence chip executes the computer program, it implements the steps of the method as described in any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, It stores a computer program that is executed by a computer device, which, when run on the computer device, causes the computer device to perform the steps of the method as described in any one of claims 1 to 8.

Images