Data screening method and device based on hardware processing capability and medium

Abstract

The invention relates to a data screening method and device based on hardware processing capacity and a medium. The invention provides a data screening method based on hardware processing capability. The data screening method comprises the following steps: acquiring N pieces of data to be processed and a target quantity K from a large language model; dividing the N data into G data groups based on the hardware processing capability, the hardware processing capability comprising a maximum number M of data that can be processed in parallel by a hardware computing unit; screening out candidate data from each data group by using the hardware computing unit based on a preset group screening ratio P, and merging the candidate data of all the data groups into a candidate data set; and screening the target number K of target data from the candidate data set by using the hardware computing unit.

Description

Technical Field

[0001] This disclosure relates to the field of large language model technology, and more specifically, to data filtering methods, apparatus and media based on hardware processing capabilities. Background Technology

[0002] In core AI applications such as large language models, Top-K filtering is an indispensable basic data processing step. Its core function is to quickly filter out the top K data points (e.g., tokens, feature vectors, etc.) with the strongest correlation to target features and semantics from massive input data, thus providing high-quality data support for subsequent downstream tasks such as attention calculation, result generation, and accurate matching. With the continuous deepening and expansion of AI applications, the scale of data to be processed is growing exponentially. However, current mainstream hardware acceleration devices (e.g., GPUs, dedicated AI chips) typically have inherent limitations in the single-processing length of their built-in Top-K operation modules in order to optimize parallel computing efficiency and on-chip storage utilization. This hardware limitation creates a sharp contradiction with the ever-increasing demand for massive data processing, becoming a core bottleneck restricting the overall processing efficiency and accuracy of the system.

[0003] Therefore, a new data filtering method, device, and medium based on hardware processing capabilities is needed. Summary of the Invention

[0004] Embodiments of this application provide a data filtering method, apparatus, and medium based on hardware processing capabilities, which can achieve a triple balance of hardware adaptability, processing efficiency, and result accuracy.

[0005] According to a first aspect of this application, a data filtering method based on hardware processing capabilities is provided, comprising: obtaining N data points to be processed and a target quantity K from a large language model; dividing the N data points into G data groups based on the hardware processing capabilities, wherein the hardware processing capabilities include the maximum number of data points M that a hardware computing unit can process in parallel, and Based on a preset grouping and filtering ratio P, the hardware computing unit is used to filter candidate data from each data group and merge the candidate data from all data groups into a candidate data set; and the hardware computing unit is used to filter the target number K target data from the candidate data set.

[0006] According to an embodiment of the first aspect of this application, the G data packets include G-1 complete data packets and 1 incomplete data packet, wherein the complete data packet has M data and the incomplete data packet has less than M data.

[0007] According to an embodiment of the first aspect of this application, the preset grouping and filtering ratio P is determined based on the target number K, the maximum data number M, and the grouping number G.

[0008] According to an embodiment of the first aspect of this application, the preset grouping and filtering ratio P is configured to maximize the total number of candidate data in the candidate data set while satisfying that the total number of candidate data in the candidate data set is less than the maximum number of data M.

[0009] According to an embodiment of the first aspect of this application, selecting the target number K target data from the candidate data set using the hardware computing unit includes: comparing the total number of candidate data in the candidate data set with the maximum data number M and the target number K; updating the grouping screening ratio P based on the target number K, the maximum data number M, and the grouping number G if the total number is greater than the maximum data number M or if the total number is less than the target number K; re-screening candidate data from each data group using the hardware computing unit based on the updated grouping screening ratio P, and merging the re-screened candidate data into a new candidate data set; and selecting the target number K target data from the new candidate data set using the hardware computing unit.

[0010] According to an embodiment of the first aspect of this application, the hardware computing unit is configured with Top-K hardware, and the Top-K hardware is configured to perform a data filtering operation.

[0011] According to an embodiment of the first aspect of this application, the data filtering operation is a Top-K sorting operation.

[0012] According to an embodiment of the first aspect of this application, the maximum data quantity M, the N, and the target quantity K are all powers of 2.

[0013] According to an embodiment of the first aspect of this application, the method further includes: sending the target number K target data to the large language model.

[0014] According to a second aspect of this application, a data filtering apparatus based on hardware processing capabilities is provided, comprising: a processor, and a memory storing instructions that, when executed by the processor, cause the processor to perform the method of the first aspect of this application.

[0015] According to a third aspect of this application, a computer-readable storage medium is provided having instructions stored thereon that, when executed by a computer, cause the computer to perform the method of the first aspect of this application.

[0016] According to embodiments of this application, a data filtering method, apparatus, and medium based on hardware processing capabilities divide large-scale data into multiple data groups based on the maximum number of data that a hardware computing unit can process in parallel. Candidate data is then filtered from each data group based on a group filtering ratio, and the candidate data from all data groups are merged to obtain a candidate data set. The total number of candidate data in this candidate data set is less than the maximum number of data. Then, a target number of target data can be filtered from the candidate data set using a hardware computing unit. This adapts to the hardware processing capabilities of the hardware computing unit, improves its processing efficiency, and appropriately ensures the accuracy of the processing results, thereby achieving a triple balance between hardware adaptability, processing efficiency, and result accuracy. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on the drawings without creative effort.

[0018] Figure 1 This is a flowchart of a data filtering method based on hardware processing capabilities according to an embodiment of this application; Figure 2 This is a schematic diagram of the hardware structure of a data filtering device based on hardware processing capabilities according to an embodiment of this application. Detailed Implementation

[0019] The features and exemplary embodiments of various aspects of this application will now be described in detail. To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only configured to explain this application and are not configured to limit this application. For those skilled in the art, this application can be implemented without some of these specific details. The following description of the embodiments is merely to provide a better understanding of this application by illustrating examples of this application.

[0020] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising..." does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes said element.

[0021] The features and exemplary embodiments of various aspects of this application will now be described in detail. Furthermore, the features, structures, or characteristics described below may be combined in any suitable manner in one or more embodiments.

[0022] Large language models have become a focal point of research in recent years. They are artificial intelligence models based on deep learning technology that solve various natural language processing tasks through dialogue. In core AI applications such as large language models, Top-K filtering is an indispensable basic data processing step. Its core function is to quickly filter out the top K data points (e.g., lexical units, feature vectors, etc.) with the strongest correlation to target features and semantics from massive input data, thus providing high-quality data support for downstream tasks such as attention calculation, result generation, and accurate matching. With the continuous deepening and expansion of AI applications, the scale of data to be processed is growing exponentially. However, current mainstream hardware acceleration devices (e.g., GPUs, dedicated AI chips) typically have inherent limitations on the single-processing length of their built-in Top-K operation modules in order to optimize parallel computing efficiency and on-chip storage utilization. This hardware limitation creates a sharp contradiction with the ever-increasing demand for massive data processing, becoming a core bottleneck restricting the overall processing efficiency and accuracy of the system. For example, the inventors found that traditional technical solutions mainly have the following defects after research.

[0023] Category 1: Hardware-Limited Direct-Connection Solution. This solution strictly adheres to the hardware's single-processing limit, uniformly grouping the massive input data according to its maximum length. Then, leveraging hardware acceleration, it independently performs Top-K selection on each group in parallel. Finally, it simply concatenates the selection results from each group as the final output. While this solution fully utilizes hardware parallelism to ensure processing speed, its core flaw lies in severely insufficient accuracy. Because the selection process for each group is completely independent, making decisions based solely on local data and lacking a global perspective, it easily leads to the omission of critical data with high global relevance but low local ranking, while retaining redundant data within certain groups. The final output deviates significantly from the true global Top-K result, failing to meet the requirements of applications with stringent accuracy demands.

[0024] The second category is the pure software full-data computation scheme. To circumvent hardware length limitations, this scheme completely abandons hardware acceleration and instead uses pure software algorithms to perform sorting or selection operations on the entire dataset to obtain an accurate global Top-K result. The advantage of this scheme is that it guarantees absolute accuracy, but its core drawback is extremely low computational efficiency. Faced with datasets of hundreds of thousands or even millions of records, the computational complexity of pure software algorithms is high, and the processing latency is enormous, making it unsuitable for scenarios with extremely high timeliness requirements, such as real-time inference and online retrieval. Furthermore, full-data computation consumes a significant amount of central processing unit (CPU) and memory resources, significantly increasing system operating costs and power consumption.

[0025] Therefore, embodiments of this application provide a data filtering method, apparatus, and medium based on hardware processing capabilities, which can achieve a triple balance of hardware adaptability, processing efficiency, and result accuracy.

[0026] Figure 1 This is a flowchart of a data filtering method based on hardware processing capabilities according to an embodiment of this application. Figure 1 As shown, the data filtering method based on hardware processing capabilities according to an embodiment of this application includes the following steps S110 to S140: S110: Obtain the N data points to be processed and the target number K from the large language model; S120: Based on the hardware processing capability, divide the N data into G data groups, wherein the hardware processing capability includes the maximum number of data M that the hardware computing unit can process in parallel, and ; S130: Based on a preset grouping screening ratio P, the hardware computing unit is used to screen candidate data from each data group and merge the candidate data of all data groups into a candidate data set. S140: Use the hardware computing unit to filter out the target number K target data from the candidate data set.

[0027] In one embodiment, the large language model can be any large language model, such as deepseek, llama, qwen, GPT, etc., and this paper does not impose any restrictions on it.

[0028] In one embodiment, the hardware processing capability may include the maximum amount of data M that the hardware computing unit can process in parallel. In one embodiment, the hardware computing unit is configured with Top-K hardware, and the Top-K hardware is configured to perform a data filtering operation. In one embodiment, the data filtering operation is a Top-K sorting operation.

[0029] For example, the maximum number of data points M can be 8192, meaning that the hardware computing unit can process 8192 data points in parallel at a time when performing a Top-K sorting operation. Furthermore, the maximum number of data points M can also be other numbers, which are not limited in this paper.

[0030] In one embodiment, N data points to be processed and a target number K can be obtained from a large language model. In one embodiment, the data to be processed can be lexical units, feature vectors, etc., and this document does not impose any restrictions on this. In one embodiment, the number N of data to be processed can be an integer greater than the maximum number of data points M, for example, an integer in the hundreds of thousands or millions, etc. For example, the number N of data to be processed could be 102400. Furthermore, the number N of data to be processed can also be other numbers, and this document does not impose any restrictions on this. In one embodiment, the target number K can be an integer less than the maximum number of data points M, for example, the target number K could be 2048. Furthermore, the target number K can also be other numbers, and this document does not impose any restrictions on this.

[0031] In one embodiment, the maximum data size M, the number of data points N, and the target number K are powers of 2. When these quantities are all powers of 2, the Top-K hardware via the hardware computing unit can perform Top-K sorting operations more quickly. For example, the maximum data size M is 8192, the number of data points N is 102400, and the target number K is 2048. However, the data size M, the number of data points N, and the target number K can also be other quantities, and this document does not impose any restrictions on them.

[0032] In one embodiment, N data points can be divided into G data groups based on hardware processing capabilities. For example, N data points can be divided into G data groups based on a maximum data size M. In other words, the number of data groups is equal to the integer obtained by rounding up the quotient of the number of data to be processed N divided by the maximum number of data M. For example, when the maximum number of data M is 8192 and the number of data is 102400, the number of data groups G can be 13.

[0033] In one embodiment, the multiple data groups include G-1 complete data groups and 1 incomplete data group. Each complete data group has M data points, and the incomplete data group has fewer than M data points, for example, N-(G-1) x M data points. That is, when grouping N data points, first, G-1 complete data groups are created based on each group having M data points, and then the remaining data is grouped into one incomplete data group. For example, when the maximum number of data points M is 8192 and the number N is 102400, the multiple data groups include 12 complete data groups and 1 incomplete data group. Each complete data group has 8192 data points, and the incomplete data group has 4096 data points.

[0034] Furthermore, in other embodiments, when grouping N data points, uniform grouping can be performed, that is, data is evenly distributed within each data group, and each data group is an incomplete data group (the number of data points is less than the maximum number of data points M). For example, when the maximum number of data points M is 8192 and the number N is 102400, multiple data groups include 13 incomplete data groups, of which 12 data groups contain 7877 data points and 1 data group contains 7876 data points. This application does not limit the method of data grouping, as long as the number of data points in each data group is less than the maximum number of data points M.

[0035] In one embodiment, a hardware computing unit can be used to filter candidate data from each data group based on a preset grouping screening ratio P, and the candidate data from all data groups can be merged into a candidate data set.

[0036] In one embodiment, the preset grouping and filtering ratio P can be pre-set, for example, the grouping and filtering ratio used in previous data filtering operations. For example, the grouping and filtering ratio P can be 4%, 5%, 6%, etc., and this document does not limit it.

[0037] For example, in one instance, with a maximum data size M of 8192, a grouping ratio P of 4%, and a group size G of 13, 327 candidate data points can be selected from 12 complete data groups, and 164 data points can be selected from 1 incomplete data group. Accordingly, the total number of candidate data points in the candidate data set is 4098.

[0038] In one embodiment, the preset group filtering ratio P can also be determined based on the target number K, the maximum number of data M, and the number of groups G. For example, K < M x P x G ≤ M. That is, the preset group filtering ratio P multiplied by the maximum number of data M and the number of groups G is greater than the target number K and less than or equal to the maximum number of data M.

[0039] In one embodiment, the preset grouping and filtering ratio P is configured to maximize the total number of candidate data points while ensuring that the total number of candidate data points in the candidate dataset is less than the maximum number of data points M. For example, the fact that the total number of candidate data points in the candidate dataset is less than the maximum number of data points M means that subsequent filtering of the candidate dataset can be performed in parallel using hardware computing units in a single operation. Furthermore, maximizing the total number of candidate data points in the candidate dataset while ensuring that it is less than the maximum number of data points M allows for the retention of as much candidate data as possible, thereby improving the accuracy of the final result.

[0040] In one embodiment, after generating a candidate data set, a hardware computing unit can be used to filter out the target number K target data points from the candidate data set. For example, after grouping and filtering each group to form a candidate data set, a secondary filtering can be performed on multiple candidate data sets to obtain the target number K target data points. For example, group filtering can be called local filtering, and secondary filtering can be called global filtering.

[0041] In one embodiment, both group filtering and secondary filtering are performed by Top-K hardware in the hardware computing unit. That is, both group filtering and secondary filtering are accelerated using hardware.

[0042] In one embodiment, using a hardware computing unit to filter out a target number of K target data from a candidate data set includes: comparing the total number of candidate data in the candidate data set with the maximum number of data M and the target number K; updating the grouping screening ratio P based on the target number K, the maximum number of data M, and the number of groups G if the total number is greater than the maximum number of data M, or if the total number is less than the target number K; using the hardware computing unit to re-filter candidate data from each data group based on the updated grouping screening ratio P, and merging the re-filtered candidate data into a new candidate data set; and using the hardware computing unit to filter out the target number of K target data from the new candidate data set.

[0043] For example, in one instance, after obtaining a candidate dataset using a preset grouping and filtering ratio P, it is found that the total number of candidate data in the dataset exceeds the maximum number of data points M. This means that subsequent filtering of the candidate dataset cannot be performed in parallel using the hardware computing unit in a single run. In this case, the grouping and filtering ratio P needs to be dynamically adjusted; that is, P is updated based on the target number K, the maximum number of data points M, and the number of groups G. Accordingly, a new candidate dataset can be obtained based on the updated grouping and filtering ratio P, and then the hardware computing unit can perform a single parallel process to filter the new candidate dataset, thereby obtaining the target number K target data points.

[0044] For example, in one instance, after obtaining a candidate data set using a preset grouping and filtering ratio P, it is found that the total number of candidate data in the set is less than the target number K. That is, subsequent filtering of the candidate data set cannot yield the target number K target data. In this case, the grouping and filtering ratio P needs to be dynamically adjusted; that is, P is updated based on the target number K, the maximum data size M, and the number of groups G. Accordingly, a new candidate data set can be obtained based on the updated grouping and filtering ratio P, and then the hardware computing unit can perform a single parallel processing step to filter the new candidate data set, thereby obtaining the target number K target data.

[0045] Therefore, before conducting secondary screening, the appropriateness of the group screening ratio P can be determined by verifying the total number of candidate data in the candidate dataset. Furthermore, if the group screening ratio P is inappropriate, it can be dynamically adjusted to obtain a new candidate dataset.

[0046] In one embodiment, the updated grouping ratio P can be stored, and in subsequent data filtering, the updated grouping ratio P can be used as the preset grouping ratio P, thereby achieving dynamic optimization of the grouping ratio P.

[0047] In one embodiment, K target data points can be sent to a large language model. For example, the large language model can use these K target data points to perform downstream tasks such as attention calculation, result generation, and precise matching.

[0048] According to the hardware processing capability-based data filtering method of the embodiments of this application, a large-scale dataset is divided into multiple data groups based on the maximum number of data that the hardware computing unit can process in parallel. Candidate data is filtered from each data group based on a group filtering ratio, and the candidate data from all data groups are merged to obtain a candidate data set. The total number of candidate data in this candidate data set is less than the maximum number of data. Then, the hardware computing unit can be used to filter out a target number of target data from the candidate data set. In this way, the hardware processing capability of the hardware computing unit can be adapted, the processing efficiency of the hardware computing unit can be improved, and the accuracy of the processing results can be appropriately guaranteed, thereby achieving a triple balance of hardware adaptability, processing efficiency, and result accuracy.

[0049] This application also provides a data filtering device based on hardware processing capabilities, including: a processor and a memory storing instructions, which, when executed by the processor, cause the processor to perform the aforementioned data filtering method based on hardware processing capabilities.

[0050] Figure 2 This is a schematic diagram of the hardware structure of a data filtering device based on hardware processing capabilities according to an embodiment of this application. Figure 2 The data filtering device based on hardware processing capabilities shown may include a processor 21 and a memory 22 storing computer program instructions.

[0051] Specifically, the processor 21 may include a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits that can be configured to implement the embodiments of this application. Furthermore, the processor 21 may also include an accelerator such as a graphics processing unit (GPU) or a tensor processor (TPU).

[0052] Memory 22 may include a large-capacity memory for data or instructions. Where appropriate, memory 22 may include removable or non-removable (or fixed) media. In a particular embodiment, memory 22 is a non-volatile solid-state memory. Memory 22 may include read-only memory (ROM), random access memory (RAM), disk storage media devices, optical storage media devices, flash memory devices, electrical, optical, or other physical / tangible memory storage devices. Thus, typically, memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the aforementioned hardware-based data filtering method.

[0053] The processor 21 implements the data filtering method based on hardware processing capabilities in the above embodiments by reading and executing computer program instructions stored in the memory 22.

[0054] In one example, the data filtering device based on hardware processing capabilities may further include a communication interface 23 and a bus 24. For example, Figure 2 As shown, the processor 21, memory 22, and communication interface 23 are connected through bus 24 and complete communication with each other.

[0055] Bus 24 includes hardware, software, or both, that couples components of a data filtering device based on hardware processing capabilities together. For example, and not limitingly, the bus may include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an Infinite Bandwidth Interconnect, a Low Pin Count (LPC) bus, a memory bus, a Microchannel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-E) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local (VLB) bus, or other suitable buses, or combinations of two or more of these. Where appropriate, bus 24 may include one or more buses. Although specific buses are described and illustrated in embodiments of this application, this application contemplates any suitable bus or interconnect.

[0056] This application also provides a computer-readable storage medium having instructions stored thereon, which, when executed by a computer, cause the computer to perform the aforementioned data filtering method based on hardware processing capabilities.

[0057] Examples of computer-readable storage media include non-transitory computer-readable storage media such as portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, etc.

[0058] It should be clarified that this application is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of this application is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of this application.

[0059] The above description is merely a specific embodiment of this application. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the device described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here. It should be understood that the protection scope of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the protection scope of this application.

Claims

1. A data filtering method based on hardware processing capabilities, characterized in that, The method includes: Obtain the N data points to be processed and the target quantity K from the large language model; Based on the hardware processing capability, the N data points are divided into G data groups, wherein the hardware processing capability includes the maximum number of data points M that the hardware computing unit can process in parallel, and ; Based on a preset grouping and filtering ratio P, the hardware computing unit filters candidate data from each data group and merges the candidate data from all data groups into a candidate data set; and The hardware computing unit is used to filter out the target number K target data from the candidate data set.

2. The method according to claim 1, characterized in that, The G data packets include G-1 complete data packets and 1 incomplete data packet. The complete data packets have M data packets, and the incomplete data packet has less than M data packets.

3. The method according to claim 1, characterized in that, The preset grouping and filtering ratio P is determined based on the target number K, the maximum number of data M, and the number of groups G.

4. The method according to claim 3, characterized in that, The preset grouping and filtering ratio P is configured to maximize the total number of candidate data in the candidate data set, provided that the total number of candidate data is less than the maximum number of data M.

5. The method according to claim 1, characterized in that, Using the hardware computing unit to filter out the target number K target data from the candidate data set includes: The total number of candidate data in the candidate data set is compared with the maximum number of data M and the target number K; If the total number is greater than the maximum data number M, or if the total number is less than the target number K, the group filtering ratio P is updated based on the target number K, the maximum data number M, and the group number G. Based on the updated grouping filtering ratio P, the hardware computing unit is used to re-filter candidate data from each data group and merge the re-filtered candidate data into a new candidate data set. The hardware computing unit is used to filter out the target number K target data from the new candidate data set.

6. The method according to claim 1, characterized in that, The hardware computing unit is configured with Top-K hardware, and the Top-K hardware is configured to perform data filtering operations.

7. The method according to claim 6, characterized in that, The data filtering operation is a Top-K sorting operation.

8. The method according to claim 1, characterized in that, The maximum number of data M, the number of data N, and the target number of data K are powers of 2.

9. The method according to claim 1, characterized in that, The method further includes: The target number K data points are sent to the large language model.

10. A data filtering device based on hardware processing capabilities, characterized in that, The device includes: processor, and A memory storing instructions that, when executed by the processor, cause the processor to perform the method according to any one of claims 1-9.

11. A computer-readable storage medium storing instructions, characterized in that, When executed by a computer, the instructions cause the computer to perform the method according to any one of claims 1-9.

Images