Data processing method, device and computer readable storage medium
By running multiple data double-tuning sorting components in parallel and merging sorting sub-results, the problem of double-tuning sorting takes time and improve sorting efficiency.
Patent Information
- Application Number
- CN202111242449.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-10-25
- Publication Date
- 2024-11-26
- Estimated Expiration
- 2041-10-25
AI Technical Summary
Double-tuning sorting requires multiple rounds of traversal during the execution of the sorting task, resulting in increased execution time and reduced sorting efficiency.
By running multiple data double-tuning sorting components in parallel, multiple data sorting sub-results are generated and merged, reducing the sorting time of the data sequence to be sorted.
Reduces the execution time of sorting tasks and improves sorting efficiency.
Smart Images

Figure CN114356512B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of Internet technology, and in particular to a data processing method, device, and computer-readable storage medium. Background Art
[0002] Sorting operations are widely used in scenarios such as databases, target recognition, and text analysis, and are common operations in artificial intelligence models and target detection models. Representative sorting algorithms include bitonic sort, odd-even sort, and radix sort. Among them, bitonic sort has important application value in parallel computing and other fields due to its unique parallelism and data index independence.
[0003] Bit-tuned sorting has a deterministic execution time and is very suitable for hardware implementation. However, during the execution of the sorting task, multiple rounds of traversal of the data to be sorted are required, which increases the execution time and reduces the sorting efficiency. Summary of the invention
[0004] The embodiments of the present application provide a data processing method, device, and computer-readable storage medium, which can not only reduce the execution time of sorting tasks, but also improve sorting efficiency.
[0005] On the one hand, an embodiment of the present application provides a data processing method, including:
[0006] Obtain a data sorting request for the data sequence to be sorted, and call C data bitonic sorting components according to the data sorting request; C is a positive integer greater than 1;
[0007] According to the data sequence to be sorted and C data bitonic sorting components, B data bitonic sorting tasks are started; B is a positive integer greater than 1; the B data bitonic sorting tasks are respectively associated with different data subsequences to be sorted, and the B data subsequences to be sorted are all generated based on the data sequence to be sorted;
[0008] According to B data bitonic sorting tasks, C data bitonic sorting components are run in parallel to obtain B data sorting sub-results output by the C data bitonic sorting components; a data sorting sub-result is used to represent the sorting result of the to-be-sorted data subsequence associated with a data bitonic sorting task;
[0009] Based on the C data bitonic sorting components, the B data sorting sub-results are merged to obtain a data sorting result for the data sequence to be sorted.
[0010] An embodiment of the present application provides a data processing device, including:
[0011] A calling component module is used to obtain a data sorting request for the data sequence to be sorted, and call C data bitonic sorting components according to the data sorting request; C is a positive integer greater than 1;
[0012] The task start module is used to start B data bitonic sorting tasks according to the data sequence to be sorted and C data bitonic sorting components; B is a positive integer greater than 1; the B data bitonic sorting tasks are respectively associated with different data subsequences to be sorted, and the B data subsequences to be sorted are all generated based on the data sequence to be sorted;
[0013] The parallel component module is used to run C data bitonic sorting components in parallel according to B data bitonic sorting tasks to obtain B data sorting sub-results output by the C data bitonic sorting components; a data sorting sub-result is used to represent the sorting result of the to-be-sorted data subsequence associated with a data bitonic sorting task;
[0014] The merging result module is used to merge B data sorting sub-results based on C data bitonic sorting components to obtain a data sorting result for the data sequence to be sorted.
[0015] The calling component module is specifically used to obtain a data sorting request and send the data sorting request to the task control component; the data sorting request carries a first total number of data to be sorted in the sequence of data to be sorted;
[0016] Then open the task module, including:
[0017] A first determining unit is used to determine, in the task control component, a number to be sorted associated with the first total number; the number to be sorted is equal to or greater than the first total number;
[0018] A first acquisition unit is used to acquire the data storage capacities corresponding to the C data bitonic sorting components, and compare the number to be sorted with the total data storage capacity to obtain a first comparison result; the total data storage capacity is equal to the sum of the C data storage capacities;
[0019] The task starting unit is used to start B data double-adjustment sorting tasks according to the first comparison result.
[0020] Among them, starting the task unit includes:
[0021] A first opening subunit is used to obtain a first quantity ratio between the number to be sorted and the total data storage capacity if the first comparison result is that the number to be sorted is greater than the total data storage capacity;
[0022] The first opening subunit is further used to open B data bitonic sorting tasks according to the first quantity ratio; the product of C and the first quantity ratio is equal to B; the second total quantity carried by a data bitonic sorting task is equal to the data storage capacity; the second total quantity is used to represent the total quantity of data to be sorted in the data subsequence to be sorted associated with a data bitonic sorting task;
[0023] The second start subunit is used for the second start subunit, and is used to start B data bitonal sorting tasks according to C data bitonal sorting components if the first comparison result is that the number to be sorted is less than or equal to the total data storage capacity; B is equal to C.
[0024] Among them, the parallel component module includes:
[0025] A first distribution unit is used for distributing B data bitonic sorting tasks to C data bitonic sorting components if B is equal to C; wherein one data bitonic sorting task is distributed to one data bitonic sorting component;
[0026] The first parallel unit is used to run B data bitonic sorting tasks in parallel through C data bitonic sorting components to obtain data sorting sub-results outputted by the C data bitonic sorting components respectively.
[0027] Among them, B data bit-tuning sorting tasks include data bit-tuning sorting task D e , e is a positive integer and e is less than or equal to B; B data subsequences to be sorted include data bitonic sorting task D e The corresponding data subsequence A to be sorted e ; C data bit-tuning sorting components include a component for performing data bit-tuning sorting tasks D e The data bitonic sorting component F e ;
[0028] The first parallel unit comprises:
[0029] The first generating subunit is used to sort the data bitune component F e In the data bit-tuning sorting task D e The corresponding data subsequence A to be sorted e , treat the sorted data subsequence A e Perform bitonic sorting to obtain the data subsequence A to be sorted e The corresponding data sorting sub-result;
[0030] The second generation subunit is used to run the data bitune sorting component F eAt the same time, B-1 data bitonic sorting tasks are run in parallel through C-1 data bitonic sorting components to obtain data sorting sub-results output by C-1 data bitonic sorting components respectively; C-1 data bitonic sorting components refer to the C data bitonic sorting components except for the data bitonic sorting component F e The data bit-tuning sorting components other than the data bit-tuning sorting task D are B-1 data bit-tuning sorting tasks. e Other data bitune sorting tasks.
[0031] The data sorting request carries the read addresses corresponding to the data to be sorted in the data sequence to be sorted, and the sorting order for the data sequence to be sorted;
[0032] The first generation subunit comprises:
[0033] The first acquisition subunit is used to sort the data in the data bitune sorting component F e In the data bit-tuning sorting task D e The target read address and the second total number carried; the read addresses corresponding to the data to be sorted in the data sequence to be sorted include the target read address; the second total number is used to represent the data subsequence A to be sorted e The total number of data to be sorted in ;
[0034] The second acquisition subunit is used to send a data acquisition request carrying a target read address to the data read-write component, and acquire an initial subsequence of data to be sorted from the target read address through the data read-write component; the initial subsequence of data to be sorted belongs to the data sequence to be sorted;
[0035] A first determining subunit is used to determine the difference between the second total number and the third total number; the third total number is used to represent the total number of data to be sorted in the initial subsequence of data to be sorted;
[0036] The first determination subunit is further used to obtain the supplementary data to be sorted according to the quantity difference and the sorting order, and add the supplementary data to be sorted to the initial subsequence of data to be sorted to obtain the subsequence A of data to be sorted. e ;
[0037] The first determining subunit is also used to sort the data subsequence A e Load into the data bitune sorting component F e The corresponding storage component.
[0038] Among them, the data double-tuning sorting component F e It includes a sorting control subcomponent, a round control subcomponent, a data acquisition subcomponent, and a bitune sorting subcomponent;
[0039] The first generation subunit comprises:
[0040] The second determination subunit is used to start the round control subcomponent when the sorting control subcomponent obtains the data loading completion notification; the data loading completion notification is used to indicate that the data bit-tuning sorting component F e The corresponding storage component has loaded the data subsequence A to be sorted e ;
[0041] The second determining subunit is further configured to determine, in the round control subcomponent, a subsequence A of the data to be sorted according to the second total number. e The corresponding bitonic sorting stage and the bitonic sorting logic parameters corresponding to the bitonic sorting stage are sent to the data acquisition subcomponent; the second total number is used to represent the subsequence A of the data to be sorted e The total number of data to be sorted in ;
[0042] The third determination subunit is used to obtain the intermediate data to be sorted in the data acquisition subcomponent according to the bitonic sorting stage and the bitonic sorting logic parameters corresponding to the bitonic sorting stage, send the intermediate data to be sorted to the bitonic sorting subcomponent, sort the intermediate data to be sorted through the bitonic sorting subcomponent, obtain the intermediate sorting subresult, and determine the data subsequence A to be sorted according to the intermediate sorting subresult. e The corresponding data sorting sub-result; the data sub-sequence A to be sorted e Including intermediate data to be sorted.
[0043] Wherein, the number of stages of the bitune sorting stage is at least two;
[0044] The third determination subunit is specifically used to read the first data sorting transition subresult from the storage component in the data acquisition subcomponent according to the i-th bitonic sorting stage in at least two bitonic sorting stages; i is a positive integer greater than 1, and the first data sorting transition subresult is the output result in the i-1-th bitonic sorting stage; the bitonic sorting logic parameters corresponding to the i-th bitonic sorting stage include n-round sorting round logic parameters; n is determined based on i, and n is a positive integer greater than 1;
[0045] The third determination subunit is further specifically used to obtain the intermediate data to be sorted from the intermediate data sorting transition subresult according to the j-th sorting round logic parameter; j is a positive integer and j is less than or equal to n; if j is 1, the intermediate data sorting transition subresult is the first data sorting transition subresult; if j is not 1, the intermediate data sorting transition subresult is the second data sorting transition subresult cached in the data acquisition subcomponent, and the second data sorting transition subresult is determined based on the j-1-th sorting round logic parameter;
[0046] The third determination subunit is also specifically used to send the intermediate data to be sorted to the bitonic sorting subcomponent, sort the intermediate data to be sorted through the bitonic sorting subcomponent, obtain the intermediate sorting subresult of the intermediate data to be sorted, and determine the subsequence A of the data to be sorted according to the intermediate sorting subresult. e The corresponding third data sorting transition sub-result;
[0047] The third determining subunit is further specifically configured to cache the third data sorting transition sub-result in the data acquiring sub-component if j is less than n;
[0048] The third determining subunit is further specifically configured to write the third data sorting transition sub-result into the storage component if j=n and the i-th bitonic sorting stage is the last bitonic sorting stage of at least two bitonic sorting stages, and determine the third data sorting transition sub-result as the data subsequence A to be sorted. e The corresponding data sorting sub-result.
[0049] Among them, the parallel component module includes:
[0050] The split task unit is used to split B data bitonic sorting tasks into K*C data bitonic sorting tasks if B is greater than C; K is the maximum running rounds corresponding to the C data bitonic sorting components respectively;
[0051] The second distribution unit is used to distribute the C data bitonic sorting tasks in the i-th running round to the C data bitonic sorting components; wherein one data bitonic sorting task is distributed to one data bitonic sorting component; i is a positive integer less than or equal to K;
[0052] The second parallel unit is used to run C data bitonic sorting tasks in the i-th running round in parallel through C data bitonic sorting components, and obtain the data sorting sub-results respectively output by the C data bitonic sorting components, until i is equal to K, and the data sorting sub-results corresponding to the B data bitonic sorting tasks are obtained.
[0053] Among them, the merge result module includes:
[0054] The second acquisition unit is used to acquire U data sorting sub-result pairs in the L-th merging; U is a positive integer; L is a positive integer; if L is 1, the U data sorting sub-result pairs are generated based on the B data sorting sub-results; if L is not 1, the U data sorting sub-result pairs are generated based on the historical merging data sorting sub-results, and the historical merging data sorting sub-results are the sub-results obtained in the L-1-th merging;
[0055] A first merging unit is used to merge U data sorting sub-result pairs based on C data bitonic sorting components to obtain U target merged data sorting sub-results;
[0056] A second determining unit is used for determining the U target merged data sorting sub-results as the data sorting result corresponding to the data sequence to be sorted if U is equal to 1;
[0057] The second merging unit is used to merge U target merged data sorting sub-results based on C data bitonic sorting components in the L+1th merge if U is greater than 1, so as to obtain a data sorting result for the data sequence to be sorted.
[0058] Among them, U data sorting sub-result pairs include data sorting sub-result pairs M p , p is a positive integer, and p is less than or equal to U;
[0059] The first merging unit comprises:
[0060] The first quantity subunit is used to determine the data sorting sub-result pair M p The fourth total number of the associated data to be sorted determines the data sorting sub-result pair M p The data storage capacity corresponding to the associated data bitonic sorting component; C data bitonic sorting components include data sorting sub-result pairs M p The associated data bitonic sorting component;
[0061] A second quantity subunit is used to compare the fourth total quantity with the data storage capacity to obtain a second comparison result;
[0062] The third generating subunit is used to generate M according to the second comparison result and the data sorting sub-result. p The associated data bit-tuning sorting component sorts the data sub-results to M p Perform bitwise sorting to obtain the data sorting result pair M p The corresponding target merged data sorting sub-results;
[0063] The fourth generating subunit is used to determine the target merged data sorting sub-results corresponding to the U data sorting sub-results as U target merged data sorting sub-results.
[0064] Among them, the data sorting sub-result pair M p including a first data sorting sub-result and a second data sorting sub-result;
[0065] The third generation subunit includes:
[0066] A first equalizing subunit is used for equally dividing the first data sorting sub-result into a first ordered data segment and a second ordered data segment, and equally dividing the second data sorting sub-result into a third ordered data segment and a fourth ordered data segment if the second comparison result is that the fourth total number is greater than the data storage capacity;
[0067] The first sorting subunit is configured to, if the data in the first ordered data segment is less than or equal to the data in the second ordered data segment, and the data in the third ordered data segment is less than or equal to the data in the fourth ordered data segment, perform bitonic sorting on the first ordered data segment and the third ordered data segment based on the first data bitonic sorting component to obtain a third data sorting subresult; the data sorting subresult is used for M p The associated data bitonic sorting components include a first data bitonic sorting component;
[0068] The second sorting subunit is used to perform bitonic sorting on the second ordered data segment and the fourth ordered data segment based on the second data bitonic sorting component to obtain a fourth data sorting sub-result; the data sorting sub-result is used to perform bitonic sorting on the second ordered data segment and the fourth ordered data segment based on the second data bitonic sorting component to obtain a fourth data sorting sub-result; p The associated data bitonic sorting component includes a second data bitonic sorting component;
[0069] A second equalizing subunit is used to equally divide the third data sorting sub-result into a fifth ordered data segment and a sixth ordered data segment, and equally divide the fourth data sorting sub-result into a seventh ordered data segment and an eighth ordered data segment;
[0070] a third sorting subunit, configured to, if the data in the fifth ordered data segment is less than or equal to the data in the sixth ordered data segment, and the data in the seventh ordered data segment is less than or equal to the data in the eighth ordered data segment, perform bitonic sorting on the sixth ordered data segment and the seventh ordered data segment based on a target data bitonic sorting component to obtain a fifth data sorting subresult; the target data bitonic sorting component is the first data bitonic sorting component or the second data bitonic sorting component;
[0071] The fourth sorting subunit is configured to obtain a data sorting subresult pair M based on the fifth ordered data segment, the fifth data sorting subresult, and the eighth ordered data sorting. p The corresponding target merged data sorting sub-result.
[0072] Among them, the C data bitonic sorting components all include an index generation subcomponent, and the index generation subcomponent is used to assign an index value to each to-be-sorted data in the to-be-sorted data sequence; the data sorting request carries a write address;
[0073] The data processing device further comprises:
[0074] A sequence generation module is used to sequentially obtain the index values corresponding to each data to be sorted in the data sorting result, and generate an index value sequence with the same length as the sequence of the data to be sorted according to the obtained index values;
[0075] The write sequence module is used to send a data write request for the index value sequence to the data read and write component, and write the index value sequence to the write address through the data read and write component.
[0076] On one hand, the present application provides a computer device, including: a processor, a memory, and a network interface;
[0077] The above-mentioned processor is connected to the above-mentioned memory and the above-mentioned network interface, wherein the above-mentioned network interface is used to provide a data communication function, the above-mentioned memory is used to store a computer program, and the above-mentioned processor is used to call the above-mentioned computer program so that the computer device executes the method in the embodiment of the present application.
[0078] On one hand, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program is suitable for being loaded by a processor and executing the method in the embodiment of the present application.
[0079] On the one hand, an embodiment of the present application provides a computer program product or a computer program, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method in the embodiment of the present application.
[0080] In an embodiment of the present application, based on C data bitonic sorting components, a computer device can generate multiple data bitonic sorting tasks for a data sequence to be sorted, and by running C data bitonic sorting components in parallel, multiple data bitonic sorting tasks can be executed simultaneously. By executing multiple data bitonic sorting tasks simultaneously, the sorting time of the data sequence to be sorted, that is, the task execution time, can be reduced; subsequently, based on C data bitonic sorting components, the computer device merges B data sorting sub-results to obtain a data sorting result for the data sequence to be sorted. It can be seen that the embodiment of the present application can not only reduce the execution time of the sorting task for the data sequence to be sorted, but also improve the sorting efficiency of the sorting task by running C data bitonic sorting components in parallel. BRIEF DESCRIPTION OF THE DRAWINGS
[0081] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0082] Figure 1It is a schematic diagram of a system architecture provided by an embodiment of the present application;
[0083] Figure 2 It is a flowchart of a data processing method provided in an embodiment of the present application;
[0084] Figure 3 It is a schematic diagram of a data processing scenario provided by an embodiment of the present application;
[0085] Figure 4 It is a schematic diagram of a data processing scenario provided by an embodiment of the present application;
[0086] Figure 5 It is a schematic diagram of a data processing scenario provided by an embodiment of the present application;
[0087] Figure 6 It is a flowchart of a data processing method provided in an embodiment of the present application;
[0088] Figure 7 is a structural schematic diagram of a sorting accelerator provided in an embodiment of the present application;
[0089] Figure 8 It is a flowchart of a data processing method provided in an embodiment of the present application;
[0090] Fig. 9 It is a schematic diagram of a data processing scenario provided by an embodiment of the present application;
[0091] Fig.10 is a structural schematic diagram of a data processing device provided in an embodiment of the present application;
[0092] Fig.11 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0093] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0094] To facilitate understanding, some nouns are briefly explained as follows:
[0095] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines so that machines have the functions of perception, reasoning and decision-making.
[0096] Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, mechatronics and other technologies. Artificial intelligence software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, as well as machine learning / deep learning, autonomous driving, smart transportation and other major directions.
[0097] See also Figure 1 , Figure 1 Schematic diagram of a system architecture provided by an embodiment of the present application. Figure 1 As shown, the system may include a business server 100 and a terminal cluster. The terminal cluster may include: terminal device 200a, terminal device 200b, terminal device 200c, ..., terminal device 200n. It can be understood that the above system may include one or more terminal devices, and this application does not limit the number of terminal devices.
[0098] Among them, there can be a communication connection between terminal clusters, for example, there is a communication connection between terminal device 200a and terminal device 200b, and there is a communication connection between terminal device 200a and terminal device 200c. At the same time, any terminal device in the terminal cluster can have a communication connection with the business server 100, for example, there is a communication connection between terminal device 200a and business server 100, wherein the above communication connection does not limit the connection method, and can be directly or indirectly connected through a wired communication method, or directly or indirectly connected through a wireless communication method, or through other methods, and this application does not limit it here.
[0099] It should be understood that Figure 1 Each terminal device in the terminal cluster shown in the figure may be installed with an application client. When the application client runs in each terminal device, it can be respectively connected to the above Figure 1The business servers 100 shown in the figure interact with each other, that is, the above-mentioned communication connection. Among them, the application client can be an application client with data sorting function such as a short video application, a video application, a live broadcast application, a social application, an instant messaging application, a game application, a music application, a shopping application, a novel application, a browser, etc. Among them, the application client can be an independent client or an embedded sub-client integrated in a client (for example, a social client, an educational client, and a multimedia client, etc.), which is not limited here.
[0100] To facilitate subsequent understanding and description, the embodiments of the present application can be Figure 1 A terminal device is selected as the target terminal device in the terminal cluster shown, for example, terminal device 200a is selected as the target terminal device. When the data sequence to be sorted is obtained and a data sorting instruction for the data sequence to be sorted is received, the terminal device 200a can send a data sorting request for the data sequence to be sorted to the business server 100. Further, after the business server 100 receives the data sorting request sent by the terminal device 200a, it can call C data bit-tuning sorting components through the data sorting request, wherein the data sorting request carries the first total number of data to be sorted in the data sequence to be sorted, C is a positive integer greater than 1, that is, at least two data bit-tuning sorting components are called; further, the business server 100 can start B data bit-tuning sorting tasks according to the first total number and the data storage capacity corresponding to the C data bit-tuning sorting components, wherein B is a positive integer greater than 1, that is, at least two data bit-tuning sorting tasks are started, and the B data bit-tuning sorting tasks are respectively associated with different subsequences of data to be sorted, B Each of the data subsequences to be sorted is generated based on the data sequence to be sorted; further, the business server 100 can run C data bitonic sorting components in parallel according to the B data bitonic sorting tasks, and obtain B data sorting sub-results output by the C data bitonic sorting components, wherein one data sorting sub-result is used to represent the sorting result of the data subsequence to be sorted associated with a data bitonic sorting task; further, based on the C data bitonic sorting components, the business server 100 can merge the B data sorting sub-results to obtain a data sorting result for the data sequence to be sorted, and finally, the data sorting result can be written to the write address carried by the data sorting request.
[0101] Optionally, if the terminal device 200a locally stores the above-mentioned C data bitonal sorting components, the terminal device 200a can perform bitonal sorting on the data to be sorted in the data sequence to be sorted through the local C data bitonal sorting components. It can be understood that this process is consistent with the process of the above-mentioned business server performing bitonal sorting on the data to be sorted in the data sequence to be sorted through C data bitonal sorting components, so it will not be repeated here. Please refer to the above description.
[0102] Optionally, it is understandable that the system architecture may include multiple business servers, a terminal device may be connected to a business server, each business server may obtain the data sorting request sent by the terminal device connected thereto, thereby performing bitonal sorting on the data sequence to be sorted targeted by the obtained data sorting request, and returning the data sorting result to the terminal device connected thereto.
[0103] The method provided in the embodiment of the present application can be applied to circuits such as application specific integrated circuits (ASIC), field programmable gate arrays (FPGA), etc. The circuit can be applied to chips. The above chips can be integrated in a central processing unit (CPU), or an AI chip, or other types of chips. The AI chip is integrated in a business server in the form of a peripheral component interconnect express (PCIE) board.
[0104] It is understandable that the method provided in the embodiments of the present application can be executed by a computer device, which includes but is not limited to a terminal device or a business server. Among them, the business server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud databases, cloud services, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Terminal devices include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, car terminals, etc. Among them, the terminal device and the business server can be directly or indirectly connected by wired or wireless means, and the embodiments of the present application are not limited here.
[0105] It is understandable that the above system architecture can be applied to data sorting scenarios such as database, target recognition, and text analysis, and specific business scenarios will not be listed one by one here.
[0106] For further information, see Figure 2 , Figure 2 is a flow chart of a data processing method provided by an embodiment of the present application. The data processing method can be performed by a business server (for example, the above Figure 1 The service server 100 shown in the figure may also be executed by a terminal device (for example, the above Figure 1The terminal device 200a shown in the figure can also be executed by the business server and the terminal device interactively. For ease of understanding, the embodiment of the present application takes the method executed by the business server as an example for explanation. In addition, the embodiment of the present application can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc. Figure 2 As shown, the data processing method may at least include the following steps S101 to S104.
[0107] Step S101, obtaining a data sorting request for a data sequence to be sorted, and calling C data bitonic sorting components according to the data sorting request; C is a positive integer greater than 1.
[0108] Specifically, a data sorting request is obtained, and the data sorting request is sent to the task control component; the data sorting request carries a first total number of data to be sorted in the sequence of data to be sorted.
[0109] In the embodiment of the present application, the data sequence to be sorted may be an unordered data sequence, such as a sequence (0, 5, 2, 6, 8, 7, 6), and the data sequence to be sorted may be a bitonic sequence, such as a sequence (0, 2, 6, 9, 8, 7, 4, 1). The embodiment of the present application does not limit the sequence type of the data sequence to be sorted. It is understandable that the first total number of data to be sorted in the data sequence to be sorted is at least three.
[0110] In the embodiment of the present application, the data sorting request may be sent by the terminal device to the service server, as described above. Figure 1 As described in, it may also be generated locally by the business server. The embodiment of the present application does not limit the source of the data sorting request. The data sorting request may carry the data to be sorted in the data sequence to be sorted. At this time, the business server may determine the first total number based on the data to be sorted in the data sequence to be sorted. Alternatively, the data sorting request carries the read addresses and the first total number corresponding to the data to be sorted in the data sequence to be sorted. Then the business server may read the data to be sorted in the data sequence to be sorted from the read address. The embodiment of the present application does not limit the data carried by the data acquisition request, and it may be set according to the actual application. In the embodiment of the present application, the data bitonic sorting component refers to a component that can bitonically sort the data sequence to be sorted.
[0111] Please also see Figure 3 , Figure 3 Schematic diagram of a data processing scenario provided by an embodiment of the present application. Figure 3As shown, the data sorting request 30a includes a first total quantity, a read address, a write address and a write type, wherein the write address is used to instruct the business server 30b to write the data sorting result to the address, and the write type refers to whether the result type written to the write address is a data result type or an index result type. Please refer to the description in step S104 below, which will not be expanded here.
[0112] After the business server 30b obtains the data sorting request 30a, it can call C (at least two) data double-tuning sorting components according to the data sorting request 30a, see below Figure 6 The description of the internal structure of the data bitonic sorting component in the corresponding embodiment is not described here. It is understandable that in actual application, the first total number can be equal to 2 or a positive integer greater than 2; it is understandable that the present application can be applied to a scenario in which a data bitonic sorting component sorts a data sequence to be sorted.
[0113] Step S102, according to the data sequence to be sorted and C data bitonic sorting components, start B data bitonic sorting tasks; B is a positive integer greater than 1; the B data bitonic sorting tasks are respectively associated with different data subsequences to be sorted, and the B data subsequences to be sorted are all generated based on the data sequence to be sorted.
[0114] Specifically, in the task control component, the number to be sorted associated with the first total number is determined; the number to be sorted is equal to or greater than the first total number; the data storage capacities corresponding to C data bitonic sorting components are obtained, and the number to be sorted is compared with the total data storage capacity to obtain a first comparison result; the total data storage capacity is equal to the sum of C data storage capacities; and B data bitonic sorting tasks are started according to the first comparison result.
[0115] Among them, the specific process of starting B data bitonic sorting tasks according to the first comparison result may include: if the first comparison result is that the number to be sorted is greater than the total data storage capacity, then obtaining a first quantity ratio between the number to be sorted and the total data storage capacity; starting B data bitonic sorting tasks according to the first quantity ratio; the product of C and the first quantity ratio is equal to B; the second total quantity carried by a data bitonic sorting task is equal to the data storage capacity; the second total quantity is used to characterize the total number of data to be sorted in a subsequence of data to be sorted associated with a data bitonic sorting task; if the first comparison result is that the number to be sorted is less than or equal to the total data storage capacity, then starting B data bitonic sorting tasks according to C data bitonic sorting components; B is equal to C.
[0116] Please see again Figure 3, the business server 30b sends the data sorting request 30a to the top-level control component, and the data sorting request 30a is sent to the task control component 30c through the top-level control component. It can also be understood that after the top-level control component configures the data carried by the data sorting request with the task control component 30c, it sends the data sorting instruction (signal, Sort request) for the data sequence to be sorted to the task control component 30c.
[0117] In the task control component 30c, the business server 30b determines the number of data to be sorted associated with the first total number. For example, the first total number corresponding to the data sequence to be sorted is 31, and 31 is closest to the fifth power of 2, so the number of data to be sorted is 2. 5 , i.e. 32. It can be understood that in the embodiment of the present application, the first total number satisfies the condition of a positive integer greater than 2, and the number to be sorted is equal to or greater than the first total number and is a power of 2; the business server 30b obtains the data storage capacity corresponding to the C data bitonic sorting components. Usually, in order not to increase the complexity of parameter configuration, the C data storage capacities are consistent, for example, all are 32; assuming that the C data storage capacities are all 32, the total data storage capacity is 32*C; further, the number to be sorted is compared with the total data storage capacity to obtain the first comparison result. It should be noted that the above examples are only for the convenience of description and understanding.
[0118] Please see again Figure 3 If the first comparison result is that the number to be sorted is greater than the total data storage capacity, the business server 30b obtains a first quantity ratio between the number to be sorted and the total data storage capacity, such as Figure 3 The ratio K shown in the example, for example, the data storage capacity corresponding to a data bitonic sorting component is 8, C is equal to 4, then the total data storage capacity is equal to 32, if the number to be sorted is 64, then K=2, at this time, the second total number carried by a data bitonic sorting task can be set to the data storage capacity, that is, the total number of data to be sorted in the data subsequence to be sorted associated with a data bitonic sorting task is equal to the data storage capacity. Obviously, the business server 30b needs to start 8 data bitonic sorting tasks through the task control component 30c.
[0119] If the first comparison result is that the number to be sorted is less than or equal to the total data storage capacity, for example, the data storage capacity corresponding to a data bit-tuned sorting component is 64, C is equal to 4, and the total data storage capacity is equal to 256. If the number to be sorted is 64, it is obvious that at this time, 4 data bit-tuned sorting tasks can be started to sort the data to be sorted in the data sequence to be sorted in one round of operation. The second total number associated with a data bit-tuned sorting task is equal to the ratio of the number to be sorted to C, that is, 64 / 4. It should be noted that the above examples are only for the convenience of description and understanding.
[0120] In summary, the task control component 30c parses the parameters (including the first total number and data storage capacity, etc.), splits the sorting task for the data sequence to be sorted into sorting subtasks (i.e., data double-tuning sorting tasks) of the same number as the data double-tuning sorting component, and sends one data double-tuning sorting task to a Sort unit (i.e., data double-tuning sorting component). In addition, the embodiment of the present application also takes into account the storage area allocated to each sort unit (i.e., data storage capacity). When the total data storage capacity is less than the number to be sorted, the sorting task can also be split into K*C subtasks (i.e., data double-tuning sorting tasks), which are completed in k rounds of operation, and C subtasks are also sent in each round, that is, one data double-tuning sorting task is sent to a Sort unit. After the task control component 30c sends the data double-tuning sorting task, it waits for the completion signal of the task.
[0121] Step S103, according to the B data bitonic sorting tasks, C data bitonic sorting components are run in parallel to obtain B data sorting sub-results output by the C data bitonic sorting components; a data sorting sub-result is used to represent the sorting result of the data subsequence to be sorted associated with a data bitonic sorting task.
[0122] Specifically, if B is equal to C, then B data bitonic sorting tasks are distributed to C data bitonic sorting components; wherein one data bitonic sorting task is distributed to one data bitonic sorting component; through C data bitonic sorting components, B data bitonic sorting tasks are run in parallel to obtain data sorting sub-results output by C data bitonic sorting components. Among them, B data bitonic sorting tasks include data bitonic sorting task D e , e is a positive integer and e is less than or equal to B; B data subsequences to be sorted include data bitonic sorting task D e The corresponding data subsequence A to be sorted e ; C data bit-tuning sorting components include a component for performing data bit-tuning sorting tasks D e The data bitonic sorting component F e .
[0123] If B is greater than C, the B data bitonic sorting tasks are split into K*C data bitonic sorting tasks; K is the maximum running rounds corresponding to the C data bitonic sorting components; the C data bitonic sorting tasks in the i-th running round are distributed to the C data bitonic sorting components; wherein one data bitonic sorting task is distributed to one data bitonic sorting component; i is a positive integer less than or equal to K; through the C data bitonic sorting components, the C data bitonic sorting tasks in the i-th running round are run in parallel to obtain the data sorting sub-results output by the C data bitonic sorting components, until i is equal to K, the data sorting sub-results corresponding to the B data bitonic sorting tasks are obtained.
[0124] As can be seen from the above step S102, there are two situations in which the data double-tuning sorting task is started in the embodiment of the present application. One is that the number to be sorted is less than or equal to the total data storage capacity, that is, B=C. At this time, the data double-tuning sorting tasks equal to the number of data double-tuning sorting components are started, that is, C data double-tuning sorting tasks; the other is that the number to be sorted is greater than the total data storage capacity, at this time K*C data double-tuning sorting tasks are started, and C data double-tuning sorting components are run in parallel for K rounds to complete the K*C data double-tuning sorting tasks; it can be understood that when the number to be sorted is greater than the total data storage capacity, the running process of a single round of data double-tuning sorting components is , which is consistent with the parallel operation process of the data bitonic sorting component when the number of data to be sorted is less than or equal to the total data storage capacity. The only difference between the two is that after the former completes the parallel operation of C data bitonic sorting components, the C data sorting sub-results generated in a single round need to be stored in the external memory to execute the next round of C data bitonic sorting tasks, while the latter does not need to store the C data sorting sub-results in the external memory, and the C data sorting sub-results can be directly merged to obtain the data sorting result corresponding to the data to be sorted. Therefore, in the embodiment of the present application, only the scenario where B is equal to C is described, and the scenario where B is greater than C can refer to the description below.
[0125] Please also see Figure 4 , Figure 4 Schematic diagram of a data processing scenario provided by an embodiment of the present application. Figure 4 As shown, in the embodiment of the present application, C is set to 4, and the task control component 40a starts 4 data double adjustment sorting tasks, such as Figure 4 In the example of data double adjustment sorting task 401a, data double adjustment sorting task 402a, data double adjustment sorting task 403a and data double adjustment sorting task 404a, the business server calls four data double adjustment sorting components, such as Figure 4 The examples include a data bitonic sorting component S1, a data bitonic sorting component S2, a data bitonic sorting component S3 and a data bitonic sorting component S4.
[0126] Among them, through the received data bitonic sorting task, the data bitonic sorting component obtains the subsequence of data to be sorted, please refer to the following Figure 6 The corresponding embodiments are not described here. Figure 4 , the data bit-tuning sorting component S1 executes the data bit-tuning sorting task 401a, and the data subsequence 401b to be sorted corresponding to the data bit-tuning sorting task 401a is (10, 20, 5, 9), the data bit-tuning sorting component S2 executes the data bit-tuning sorting task 402a, and the data subsequence 402b to be sorted corresponding to the data bit-tuning sorting task 402a is (3, 8, 12, 14), the data bit-tuning sorting component S3 executes the data bit-tuning sorting task 403a, and the data subsequence 403b to be sorted corresponding to the data bit-tuning sorting task 403a is (90, 0, 60, 40), and the data bit-tuning sorting component S4 executes the data bit-tuning sorting task 404a, and the data subsequence 404b to be sorted corresponding to the data bit-tuning sorting task 404a is (23, 35, 95, 18). In the embodiment of the present application, the business server may first generate a unique index identifier corresponding to each to-be-sorted data in the to-be-sorted data sequence in order. The embodiment of the present application does not limit the form of the index identifier, and may be any information that can be used to identify each to-be-sorted data in the to-be-sorted data sequence. For ease of description and understanding, the present application takes a 4-bit binary system as an example, such as Figure 4 As shown, the index of 10 is 0000, the index of 20 is 0001, the index of 5 is 0010, the index of 9 is 0011, the index of 3 is 0100, the index of 8 is 0101, the index of 12 is 0110, the index of 14 is 0111, the index of 90 is 1000, the index of 0 is 1001, the index of 60 is 1010, the index of 1011 is 40, the index of 23 is 1100, the index of 35 is 1101, the index of 95 is 1110, and the index of 18 is 1111.
[0127] Please see again Figure 4 , the business server passes C (such as Figure 4 4) data bitune sorting components, running B in parallel (such as Figure 44) data bitonic sorting tasks, and obtain the data sorting sub-results output by C data bitonic sorting components respectively. It can be understood that one bitonic sorting component performs one data bitonic sorting task, and the sorting logic of each data bitonic sorting component is consistent, so here the data bitonic sorting component S1 performing the data bitonic sorting task 401a is described as an example, and the execution process of the remaining data bitonic sorting components can be referred to the description below, which is not repeated. The embodiment of the present application compares the data to be sorted (original data to be sorted) corresponding to the two index identifiers respectively, and updates the data to be sorted (compared data to be sorted) corresponding to the two index identifiers respectively according to the comparison result.
[0128] Figure 4 Symbols in Indicates ascending order, two Connected means that the data to be sorted corresponding to the two connected index identifiers are sorted in ascending order. Indicates descending order, two Connected means that the data to be sorted corresponding to the two connected index identifiers are sorted in descending order. Figure 4 As shown, in the data bitonic sorting component S1, the data subsequence 401b to be sorted is (10, 20, 5, 9), so the second total number is equal to 4, and there are two bitonic sorting stages, as shown in FIG. Figure 4 The first double-tuning sorting stage 401h and the second double-tuning sorting stage 402h in the second double-tuning sorting stage 402h include two sorting rounds, such as Figure 4 The first sorting round 401i and the second sorting round 402i in the first two-tone sorting stage 401h, a sorting round logic parameter includes an ascending comparison between index identifier 0000 and index identifier 0001, and a descending comparison between index identifier 0010 and index identifier 0011. Therefore, the output result of the first two-tone sorting stage 401h is that index identifier 0000 is updated to data to be sorted 10, index identifier 0001 is updated to data to be sorted 20, index identifier 0010 is updated to data to be sorted 9, and index identifier 0011 is updated to data to be sorted 5. At this time, the business server writes the above output results to the storage component 401d corresponding to the data two-tone sorting component S1. After the writing is successful, the second two-tone sorting stage 402h is started.
[0129] Please see again Figure 4In the second bit-tuning sorting stage 402h, the sorting round logic parameters of the first sorting round 401i include ascending comparison of index identifier 0000 and index identifier 0010, and ascending comparison of index identifier 0001 and index identifier 0011. Therefore, the output result of the first sorting round 401i is that index identifier 0000 is updated to data 9 to be sorted, index identifier 0010 is updated to data 10 to be sorted, index identifier 0001 is updated to data 5 to be sorted, and index identifier 0011 is updated to data 20 to be sorted. At this time, the business server caches the output result of the first sorting round 401i in the data acquisition subcomponent of the data bit-tuning sorting component S1, and does not need to be written to the storage component 401d. Therefore, when the second sorting round 402i is started, the data bit-tuning sorting component S1 does not need to obtain the output result of the previous sorting round from the storage component 401d, and can directly obtain it from the local data acquisition subcomponent.
[0130] Please see again Figure 4 , the sorting round logic parameters of the second sorting round 402i include the index identifier 0000 and the index identifier 0001 are compared in ascending order, and the index identifier 0010 and the index identifier 0011 are compared in ascending order. Therefore, the output result of the second sorting round 402i is that the index identifier 0000 is updated to the data to be sorted 5, the index identifier 0001 is updated to the data to be sorted 9, the index identifier 0010 is updated to the data to be sorted 10, and the index identifier 0011 is updated to the data to be sorted 20. The second double-adjustment sorting stage 402h ends. At this time, the business server outputs the output result of the second sorting round 402i (i.e. Figure 4 The data sorting sub-result 401c) is written to the storage component 401d. It should be noted that, Figure 4 The storage component 401d, storage component 402d, storage component 403d and storage component 404d shown in the example are all local storages of the data bitonic sorting component.
[0131] According to the above sorting process, the output result corresponding to the data bitonic sorting component S1 is the data sorting sub-result 401c, the output result corresponding to the data bitonic sorting component S2 is the data sorting sub-result 402c, the output result corresponding to the data bitonic sorting component S3 is the data sorting sub-result 403c, and the output result corresponding to the data bitonic sorting component S4 is the data sorting sub-result 404c.
[0132] After the sorting is completed, the sort unit writes the output result back to the memory and sends a sorting completion signal to the task control component 40a, indicating that the data bit-tuning sorting task is completed.
[0133] Step S104: Based on the C data bitonic sorting components, the B data sorting sub-results are merged to obtain a data sorting result for the data sequence to be sorted.
[0134] Specifically, U data sorting sub-result pairs are obtained in the Lth merge; U is a positive integer; L is a positive integer; if L is 1, then the U data sorting sub-result pairs are generated based on B data sorting sub-results; if L is not 1, then the U data sorting sub-result pairs are generated based on the historical merged data sorting sub-results, and the historical merged data sorting sub-results are the sub-results obtained in the L-1th merge; based on C data bitonic sorting components, the U data sorting sub-result pairs are merged to obtain U target merged data sorting sub-results; if U is equal to 1, then the U target merged data sorting sub-results are determined as the data sorting results corresponding to the data sequence to be sorted; if U is greater than 1, then in the L+1th merge, based on C data bitonic sorting components, the U target merged data sorting sub-results are merged to obtain the data sorting result for the data sequence to be sorted. Among them, the U data sorting sub-result pairs include data sorting sub-result pairs M p , p is a positive integer, and p is less than or equal to U.
[0135] After the task control component receives the completion signals of all subtasks (data bit-sorting tasks), it starts the merge instruction for the C data sorting sub-results. Similarly, the business server calls the sort unit to complete, which is consistent with the sorting process in step S103. The difference between the two is that the data subsequence to be sorted may be an unordered sequence, while the sequence corresponding to the data sorting subresult is an ordered sequence.
[0136] Please also see Figure 5 as well as Figure 4 , Figure 5 Schematic diagram of a data processing scenario provided by an embodiment of the present application. Figure 5 As shown, the business server is for Figure 4 The four data sorting sub-results in the example can obtain two data sorting sub-result pairs, such as Figure 5 Shown is a data sorting sub-result pair 401f generated by the data sorting sub-result 401c and the data sorting sub-result 402c, and a data sorting sub-result pair 402f generated by the data sorting sub-result 403c and the data sorting sub-result 404c.
[0137] Through the task control component 40a, the business server starts two data merging tasks, such as Figure 5 The data merging task 401e and the data merging task 402e, the data merging task 401e refers to the task of merging the data sorting sub-result 401c and the data sorting sub-result 402c, and the data merging task 402e refers to the task of merging the data sorting sub-result 403c and the data sorting sub-result 404c.
[0138] If the data storage capacity is greater than or equal to the fourth total number of to-be-sorted data associated with the data sorting sub-result pair, the task control component 40a may distribute the data merging task 401e to one or both of the two data bitonic sorting components associated with the data sorting sub-result pair 401f, that is, Figure 4 The bitonic sorting component S1 and the bitonic sorting component S2 shown in the embodiment of the present application describe that the task control component 40a distributes the data merging task 401e to one of the two data bitonic sorting components associated with the data sorting sub-result pair 401f (such as Figure 5 If the data storage capacity is less than the fourth total number of to-be-sorted data associated with the data sorting sub-result pair, please refer to the following Figure 8 The description in the corresponding embodiment will not be expanded here.
[0139] Please combine Figure 4 as well as Figure 5 In the data bitonic sorting component S1, the data sorting sub-result pair 401f is a bitonic sequence, which includes (5, 9, 10, 20, 14, 12, 8, 3), so the fourth total number is equal to 8, with three sorting rounds, and the sorting round logic parameters of each sorting round are as follows: Figure 5 As shown, in the first round of sorting, the first round of sorting logic parameters include index identifier 0000 and index identifier 0100 are compared in ascending order, index identifier 0001 and index identifier 0101 are compared in ascending order, index identifier 0010 and index identifier 0110 are compared in ascending order, and index identifier 0011 and index identifier 0111 are compared in ascending order. The output result of the first round of sorting is that index identifier 0000 is updated to data to be sorted 5, index identifier 0100 is updated to data to be sorted 14, index identifier 0001 is updated to data to be sorted 9, index identifier 0101 is updated to data to be sorted 12, and index identifier 0010 is updated to data to be sorted 13. 010 is updated to data 8 to be sorted, index identifier 0110 is updated to data 10 to be sorted, index identifier 0011 is updated to data 3 to be sorted, and index identifier 0111 is updated to data 20 to be sorted; at this time, the business server caches the output results of the first sorting round (i.e., 5, 9, 8, 3, 14, 12, 10, 20) in the data acquisition subcomponent of the data bit-tuning sorting component S1, and does not need to be written to the storage component 401d. Therefore, when the second sorting round is started, the data bit-tuning sorting component S1 does not need to obtain the output results of the previous sorting round from the storage component 401d, and can directly obtain them from the local data acquisition subcomponent.
[0140] Please see again Figure 5The second round of sorting logic parameters include the ascending comparison of index identifier 0000 and index identifier 0010, the ascending comparison of index identifier 0001 and index identifier 0011, the ascending comparison of index identifier 0100 and index identifier 0110, and the ascending comparison of index identifier 0101 and index identifier 0111. The output result of the second round of sorting is that index identifier 0000 is updated to the data to be sorted 5, index identifier 0010 is updated to the data to be sorted 8, index identifier 0001 is updated to the data to be sorted 3, index identifier 0011 is updated to the data to be sorted 9, and index identifier 0100 is updated to the data to be sorted. The data to be sorted is 10, the index identifier 0110 is updated to the data to be sorted 14, the index identifier 0101 is updated to the data to be sorted 12, and the index identifier 0111 is updated to the data to be sorted 20; at this time, the business server caches the output results of the second sorting round (i.e., 5, 3, 8, 9, 10, 12, 14, 20) in the data acquisition subcomponent of the data bit-tuned sorting component S1, and does not need to be written to the storage component 401d. Therefore, when the third sorting round is started, the data bit-tuned sorting component S1 does not need to obtain the output results of the previous sorting round from the storage component 401d, and can directly obtain them from the local data acquisition subcomponent.
[0141] Please see again Figure 5 The third round of sorting logic parameters include index identifier 0000 and index identifier 0001 in ascending order, index identifier 0010 and index identifier 0011 in ascending order, index identifier 0100 and index identifier 0101 in ascending order, and index identifier 0110 and index identifier 0111 in ascending order. The output result of the third round of sorting is that index identifier 0000 is updated to the data to be sorted 3, index identifier 0001 is updated to the data to be sorted 5, index identifier 0010 is updated to the data to be sorted 8, and index identifier 0101 is updated to the data to be sorted 9. Index identifier 0011 is updated to data 9 to be sorted, index identifier 0100 is updated to data 10 to be sorted, index identifier 0101 is updated to data 12 to be sorted, index identifier 0110 is updated to data 14 to be sorted, and index identifier 0111 is updated to data 20 to be sorted; at this time, the business server uses the output result of the third round of sorting (i.e., 3, 5, 8, 9, 10, 12, 14, 20) as the target merged data sorting sub-result 401g, and writes the target merged data sorting sub-result 401g to the storage component 401d.
[0142] It can be understood that the process of the data bit-adjusting sorting component S3 executing the data merging task 402e is consistent with the process described above, the difference is that the data bit-adjusting sorting component S1 executes ascending sorting, and the data bit-adjusting sorting component S3 executes descending sorting, so it will not be repeated, and the target merged data sorting sub-result 402g can be obtained.
[0143] After one merge, two target merged data sorting sub-results are obtained, so the task control component 40a distributes another data merge task to the data bitune sorting component according to the two target merged data sorting sub-results until all data sorting sub-results are merged to obtain a set of data sorting results.
[0144] The C data bitonic sorting components all include an index generation subcomponent, which is used to assign an index value to each to-be-sorted data in the to-be-sorted data sequence; the data sorting request carries a write address; the index values corresponding to each to-be-sorted data in the data sorting result are obtained in sequence, and an index value sequence with the same length as the to-be-sorted data sequence is generated according to the obtained index values; a data write request for the index value sequence is sent to the data read-write component, and the index value sequence is written to the write address through the data read-write component.
[0145] The return form of the data sorting result can be to rearrange the data to be sorted in the original sequence in the order from small to large or from large to small and then return it; optionally, keep the original sequence data unchanged, and return the index value (index) of each data to be sorted in the sorted sequence (i.e., the data sorting result) in the data sequence to be sorted, thereby forming an index value sequence with the same length as the data sequence to be sorted.
[0146] Through architectural parallelism and process optimization, the embodiment of the present application proposes a computing architecture for hardware-accelerated sorting, which implements multiple sorting operations per clock cycle (clock), thereby improving computing efficiency compared to a central processing unit. Architectural parallelism refers to increasing the parallelism of the sorting unit (i.e., the data bitune sorting component) and using multiple sorting units. In this case, the local memory of the sorting accelerator can provide multiple read and write ports that can be accessed simultaneously; process optimization refers to modifying the execution order of the algorithm, merging separate calculation processes, combining the calculation pipeline, and reducing the number of memory read-compare-memory write rounds to achieve acceleration. By using process optimization and architectural parallelism, the sorting speed can be increased, and the sorting time T can be reduced from formula (1),
[0147]
[0148] Reduced to formula (2),
[0149]
[0150] In formula (1) and formula (2), T is the execution time, n is the number of items to be sorted, N = ceil (log2 (n)), C and P S is the speedup ratio achieved by architecture parallelism and process optimization, and ceil() is the upward rounding function. S The acceleration method and execution architecture of PS >=2,C>=2,when P S =2, C = 2, the performance can be improved by nearly 4 times. In implementation, the number of sort units commonly used can be 2 / 4 / 8, etc.
[0151] In an embodiment of the present application, based on C data bitonic sorting components, a computer device can generate multiple data bitonic sorting tasks for a data sequence to be sorted, and by running C data bitonic sorting components in parallel, multiple data bitonic sorting tasks can be executed simultaneously. By executing multiple data bitonic sorting tasks simultaneously, the sorting time of the data sequence to be sorted, that is, the task execution time, can be reduced; subsequently, based on C data bitonic sorting components, the computer device merges B data sorting sub-results to obtain a data sorting result for the data sequence to be sorted. It can be seen that the embodiment of the present application can not only reduce the execution time of the sorting task for the data sequence to be sorted, but also improve the sorting efficiency of the sorting task by running C data bitonic sorting components in parallel.
[0152] See also Figure 6 , Figure 6 is a flow chart of a data processing method provided by an embodiment of the present application. The method can be performed by a business server (for example, the above Figure 1 The service server 100 shown in the figure may also be executed by a terminal device (for example, the above Figure 1 The terminal device 200a) shown in the figure can also be executed by the service server and the terminal device interactively. For ease of understanding, the embodiment of the present application takes the method executed by the service server as an example for explanation. Figure 6 As shown, the process of the data processing method includes the following steps S201-step S203, and steps S201-step S203 are Figure 2 Another specific embodiment of step S103 in the corresponding embodiment.
[0153] Step S201: In the data bitwise sorting component F e In the data bit-tuning sorting task D e The corresponding data subsequence A to be sorted e .
[0154] Specifically, the data sorting request carries the read addresses corresponding to the data to be sorted in the data sequence to be sorted, and the sorting order for the data sequence to be sorted; in the data bitune sorting component F e In the data bit-tuning sorting task D e The target read address and the second total number carried; the read addresses corresponding to the data to be sorted in the data sequence to be sorted include the target read address; the second total number is used to represent the data subsequence A to be sorted eThe total number of data to be sorted in the data to be sorted; sending a data acquisition request carrying a target read address to the data read-write component, and obtaining an initial subsequence of data to be sorted from the target read address through the data read-write component; the initial subsequence of data to be sorted belongs to the data sequence to be sorted; determining the quantity difference between the second total quantity and the third total quantity; the third total quantity is used to characterize the total number of data to be sorted in the initial subsequence of data to be sorted; obtaining supplementary data to be sorted according to the quantity difference and the sorting order, and adding the supplementary data to be sorted to the initial subsequence of data to be sorted to obtain a subsequence of data to be sorted A. e ; Subsequence A of the data to be sorted e Load into the data bitune sorting component F e The corresponding storage component.
[0155] Please also see Figure 7 , Figure 7 is a schematic diagram of the structure of a sorting accelerator provided in an embodiment of the present application. Figure 7 As shown, the sorting accelerator includes a task control component, C data bit-tuning sorting components, storage components corresponding to the C data bit-tuning sorting components, a data selection component, and a data reading and writing component.
[0156] Among them, the data read and write component is used to obtain a data transfer request and implement data transfer according to the data transfer request. The data transfer request can be a data acquisition request. According to the target read address in the data acquisition request, the data read and write component obtains the initial data subsequence to be sorted; the data transfer request can be a data write request. According to the data write request, the data read and write component writes the index value sequence to the write address.
[0157] The data selection component is used to allocate the read and write channels of the local memory to the data read and write components or multiple data bitune sorting components as needed.
[0158] A storage component is used to write data and read data. The written data may include the data subsequence to be sorted written from the external storage component through the data read-write component, the intermediate sorting subresults returned by the write data bitonic sorting component, and the data sorting subresults, etc.; the read data may include the data subsequence to be sorted, the intermediate sorting subresults, the data sorting subresults, etc.
[0159] A data bitonic sorting component is used to bitonically sort the data subsequence to be sorted to obtain data sorting subresults; and is also used to merge the data sorting subresults to obtain the data sorting result for the data sequence to be sorted. The data bitonic sorting component can also be called a sorting unit.
[0160] The task control component is used to obtain a data sorting request, start B data bit-tuning sorting tasks according to the data sorting request, and distribute one data bit-tuning sorting task to a data bit-tuning sorting component; it is also used to obtain the sorting completion signal returned by each data bit-tuning sorting component, start multiple data merging tasks based on the B sorting completion signals, and distribute one data merging task to the data bit-tuning sorting component associated with the data sorting sub-result; it is also used to obtain the merging completion signal and send the merging completion signal to the top-level control component; wherein the merging completion signal is used to indicate that the data to be sorted in the data sequence to be sorted has been sorted. The task control component can also be called a control unit.
[0161] The data reading and writing component may be Direct Memory Access (DMA), which can load the data to be sorted from the external memory (i.e., external storage component) outside the sorting (SORT) accelerator to the internal memory (i.e., storage component) of the sorting accelerator, and write (store) the specified data in the internal memory of the sorting accelerator to the memory outside the sorting accelerator (i.e., external storage component). DMA can be executed in the form of an instruction queue or in a polling manner, which is not limited in the embodiments of the present application.
[0162] The data selection component (multiplexer, MUX) is an internal selector used by the sort accelerator. The business server allocates the read and write channels of the memory to DMA or multiple sort components as needed. The local memory supports C read channels and C write channels, which can be composed of multiple physical storage units to achieve simultaneous reading and writing of multiple channels. There is no limitation here.
[0163] The storage component corresponding to the data bitonic sorting component refers to the local memory of the sorting accelerator. The data to be sorted in the data sequence to be sorted is stored in the local memory or external memory of the sorting accelerator. If it is stored in the external memory, the business server reads the data to be sorted in the external memory to the local memory through the data reading component to perform bitonic sorting on the data to be sorted.
[0164] It can be understood that if the sorting order is from small to large, the supplementary data to be sorted will give priority to positive infinity, and the supplementary data to be sorted will be added to the high-order address side of the data to be sorted of the initial subsequence of data to be sorted, that is, positive infinity will be placed behind the data to be sorted; if the sorting order is from large to small, the supplementary data to be sorted will give priority to negative infinity, and the supplementary data to be sorted will be added to the high-order address side of the data to be sorted of the initial subsequence of data to be sorted, that is, negative infinity will be placed behind the data to be sorted.
[0165] Step S202: the data subsequence A to be sortede Perform bitonic sorting to obtain the data subsequence A to be sorted e The corresponding data sorting sub-result.
[0166] Specifically, the data bitune sorting component F e It includes a sorting control subcomponent, a round control subcomponent, a data acquisition subcomponent, and a bitonic sorting subcomponent; when the sorting control subcomponent obtains the data loading completion notification, the round control subcomponent is started; the data loading completion notification is used to represent the data bitonic sorting component F e The corresponding storage component has loaded the data subsequence A to be sorted e In the round control subcomponent, according to the second total number, determine the data subsequence A to be sorted e The corresponding bitonic sorting stage and the bitonic sorting logic parameters corresponding to the bitonic sorting stage are sent to the data acquisition subcomponent; the second total number is used to represent the subsequence A of the data to be sorted e The total number of data to be sorted in the bitonic sorting subcomponent; in the data acquisition subcomponent, according to the bitonic sorting stage and the bitonic sorting logic parameters corresponding to the bitonic sorting stage, the intermediate data to be sorted is obtained, and the intermediate data to be sorted is sent to the bitonic sorting subcomponent. The intermediate data to be sorted is sorted through the bitonic sorting subcomponent to obtain the intermediate sorting subresult, and the subsequence A of the data to be sorted is determined according to the intermediate sorting subresult e The corresponding data sorting sub-result; the data sub-sequence A to be sorted e Including intermediate data to be sorted.
[0167] Among them, determine the data subsequence A to be sorted eThe specific process of the corresponding data sorting sub-result may include: the number of stages of the bitonic sorting stage is at least two; in the data acquisition sub-component, according to the i-th bitonic sorting stage in at least two bitonic sorting stages, the first data sorting transition sub-result is read from the storage component; i is a positive integer greater than 1, and the first data sorting transition sub-result is the output result in the i-1-th bitonic sorting stage; the bitonic sorting logic parameters corresponding to the i-th bitonic sorting stage include n-round sorting round logic parameters; n is determined based on i, and n is a positive integer greater than 1; according to the j-th round sorting round logic parameters, the first data sorting transition sub-result is read from the intermediate data sorting transition sub-result The intermediate data to be sorted is obtained from n; j is a positive integer and j is less than or equal to n; if j is 1, the intermediate data sorting transition sub-result is the first data sorting transition sub-result; if j is not 1, the intermediate data sorting transition sub-result is the second data sorting transition sub-result cached in the data acquisition sub-component, and the second data sorting transition sub-result is determined based on the j-1th round of sorting round logic parameters; the intermediate data to be sorted is sent to the bitonic sorting sub-component, and the intermediate data to be sorted is sorted by the bitonic sorting sub-component to obtain the intermediate sorting sub-result of the intermediate data to be sorted, and the data subsequence A to be sorted is determined according to the intermediate sorting sub-result e The corresponding third data sorting transition sub-result; if j is less than n, the third data sorting transition sub-result is cached in the data acquisition sub-component; if j=n, and the i-th bitonic sorting stage is the last bitonic sorting stage of at least two bitonic sorting stages, the third data sorting transition sub-result is written to the storage component, and the third data sorting transition sub-result is determined as the data subsequence A to be sorted e The corresponding data sorting sub-result.
[0168] Please see again Figure 7 ,Multiple data bit-tuned sorting components, task control components, and data reading and writing components can form a sorting accelerator. It can be understood that the basic structure of each data bit-tuned sorting component is the same. Figure 7 In the figure, the basic structure of each data bitonic sorting component is illustrated by the data bitonic sorting component 1. The data bitonic sorting component 1 may include a sorting control subcomponent, a round control subcomponent, an index generation subcomponent, a data acquisition subcomponent, and a bitonic sorting subcomponent. In the bitonic sorting scenario, the specific implementation functions of these subcomponents can be found in the above S202.
[0169] Among them, the sorting control subcomponent is used to obtain the data bit-tuning sorting task distributed by the task control component, and start the round control component based on the data bit-tuning sorting task; it is also used to obtain the completion signal (including the sorting completion signal and the merging completion signal) sent by the round control subcomponent, and send the merging completion signal to the task control component. As can be seen from the above, the sorting control subcomponent realizes the signal interaction between the data bit-tuning sorting component and the task control component, and starts the round control subcomponent after receiving the enable signal. The sorting control subcomponent can be called a sorting control module.
[0170] The round control subcomponent is used to determine the bitonic sorting stage and the bitonic sorting logic parameters corresponding to the bitonic sorting stage according to the subsequence of the data to be sorted, and send corresponding enable signals to the index generation subcomponent, the data acquisition subcomponent, and the bitonic sorting subcomponent. The round control subcomponent can be called a round control module.
[0171] The index generation subcomponent is used to read the data to be sorted for the first time, generate a unique index identifier for each group of data, and write it into the data buffer. The index generation subcomponent can be called an index generation module.
[0172] The data acquisition subcomponent is used to implement a round of traversal of the data to be sorted in the subsequence of data to be sorted. The data acquisition subcomponent can be called a single-round data control and memory access address calculation module.
[0173] The bitonic sorting subcomponent is used to realize fast data comparison for the computing subcomponent in the bitonic sorting process. The bitonic sorting subcomponent can be called a comparison module.
[0174] The embodiment of the present application can adopt a process optimization method to merge the operation process of the same data in two or more rounds, so as to realize one-time reading and one-time writing of multiple local data, instead of multiple reading and writing processes, thus reducing the overall reading and writing rounds of data and achieving acceleration effect. For details, please refer to the above Figure 3 The description of step S103 in the corresponding embodiment.
[0175] Step S203, running the data bit-adjustment sorting component F e At the same time, B-1 data bitonic sorting tasks are run in parallel through C-1 data bitonic sorting components to obtain data sorting sub-results output by C-1 data bitonic sorting components respectively; C-1 data bitonic sorting components refer to the C data bitonic sorting components except for the data bitonic sorting component F e The data bit-tuning sorting components other than the data bit-tuning sorting task D are B-1 data bit-tuning sorting tasks. e Other data bitonic sorting tasks.
[0176] The embodiment of the present application can reduce the execution time of double-tuning sorting through architecture parallelism and process optimization. Figure 3 In the corresponding embodiment, it can be seen that the top-level control unit issues a sorting instruction, and after the sorting accelerator receives the instruction, it reads the data to be sorted from the external memory to the local memory, calls multiple data bitune sorting components to complete the sorting, writes the data sorting results from the local memory back to the external memory, and sends a response to the top-level control unit indicating that the instruction is completed.
[0177] In an embodiment of the present application, based on C data bitonic sorting components, a computer device can generate multiple data bitonic sorting tasks for a data sequence to be sorted, and by running C data bitonic sorting components in parallel, multiple data bitonic sorting tasks can be executed simultaneously. By executing multiple data bitonic sorting tasks simultaneously, the sorting time of the data sequence to be sorted, that is, the task execution time, can be reduced; subsequently, based on C data bitonic sorting components, the computer device merges B data sorting sub-results to obtain a data sorting result for the data sequence to be sorted. It can be seen that the embodiment of the present application can not only reduce the execution time of the sorting task for the data sequence to be sorted, but also improve the sorting efficiency of the sorting task by running C data bitonic sorting components in parallel.
[0178] See also Figure 8 , Figure 8 is a flow chart of a data processing method provided by an embodiment of the present application. The method can be performed by a business server (for example, the above Figure 1 The service server 100 shown in the figure may also be executed by a terminal device (for example, the above Figure 1 The terminal device 200a) shown in the figure can also be executed by the service server and the terminal device interactively. For ease of understanding, the embodiment of the present application takes the method executed by the service server as an example for explanation. Figure 8 As shown, the process of the data processing method includes the following steps S301-step S304, and steps S301-step S304 are Figure 2 Another specific embodiment of step S104 in the corresponding embodiment.
[0179] Step S301, determine the data sorting sub-result pair M p The fourth total number of the associated data to be sorted determines the data sorting sub-result pair M p The data storage capacity corresponding to the associated data bitonic sorting component; C data bitonic sorting components include data sorting sub-result pairs M p The associated data bitonic sorting component.
[0180] Step S302: Compare the fourth total quantity with the data storage capacity to obtain a second comparison result.
[0181] For the specific process of step S301-step S302, please refer to the above Figure 2 The description of step S104 in the corresponding embodiment is not repeated here.
[0182] Step S303: sort M according to the second comparison result and the data sorting sub-result. p The associated data bit-tuning sorting component sorts the data sub-results to M p Perform bitwise sorting to obtain the data sorting result pair M p The corresponding target merged data sorting sub-result.
[0183] Specifically, the data sorting sub-results are M p The method comprises a first data sorting sub-result and a second data sorting sub-result; if the second comparison result is that the fourth total number is greater than the data storage capacity, the first data sorting sub-result is evenly divided into a first ordered data segment and a second ordered data segment, and the second data sorting sub-result is evenly divided into a third ordered data segment and a fourth ordered data segment; if the data in the first ordered data segment is less than or equal to the data in the second ordered data segment, and the data in the third ordered data segment is less than or equal to the data in the fourth ordered data segment, then based on the first data bitonic sorting component, the first ordered data segment and the third ordered data segment are bitonically sorted to obtain a third data sorting sub-result; the data sorting sub-results are compared with M p The associated data bitonic sorting component comprises a first data bitonic sorting component; based on the second data bitonic sorting component, the second ordered data segment and the fourth ordered data segment are bitonic sorted to obtain a fourth data sorting sub-result; the data sorting sub-result is used to sort the M p The associated data bitonal sorting component includes a second data bitonal sorting component; the third data sorting sub-result is evenly divided into a fifth ordered data segment and a sixth ordered data segment, and the fourth data sorting sub-result is evenly divided into a seventh ordered data segment and an eighth ordered data segment; if the data in the fifth ordered data segment is less than or equal to the data in the sixth ordered data segment, and the data in the seventh ordered data segment is less than or equal to the data in the eighth ordered data segment, then based on the target data bitonal sorting component, the sixth ordered data segment and the seventh ordered data segment are bitonally sorted to obtain a fifth data sorting sub-result; the target data bitonal sorting component is the first data bitonal sorting component or the second data bitonal sorting component; based on the fifth ordered data segment, the fifth data sorting sub-result and the eighth ordered data sorting, the data sorting sub-result pair M is obtained. p The corresponding target merged data sorting sub-result.
[0184] Please also see Fig. 9 , Fig. 9Schematic diagram of a data processing scenario provided by an embodiment of the present application. Fig. 9 As shown, the business server divides the two ordered sequences into two parts respectively, and obtains four ordered sequences, that is, the ordered sequence S0-1 (equivalent to the first data sorting sub-result) is divided into the first ordered data segment and the second ordered data segment, and the ordered sequence S0-2 (equivalent to the second data sorting sub-result) is divided into the third ordered data segment and the fourth ordered data segment; based on the first data bitonic sorting component, the first ordered data segment and the third ordered data segment are bitonically sorted to obtain the third data sorting sub-result (equivalent to Fig. 9 Based on the second data bitonic sorting component, the second ordered data segment and the fourth ordered data segment are bitonically sorted to obtain a fourth data sorting sub-result (equivalent to Fig. 9 The third data sorting sub-result is divided equally into the fifth ordered data segment and the sixth ordered data segment, and the fourth data sorting sub-result is divided equally into the seventh ordered data segment and the eighth ordered data segment. Based on the target data bitonic sorting component, the sixth ordered data segment and the seventh ordered data segment are bitonic sorted to obtain the fifth data sorting sub-result (equivalent to Fig. 9 The fifth ordered data fragment is used as the ordered sequence S2-1, the eighth ordered data fragment is used as the ordered sequence S2-3, and the three ordered sequences S2-1, S2-2, and S2-3 from small to large are concatenated through a concat operation to obtain the final output.
[0185] Step S304: The target merged data sorting sub-results corresponding to the U data sorting sub-results are determined as U target merged data sorting sub-results.
[0186] The specific implementation process of step S304 is shown in the above Figure 2 The corresponding step S104 in the embodiment will not be described in detail here.
[0187] In an embodiment of the present application, based on C data bitonic sorting components, a computer device can generate multiple data bitonic sorting tasks for a data sequence to be sorted, and by running C data bitonic sorting components in parallel, multiple data bitonic sorting tasks can be executed simultaneously. By executing multiple data bitonic sorting tasks simultaneously, the sorting time of the data sequence to be sorted, that is, the task execution time, can be reduced; subsequently, based on C data bitonic sorting components, the computer device merges B data sorting sub-results to obtain a data sorting result for the data sequence to be sorted. It can be seen that the embodiment of the present application can not only reduce the execution time of the sorting task for the data sequence to be sorted, but also improve the sorting efficiency of the sorting task by running C data bitonic sorting components in parallel.
[0188] For further information, see Fig.10 , Fig.10 : is a schematic diagram of the structure of a data processing device provided in an embodiment of the present application. The above-mentioned data processing device may be a computer program (including program code) running in a computer device, for example, the data processing device is an application software; the device may be used to execute the corresponding steps in the method provided in an embodiment of the present application. Fig.10 As shown, the data processing device 1 may include: a calling component module 11 , a starting task module 12 , a parallel component module 13 and a merging result module 14 .
[0189] The calling component module 11 is used to obtain a data sorting request for the data sequence to be sorted, and call C data bitonic sorting components according to the data sorting request; C is a positive integer greater than 1;
[0190] The task start module 12 is used to start B data bitonic sorting tasks according to the data sequence to be sorted and C data bitonic sorting components; B is a positive integer greater than 1; the B data bitonic sorting tasks are respectively associated with different data subsequences to be sorted, and the B data subsequences to be sorted are all generated based on the data sequence to be sorted;
[0191] The parallel component module 13 is used to run C data bitonic sorting components in parallel according to B data bitonic sorting tasks, and obtain B data sorting sub-results output by the C data bitonic sorting components; a data sorting sub-result is used to represent the sorting result of the to-be-sorted data subsequence associated with a data bitonic sorting task;
[0192] The merging result module 14 is used to merge B data sorting sub-results based on C data bitonic sorting components to obtain a data sorting result for the data sequence to be sorted.
[0193] The specific functional implementation of the calling component module 11, the starting task module 12, the parallel component module 13 and the merging result module 14 can be found in the above Figure 2 Steps S101 to S104 in the corresponding embodiment will not be described in detail here.
[0194] See also Fig.10 , calling component module 11, specifically used to obtain a data sorting request, and send the data sorting request to the task control component; the data sorting request carries a first total number of data to be sorted in the sequence of data to be sorted;
[0195] The task start module 12 may include: a first determining unit 121 , a first acquiring unit 122 and a task start unit 123 .
[0196] A first determining unit 121 is used to determine, in the task control component, a number to be sorted associated with the first total number; the number to be sorted is equal to or greater than the first total number;
[0197] The first acquisition unit 122 is used to acquire the data storage capacities corresponding to the C data bitonic sorting components, and compare the number to be sorted with the total data storage capacity to obtain a first comparison result; the total data storage capacity is equal to the sum of the C data storage capacities;
[0198] The task starting unit 123 is used to start B data bitonal sorting tasks according to the first comparison result.
[0199] The specific functional implementation of the first determining unit 121, the first obtaining unit 122 and the starting task unit 123 can be referred to above. Figure 2 The steps S101 and S102 in the corresponding embodiment will not be described in detail here.
[0200] See also Fig.10 The opening task unit 123 may include: a first opening subunit 1231 and a second opening subunit 1232 .
[0201] The first opening subunit 1231 is used to obtain a first quantity ratio between the number to be sorted and the total data storage capacity if the first comparison result is that the number to be sorted is greater than the total data storage capacity;
[0202] The first opening subunit 1231 is further used to open B data bitonic sorting tasks according to the first quantity ratio; the product of C and the first quantity ratio is equal to B; the second total quantity carried by a data bitonic sorting task is equal to the data storage capacity; the second total quantity is used to represent the total quantity of the to-be-sorted data in the to-be-sorted data subsequence associated with a data bitonic sorting task;
[0203] The second opening subunit 1232 is used for the second opening subunit, and is used to open B data bitonal sorting tasks according to C data bitonal sorting components if the first comparison result is that the number to be sorted is less than or equal to the total data storage capacity; B is equal to C.
[0204] The specific functional implementation of the first opening subunit 1231 and the second opening subunit 1232 can be referred to above. Figure 2 The step S102 in the corresponding embodiment will not be described in detail here.
[0205] See also Fig.10 The parallel component module 13 may include: a first distribution unit 131 and a first parallel unit 132 .
[0206] The first distribution unit 131 is used for distributing B data bitonic sorting tasks to C data bitonic sorting components if B is equal to C; wherein one data bitonic sorting task is distributed to one data bitonic sorting component;
[0207] The first parallel unit 132 is used to run B data bitonic sorting tasks in parallel through C data bitonic sorting components to obtain data sorting sub-results outputted by the C data bitonic sorting components respectively.
[0208] The specific functional implementation of the first distribution unit 131 and the first parallel unit 132 can be found in the above Figure 2 The step S103 in the corresponding embodiment will not be described in detail here.
[0209] See also Fig.10 , B data bit-tuning sorting tasks include data bit-tuning sorting task D e , e is a positive integer and e is less than or equal to B; B data subsequences to be sorted include data bitonic sorting task D e The corresponding data subsequence A to be sorted e ; C data bit-tuning sorting components include a component for performing data bit-tuning sorting tasks D e The data bitonic sorting component F e ;
[0210] The first parallel unit 132 may include a first generating subunit 1321 and a second generating subunit 1322 .
[0211] The first generating subunit 1321 is used to generate a bitwise sorting component F in the data e In the data bit-tuning sorting task D e The corresponding data subsequence A to be sorted e , treat the sorted data subsequence A e Perform bitonic sorting to obtain the data subsequence A to be sorted e The corresponding data sorting sub-result;
[0212] The second generating subunit 1322 is used to run the data bitune sorting component F e At the same time, B-1 data bitonic sorting tasks are run in parallel through C-1 data bitonic sorting components to obtain data sorting sub-results output by C-1 data bitonic sorting components respectively; C-1 data bitonic sorting components refer to the C data bitonic sorting components except for the data bitonic sorting component F e The data bit-tuning sorting components other than the data bit-tuning sorting task D are B-1 data bit-tuning sorting tasks. e Other data bitonic sorting tasks.
[0213] The specific functional implementation of the first generation subunit 1321 and the second generation subunit 1322 can be found in the above Figure 6 Steps S201 to S203 in the corresponding embodiment are not described in detail here.
[0214] See also Fig.10 , the data sorting request carries the read addresses corresponding to the data to be sorted in the data sequence to be sorted, and the sorting order for the data sequence to be sorted;
[0215] The first generating subunit 1321 may include: a first acquiring subunit 13211 , a second acquiring subunit 13212 , and a first determining subunit 13213 .
[0216] The first acquisition subunit 13211 is used to sort the data in the data bitune sorting component F e In the data bit-tuning sorting task D e The target read address and the second total number carried; the read addresses corresponding to the data to be sorted in the data sequence to be sorted include the target read address; the second total number is used to represent the data subsequence A to be sorted e The total number of data to be sorted in ;
[0217] The second acquisition subunit 13212 is used to send a data acquisition request carrying a target read address to the data read-write component, and acquire an initial subsequence of data to be sorted from the target read address through the data read-write component; the initial subsequence of data to be sorted belongs to the data sequence to be sorted;
[0218] The first determining subunit 13213 is used to determine the difference between the second total number and the third total number; the third total number is used to represent the total number of data to be sorted in the initial subsequence of data to be sorted;
[0219] The first determining subunit 13213 is further used to obtain the supplementary data to be sorted according to the quantity difference and the sorting order, and add the supplementary data to be sorted to the initial subsequence of data to be sorted to obtain the subsequence A of data to be sorted. e ;
[0220] The first determining subunit 13213 is further configured to: e Load into the data bitune sorting component F e The corresponding storage component.
[0221] The specific functional implementation of the first acquisition subunit 13211, the second acquisition subunit 13212 and the first determination subunit 13213 can be referred to above. Figure 6 Step S201 in the embodiment will not be described in detail here.
[0222] See also Fig.10 , data bitune sorting component F e It includes a sorting control subcomponent, a round control subcomponent, a data acquisition subcomponent, and a bitune sorting subcomponent;
[0223] The first generating subunit 1321 may include: a second determining subunit 13214 and a third determining subunit 13215 .
[0224] The second determining subunit 13214 is used to start the round control subcomponent when the sorting control subcomponent obtains the data loading completion notification; the data loading completion notification is used to indicate that the data bit-tuning sorting component F e The corresponding storage component has loaded the data subsequence A to be sorted e ;
[0225] The second determining subunit 13214 is further configured to determine, in the round control subcomponent, the subsequence A of the data to be sorted according to the second total number. e The corresponding bitonic sorting stage and the bitonic sorting logic parameters corresponding to the bitonic sorting stage are sent to the data acquisition subcomponent; the second total number is used to represent the subsequence A of the data to be sorted e The total number of data to be sorted in ;
[0226] The third determination subunit 13215 is used to obtain the intermediate data to be sorted according to the bitonic sorting stage and the bitonic sorting logic parameters corresponding to the bitonic sorting stage in the data acquisition subcomponent, send the intermediate data to be sorted to the bitonic sorting subcomponent, sort the intermediate data to be sorted through the bitonic sorting subcomponent, obtain the intermediate sorting subresult, and determine the data subsequence A to be sorted according to the intermediate sorting subresult. e The corresponding data sorting sub-result; the data sub-sequence A to be sorted e Including intermediate data to be sorted.
[0227] The specific functional implementation of the second determining subunit 13214 and the third determining subunit 13215 can be found in the above Figure 6 The step S202 in the corresponding embodiment will not be described in detail here.
[0228] See also Fig.10 , the number of stages in the bitonic sorting phase is at least two;
[0229] The third determining subunit 13215 is specifically configured to read a first data sorting transition subresult from a storage component in a data acquisition subcomponent according to an i-th bitonic sorting stage in at least two bitonic sorting stages; i is a positive integer greater than 1, and the first data sorting transition subresult is an output result in an i-1-th bitonic sorting stage; the bitonic sorting logic parameters corresponding to the i-th bitonic sorting stage include n-round sorting round logic parameters; n is determined based on i, and n is a positive integer greater than 1;
[0230] The third determination subunit 13215 is further specifically used to obtain the intermediate data to be sorted from the intermediate data sorting transition subresult according to the j-th sorting round logic parameter; j is a positive integer and j is less than or equal to n; if j is 1, the intermediate data sorting transition subresult is the first data sorting transition subresult; if j is not 1, the intermediate data sorting transition subresult is the second data sorting transition subresult cached in the data acquisition subcomponent, and the second data sorting transition subresult is determined based on the j-1-th sorting round logic parameter;
[0231] The third determining subunit 13215 is further specifically used to send the intermediate data to be sorted to the bitonic sorting subcomponent, sort the intermediate data to be sorted through the bitonic sorting subcomponent, obtain the intermediate sorting subresult of the intermediate data to be sorted, and determine the subsequence A of the data to be sorted according to the intermediate sorting subresult. e The corresponding third data sorting transition sub-result;
[0232] The third determining subunit 13215 is further specifically configured to cache the third data sorting transition sub-result in the data acquiring sub-component if j is less than n;
[0233] The third determining subunit 13215 is further specifically configured to write the third data sorting transition sub-result to the storage component and determine the third data sorting transition sub-result as the data subsequence A to be sorted if j=n and the i-th bitonic sorting stage is the last bitonic sorting stage of at least two bitonic sorting stages. e The corresponding data sorting sub-result.
[0234] The specific functional implementation of the third determining subunit 13215 can refer to the above Figure 6 The step S202 in the corresponding embodiment will not be described in detail here.
[0235] See also Fig.10 The parallel component module 13 may include: a task splitting unit 133 , a second distribution unit 134 and a second parallel unit 135 .
[0236] The task splitting unit 133 is used for splitting B data bitonic sorting tasks into K*C data bitonic sorting tasks if B is greater than C; K is the maximum running rounds corresponding to the C data bitonic sorting components respectively;
[0237] The second distribution unit 134 is used to distribute the C data bitonic sorting tasks in the i-th running round to the C data bitonic sorting components; wherein one data bitonic sorting task is distributed to one data bitonic sorting component; i is a positive integer less than or equal to K;
[0238] The second parallel unit 135 is used to run C data bitonic sorting tasks in the i-th running round in parallel through C data bitonic sorting components, and obtain the data sorting sub-results respectively output by the C data bitonic sorting components, until i is equal to K, and the data sorting sub-results respectively corresponding to the B data bitonic sorting tasks are obtained.
[0239] The specific functional implementation of the split task unit 133, the second distribution unit 134 and the second parallel unit 135 can be found in the above Figure 2 The step S103 in the corresponding embodiment will not be described in detail here.
[0240] See also Fig.10 The merging result module 14 may include: a second acquiring unit 141 , a first merging unit 142 , a second determining unit 143 and a second merging unit 144 .
[0241] The second acquisition unit 141 is used to acquire U data sorting sub-result pairs in the L-th merging; U is a positive integer; L is a positive integer; if L is 1, the U data sorting sub-result pairs are generated based on the B data sorting sub-results; if L is not 1, the U data sorting sub-result pairs are generated based on the historical merging data sorting sub-results, and the historical merging data sorting sub-results are the sub-results obtained in the L-1-th merging;
[0242] A first merging unit 142 is used to merge U data sorting sub-result pairs based on the C data bitonic sorting components to obtain U target merged data sorting sub-results;
[0243] The second determining unit 143 is configured to determine the U target merged data sorting sub-results as the data sorting result corresponding to the data sequence to be sorted if U is equal to 1;
[0244] The second merging unit 144 is used to merge U target merged data sorting sub-results based on C data bitonic sorting components in the L+1th merge if U is greater than 1, to obtain a data sorting result for the data sequence to be sorted.
[0245] The specific functional implementation of the second acquisition unit 141, the first merging unit 142, the second determination unit 143 and the second merging unit 144 can refer to the above Figure 2 The step S104 in the corresponding embodiment will not be described in detail here.
[0246] See also Fig.10 , U data sorting sub-result pairs include data sorting sub-result pairs M p , p is a positive integer, and p is less than or equal to U;
[0247] The first merging unit 142 may include: a first quantity subunit 1421 , a second quantity subunit 1422 , a third generation subunit 1423 , and a fourth generation subunit 1424 .
[0248] The first quantity subunit 1421 is used to determine the data sorting sub-result pair M. p The fourth total number of the associated data to be sorted determines the data sorting sub-result pair M p The data storage capacity corresponding to the associated data bitonic sorting component; C data bitonic sorting components include data sorting sub-result pairs M p The associated data bitonic sorting component;
[0249] The second quantity subunit 1422 is used to compare the fourth total quantity with the data storage capacity to obtain a second comparison result;
[0250] The third generating subunit 1423 is used to generate M according to the second comparison result and the data sorting sub-result. p The associated data bit-tuning sorting component sorts the data sub-results to M p Perform bitwise sorting to obtain the data sorting result pair M p The corresponding target merged data sorting sub-results;
[0251] The fourth generating subunit 1424 is used to determine the target merged data sorting sub-results corresponding to the U data sorting sub-results as U target merged data sorting sub-results.
[0252] The specific functional implementation of the first quantity subunit 1421, the second quantity subunit 1422, the third generation subunit 1423 and the fourth generation subunit 1424 can be referred to above. Figure 8 Steps S301 to S304 in the corresponding embodiment are not described in detail here.
[0253] See also Fig.10 , data sorting sub-result pair M p including a first data sorting sub-result and a second data sorting sub-result;
[0254] The third generating subunit 1423 may include: a first averaging subunit 14231 , a first sorting subunit 14232 , a second sorting subunit 14233 , a second averaging subunit 14234 , a third sorting subunit 14235 and a fourth sorting subunit 14236 .
[0255] The first equalizing subunit 14231 is configured to equally divide the first data sorting sub-result into a first ordered data segment and a second ordered data segment, and equally divide the second data sorting sub-result into a third ordered data segment and a fourth ordered data segment if the second comparison result is that the fourth total number is greater than the data storage capacity;
[0256] The first sorting subunit 14232 is configured to, if the data in the first ordered data segment is less than or equal to the data in the second ordered data segment, and the data in the third ordered data segment is less than or equal to the data in the fourth ordered data segment, perform bitonic sorting on the first ordered data segment and the third ordered data segment based on the first data bitonic sorting component to obtain a third data sorting subresult; the data sorting subresult is used for M p The associated data bitonic sorting components include a first data bitonic sorting component;
[0257] The second sorting subunit 14233 is used to perform bitonic sorting on the second ordered data segment and the fourth ordered data segment based on the second data bitonic sorting component to obtain a fourth data sorting sub-result; the data sorting sub-result is used to perform bitonic sorting on the second ordered data segment and the fourth ordered data segment based on the second data bitonic sorting component to obtain a fourth data sorting sub-result; p The associated data bitonic sorting component includes a second data bitonic sorting component;
[0258] The second equalizing subunit 14234 is used to equally divide the third data sorting sub-result into a fifth ordered data segment and a sixth ordered data segment, and equally divide the fourth data sorting sub-result into a seventh ordered data segment and an eighth ordered data segment;
[0259] The third sorting subunit 14235 is configured to, if the data in the fifth ordered data segment is less than or equal to the data in the sixth ordered data segment, and the data in the seventh ordered data segment is less than or equal to the data in the eighth ordered data segment, perform bitonic sorting on the sixth ordered data segment and the seventh ordered data segment based on a target data bitonic sorting component to obtain a fifth data sorting subresult; the target data bitonic sorting component is the first data bitonic sorting component or the second data bitonic sorting component;
[0260] The fourth sorting subunit 14236 is used to obtain a data sorting sub-result pair M based on the fifth ordered data segment, the fifth data sorting sub-result and the eighth ordered data sorting. p The corresponding target merged data sorting sub-result.
[0261] The specific functional implementation of the first averaging subunit 14231, the first sorting subunit 14232, the second sorting subunit 14233, the second averaging subunit 14234, the third sorting subunit 14235 and the fourth sorting subunit 14236 can be referred to above. Figure 8 The step S303 in the corresponding embodiment will not be described in detail here.
[0262] See also Fig.10 , the C data bitonic sorting components all include an index generation subcomponent, and the index generation subcomponent is used to assign an index value to each to-be-sorted data in the to-be-sorted data sequence; the data sorting request carries a write address;
[0263] The data processing device 1 may further include: a sequence generating module 15 and a sequence writing module 16 .
[0264] A sequence generation module 15 is used to sequentially obtain the index values corresponding to each data to be sorted in the data sorting result, and generate an index value sequence with the same length as the sequence of the data to be sorted according to the obtained index values;
[0265] The write sequence module 16 is used to send a data write request for the index value sequence to the data read-write component, and write the index value sequence to the write address through the data read-write component.
[0266] The specific functional implementation of the sequence generation module 15 and the sequence writing module 16 can be found in the above Figure 2 The step S104 in the corresponding embodiment will not be described in detail here.
[0267] In an embodiment of the present application, based on C data bitonic sorting components, a computer device can generate multiple data bitonic sorting tasks for a data sequence to be sorted, and by running C data bitonic sorting components in parallel, multiple data bitonic sorting tasks can be executed simultaneously. By executing multiple data bitonic sorting tasks simultaneously, the sorting time of the data sequence to be sorted, that is, the task execution time, can be reduced; subsequently, based on C data bitonic sorting components, the computer device merges B data sorting sub-results to obtain a data sorting result for the data sequence to be sorted. It can be seen that the embodiment of the present application can not only reduce the execution time of the sorting task for the data sequence to be sorted, but also improve the sorting efficiency of the sorting task by running C data bitonic sorting components in parallel.
[0268] For further information, see Fig.11 , Fig.11 Schematic diagram of the structure of a computer device provided in an embodiment of the present application. Fig.11As shown, the computer device 1000 may include: at least one processor 1001, such as a CPU, at least one network interface 1004, a user interface 1003, a memory 1005, and at least one communication bus 1002. The communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display) and a keyboard (Keyboard), and the network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may optionally also be at least one storage device located away from the aforementioned processor 1001. As shown in FIG. Fig.11 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and a device control application.
[0269] exist Fig.11 In the computer device 1000 shown, the network interface 1004 can provide a network communication function; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the device control application stored in the memory 1005 to achieve:
[0270] Obtain a data sorting request for the data sequence to be sorted, and call C data bitonic sorting components according to the data sorting request; C is a positive integer greater than 1;
[0271] According to the data sequence to be sorted and C data bitonic sorting components, B data bitonic sorting tasks are started; B is a positive integer greater than 1; the B data bitonic sorting tasks are respectively associated with different data subsequences to be sorted, and the B data subsequences to be sorted are all generated based on the data sequence to be sorted;
[0272] According to B data bitonic sorting tasks, C data bitonic sorting components are run in parallel to obtain B data sorting sub-results output by the C data bitonic sorting components; a data sorting sub-result is used to represent the sorting result of the to-be-sorted data subsequence associated with a data bitonic sorting task;
[0273] Based on the C data bitonic sorting components, the B data sorting sub-results are merged to obtain a data sorting result for the data sequence to be sorted.
[0274] It should be understood that the computer device 1000 described in the embodiment of the present application can execute the above Figure 2 , Figure 6 ,as well as Figure 8The description of the data processing method in the corresponding embodiment can also be performed as described above. Fig.10 The description of the data processing device 1 in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here either.
[0275] The present application also provides a computer-readable storage medium storing a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, Figure 2 , Figure 6 ,as well as Figure 8 For details on the data processing methods provided in each step, please refer to the above Figure 2 , Figure 6 ,as well as Figure 8 The implementation methods provided by each step will not be described in detail here. In addition, the description of the beneficial effects of adopting the same method will not be described in detail here either.
[0276] The computer-readable storage medium may be the data processing device provided in any of the aforementioned embodiments or the internal storage unit of the computer device, such as the hard disk or memory of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, a smart memory card (smart media card, SMC), a secure digital (secure digital, SD) card, a flash card (flash card), etc. equipped on the computer device. Further, the computer-readable storage medium may also include both the internal storage unit of the computer device and an external storage device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.
[0277] The embodiment of the present application also provides a computer program product or a computer program, the computer program product or the computer program includes a computer instruction, the computer instruction is stored in a computer-readable storage medium. The processor of the computer device reads the computer instruction from the computer-readable storage medium, and the processor executes the computer instruction, so that the computer device can execute the above Figure 2 , Figure 6 ,as well as Figure 8 The description of the data processing method in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of adopting the same method will not be repeated here.
[0278] The terms "first", "second", etc. in the description, claims, and drawings of the embodiments of the present application are used to distinguish different objects, rather than to describe a specific order. In addition, the term "comprising" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, device, product, or equipment that includes a series of steps or units is not limited to the listed steps or modules, but optionally includes steps or modules that are not listed, or optionally includes other step units inherent to these processes, methods, devices, products, or equipment.
[0279] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.
[0280] The method and related apparatus provided by the embodiment of the present application are described with reference to the method flow chart and / or structural diagram provided by the embodiment of the present application. Specifically, each process and / or box in the method flow chart and / or structural diagram, as well as the combination of the processes and / or boxes in the flow chart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A flow or multiple flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.
[0281] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.
Claims
1. A data processing method, characterized in that: include: Obtain a data sorting request, and call C data bitonic sorting components according to the data sorting request; C is a positive integer greater than 1; the data sorting request carries a first total number of data to be sorted in the data sequence to be sorted; Sending the data sorting request to a task control component, in which a number to be sorted associated with the first total number is determined; the number to be sorted is equal to or greater than the first total number; Obtaining the data storage capacities corresponding to the C data bitonic sorting components respectively, and comparing the number to be sorted with the total data storage capacity to obtain a first comparison result; the total data storage capacity is equal to the sum of the C data storage capacities; Starting B data bitonic sorting tasks according to the first comparison result; B is a positive integer greater than 1; the B data bitonic sorting tasks are respectively associated with different subsequences of data to be sorted, and the B subsequences of data to be sorted are all generated based on the data sequence to be sorted; According to the B data bitonic sorting tasks, the C data bitonic sorting components are run in parallel to obtain B data sorting sub-results output by the C data bitonic sorting components; one data sorting sub-result is used to represent the sorting result of the to-be-sorted data subsequence associated with one data bitonic sorting task; Based on the C data bitonic sorting components, the B data sorting sub-results are merged to obtain a data sorting result for the data sequence to be sorted.
2. The method according to claim 1, characterized in that The starting the B data bitonal sorting tasks according to the first comparison result includes: If the first comparison result is that the number to be sorted is greater than the total data storage capacity, obtaining a first quantity ratio between the number to be sorted and the total data storage capacity; The B data bitonic sorting tasks are started according to the first quantity ratio; the product of C and the first quantity ratio is equal to B; the second total quantity carried by the one data bitonic sorting task is equal to the data storage capacity; the second total quantity is used to represent the total quantity of the to-be-sorted data in the to-be-sorted data subsequence associated with the one data bitonic sorting task; If the first comparison result is that the number to be sorted is less than or equal to the total data storage capacity, the B data bitonal sorting tasks are started according to the C data bitonal sorting components; B is equal to C.
3. The method according to claim 1, characterized in that According to the B data bitonic sorting tasks, the C data bitonic sorting components are run in parallel to obtain B data sorting sub-results output by the C data bitonic sorting components, including: If B is equal to C, the B data bitonic sorting tasks are distributed to the C data bitonic sorting components; wherein one data bitonic sorting task is distributed to one data bitonic sorting component; The B data bitonic sorting tasks are run in parallel through the C data bitonic sorting components to obtain data sorting sub-results outputted by the C data bitonic sorting components respectively.
4. The method according to claim 3, characterized in that The B data bit-tuning sorting tasks include a data bit-tuning sorting task D e , e is a positive integer and e is less than or equal to B; the B data subsequences to be sorted include the data bitonic sorting task D e The corresponding data subsequence A to be sorted e The C data bit-tuning sorting components include a component for performing the data bit-tuning sorting task D e The data bitonic sorting component F e ; The step of running the B data bitonic sorting tasks in parallel through the C data bitonic sorting components to obtain data sorting sub-results outputted by the C data bitonic sorting components respectively includes: In the data bitonic sorting component F e , obtain the data bit-tuning sorting task D e The corresponding data subsequence A to be sorted e , for the data subsequence A to be sorted e Perform bitonic sorting to obtain the data subsequence A to be sorted e The corresponding data sorting sub-results; When running the data bitune sorting component F e At the same time, B-1 data bitonic sorting tasks are run in parallel through C-1 data bitonic sorting components to obtain data sorting sub-results output by the C-1 data bitonic sorting components respectively; the C-1 data bitonic sorting components refer to the C data bitonic sorting components except the data bitonic sorting component F e The B-1 data bit-tuning sorting tasks refer to the B data bit-tuning sorting tasks except the data bit-tuning sorting task D e Other data bitune sorting tasks.
5. The method according to claim 4, characterized in that The data sorting request carries the read addresses corresponding to the data to be sorted in the data sequence to be sorted, and the sorting order for the data sequence to be sorted; The bitonic sorting component F e , obtain the data bit-tuning sorting task D e The corresponding data subsequence A to be sorted e ,include: In the data bitonic sorting component F e , obtain the data bit-tuning sorting task D e The target read address and the second total number carried; the read addresses corresponding to the data to be sorted in the data sequence to be sorted include the target read address; the second total number is used to represent the data subsequence A to be sorted e The total number of data to be sorted in ; Sending a data acquisition request carrying the target read address to a data read-write component, and acquiring an initial subsequence of data to be sorted from the target read address through the data read-write component; the initial subsequence of data to be sorted belongs to the data sequence to be sorted; Determine a quantity difference between the second total quantity and a third total quantity; the third total quantity is used to represent the total quantity of the data to be sorted in the initial subsequence of data to be sorted; The supplementary data to be sorted is acquired according to the quantity difference and the sorting order, and the supplementary data to be sorted is added to the initial subsequence of data to be sorted to obtain the subsequence A of data to be sorted. e ; The data subsequence A to be sorted e Load into the data bitune sorting component F e The corresponding storage component.
6. The method according to claim 4, characterized in that The data bitune sorting component F e It includes a sorting control subcomponent, a round control subcomponent, a data acquisition subcomponent, and a bitune sorting subcomponent; The to-be-sorted data subsequence A e Perform bitonic sorting to obtain the data subsequence A to be sorted e The corresponding data sorting sub-results include: When the sorting control subcomponent obtains the data loading completion notification, the round control subcomponent is started; the data loading completion notification is used to indicate the data bit-tuning sorting component F e The corresponding storage component has loaded the subsequence A of the data to be sorted e ; In the round control subcomponent, the data subsequence A to be sorted is determined according to the second total number. e The corresponding bitonic sorting stage and the bitonic sorting logic parameters corresponding to the bitonic sorting stage are sent to the data acquisition subcomponent; the second total number is used to represent the data subsequence A to be sorted e The total number of data to be sorted in ; In the data acquisition subcomponent, according to the bitonic sorting stage and the bitonic sorting logic parameters corresponding to the bitonic sorting stage, the intermediate data to be sorted is acquired, the intermediate data to be sorted is sent to the bitonic sorting subcomponent, the intermediate data to be sorted is sorted by the bitonic sorting subcomponent to obtain the intermediate sorting subresult, and the subsequence A of the data to be sorted is determined according to the intermediate sorting subresult e The corresponding data sorting sub-result; the data sub-sequence A to be sorted e Including the intermediate data to be sorted.
7. The method according to claim 6, characterized in that The number of stages of the bitonic sorting stage is at least two; In the data acquisition subcomponent, according to the bitonic sorting stage and the bitonic sorting logic parameters corresponding to the bitonic sorting stage, the intermediate data to be sorted is acquired, the intermediate data to be sorted is sent to the bitonic sorting subcomponent, the intermediate data to be sorted is sorted by the bitonic sorting subcomponent to obtain an intermediate sorting subresult, and the subsequence A of the data to be sorted is determined according to the intermediate sorting subresult e The corresponding data sorting sub-results include: In the data acquisition subcomponent, according to the i-th bitonic sorting stage in at least two bitonic sorting stages, a first data sorting transition subresult is read from the storage component; i is a positive integer greater than 1, and the first data sorting transition subresult is an output result in the i-1-th bitonic sorting stage; the bitonic sorting logic parameters corresponding to the i-th bitonic sorting stage include n-round sorting round logic parameters; n is determined based on i, and n is a positive integer greater than 1; According to the j-th sorting round logic parameter, the intermediate data to be sorted is obtained from the intermediate data sorting transition sub-result; j is a positive integer and j is less than or equal to n; if j is 1, the intermediate data sorting transition sub-result is the first data sorting transition sub-result; if j is not 1, the intermediate data sorting transition sub-result is the second data sorting transition sub-result cached in the data acquisition sub-component, and the second data sorting transition sub-result is determined based on the j-1-th sorting round logic parameter; The intermediate data to be sorted is sent to the bitonic sorting subcomponent, and the intermediate data to be sorted is sorted by the bitonic sorting subcomponent to obtain an intermediate sorting subresult of the intermediate data to be sorted, and the subsequence A of the data to be sorted is determined according to the intermediate sorting subresult. e The corresponding third data sorting transition sub-result; If j is less than n, the third data sorting transition sub-result is cached in the data acquisition sub-component; If j=n, and the i-th bitonic sorting stage is the last bitonic sorting stage of the at least two bitonic sorting stages, then the third data sorting transition sub-result is written to the storage component, and the third data sorting transition sub-result is determined as the data sub-sequence A to be sorted. e The corresponding data sorting sub-result.
8. The method according to claim 1, characterized in that According to the B data bitonic sorting tasks, the C data bitonic sorting components are run in parallel to obtain B data sorting sub-results output by the C data bitonic sorting components, including: If B is greater than C, the B data bitonic sorting tasks are split into K*C data bitonic sorting tasks; K is the maximum running rounds corresponding to the C data bitonic sorting components respectively; Distribute the C data bitonic sorting tasks in the i-th running round to the C data bitonic sorting components; wherein one data bitonic sorting task is distributed to one data bitonic sorting component; i is a positive integer less than or equal to K; The C data bitonic sorting tasks in the i-th running round are run in parallel through the C data bitonic sorting components to obtain data sorting sub-results respectively output by the C data bitonic sorting components, until i is equal to K, the data sorting sub-results respectively corresponding to the B data bitonic sorting tasks are obtained.
9. The method according to claim 1, characterized in that: The step of merging the B data sorting sub-results based on the C data bitonic sorting components to obtain a data sorting result for the data sequence to be sorted includes: In the Lth merging, U data sorting sub-result pairs are obtained; U is a positive integer; L is a positive integer; if L is 1, the U data sorting sub-result pairs are generated based on the B data sorting sub-results; if L is not 1, the U data sorting sub-result pairs are generated based on the historical merging data sorting sub-results, and the historical merging data sorting sub-results are the sub-results obtained in the L-1th merging; Based on the C data bitonic sorting components, the U data sorting sub-result pairs are merged to obtain U target merged data sorting sub-results; If U is equal to 1, the U target merged data sorting sub-results are determined as the data sorting result corresponding to the data sequence to be sorted; If U is greater than 1, then in the L+1th merge, based on the C data bitonic sorting components, the U target merged data sorting sub-results are merged to obtain a data sorting result for the data sequence to be sorted.
10. The method according to claim 9, characterized in that The U data sorting sub-result pairs include data sorting sub-result pairs M p , p is a positive integer, and p is less than or equal to U; The step of merging the U data sorting sub-result pairs based on the C data bitonic sorting components to obtain U target merged data sorting sub-results includes: Determine the data sorting sub-result pair M p The fourth total number of the associated data to be sorted is used to determine the data sorting sub-result pair M p The data storage capacity corresponding to the associated data bitonic sorting component; the C data bitonic sorting components include the data sorting sub-result pair M p The associated data bitonic sorting component; Comparing the fourth total quantity with the data storage capacity to obtain a second comparison result; According to the second comparison result and the data sorting sub-result, M p The associated data bitune sorting component sorts the data sub-results M p Perform bitwise sorting to obtain the data sorting sub-result pair M p The corresponding target merged data sorting sub-results; The target merged data sorting sub-results corresponding to the U data sorting sub-results are determined as the U target merged data sorting sub-results.
11. The method according to claim 10, characterized in that The data sorting sub-result pair M p including a first data sorting sub-result and a second data sorting sub-result; The comparison of the data according to the second comparison result and the data sorting sub-result p The associated data bitune sorting component sorts the data sub-results M p Perform bitwise sorting to obtain the data sorting sub-result pair M p The corresponding target merged data sorting sub-results include: If the second comparison result is that the fourth total number is greater than the data storage capacity, the first data sorting sub-result is evenly divided into a first ordered data segment and a second ordered data segment, and the second data sorting sub-result is evenly divided into a third ordered data segment and a fourth ordered data segment; If the data in the first ordered data segment is less than or equal to the data in the second ordered data segment, and the data in the third ordered data segment is less than or equal to the data in the fourth ordered data segment, then based on the first data bitonic sorting component, the first ordered data segment and the third ordered data segment are bitonically sorted to obtain a third data sorting sub-result; the data sorting sub-result is used for M p The associated data bitonic sorting component includes the first data bitonic sorting component; Based on the second data bitonic sorting component, the second ordered data segment and the fourth ordered data segment are bitonically sorted to obtain a fourth data sorting sub-result; the data sorting sub-result is sorted by M p The associated data bitonic sorting component includes the second data bitonic sorting component; The third data sorting sub-result is equally divided into a fifth ordered data segment and a sixth ordered data segment, and the fourth data sorting sub-result is equally divided into a seventh ordered data segment and an eighth ordered data segment; If the data in the fifth ordered data segment is less than or equal to the data in the sixth ordered data segment, and the data in the seventh ordered data segment is less than or equal to the data in the eighth ordered data segment, then based on a target data bitonic sorting component, bitonic sorting is performed on the sixth ordered data segment and the seventh ordered data segment to obtain a fifth data sorting sub-result; the target data bitonic sorting component is the first data bitonic sorting component or the second data bitonic sorting component; Based on the fifth ordered data segment, the fifth data sorting sub-result and the eighth ordered data sorting, the data sorting sub-result pair M is obtained. p The corresponding target merged data sorting sub-result.
12. The method according to claim 1, characterized in that The C data bitonic sorting components each include an index generation subcomponent, and the index generation subcomponent is used to assign an index value to each to-be-sorted data in the to-be-sorted data sequence; The data sorting request carries a write address; The method further comprises: Sequentially obtain the index values corresponding to each data to be sorted in the data sorting result, and generate an index value sequence with the same length as the sequence of the data to be sorted according to the obtained index values; A data write request for the index value sequence is sent to a data read-write component, and the index value sequence is written to the write address through the data read-write component.
13. A computer device, characterized in that: include: A processor, a memory and a network interface; the processor is connected to the memory and the network interface, wherein the network interface is used to provide a data communication function, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method described in any one of claims 1 to 12.
14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 12.
15. A computer program product, characterized in that The computer program product comprises computer instructions, which are stored in a computer-readable storage medium and are suitable for being read and executed by a processor, so as to cause a computer device having the processor to perform the method according to any one of claims 1 to 12.
