Data sorting method and device, electronic equipment and storage medium

Abstract

The application is suitable for the technical field of data processing, and provides a data sorting method and device, electronic equipment and storage medium. The data sorting method comprises the following steps: converting a one-dimensional array to be sorted into a matrix; dividing the matrix into a plurality of sub-matrices, and determining reference elements in each sub-matrix; performing partitioning operation on each sub-matrix based on a single instruction multiple data stream (SIMD) instruction set, to obtain a first partition and a second partition of each sub-matrix; sorting data elements in the first partition and the second partition of each sub-matrix, to obtain a target sub-matrix of each sub-matrix after sorting is completed; recombining the target sub-matrices of the plurality of sub-matrices, and converting a target matrix obtained by recombination into an ordered target one-dimensional array. The application can efficiently sort a large-scale data set, and improve the processing efficiency of the large-scale data set.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data sorting method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the continuous advancement of information technology, large-scale dataset processing has become a core requirement in numerous fields. In government, industry, finance, transportation, telecommunications, and geospatial fields, big data has been widely applied as the underlying support for industry solutions such as smart cities, smart government, smart parks, and smart public security. For example, in urban management, big data can be used to achieve intelligent transportation, environmental monitoring, urban planning, and intelligent security. Big data analysis can predict traffic congestion times and formulate traffic congestion mitigation plans. The ability to process large-scale datasets directly determines the speed of innovation and operational efficiency in these fields.

[0003] While the traditional quicksort algorithm is highly efficient in processing large datasets, the sheer size of the datasets means that the required memory space may far exceed the computer's memory capacity. This leads to frequent disk I / O operations during the sorting process, further reducing sorting efficiency.

[0004] Therefore, how to efficiently sort large-scale datasets and improve the processing efficiency of large-scale datasets is a technical problem that needs to be solved. Summary of the Invention

[0005] This application provides a data sorting method, apparatus, electronic device, and storage medium that can efficiently sort large-scale datasets and improve the processing efficiency of large-scale datasets.

[0006] In a first aspect, embodiments of this application provide a data sorting method, including:

[0007] Convert a one-dimensional array to be sorted into a matrix;

[0008] The matrix is ​​divided into several sub-matrices, and the pivot element within each sub-matrix is ​​determined;

[0009] Based on the Single Instruction Multiple Data (SIMD) instruction set, each of the sub-matrices is partitioned to obtain a first partition and a second partition for each of the sub-matrices. The data elements in the first partition are not greater than the base element inside the sub-matrix, and the data elements in the second partition are greater than the base element inside the sub-matrix.

[0010] Sort the data elements in the first and second partitions of each submatrix to obtain the target submatrix after sorting.

[0011] The target submatrix of the aforementioned submatrixes is reorganized, and the reorganized target matrix is ​​converted into an ordered one-dimensional array of targets.

[0012] In one possible implementation of the first aspect, the step of partitioning each of the sub-matrices based on the Single Instruction Multiple Data (SIMD) instruction set to obtain a first partition and a second partition of each of the sub-matrices includes:

[0013] The data elements in the submatrix are loaded into the SIMD register using the SIMD load instruction;

[0014] The data element in the SIMD register is compared with the reference element using a SIMD comparison instruction;

[0015] Based on the comparison results, the data elements in the submatrix are logically partitioned using SIMD conditional selection instructions. Data elements not greater than the reference element are selected to the first partition, and data elements greater than the reference element are selected to the second partition.

[0016] The partitioning results are then stored back into memory using SIMD storage instructions, thus completing the partitioning of the submatrix.

[0017] In one possible implementation of the first aspect, the step of sorting the data elements in the first and second partitions of each of the submatrices to obtain the sorted target submatrices includes:

[0018] Determine whether the number of data elements in the first partition and the second partition of the submatrix exceeds a preset threshold.

[0019] If the number of data elements in a partition exceeds the preset threshold, the data elements in the partition are recursively sorted.

[0020] If the number of data elements in a partition does not exceed the preset threshold, then the data elements in the partition are simply sorted.

[0021] The sorted first partition and the second partition are merged to obtain the target submatrix of the submatrix.

[0022] In one possible implementation of the first aspect, the step of recursively sorting the data elements within the partition includes:

[0023] The partition is divided into several matrix units, and the reference element within each matrix unit is determined.

[0024] Based on the SIMD instruction set, each matrix unit is partitioned to obtain a first sub-partition and a second sub-partition for each matrix unit. The data elements in the first sub-partition are not greater than the reference element inside the matrix unit, and the data elements in the second sub-partition are greater than the reference element inside the matrix unit.

[0025] The data elements in the first and second sub-partitions of each matrix unit are sorted to obtain the target matrix unit after sorting.

[0026] The target matrix units of the aforementioned matrix units are reorganized to obtain sorted partitions.

[0027] In one possible implementation of the first aspect, the step of dividing the matrix into several sub-matrices and determining the reference element within each of the sub-matrices includes:

[0028] The matrix is ​​divided based on its rows or columns, and a row or column of the matrix is ​​defined as a submatrix.

[0029] Determine the pivot element for each row or column of the matrix.

[0030] In one possible implementation of the first aspect, the step of dividing the matrix into several sub-matrices and determining the reference element within each of the sub-matrices includes:

[0031] The matrix is ​​divided into several matrix blocks, and each matrix block is a sub-matrix;

[0032] Determine the pivot element in each of the submatrices.

[0033] In one possible implementation of the first aspect, the step of determining the reference element within each of the sub-matrices includes:

[0034] The data element at the center position of the sub-matrix is ​​determined as the reference element inside the sub-matrix;

[0035] Alternatively, a data element may be randomly selected from the submatrix and determined as the reference element within the submatrix.

[0036] Secondly, embodiments of this application provide a data sorting apparatus, including:

[0037] A matrix processing unit is used to convert a one-dimensional array to be sorted into a matrix.

[0038] A reference determination unit is used to divide the matrix into several sub-matrices and determine the reference element within each sub-matrix.

[0039] A matrix partitioning unit is used to perform partitioning operations on each of the sub-matrices based on the Single Instruction Multiple Data Stream (SIMD) instruction set to obtain a first partition and a second partition of each of the sub-matrices, wherein the data elements in the first partition are not greater than the reference element inside the sub-matrices, and the data elements in the second partition are greater than the reference element inside the sub-matrices.

[0040] A sorting unit is used to sort the data elements in the first and second partitions of each of the submatrices to obtain the target submatrices after the sorting of each of the submatrices;

[0041] The recombination unit is used to recombine the target submatrices of the plurality of submatrices and convert the recombined target matrix into an ordered one-dimensional array of targets.

[0042] Thirdly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the data sorting method as described in the first aspect above.

[0043] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the data sorting method described in the first aspect above.

[0044] Fifthly, embodiments of this application provide a computer program product that, when run on an electronic device, causes the electronic device to execute the data sorting method described in the first aspect above.

[0045] In this embodiment, by converting the one-dimensional array to be sorted into a matrix and dividing the matrix into several sub-matrices, the amount of data processed in a single operation can be effectively reduced. Then, the pivot element within each sub-matrix is ​​determined, and the sub-matrices are partitioned in parallel based on the Single Instruction Multiple Data Stream (SIMD) instruction set to obtain the first and second partitions of each sub-matrix. This accelerates the sorting process. The data elements in the first and second partitions of each sub-matrix are sorted to obtain the target sub-matrix after sorting. The target sub-matrices of the several sub-matrices are then reassembled, and the reassembled target matrix is ​​converted into an ordered target one-dimensional array. This solution converts the one-dimensional array corresponding to a large-scale dataset into a matrix and divides it into sub-matrices for sorting, reducing memory usage and memory access latency. This helps avoid frequent disk I / O operations due to insufficient memory when processing large-scale datasets. It also makes the sorting process more localized. Furthermore, combining matrix partitioning with the SIMD instruction set leverages the SIMD characteristics of modern processors to improve the efficiency of parallel processing of multiple data elements, accelerating key operations in the sorting process and achieving efficient sorting of large-scale datasets, thereby improving the processing efficiency of large-scale datasets. Attached Figure Description

[0046] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0047] Figure 1 This is a flowchart illustrating the implementation of the data sorting method provided in the embodiments of this application;

[0048] Figure 2 This is a flowchart illustrating a specific implementation of step S103 in the data sorting method provided in this application embodiment;

[0049] Figure 3 This is a flowchart illustrating a specific implementation of step S104 in the data sorting method provided in this application embodiment;

[0050] Figure 4 This is a flowchart illustrating a specific implementation of the data sorting method provided in this application, which recursively sorts data elements within a partition.

[0051] Figure 5 This is a structural block diagram of the data sorting device provided in the embodiments of this application;

[0052] Figure 6 This is a schematic diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0053] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0054] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0055] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0056] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as meaning "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0057] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0058] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0059] By way of example and not limitation, the data sorting method provided in this application can be applied to smart devices and servers that require data sorting for various types of data. Specifically, smart devices may include mobile phones, tablets, laptops, ultra-mobile personal computers (UMPCs), desktop computers, and other smart devices capable of data processing. This application does not impose any limitations on the specific type of smart device.

[0060] Figure 1 The implementation flow of the data sorting method provided in this application embodiment is illustrated. In this application embodiment, the execution end is an electronic device, and the method flow includes steps S101 to S105. The specific implementation principle of each step is as follows:

[0061] Step S101: Convert the one-dimensional array to be sorted into a matrix.

[0062] In data processing, large-scale datasets are stored in the form of one-dimensional data arrays. These one-dimensional arrays contain a large number of data elements to be sorted. Directly sorting these one-dimensional arrays can become inefficient due to the sheer size of the data. To improve the performance of data sorting, embodiments of this application convert the one-dimensional array to be sorted into a matrix. This conversion leverages the locality and parallelism of matrix operations to accelerate the sorting process.

[0063] In this embodiment, the length of the one-dimensional array must match the dimension of the matrix, and the conversion process involves determining the number of rows and columns of the two-dimensional matrix. Furthermore, converting a one-dimensional array to a two-dimensional matrix requires ensuring that the data type of the converted two-dimensional matrix is ​​consistent with the data type of the original one-dimensional array. For example, if the one-dimensional array is an integer array, then the two-dimensional matrix should also be an integer matrix.

[0064] In this embodiment, converting the one-dimensional array to be sorted into a two-dimensional matrix requires determining the number of columns in the two-dimensional matrix. Based on this number of columns and the length of the one-dimensional array, the number of rows in the two-dimensional matrix is ​​determined. If the length of the one-dimensional array is not divisible by the number of columns, padding values ​​can be added to the last row.

[0065] For example, given a one-dimensional array of length N, and setting the number of columns in the two-dimensional matrix to n, the number of rows in the two-dimensional matrix, m = N / n, is calculated using the upward division method. The indices of the data elements in the one-dimensional array are mapped to the two-dimensional matrix, either row by row or column by column according to the initial sorting of the data elements in the one-dimensional array. When N is not an integer multiple of n, it means that the last row of the two-dimensional matrix cannot be filled; it can be filled with padding values ​​such as 0.

[0066] One possible implementation is to convert the one-dimensional array to be sorted into a three-dimensional matrix.

[0067] In certain specific situations, to meet specific algorithmic requirements, optimize computational performance, or represent more complex data structures, one-dimensional arrays can be converted into three-dimensional matrices. For example, when processing data in three-dimensional space (such as three-dimensional images, volumetric data, etc.), using three-dimensional matrices may be more appropriate.

[0068] Converting a one-dimensional array to a three-dimensional matrix involves more dimensional choices. The conversion requires determining the depth (or the size of the third dimension), number of rows, and number of columns of the three-dimensional matrix.

[0069] For example, given a one-dimensional array of length N, with a given depth d and number of columns n, calculate the number of rows m = n / / (d * n) using the upward division method. Map the indices of the data elements in the one-dimensional array to the three-dimensional matrix. Similarly, when N is not an integer multiple of (d * n), padding values ​​such as 0 can be used to fill the three-dimensional matrix.

[0070] Step S102: Divide the matrix into several sub-matrices and determine the reference element within each sub-matrix.

[0071] In one possible implementation, the matrix is ​​divided based on its rows or columns, with each row or column defined as a submatrix; a pivot element is then determined for each row or column. Specifically, each row is designated as a submatrix, the number of submatrixes equals the number of rows, and all submatrixes corresponding to all rows form the matrix corresponding to the one-dimensional array, with the pivot element in each row determined. Alternatively, each column is designated as a submatrix, the number of submatrixes equals the number of columns, and all submatrixes corresponding to all columns form the matrix corresponding to the one-dimensional array, with the pivot element in each column determined.

[0072] In one possible implementation, the matrix is ​​divided into several matrix blocks, each matrix block being a submatrix; a pivot element is determined in each matrix block. That is, each matrix block is considered a submatrix, and the matrix corresponding to the one-dimensional array is obtained by concatenating the several matrix blocks. The sizes of the several matrix blocks can be the same or different. Size refers to the number of rows and columns in the submatrix. Submatrices corresponding to different matrix blocks can have the same or different numbers of rows and columns.

[0073] By dividing a matrix into several submatrices, different submatrices can be assigned to different processors or computing nodes for individual processing. The parallelism of matrix operations can be used to accelerate the sorting process, thereby improving sorting efficiency and the processing efficiency of large-scale datasets.

[0074] In this embodiment of the application, the data element at the center position of the sub-matrix is ​​determined as the reference element inside the sub-matrix; or, a data element is randomly selected from the sub-matrix and determined as the reference element inside the sub-matrix.

[0075] For a submatrix obtained by dividing by rows or columns, the data element at the center of a row or column is determined as the pivot element of the submatrix; or, a data element is randomly selected from a row or column and determined as the pivot element of the submatrix.

[0076] For submatrices obtained by dividing the matrix into blocks, in one possible implementation, the pivot element can be determined by row or column, but a submatrix may correspond to more than one pivot element. For example, the data element at the center of a row or column within the submatrix can be determined as the pivot element for that row or column; alternatively, a data element can be randomly selected from a row or column to be determined as the pivot element for that row or column. That is, the submatrix includes multiple pivot elements, and the number of pivot elements in the submatrix is ​​the same as the number of rows or columns within the submatrix. In another possible implementation, data elements in the submatrix that meet specific characteristics can be determined as pivot elements, with each submatrix corresponding to one pivot element.

[0077] Step S103: Perform a partitioning operation on each of the sub-matrices based on the Single Instruction Multiple Data (SIMD) instruction set to obtain the first partition and the second partition of each of the sub-matrices.

[0078] Wherein, the data elements in the first partition are not greater than the reference element inside the submatrix, and the data elements in the second partition are greater than the reference element inside the submatrix.

[0079] The first and second partitions are determined based on the position of the reference element. For example, the first partition could be to the left of the reference element, and the second partition to the right of the reference element. Alternatively, the first partition could be above the reference element, and the second partition to the bottom of the reference element.

[0080] Converting a one-dimensional array into a matrix makes it easier and faster to process the data elements within the one-dimensional array. After dividing the matrix into several sub-matrices, this embodiment of the application uses the Single Instruction, Multiple Data (SIMD) instruction set to independently partition the data elements within the sub-matrices, which can effectively speed up the sorting process.

[0081] As one possible implementation of this application, the SIMD instruction set includes SIMD load instructions, SIMD compare instructions, SIMD condition select instructions, and SIMD store instructions. Figure 2A specific implementation flow of step S103 in the data sorting method provided in this application embodiment is shown below:

[0082] A1: Load the data elements in the submatrix into the SIMD register using the SIMD load instruction.

[0083] Generally, a SIMD register can only store a fixed number of data elements at a time. Depending on the size of the submatrix, multiple SIMD registers can be used to load and store the data elements of the submatrix.

[0084] A2: The SIMD comparison instruction compares the data element in the SIMD register with the reference element. Specifically, it determines whether the data element loaded into the SIMD register is greater than the reference element. The SIMD comparison instruction can compare multiple data elements in one instruction cycle. Using Intel's AVX2 instruction set, eight 32-bit integers can be compared in one instruction cycle.

[0085] A3: Based on the comparison results, the data elements in the submatrix are logically partitioned using SIMD conditional selection instructions. Data elements not greater than the reference element are selected to the first partition, and data elements greater than the reference element are selected to the second partition. Specifically, the SIMD conditional selection instructions can be parallel swap instructions, enabling the swapping of multiple data elements within a single instruction cycle. The logical partitioning of data elements does not involve physically moving the data elements to different locations in memory; instead, it logically groups the data elements into different vectors based on the comparison results and stores them in the SIMD register as vectors.

[0086] A4: The partitioning results are stored back into memory using SIMD storage instructions to complete the partitioning of the submatrix.

[0087] In practical applications, the decision to store the results back in memory or in a SIMD register depends on the algorithm's requirements and the size of the data. If the data is large, or if the original data needs to be retained for multiple iterations, it is generally advisable to store the results back in memory. If the data is small, or if only simple data processing is being performed, it is preferable to store the results in a separate SIMD register to reduce memory access overhead.

[0088] In some scenarios, if iterative processes do not require immediate access to memory, the partitioning results can be stored in another SIMD register, and the final result can be stored in memory to complete the entire processing.

[0089] In this embodiment, when comparing the data elements in the SIMD register with the reference element using a SIMD comparison instruction, a mask is generated. Based on the mask, data elements less than or equal to the reference element are written to the left of the reference element, and data elements greater than the reference element are written to the right of the reference element. Each bit in the mask indicates whether the corresponding data element is greater than the reference element.

[0090] Prepare two SIMD vectors: one with all elements zero (the zero vector), and the other with all elements representing the possible maximum values ​​(used for padding). Using a mask, selectively write data elements less than or equal to the pivot element into the zero vector (because they are 1 in the mask) using SIMD conditional selection instructions, and selectively write data elements greater than the pivot element into the maximum value vector (because they are 0 in the mask). Merge the two vectors into a new vector, with elements less than the pivot element on the left and elements greater than the pivot element on the right. Then store the partitioned result back into memory or a specified location such as a register.

[0091] In this embodiment, the SIMD instruction set can be used to process multiple data elements simultaneously within a single instruction cycle, thereby significantly improving the efficiency of partitioning operations.

[0092] Step S104: Sort the data elements in the first and second partitions of each submatrix to obtain the target submatrix after sorting.

[0093] Sort the data elements in each submatrix separately to obtain the target submatrix after sorting each submatrix.

[0094] As one possible implementation of this application Figure 3 A specific implementation flow of step S104 in the data sorting method provided in this application embodiment is shown below:

[0095] B1: Determine whether the number of data elements in the first partition and the second partition of the submatrix exceeds a preset threshold.

[0096] B2: If the number of data elements in a partition exceeds the preset threshold, the data elements in the partition are recursively sorted.

[0097] As one possible implementation of this application Figure 4 This application illustrates a specific implementation flow of recursively sorting data elements within a partition in the data sorting method provided in this embodiment, detailed below:

[0098] B21: Divide the partition into several matrix units and determine the reference element within each matrix unit.

[0099] B22: Based on the SIMD instruction set, perform a partitioning operation on each of the matrix units to obtain the first sub-partition and the second sub-partition of each of the matrix units.

[0100] In this configuration, the data elements in the first sub-partition are no greater than the reference element within the matrix unit, while the data elements in the second sub-partition are greater than the reference element within the matrix unit.

[0101] B23: Sort the data elements in the first and second sub-partitions of each matrix unit to obtain the target matrix unit after sorting.

[0102] B24: Reorganize the target matrix units of the aforementioned matrix units to obtain sorted partitions. Reorganize the target matrix units of the several matrix units in the first partition to obtain the sorted first partition, and reorganize the target matrix units of the several matrix units in the second partition to obtain the sorted second partition.

[0103] In this embodiment, recursive sorting is used for partitions in the submatrix that exceed a preset threshold number. This involves recursively performing submatrix partitioning, determining the pivot element, partitioning operations, and sorting the data elements. For large-scale datasets, recursive sorting can effectively improve sorting efficiency. The specific content of steps B21-B24 can be found in steps S102 to S104, and will not be repeated here.

[0104] B3: If the number of data elements in a partition does not exceed the preset threshold, then the data elements in the partition are sorted using a simple sorting algorithm. Common simple sorting algorithms include bubble sort, selection sort, and insertion sort.

[0105] B4: Merge the sorted first partition with the second partition to obtain the target submatrix of the submatrix.

[0106] In this embodiment, for partitions with more than a preset threshold number of data elements, sorting efficiency can be accelerated, while for partitions with fewer than the preset threshold number of data elements, simple sorting is faster than recursive sorting.

[0107] Step S105: Reorganize the target submatrix of the plurality of submatrices, and convert the reorganized target matrix into an ordered one-dimensional target array.

[0108] In this embodiment, the several sub-matrices obtained by division are reorganized according to the division order to obtain the target matrix. The size and scale of the target matrix are the same as the original matrix corresponding to the one-dimensional array. Then, the target matrix is ​​converted into an array to obtain the sorted target one-dimensional array, thus completing the sorting of the one-dimensional array.

[0109] As can be seen from the above, in this embodiment, by converting the one-dimensional array to be sorted into a matrix and dividing the matrix into several sub-matrices, the amount of data processed in a single operation can be effectively reduced. Then, the base element within each sub-matrix is ​​determined, and the sub-matrix is ​​partitioned in parallel based on the Single Instruction Multiple Data Stream (SIMD) instruction set to obtain the first and second partitions of each sub-matrix. This accelerates the sorting process. The data elements in the first and second partitions of each sub-matrix are sorted to obtain the target sub-matrix after sorting. The target sub-matrixes of the several sub-matrixes are reorganized, and the reorganized target matrix is ​​converted into an ordered target one-dimensional array. This solution converts the one-dimensional array corresponding to a large-scale dataset into a matrix and divides it into sub-matrices for sorting. This reduces memory usage and memory access latency, helps avoid frequent disk I / O operations due to insufficient memory when processing large-scale datasets, and makes the sorting process more localized. Combined with matrix partitioning using the SIMD instruction set, the SIMD characteristics of modern processors are utilized to improve the efficiency of parallel processing of multiple data elements, accelerate key operations in the sorting process, achieve efficient sorting of large-scale datasets, and thus improve the processing efficiency of large-scale datasets.

[0110] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0111] Corresponding to the data sorting method described in the above embodiments, Figure 5 A structural block diagram of a data sorting apparatus provided in an embodiment of this application is shown. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0112] Reference Figure 5 The data sorting device includes: a matrix processing unit 51, a benchmark determination unit 52, a matrix partitioning unit 53, a sorting unit 54, and a reorganization unit 55, wherein:

[0113] The matrix processing unit 51 is used to convert a one-dimensional array to be sorted into a matrix.

[0114] The reference determination unit 52 is used to divide the matrix into several sub-matrices and determine the reference element within each sub-matrix.

[0115] Matrix partitioning unit 53 is used to perform partitioning operations on each of the sub-matrices based on the Single Instruction Multiple Data Stream (SIMD) instruction set to obtain a first partition and a second partition of each of the sub-matrices, wherein the data elements in the first partition are not greater than the reference element inside the sub-matrices, and the data elements in the second partition are greater than the reference element inside the sub-matrices.

[0116] The sorting unit 54 is used to sort the data elements in the first and second partitions of each of the submatrices to obtain the target submatrices after the sorting of each of the submatrices.

[0117] The recombination unit 55 is used to recombine the target submatrices of the plurality of submatrices and convert the recombined target matrix into an ordered one-dimensional array of targets.

[0118] As one possible implementation of this application, the matrix partitioning unit 53 is used for:

[0119] The data elements in the submatrix are loaded into the SIMD register using the SIMD load instruction;

[0120] The data element in the SIMD register is compared with the reference element using a SIMD comparison instruction;

[0121] Based on the comparison results, the data elements in the submatrix are logically partitioned using SIMD conditional selection instructions. Data elements not greater than the reference element are selected to the first partition, and data elements greater than the reference element are selected to the second partition.

[0122] The partitioning results are then stored back into memory using SIMD storage instructions, thus completing the partitioning of the submatrix.

[0123] As one possible implementation of this application, the sorting unit 54 specifically includes:

[0124] The element count determination module is used to determine whether the number of data elements in the first partition and the second partition of the submatrix exceeds a preset number threshold.

[0125] The first sorting module is used to recursively sort the data elements in the partition if the number of data elements in the partition exceeds the preset number threshold.

[0126] The second sorting module is used to perform a simple sorting of the data elements in the partition if the number of data elements in the partition does not exceed the preset number threshold.

[0127] The partition merging module is used to merge the sorted first partition with the second partition to obtain the target submatrix of the submatrix.

[0128] As one possible implementation of this application, the first sorting module is specifically used for:

[0129] The partition is divided into several matrix units, and the reference element within each matrix unit is determined.

[0130] Based on the SIMD instruction set, each matrix unit is partitioned to obtain a first sub-partition and a second sub-partition for each matrix unit. The data elements in the first sub-partition are not greater than the reference element inside the matrix unit, and the data elements in the second sub-partition are greater than the reference element inside the matrix unit.

[0131] The data elements in the first and second sub-partitions of each matrix unit are sorted to obtain the target matrix unit after sorting.

[0132] The target matrix units of the aforementioned matrix units are reorganized to obtain sorted partitions.

[0133] As one possible implementation of this application, the reference determination unit 52 is specifically used for:

[0134] The matrix is ​​divided based on its rows or columns, and a row or column of the matrix is ​​defined as a submatrix.

[0135] Determine the pivot element for each row or column of the matrix.

[0136] As one possible implementation of this application, the reference determination unit 52 is further configured to:

[0137] The matrix is ​​divided into several matrix blocks, and each matrix block is a sub-matrix;

[0138] Determine the pivot element in each of the submatrices.

[0139] As one possible implementation of this application, the reference determination unit 52 includes:

[0140] The first reference determination module is used to determine the data element at the center position of the sub-matrix as the reference element inside the sub-matrix;

[0141] Alternatively, the first reference determination module is used to randomly select a data element from the submatrix and determine it as the reference element inside the submatrix.

[0142] As can be seen from the above, in this embodiment, by converting the one-dimensional array to be sorted into a matrix and dividing the matrix into several sub-matrices, the amount of data processed in a single operation can be effectively reduced. Then, the base element within each sub-matrix is ​​determined, and the sub-matrix is ​​partitioned in parallel based on the Single Instruction Multiple Data Stream (SIMD) instruction set to obtain the first and second partitions of each sub-matrix. This accelerates the sorting process. The data elements in the first and second partitions of each sub-matrix are sorted to obtain the target sub-matrix after sorting. The target sub-matrixes of the several sub-matrixes are reorganized, and the reorganized target matrix is ​​converted into an ordered target one-dimensional array. This solution converts the one-dimensional array corresponding to a large-scale dataset into a matrix and divides it into sub-matrices for sorting. This reduces memory usage and memory access latency, helps avoid frequent disk I / O operations due to insufficient memory when processing large-scale datasets, and makes the sorting process more localized. Combined with matrix partitioning using the SIMD instruction set, the SIMD characteristics of modern processors are utilized to improve the efficiency of parallel processing of multiple data elements, accelerate key operations in the sorting process, achieve efficient sorting of large-scale datasets, and thus improve the processing efficiency of large-scale datasets.

[0143] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0144] This application embodiment also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements... Figures 1 to 4 This represents the steps of any data sorting method.

[0145] This application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements... Figures 1 to 4 This represents the steps of any data sorting method.

[0146] This application also provides a computer program product that, when run on an electronic device, causes the electronic device to perform the following: Figures 1 to 4 This represents the steps of any data sorting method.

[0147] Figure 6 This is a schematic diagram of an electronic device provided in an embodiment of this application. Figure 6As shown, the electronic device 6 of this embodiment includes: a processor 60, a memory 61, and a computer program 62 stored in the memory 61 and executable on the processor 60. When the processor 60 executes the computer program 62, it implements the steps in the various data sorting method embodiments described above, for example... Figure 1 Steps S101 to S105 are shown. Alternatively, when the processor 60 executes the computer program 62, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 5 The functions of units 51 to 55 shown.

[0148] For example, the computer program 62 may be divided into one or more modules / units, which are stored in the memory 61 and executed by the processor 60 to complete this application. The one or more modules / units may be a series of computer-readable instruction segments capable of performing a specific function, which describe the execution process of the computer program 62 in the electronic device 6.

[0149] The electronic device 6 may include, but is not limited to, a processor 60 and a memory 61. Those skilled in the art will understand that... Figure 6 This is merely an example of electronic device 6 and does not constitute a limitation on electronic device 6. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device 6 may also include input / output devices, network access devices, buses, etc.

[0150] The processor 60 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0151] The memory 61 can be an internal storage unit of the electronic device 6, such as a hard disk or memory. The memory 61 can also be an external storage device of the electronic device 6, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 61 can include both internal and external storage units of the electronic device 6. The memory 61 is used to store the computer program and other programs and data required by the electronic device. The memory 61 can also be used to temporarily store data that has been output or will be output.

[0152] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0153] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0154] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / terminal equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0155] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0156] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A data sorting method, characterized in that, include: Convert a one-dimensional array to be sorted into a matrix; The matrix is ​​divided into several sub-matrices, and the pivot element within each sub-matrix is ​​determined; Based on the Single Instruction Multiple Data (SIMD) instruction set, each of the sub-matrices is partitioned to obtain a first partition and a second partition for each of the sub-matrices. The data elements in the first partition are not greater than the base element inside the sub-matrix, and the data elements in the second partition are greater than the base element inside the sub-matrix. Sort the data elements in the first and second partitions of each of the submatrices to obtain the target submatrix after sorting, including: determining whether the number of data elements in the first and second partitions of the submatrix exceeds a preset threshold; if the number of data elements in the partition exceeds the preset threshold, recursively sorting the data elements in the partition; if the number of data elements in the partition does not exceed the preset threshold, performing a simple sorting of the data elements in the partition; merging the sorted first and second partitions to obtain the target submatrix; wherein, the recursive sorting of the data elements in the partitions... The steps include: dividing the partition into several matrix units and determining the reference element within each matrix unit; performing a partitioning operation on each matrix unit based on the SIMD instruction set to obtain a first sub-partition and a second sub-partition for each matrix unit, wherein the data elements in the first sub-partition are not greater than the reference element within the matrix unit, and the data elements in the second sub-partition are greater than the reference element within the matrix unit; sorting the data elements in the first and second sub-partitions of each matrix unit to obtain the sorted target matrix unit; and reorganizing the target matrix units of the several matrix units to obtain the sorted partition. The target submatrix of the aforementioned submatrixes is reorganized, and the reorganized target matrix is ​​converted into an ordered one-dimensional array of targets.

2. The method according to claim 1, characterized in that, The step of partitioning each of the sub-matrices based on the Single Instruction Multiple Data (SIMD) instruction set to obtain the first and second partitions of each of the sub-matrices includes: The data elements in the submatrix are loaded into the SIMD register using the SIMD load instruction; The data element in the SIMD register is compared with the reference element using a SIMD comparison instruction; Based on the comparison results, the data elements in the submatrix are logically partitioned using SIMD conditional selection instructions. Data elements not greater than the reference element are selected to the first partition, and data elements greater than the reference element are selected to the second partition. The partitioning results are then stored back into memory using SIMD storage instructions, thus completing the partitioning of the submatrix.

3. The method according to claim 1, characterized in that, The step of dividing the matrix into several sub-matrices and determining the reference element within each sub-matrix includes: The matrix is ​​divided based on its rows or columns, and a row or column of the matrix is ​​defined as a submatrix. Determine the pivot element for each row or column of the matrix.

4. The method according to claim 1, characterized in that, The step of dividing the matrix into several sub-matrices and determining the reference element within each sub-matrix includes: The matrix is ​​divided into several matrix blocks, and each matrix block is a sub-matrix; Determine the pivot element in each of the submatrices.

5. The method according to any one of claims 1 to 4, characterized in that, The step of determining the reference element within each of the sub-matrices includes: The data element at the center of the submatrix is ​​determined as the reference element inside the submatrix; Alternatively, a data element may be randomly selected from the submatrix and determined as the reference element within the submatrix.

6. A data sorting device, characterized in that, include: A matrix processing unit is used to convert a one-dimensional array to be sorted into a matrix. A reference determination unit is used to divide the matrix into several sub-matrices and determine the reference element within each sub-matrix. A matrix partitioning unit is used to perform partitioning operations on each of the sub-matrices based on the Single Instruction Multiple Data Stream (SIMD) instruction set to obtain a first partition and a second partition of each of the sub-matrices, wherein the data elements in the first partition are not greater than the reference element inside the sub-matrices, and the data elements in the second partition are greater than the reference element inside the sub-matrices. A sorting unit is used to sort the data elements in the first and second partitions of each of the submatrices to obtain the target submatrices after sorting; the sorting unit includes: The element count determination module is used to determine whether the number of data elements in the first partition and the second partition of the submatrix exceeds a preset number threshold. The first sorting module is used to recursively sort the data elements in the partition if the number of data elements in the partition exceeds the preset number threshold. This includes: dividing the partition into several matrix units and determining the reference element within each matrix unit; performing a partitioning operation on each matrix unit based on the SIMD instruction set to obtain a first sub-partition and a second sub-partition for each matrix unit, where the data elements in the first sub-partition are not greater than the reference element within the matrix unit, and the data elements in the second sub-partition are greater than the reference element within the matrix unit; sorting the data elements in the first and second sub-partitions of each matrix unit to obtain the target matrix unit after sorting; and reorganizing the target matrix units of the several matrix units to obtain the sorted partition. The second sorting module is used to perform a simple sorting of the data elements in the partition if the number of data elements in the partition does not exceed the preset number threshold. The partition merging module is used to merge the sorted first partition with the second partition to obtain the target submatrix of the submatrix. The recombination unit is used to recombine the target submatrices of the plurality of submatrices and convert the recombined target matrix into an ordered one-dimensional array of targets.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the data sorting method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the data sorting method as described in any one of claims 1 to 5.

Images