Data processing method, device and computer-readable storage medium

Abstract

To provide a data processing method, a device, an apparatus, and a computer-readable storage medium that increase calculation power and furthermore improve a data processing capability.SOLUTION: The method includes: receiving input data transmitted by a central processing unit; performing a hash join operation on the input data to obtain a target operation result and to set the target operation result as output data; and transmitting the output data to the central processing unit. The hash join operation is performed on the input data by an acceleration card, that is, parallel operation is used to increase calculation power, thereby improving the efficiency of data processing.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] (cross reference) This disclosure claims priority to Chinese patent application Ser. No. 202310429904.5, filed on April 20, 2023, entitled "Data Processing Method, Apparatus, and Computer-Readable Storage Medium," the entire contents of which are incorporated herein by reference.

[0002] (Technical field) The present disclosure relates to the field of computer technology, and in particular to a data processing method, apparatus, device and computer-readable storage medium. [Background technology]

[0003] Databases are an important workload for artificial intelligence, cloud computing, big data processing, etc. With the rapid development of Internet technology and the massive generation of data, the demands on database performance are increasing. With the development of technology, disk read / write capacity and network bandwidth performance are constantly improving, and as a result, the performance bottleneck in the database field has shifted to computing power. Summary of the Invention [Problem to be solved by the invention]

[0004] In the prior art, data is generally processed by a central processing unit (CPU), but this processing method does not provide sufficient computing power to support large-volume data services. Therefore, how to increase computing power and improve data processing capacity is an issue that needs to be addressed urgently.

[0005] To solve or at least partially solve the above problems, the present disclosure provides a data processing method, apparatus, device and computer-readable storage medium, which enhances computing power and further improves data processing capabilities. [Means for solving the problem]

[0006] In a first aspect, an embodiment of the present disclosure provides a data processing method, comprising: receiving input data transmitted by a central processing unit; performing a hash join operation on the input data to obtain a target operation result, and setting the target operation result as output data; transmitting the output data to the central processing unit.

[0007] In a second aspect, an embodiment of the present disclosure provides a data processing apparatus, comprising: a receiving module for receiving input data transmitted by the central processing unit; an operation module for performing a hash join operation on the input data to obtain a target operation result and outputting the target operation result; a transmission module for transmitting the output data to the central processing unit.

[0008] In a third aspect, an embodiment of the present disclosure provides an electronic device, Memory and a processor; a computer program; The computer program is stored in the memory and configured to be executed by the processor to implement the method according to the first aspect.

[0009] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable recording medium having a computer program stored thereon, the computer program being executed by a processor to implement the method according to the first aspect.

[0010] In a fifth aspect, embodiments of the present disclosure further provide a computer program product including a computer program or instructions, which when executed by a processor, implements the method of the first aspect. [Effects of the Invention]

[0011] The data processing method, device, equipment, and computer-readable recording medium provided by the embodiments of the present disclosure receive input data transmitted by a central processing unit, transmit the input data to an acceleration card, and perform a hash join operation on the input data, i.e., improve computational power through parallel operation, ensure that the acceleration card processes the input data in real time, obtain a target operation result for the input data, and use the target operation result as output data, thereby improving data processing efficiency, and transmit the output data to the central processing unit, allowing the central processing unit to obtain the output data, facilitating the storage and management of the output data by the central processing unit, and improving data management capabilities. [Brief explanation of the drawings]

[0012] The drawings herein are incorporated in and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description, serve to explain the principles of the present disclosure.

[0013] In order to more clearly describe the technical solutions in the embodiments of the present disclosure or the prior art, the following briefly describes the drawings that need to be used in the description of the embodiments or the prior art, and obviously, those skilled in the art can obtain other drawings based on these drawings without paying creative labor.

[0014] [Figure 1] 1 is a flowchart of a data processing method provided by an embodiment of the present disclosure. [Figure 2] 1 is a schematic diagram of an application scenario provided by an embodiment of the present disclosure; [Figure 3] 1 is a schematic diagram of an application scenario provided by an embodiment of the present disclosure; [Figure 4] FIG. 2 is a schematic diagram of input data provided by an embodiment of the present disclosure. [Figure 5] FIG. 2 is a schematic diagram of a hash table provided by an embodiment of the present disclosure. [Figure 6] 4 is a flowchart of a data processing method provided by another embodiment of the present disclosure. [Figure 7] FIG. 2 is an architecture diagram of an acceleration core provided by an embodiment of the present disclosure. [Figure 8] 1 is a schematic configuration diagram of a data processing device provided by an embodiment of the present disclosure. [Figure 9] FIG. 1 is a schematic configuration diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0015] In order to make the above-mentioned objectives, features and advantages of the present disclosure more clearly understood, the solutions of the present disclosure are further described below. It should be noted that, without contradiction, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0016] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure, but the present disclosure may be practiced in ways other than those described herein, and it is apparent that the examples in this specification are merely some examples of the present disclosure, and not all examples.

[0017] Databases are an important workload for artificial intelligence, cloud computing, big data processing, etc. With the rapid development of Internet technology and the massive generation of data, the demands on database performance are increasing. With the development of technology, disk read / write capacity and network bandwidth performance are constantly improving, and as a result, the performance bottleneck in the database field has shifted to computing power.

[0018] In the prior art, data is generally processed by a central processing unit (CPU), but this processing method does not provide sufficient computing power to support large-volume data services. Therefore, how to increase computing power and improve data processing capacity is an urgent issue to be resolved. To address this issue, an embodiment of the present disclosure provides a data processing method, which will be described below with reference to a specific embodiment.

[0019] The method can be executed by a data processing device that can be implemented in software and / or hardware, and the data processing device can be installed in an electronic device such as a server or a terminal, where the terminal specifically includes a mobile phone, a personal computer, a tablet computer, etc. The method can also be applied to an application scenario shown in Figure 2, which includes a central processing unit (CPU) 21 and an acceleration card 22. It can be understood that the data processing method provided by the embodiments of the present disclosure can also be applied to other scenarios.

[0020] 1 will be described below in accordance with the application scenario shown in Fig. 2, for example, the acceleration card 22 in Fig. 2 can implement the method. The method includes the following specific steps:

[0021] S101, receiving input data transmitted by a central processing unit;

[0022] The central processing unit (CPU) is the core of a computer system's calculations and control, and is the final execution unit for information processing and program execution.

[0023] As shown in FIG. 2, after the CPU obtains the input data, it transmits the input data to the acceleration card, and the acceleration card receives the input data transmitted by the CPU.

[0024] S102: A hash join operation is performed on the input data to obtain a target operation result, and the target operation result is set as output data.

[0025] Hash join algorithms are used to perform natural joins and other join operations. The idea behind hash join algorithms is to partition the tuples of each given relation into sets. The partitions are those that are completed based on the same hash value on the join property. A hash function provides the hash value. The main purpose of using a hash function in an algorithm is to reduce the number of comparisons and improve the efficiency of the join operation on the completed relation.

[0026] The acceleration card performs a hash join operation on the input data to obtain a target operation result, and sets the target operation result as output data.

[0027] S103, transmitting the output data to the central processing unit.

[0028] The acceleration card transmits the output data to the CPU, makes the CPU obtain the output data, and performs storage and management for the output data.

[0029] In an embodiment of the present disclosure, by receiving input data transmitted by a central processing unit, the input data is transmitted to an acceleration card, and a hash join operation is performed on the input data, i.e., computing power is improved through parallel computing, ensuring that the acceleration card processes the input data in real time, obtaining a target computation result for the input data, and using the target computation result as output data, improving data processing efficiency, transmitting the output data to the central processing unit, allowing the central processing unit to obtain the output data, facilitating the storage and management of the output data by the central processing unit, and improving data management capabilities.

[0030] In some embodiments, the input data and the output data are column data, and the input data and the output data are in an array format.

[0031] The input data and output data are column data, specifically, one-dimensional column data such as Parquet. The input data and output data are array-shaped, specifically, the input data type includes the data length of the input data, the operation operator type of the input data, and the data type of the input data. The output data type includes the data length of the output data, the operation operator of the output data, and the data type of the output data.

[0032] Here, the method further includes connecting to a central processing unit before receiving input data transmitted by the central processing unit, the central processing unit being used to obtain the input data and transmit the input data to the acceleration card.

[0033] PCIE is a high-speed serial point-to-point dual-channel high-bandwidth transmission standard. Connected devices are assigned dedicated channel bandwidth and do not share bus bandwidth. It mainly supports features such as active power management, error reporting, end-to-end reliable transmission, hot swapping, and Quality of Service (QOS), such as video cards.

[0034] The acceleration card is inserted into the CPU server through a standard peripheral component interconnect express (PCIE) slot to establish a connection between the acceleration card and the central processing unit. After the CPU acquires input data, it transmits the input data to the acceleration card, and the acceleration card receives the input data transmitted by the CPU.

[0035] Optionally, said acceleration card includes an acceleration core and a memory, wherein first cache data is stored in said acceleration core, and said input data, said output data and second cache data are stored in said memory.

[0036] Specifically, as shown in Figure 3, the acceleration card includes an acceleration core and a memory, specifically, the memory may be a double-rate synchronous dynamic random access memory (DDR), and the acceleration core may be a hash-combined core, where the first cache data is stored in the acceleration core, and the input data, output data, and second cache data are stored in the memory DDR.

[0037] Here, performing a hash join operation on the input data to obtain a target operation result includes the acceleration core reading the input data from the memory, calculating a hash value of the input data and the number of the hash values, establishing a hash table based on the hash value and the number of the hash values, and performing a hash join operation on the input data based on the hash table to obtain a target operation result.

[0038] Specifically, as shown in Figure 3, after the CPU receives input data, it transmits the input data to the memory in the acceleration card. The memory receives the input data transmitted from the CPU. The acceleration core in the acceleration card reads the input data and calculates the hash value and the number of hash values ​​of the input data. The number of hash values ​​can be said to be the depth of the hash bucket. According to the hash value and the number of hash values, a hash table is established. A hash join operation is performed on the input data based on the hash table, thereby obtaining intermediate operation results and target operation results. The intermediate operation results include primary cache data and secondary cache data. The primary cache data is stored in the acceleration core, and the secondary cache data is stored in the memory DDR. The target operation result is output data. The acceleration core sends the output data to the memory in the acceleration card, so that the CPU can obtain the output data through the acceleration card.

[0039] Optionally, calculating a hash value of the input data and the number of hash values ​​comprises calculating a hash value of the input data and the number of hash values ​​based on the input data and a parameter set.

[0040] Specifically, the parameter set is set in advance, and the hash value of the input data and the number of hash values ​​can be calculated based on the input data and the parameter set.

[0041] For example, as shown in FIG. 4, if the input data is [1,2,7,9,5,1,4,2], the index value of the input data is [0,1,2,3,4,5,6,7]. Based on the parameter set and the input data, the number of hash values ​​is determined to be 4, and the hash value of the input data is calculated as [1,2,3,1,1,1,0,2].

[0042] Optionally, establishing a hash table based on the hash value and the number of hash values ​​includes determining a first cache depth and a second cache depth for the input data based on the hash value and the number of hash values, the first cache depth being a hash bucket corresponding to cache data and the second cache depth being a hash bucket corresponding to overflow data, and establishing a hash table based on the hash value, the first cache depth, and the second cache depth.

[0043] Specifically, the first cache depth and the second cache depth of the input data are determined based on the input data, the hash value of the input data, and the index value of the input data, where the first cache depth is the hash bucket corresponding to the first cache data, the second cache depth is the hash bucket corresponding to the second cache data, and the second cache data can also be called overflow data, and a hash table is established based on the hash value, the first cache depth, and the second cache depth.

[0044] For example, if the input data is [1,2,7,9,5,1,4,2] as shown in FIG. 4, a hash table is established based on the hash value of the input data, the primary cache depth, and the secondary cache depth. As shown in FIG. 5, the number of hash values ​​in the hash table is 4, the primary cache depth is 2, and the secondary cache depth is 4. An input data depth counter, i.e., a hash bucket depth counter, is calculated based on the primary cache depth of the input data and the secondary cache depth of the input data. Specifically, the input data depth count is the sum of the primary cache depth of the input data and the secondary cache depth of the input data.

[0045] The embodiments of the present disclosure improve computing power through acceleration card parallel operation, realize pipeline processing through acceleration card hardware logic rather than instructions, ensure that the acceleration core processes data every clock cycle, i.e., the acceleration core processes data in real time, and combine memory and cache to ensure that the acceleration performance of the system is improved without data overflow.

[0046] FIG. 6 is a flowchart of a data processing method provided by another embodiment of the present disclosure. As shown in FIG. 6, the method includes the following steps:

[0047] S601, connected to a central processing unit, the central processing unit is used to obtain input data and transmit the input data to an acceleration card, the acceleration card including an acceleration core and a memory;

[0048] The acceleration card is inserted into the CPU server through a standard peripheral component interconnect express (PCIE) slot to establish a connection between the acceleration card and the central processing unit. After the CPU acquires input data, it transmits the input data to the acceleration card, which then receives the input data transmitted by the CPU. Here, the acceleration card includes an acceleration core and a memory. Specifically, the memory may be a double-rate synchronous dynamic random access memory (DDR), and the acceleration core may be a hash-combined core.

[0049] S602, receiving input data transmitted by the central processing unit.

[0050] After the CPU obtains the input data, it transmits the input data to the memory in the acceleration card, and the memory receives the input data transmitted by the CPU.

[0051] S603, the acceleration core reads the input data from the memory.

[0052] The acceleration core in the acceleration card reads input data from the memory.

[0053] S604: calculating a hash value of the input data and the number of hash values ​​according to the input data and the parameter set;

[0054] The parameter set is set in advance, and the hash value of the input data and the number of hash values ​​can be calculated based on the input data and the parameter set.

[0055] For example, as shown in FIG. 4, if the input data is [1,2,7,9,5,1,4,2], the index value of the input data is [0,1,2,3,4,5,6,7]. Based on the parameter set and the input data, the number of hash values ​​is determined to be 4, and the hash value of the input data is calculated as [1,2,3,1,1,1,0,2].

[0056] S605, establishing a hash table based on the hash value and the number of the hash values.

[0057] Optionally, determining a first cache depth and a second cache depth for the input data based on the hash value and the number of hash values, wherein the first cache depth is a hash bucket corresponding to first cache data and the second cache depth is a hash bucket corresponding to second overflow data, and establishing a hash table based on the hash value, the first cache depth and the second cache depth.

[0058] Specifically, the first cache depth and the second cache depth of the input data are determined based on the input data, the hash value of the input data, and the index value of the input data, where the first cache depth is the hash bucket corresponding to the first cache data, the second cache depth is the hash bucket corresponding to the second cache data, and the second cache data can also be called overflow data, and a hash table is established based on the hash value, the first cache depth, and the second cache depth.

[0059] For example, if the input data is [1,2,7,9,5,1,4,2] as shown in FIG. 4, a hash table is established based on the hash value of the input data, the primary cache depth, and the secondary cache depth. As shown in FIG. 5, the number of hash values ​​in the hash table is 4, the primary cache depth is 2, and the secondary cache depth is 4. An input data depth counter, i.e., a hash bucket depth counter, is calculated based on the primary cache depth of the input data and the secondary cache depth of the input data. Specifically, the input data depth count is the sum of the primary cache depth of the input data and the secondary cache depth of the input data.

[0060] S606: A hash join operation is performed on the input data based on the hash table to obtain a target operation result, and the target operation result is set as output data.

[0061] Based on the hash table, a hash join operation is performed on the input data, thereby obtaining an intermediate operation result and a target operation result, the intermediate operation result being the first cache data and the second cache data, the first cache data being stored in the acceleration core, and the second cache data being stored in the memory DDR, the target operation result being obtained by calculation using the input data, the first cache data, and the second cache data, the target operation result being the output data, and the acceleration core sends the output data to the memory in the acceleration card.

[0062] S607, transmitting the output data to the central processing unit.

[0063] As the industry's demand for computing power grows, more and more computing platforms have begun to introduce a variety of different computing units, such as CPUs, GPUs, ASICs, and FPGAs, to accelerate computing, and heterogeneous computing has emerged accordingly. The core of heterogeneous computing is the word "heterogeneous," which means combining hardware with different process architectures, different instruction sets, and different functions to improve computing power. Examples include chip-level (SoC) heterogeneous computing, board-level integrated heterogeneous computing, and system-level heterogeneous computing.

[0064] The memory transmits the output data to the central processing unit through the PCIE slot, allowing the CPU to obtain the output data, thereby completing heterogeneous computing and performing storage and management for the output data.

[0065] In an embodiment of the present disclosure, by receiving input data transmitted by a central processing unit, the input data is transmitted to an acceleration card, and a hash join operation is performed on the input data, i.e., computing power is improved through parallel computing, ensuring that the acceleration card processes the input data in real time, obtaining a target computation result for the input data, and using the target computation result as output data, improving data processing efficiency, transmitting the output data to the central processing unit, allowing the central processing unit to obtain the output data, facilitating the storage and management of the output data by the central processing unit, and improving data management capabilities.

[0066] FIG. 7 is an architecture diagram of an acceleration core provided by an embodiment of the present disclosure. As shown in FIG. 7, the hash join operator includes input data A, input data B, a parameter set, second-level cache data, output data C, and output data D. Here, the input data A, input data B, output data C, and output data D are all column data, and may be one-dimensional column data, such as Puaquet. The second-level cache data is a two-dimensional data structure hash table that can be indexed by hash values. The parameter set includes the length of the data and an arithmetic operator.

[0067] The hash join operation of the acceleration core is divided into two stages: the establishment stage and the detection stage.

[0068] In the establishment phase, a hash value and the number of hash values ​​are calculated based on the input data A and a parameter set. The number of hash values ​​is the depth of the hash bucket. Based on the hash value, the input data A and the index number where the input data A is located are written into a hash table. The hash table is stored in the primary cache data and the secondary cache data. When the hash bucket corresponding to the primary cache data is full, the hash table overflows and stores the secondary cache data. At the same time, the depth count records the hash bucket depth corresponding to the primary cache data and the secondary cache data, i.e., the primary cache depth and the secondary cache depth. Here, the primary cache data and the secondary cache data have the same data structure. Generally, the capacity of the accelerator core processor is smaller than the capacity of memory, and the accelerator core processor has a faster access speed than memory. The purpose of the secondary cache data is to prevent hash bucket overflow. If the hash table does not overflow into the secondary cache data, better performance is achieved.

[0069] In the detection stage, a hash value is calculated based on input data B, the first cache data, and the second cache data, and a depth count is searched for based on the hash value. If the depth count value is greater than 0, input data A and its index number, input data B and its index number are obtained from the first cache data and the second cache data. If the output data and the input data are equal, the index number of input data A is output to output data C, and the index number of input data B is output to output data D.

[0070] For example, input data A is [1,2,7,9,5,1,4,2], input data B is [3,5,1,10,3,1], and it is determined based on the input data A and the parameter set that the number of hash values ​​is 4, the primary cache depth is 2, and the secondary cache depth is 4. The hash join algorithm is set to hash=data%4. The hash table generated in the establishment phase will be as shown in FIG. 5, where each row is a hash bucket corresponding to a hash value. A hash value is calculated for input data A and then stored in the corresponding hash bucket. Each rectangular box represents a storage location, the value in the upper left is the input data, and the value in the lower right is an index value in the array. For example, the hash bucket corresponding to hash value 0 is 1, the hash bucket corresponding to hash value 1 is 4, the hash bucket corresponding to hash value 2 is 2, and the hash bucket corresponding to hash value 3 is 1. In this case, the hash bucket corresponding to the second row (i.e., the hash value is 1) is used. When the primary cache is full, new data will overflow and be stored in the secondary cache. Input data A goes through the establishment phase to obtain the above hash table structure, and the depth calculation also obtains an array result, corresponding to the depth of each hash bucket: [1, 4, 2, 1]. In the detection phase, based on the hash value calculated based on input data B, the first cache data, and the second cache data, the hash table is searched for a matching result, and output data C: [4, 0, 5, 0, 5] corresponding to input data A and output data D: [1, 2, 2, 5, 5] corresponding to input data B are obtained.

[0071] FIG. 8 is a schematic diagram of a data processing device provided by an embodiment of the present disclosure. The data processing device may be the terminal described in the above embodiment, or a component or assembly within the terminal. The data processing device provided by the embodiment of the present disclosure can execute the processing flow provided by the embodiment of the data processing method. As shown in FIG. 8, a data processing device 80 includes a receiving module 81, an operation module 82, and a transmission module 83. Here, the receiving module 81 is used to receive input data transmitted by a central processing unit, the operation module 82 is used to perform a hash join operation on the input data to obtain a target operation result and use the target operation result as output data, and the transmission module 83 is used to transmit the output data to the central processing unit.

[0072] Optionally, said input data and said output data are column data, and the form of said input data and said output data is array form.

[0073] Optionally, the data processing device 80 further includes a connection module 84 that is also used to connect with a central processing unit, which is used to obtain input data and transmit the input data to the acceleration card.

[0074] Optionally, said acceleration card includes an acceleration core and a memory, wherein first cache data is stored in said acceleration core, and said input data, said output data and said second cache data are stored in memory.

[0075] Optionally, the calculation module 82 is also used by the acceleration core to read the input data from the memory, calculate a hash value and the number of hash values ​​of the input data, establish a hash table based on the hash value and the number of hash values, and perform a hash join operation on the input data based on the hash table to obtain a target calculation result.

[0076] Optionally, the calculation module 82 is also used to calculate a hash value and a number of hash values ​​of the input data based on the input data and a parameter set.

[0077] Optionally, the calculation module 82 determines a first cache depth and a second cache depth of the input data based on the hash value and the number of the hash values, where the first cache depth is a hash bucket corresponding to the first cache data, and the second cache depth is a hash bucket corresponding to the second overflow data, and is also used to establish a hash table based on the hash value, the first cache depth, and the second cache depth.

[0078] The data processing device of the embodiment shown in FIG. 8 can be used to implement the technical solutions of the data processing method embodiment described above, and the principles and technical effects thereof are similar, so they will not be further detailed here.

[0079] 7 is a schematic diagram of an electronic device provided by an embodiment of the present disclosure. The electronic device may be a terminal described in the above embodiment. The electronic device provided by the embodiment of the present disclosure can execute the processing flow provided by the embodiment of the data processing method. As shown in FIG. 7, an electronic device 70 includes a memory 71, a processor 72, a computer program, and a communication interface 73. Here, the computer program is stored in the memory 71 and configured to execute the above-mentioned data processing method by the processor 72.

[0080] Furthermore, the embodiments of the present disclosure further provide a computer-readable recording medium having a computer program stored thereon, the computer program being executed by a processor to realize the data processing method described in the above embodiments.

[0081] Moreover, embodiments of the present disclosure further provide a computer program product including a computer program or instructions, which, when executed by a processor, implements the data processing method described above.

[0082] It should be noted that the computer-readable medium described above in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of the computer-readable storage medium may be, but are not limited to, one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program, and the program is used in or in combination with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a propagating data signal in baseband or as part of a carrier wave that carries computer-readable program code. The propagated data signal may take a variety of forms, including, but not limited to, electromagnetic, optical, or any suitable combination of the above.

[0083] A computer-readable signal medium may be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained in the computer-readable medium may be transmitted over any suitable medium, including, but not limited to, wire, fiber optic cable, RF (radio frequency), etc., or any suitable combination of the above.

[0084] In some embodiments, clients and servers may communicate using any network protocol now known or later developed, such as HyperText Transfer Protocol (HTTP), and may be interconnected for digital data and communications in any form or medium (e.g., a communications network). Examples of communications networks include local area networks (LANs), wide area networks (WANs), extranets (e.g., the Internet), and end-to-end networks (e.g., ad-hoc end-to-end networks), now known or later developed networks.

[0085] The computer-readable medium may be included in the electronic device, or may exist independently of the electronic device.

[0086] The computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to receive input data transmitted by a central processing unit, perform a hash join operation on the input data to obtain a target operation result, obtain the target operation result as output data, and transmit the output data to the central processing unit.

[0087] Furthermore, the electronic device can also perform other steps in the data processing methods described above.

[0088] Computer program code for carrying out the operations of the present disclosure can be written in one or more programming languages, or combinations thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer, partially on a remote computer, or entirely on a remote computer or server. When referring to a remote computer, the remote computer can be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., connected via the Internet using an Internet Service Provider).

[0089] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code, which includes one or more executable instructions for implementing a given logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in an order different from that shown in the figures. For example, two blocks shown in succession may actually be executed substantially in parallel or may even be executed in the reverse order, as dictated by the functionality involved. Furthermore, each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented in a dedicated hardware-based system that performs the specified functions or operations, or may be implemented in a combination of dedicated hardware and computer instructions.

[0090] The units according to the embodiments of the present invention may be realized by software or hardware, and the names of the units may not necessarily be limiting of the units themselves.

[0091] The functions described herein may be performed at least in part by one or more hardware logic elements. For example, without limitation, exemplary types of hardware logic elements that may be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), etc.

[0092] In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain or store a program used by an instruction execution system, device, or apparatus, or used in conjunction with an instruction execution system or in combination with a device or apparatus. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination of the above. More specific examples of machine-readable storage media include one or more wire-based electrical connections, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0093] As used herein, relational terms such as "first" and "second" are used solely to distinguish one entity or operation from another and do not necessarily require or imply that any such actual relationship or order exists between those entities or operations. Furthermore, the terms "comprise," "comprises," or any other variation thereof are intended to cover a non-exclusive inclusion; thus, a process, method, article, or apparatus comprising a set of elements may include not only those elements but also other elements not expressly listed or elements inherent in such process, method, article, or apparatus. Absent further limitations, an element qualified by the phrase "comprises a..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0094] The foregoing description is merely a specific embodiment of the present disclosure, so that those skilled in the art can understand or practice the present disclosure. Although various modifications to these examples will be apparent to those skilled in the art, the general principles defined herein may be implemented in other examples without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure is not limited to the examples described herein, but rather is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0095] (Addendum) (Appendix 1) 1. A data processing method comprising: receiving input data transmitted by a central processing unit; performing a hash join operation on the input data to obtain a target operation result, and setting the target operation result as output data; transmitting said output data to said central processing unit.

[0096] (Appendix 2) 2. The method according to claim 1, wherein the input data and the output data are column data, and the input data and the output data are in array format.

[0097] (Appendix 3) The method described in Appendix 2, characterized in that the method further includes connecting to a central processing unit before receiving the input data transmitted by the central processing unit, the central processing unit being used to obtain the input data and transmit the input data to the acceleration card.

[0098] (Appendix 4) The method described in Appendix 3, characterized in that the acceleration card includes an acceleration core and memory, first cache data is stored in the acceleration core, and the input data, output data, and second cache data are stored in the memory.

[0099] (Appendix 5) performing a hash join operation on the input data to obtain a target operation result, the acceleration core reading the input data from the memory; calculating a hash value of the input data and a number of the hash values; establishing a hash table based on the hash values ​​and the number of hash values; 5. The method of claim 4, further comprising: performing a hash join operation on the input data based on the hash table to obtain a target operation result.

[0100] (Appendix 6) 6. The method of claim 5, wherein calculating the hash value and the number of hash values ​​of the input data includes calculating the hash value and the number of hash values ​​of the input data based on the input data and a parameter set.

[0101] (Appendix 7) Establishing a hash table based on the hash values ​​and the number of hash values ​​includes: Determine a first cache depth and a second cache depth of the input data according to the hash value and the number of the hash values, where the first cache depth is a hash bucket corresponding to the first cache data, and the second cache depth is a hash bucket corresponding to the second cache data; and establishing a hash table based on the hash value, the first cache depth, and the second cache depth.

[0102] (Appendix 8) An electronic device, Memory and a processor; a computer program; The electronic device is characterized in that the computer program is stored in the memory and is configured to be executed by the processor to realize the data processing method according to any one of Supplementary Notes 1 to 7.

[0103] (Appendix 9) A computer-readable recording medium having a computer program stored thereon, the computer program being characterized in that, when executed by a processor, it realizes the data processing method described in any one of Appendices 1 to 7.

Claims

1. 1. A data processing method comprising: receiving input data transmitted by a central processing unit; performing a hash join operation on the input data to obtain a target operation result, and setting the target operation result as output data; and transmitting the output data to the central processing unit; the input data and the output data are column data, and the input data and the output data are in an array format; before receiving the input data transmitted by the central processing unit, further including connecting to a central processing unit, the central processing unit being used to obtain the input data and transmit the input data to an acceleration card; the acceleration card includes an acceleration core and a memory, the first cache data is stored in the acceleration core, and the input data, the output data, and the second cache data are stored in the memory; A data processing method, wherein performing a hash join operation on the input data to obtain a target operation result includes the acceleration core reading the input data from the memory, calculating a hash value of the input data and the number of the hash values, establishing a hash table based on the hash value and the number of the hash values, and performing a hash join operation on the input data based on the hash table to obtain a target operation result.

2. 2. The method of claim 1, wherein calculating the hash value and the number of hash values ​​of the input data comprises calculating the hash value and the number of hash values ​​of the input data based on the input data and a parameter set.

3. Establishing a hash table based on the hash values ​​and the number of hash values ​​includes: Determine a first cache depth and a second cache depth of the input data according to the hash value and the number of the hash values, where the first cache depth is a hash bucket corresponding to the first cache data, and the second cache depth is a hash bucket corresponding to the second cache data; 2. The method of claim 1, further comprising establishing a hash table based on the hash value, the first cache depth, and the second cache depth.

4. An electronic device, Memory and a processor; a computer program; The computer program is stored in the memory and configured to be executed by the processor to implement the data processing method according to any one of claims 1 to 3.

5. 4. A computer-readable recording medium having a computer program stored thereon, the computer program being characterized in that, when executed by a processor, the computer program realizes the data processing method according to any one of claims 1 to 3.

Images