Data distribution method and device, equipment, medium and product

Abstract

The invention discloses a data distribution method and device, equipment, a medium and a product, and the data distribution method comprises the steps: in response to the completion of hash table caching at a data caching side and the total number of data caching being zero, checking whether a first data distribution operator exists at a data detection side; if the first data distribution operator exists and the first distribution type of the first data distribution operator is the target type, modifying the state of the to-be-modified signal mark of the thread to which the first data distribution operator belongs; in response to a second data distribution operator on the data detection side, if a second distribution type of the second data distribution operator is a target type, obtaining a target signal mark of a target thread of a data receiver; and according to the state marked by the target signal, determining a distribution operation on the data sent to the target thread. According to the method, the signal mark state of the thread with the total data cache number being zero is modified so as to prompt in advance through the signal mark, unnecessary data sending is reduced, and then the query performance is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of distributed databases, and in particular to a data distribution method, device, equipment, medium and product. Background Art

[0002] A hash join is a database operation that uses a hash table to implement the join operation between two tables. It is a method used to join two tables when executing SQL queries, mainly using a hash algorithm to achieve efficient data matching.

[0003] The parallel execution of hash joins requires the collaboration of multiple threads (or multiple processes) to complete the matching of records on both sides of the join. In order to ensure that the records on both sides that meet the join conditions are located in the same thread, the processor usually needs to insert a data exchange operator on one or both sides of the join.

[0004] However, in a parallel execution framework, since the data obtained by the threads is filtered in advance or due to the distribution characteristics of the data, there may be a situation where the cache-side data of some threads is empty but the overall data is not empty. The existing technology usually still copies and sends the data in this case, which reduces the query performance. Summary of the invention

[0005] The present invention provides a data distribution method, device, equipment, medium and product to realize the identification and advance processing of empty set threads.

[0006] According to a first aspect of the present invention, there is provided a data distribution method, comprising:

[0007] In response to the hash table caching on the data caching side being finished and the total number of data caches being zero, checking whether there is a first data distribution operator on the data detection side;

[0008] If it exists and the first distribution type of the first data distribution operator is the target type, modifying the state of the to-be-modified signal flag of the thread to which the first data distribution operator belongs;

[0009] In response to the second data distribution operator of the data detection side, if the second distribution type of the second data distribution operator is the target type, obtaining a target signal flag of a target thread of a data receiver;

[0010] A distribution operation of the data sent to the target thread is determined according to the state of the target signal flag.

[0011] According to a second aspect of the present invention, there is provided a data distribution device, comprising:

[0012] An operator determination module, configured to check whether a first data distribution operator exists on the data detection side in response to the hash table cache on the data cache side ending and the total number of data caches being zero;

[0013] a state modification module, configured to modify the state of the to-be-modified signal flag of the thread to which the first data distribution operator belongs, if the first distribution type of the first data distribution operator exists and is the target type;

[0014] a tag acquisition module, configured to respond to the second data distribution operator of the data detection side, and if the second distribution type of the second data distribution operator is the target type, acquire the target signal tag of the target thread of the data receiver;

[0015] The data distribution module is used to determine the distribution operation of the data sent to the target thread according to the state of the target signal mark.

[0016] According to a third aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the data distribution method described in any embodiment of the present invention.

[0020] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the data distribution method described in any embodiment of the present invention when executed by a processor.

[0021] According to a fifth aspect of the present invention, an embodiment of the present invention further provides a computer program product, the computer program product comprising a computer program, and when the computer program is executed by a processor, the data distribution method of any embodiment of the present invention is implemented.

[0022] The technical solution of the embodiment of the present invention is to check whether there is a first data distribution operator on the data detection side in response to the end of the hash table cache on the data cache side and the total number of data caches being zero; if there is and the first distribution type of the first data distribution operator is the target type, then modify the state of the signal mark to be modified of the thread to which the first data distribution operator belongs; in response to the second data distribution operator on the data detection side, if the second distribution type of the second data distribution operator is the target type, obtain the target signal mark of the target thread of the data receiver; and determine the distribution operation of the data sent to the target thread according to the state of the target signal mark. By modifying the signal mark state of the thread whose total number of data caches is zero, the signal mark is used to give advance prompts to other threads that target the target thread, thereby reducing unnecessary data transmission, reducing memory copying and transmission overhead, thereby improving resource utilization efficiency and improving system response time.

[0023] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 is a flow chart of a data distribution method provided according to Embodiment 1 of the present invention;

[0026] Figure 2 is a flow chart of a data distribution method provided according to Embodiment 2 of the present invention;

[0027] Figure 3 is an example flow chart of a data distribution method provided according to Embodiment 2 of the present invention;

[0028] Figure 4 is another exemplary flow chart of a data distribution method provided according to the second embodiment of the present invention;

[0029] Figure 5 is a structural diagram of a data distribution device provided according to Embodiment 3 of the present invention;

[0030] Figure 6 It is a schematic diagram of the structure of an electronic device implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0031] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0033] Embodiment 1

[0034] Figure 1 A flowchart of a data distribution method is provided for the first embodiment of the present invention. This embodiment is applicable to the case of an empty thread set when a hash connection is performed under a parallel framework. The method can be performed by a data distribution device, which can be implemented in the form of hardware and / or software, and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0035] S110 , in response to the hash table caching on the data cache side being finished and the total number of data caches being zero, checking whether there is a first data distribution operator on the data detection side.

[0036] In this embodiment, the hash connection usually divides the two tables to be connected into the data cache side and the data detection side, and combines the partition Partition to obtain the target result set. The data cache side can be understood as a table with a small number of records. The data detection side can be understood as a table with a large number of records relative to the data cache side. The hash table can be understood as a structure that stores data through key-value pairs built in memory through the table on the data cache side, and the key is mapped to the position in the hash table through a hash function to achieve fast data access. The total data cache can be understood as the amount of cached data in the hash table.

[0037] It is known that when hash join is executed in parallel, it needs to rely on multiple threads (same for multiple processes) to cooperate to complete the matching of records on both sides of the connection. In order to ensure that the records on both sides that meet the connection conditions are located in the same thread, a data exchange operator can usually be inserted on one or both sides of the connection. For example, the Send / Recv operator can be used as a data exchange operator. Then the first data distribution operator can be the Send operator.

[0038] Specifically, a hash connection can be established based on the statement input by the user. The hash connection operator can obtain records from the data cache side. For each record obtained, it is placed in the hash table of the hash connection operator itself for cache, and it is determined whether the record acquisition on the data cache side has been completed. If so, in response to the hash table cache on the data cache side being completed and the total data cache being zero, it is checked whether there is a first data distribution operator on the data detection side; if not, the records continue to be obtained from the data cache side.

[0039] S120: If it exists and the first distribution type of the first data distribution operator is the target type, modify the state of the to-be-modified signal flag of the thread to which the first data distribution operator belongs.

[0040] In this embodiment, the first distribution type can be understood as a type used to characterize the first data distribution operator. The target type can be understood as a type that requires a data distribution operation judgment operation, for example, it can include any one of by hash, by list, by range, by N. The thread can be understood as the thread where the first data distribution operator is located when the hash connection is executed in parallel. The signal flag to be modified can be understood as a flag used to characterize and distinguish whether different receiving operators receive data. The state can be understood as a state that characterizes whether data can be received.

[0041] Specifically, if it is detected that there is a first data distribution operator on the data detection side, and the first distribution type of the first data distribution operator is the target type, the thread to which the first data distribution operator belongs can be determined, and the state of the corresponding signal flag to be modified can be modified.

[0042] S130 , in response to the second data distribution operator on the data detection side, if the second distribution type of the second data distribution operator is a target type, obtaining a target signal flag of a target thread of a data receiver.

[0043] In this embodiment, the second data distribution operator can be understood as the data distribution operator on the data detection side when performing data distribution. The second distribution type can be understood as the distribution type corresponding to the second data distribution operator. The data receiver can be understood as the object receiving the query data. The target thread can be understood as the thread that feeds back the query data. The target signal mark can be understood as the signal mark of the received signal in the target thread.

[0044] Specifically, when performing data query, in response to the second data distribution operator on the data detection side, if the second distribution type of the second data distribution operator is a target type, the target signal mark of the target thread of the data receiver receiving the data can be determined first.

[0045] S140: Determine a distribution operation for data sent to a target thread according to a state of a target signal mark.

[0046] In this embodiment, the distribution operation may include discarding data or sending data.

[0047] Specifically, the second data distribution operator can notify other working threads that have data to send to the target thread according to the state of the target signal mark, and perform the distribution operation according to the state of the target signal mark. For example, if the state indicates that the target thread no longer receives data, other working threads can directly discard the data and do not perform the data publishing operation. If the state indicates that reception is allowed, other working threads can send data to the target thread normally.

[0048] The technical solution of the embodiment of the present invention is to check whether there is a first data distribution operator on the data detection side in response to the end of the hash table cache on the data cache side and the total number of data caches being zero; if there is and the first distribution type of the first data distribution operator is the target type, then modify the state of the signal mark to be modified of the thread to which the first data distribution operator belongs; in response to the second data distribution operator on the data detection side, if the second distribution type of the second data distribution operator is the target type, obtain the target signal mark of the target thread of the data receiver; determine the distribution operation of the data sent to the target thread according to the state of the target signal mark. By modifying the signal mark state of the thread whose total number of data caches is zero, and giving advance prompts to other threads that target the target thread through the signal mark, unnecessary data transmission is reduced, memory copying and transmission overhead are reduced, thereby improving resource utilization efficiency and improving system response time.

[0049] As a first optional embodiment of the first embodiment, based on the above embodiment, it also includes:

[0050] If the second distribution type of the second data distribution operator is not the target type, the data is sent to the target thread of the data receiver.

[0051] Specifically, if the second distribution type of the second data distribution operator is not a target type, other threads can normally send data to the target thread of the data receiver.

[0052] In the first optional embodiment of the first embodiment, through such a setting, when the second distribution type is not the target type, a normal data sending process is executed, thereby ensuring normal data sending.

[0053] As a second optional embodiment of the first embodiment, in response to the hash table cache on the data cache side being finished and the total number of data caches being zero, before checking whether there is a first data distribution operator on the data detection side, the method further includes:

[0054] Change the signal flags of all data receiving operators to the initial state, which is the receiving permission state.

[0055] In this embodiment, the initial state can be understood as a state indicating that the thread where the data receiving operator is located is allowed to receive data.

[0056] Specifically, in response to the hash table cache on the data cache side ending and the total data cache being zero, before checking whether there is a first data distribution operator on the data detection side, the processor can collect the data receiving operators of all subtasks and modify their corresponding signal flags to an initial state, which is a state that allows reception.

[0057] In a second optional embodiment of the first embodiment, through such a setting, the signal flag of the data receiving operator is first set to an initial state to ensure normal data transmission of other threads except when the total number of data caches is zero.

[0058] Embodiment 2

[0059] Figure 2 This is a flow chart of a data distribution method provided in Embodiment 2 of the present invention. This embodiment is a further refinement of the above embodiment. Figure 2 As shown, the method includes:

[0060] S210 : In response to the hash table caching on the data cache side being completed and the total number of data caches being zero, checking whether there is a first data distribution operator on the data detection side.

[0061] S220: If it exists and the first distribution type of the first data distribution operator is the target type, determine the thread to which the first data distribution operator belongs.

[0062] Specifically, when the hash table cache on the data cache side ends and the total number of data caches is zero, and there is a first data distribution operator on the data detection side, the processor may first determine the thread to which the first data distribution operator belongs.

[0063] S230 , searching the signal tags of each data receiving operator for a signal tag to be modified that is identical to the thread program number of the thread.

[0064] In this embodiment, the thread program number can be understood as a serial number used to distinguish different threads. For example, the signal tag can be represented by Signal(nth,s,r), where nth represents the serial number of the current thread, s represents the serial number of the current subtask, and r represents the nth Recv operator of the current subtask, and the signal tag can be seen by different threads of the same instance.

[0065] Specifically, the processor may search for a signal tag to be modified that is the same as the thread program number of the thread in the signal tags of each data receiving operator, that is, find out the signal tag to be modified that is in the same thread as the first data distribution operator.

[0066] S240: Change the state of the signal mark to be modified to a non-receiving state.

[0067] In this embodiment, the non-receiving state can be understood as a state identifier indicating that the target thread does not receive data.

[0068] Specifically, the executor may modify the state of the signal tag to be modified to a non-receiving state, so that other threads may query the non-receiving state of the signal tag to be modified and respond.

[0069] S250 , in response to the second data distribution operator on the data detection side, if the second distribution type of the second data distribution operator is a target type, obtain a cache record from the child operator.

[0070] In this embodiment, the sub-operator can be understood as the first sub-node operator under the second data distribution operator. The cache record can be understood as information related to the table and column connected to the cache, such as the partition definition or column value connected to the cache.

[0071] Specifically, when executing the data detection side, in response to the second data distribution operator of the data detection side, if the second distribution type of the second data distribution operator is the target type, the data detection side may obtain the cache record from the sub-operator.

[0072] S260: Determine the target line program number and the target thread to which the data receiver belongs for receiving the data according to the second distribution type and the cache record.

[0073] In this embodiment, the target line program number can be understood as representing the receiving thread corresponding to the data receiver.

[0074] Specifically, the data detection side can determine how to calculate the target line program number for the data receiver to receive data according to the second distribution type, and calculate the target line program number and the target thread to which the target line program number belongs through information related to the table and column connected to the cache.

[0075] For example, if the second distribution type is by hash, which means to calculate which receiver the data belongs to according to the hash partition definition of the t1 table on the other side, the target line program number can be calculated as: hash_fold(d)% number of t1 table partitions. d is the value of the connection column. Other distribution methods include by range (if t1 is range partition) and by list (if t1 is list partition). And by N method, the corresponding target line program number calculation formula is: hash_fold(d)% parallelism of subtask.

[0076] S270: Determine the target signal tag under the target thread according to the target thread program number.

[0077] Specifically, the data detection side may search based on the target thread program number to determine the target signal tag that is the same as the target thread program number (ie, belongs to the target thread).

[0078] S280: If the state of the target signal mark is a non-receiving state, determine that the target thread is an empty set thread, and discard the data sent to the target thread.

[0079] In this embodiment, the empty set thread can be understood as a thread without cache data.

[0080] Specifically, the data detection side can read the state of the target signal mark. If the state is a non-receiving state, the target thread corresponds to an empty set thread, and other threads can discard the data sent to the target thread.

[0081] S290: Otherwise, obtain the matching data to be sent from the data detection side, and send the data to be sent to the target thread.

[0082] Specifically, if the state of the target signal mark is not a non-receiving state, that is, it corresponds to a receiving-allowed state, the data to be sent can be sent to the target thread normally.

[0083] In actual execution, it is possible that data has been sent out before the status of Signal(nth,s,r) is set to "no longer received". This is because the hash table cache construction and data detection subtasks are executed in parallel. Even if the above situation occurs, it will not affect the correctness of the hash join operation, because the cache record is empty at this time, and the record received by the data detection side will be ignored.

[0084] The technical solution of the embodiment of the present invention is to check whether there is a first data distribution operator on the data detection side in response to the end of the hash table cache on the data cache side and the total number of data caches is zero. When the first data distribution operator exists on the data detection side and the type is the target type, the signal mark of the data receiving operator under the same thread as the first data distribution operator is determined as the signal mark to be modified, and the signal mark to be modified is modified to a non-receiving state as a prompt to end data transmission in advance. In response to the second data distribution operator on the data detection side, if the second distribution type of the second data distribution operator is the target type, the cache record is obtained from the sub-operator, and the target line program number of the data receiver is determined by the second distribution type and the cache record, and then the target signal mark under the target thread is determined. When the state of the target signal mark is in a non-receiving state, the distribution operation of the data sent to the target thread is determined. By setting the signal mark state of whether the data at the thread level is received, the sending operator on the data detection side of the hash connection operator can stop sending unnecessary data in a timely and effective manner, reduce memory copying and sending overhead, thereby improving resource utilization efficiency and improving system response time.

[0085] For example, taking a specific example as a demonstration, the following statements create a table t1 with columns c and c2 and 4 hash join partitions, and a table t2 with columns d and d2 and 3 hash join partitions.

[0086] create table t1(c int,c2 int)partition by hash(c)partitions4;

[0087] create table t2(d int,d2 int)partition by hash(d)partitions3;

[0088] Example 1, Figure 3 This is an example flow chart of a data distribution method provided in Embodiment 2 of the present invention, such as Figure 3 As shown, a data distribution operator can be inserted on the right side of the hash join operator, wherein the data distribution operator is represented by Send.

[0089] Based on the tables t1 and t2 in the previous article, execute the following statements:

[0090] Select*from t1,t2 where c=d;

[0091] When executing this statement, the database optimizes the execution plan of the statement. A possible plan is: data distribution is added to one side of the connection (subtask 2), and the optimization of partial partition wise join (Partition Wise Join, PWJ) is enabled. Partition wise join is used for data on both sides of the connection. It has been distributed in advance according to the connection columns in the connection equal filter conditions, and the partitioning mode and data storage on both sides are completely consistent. In this case, the processor can identify and directly connect the data on both sides without having to distribute the data. The meaning of the above plan is that the plan is split into two independent subtasks. Subtask 2 is executed by 3 threads together. After they obtain data from the t2 table, the send operator calculates which partition of the t1 table it belongs to according to the value of the d column in the connection condition, and then the data is handed over to the data receiving operator of the corresponding thread of subtask 1. The data receiving operator is represented by the Recv operator. The parallelism of subtask 1 is set to 4, which is the same as the number of subtables of the t1 table. This ensures that each thread scans a subtable of t1, and after constructing the hash table of Hash innerjoin, it receives the records belonging to the current thread sent by the three threads in subtask 2 at the Recv operator. In the above example, the send distribution method of subtask 2 is by hash, which means that the data belongs to which receiver according to the hash partition definition of the other side t1 table. The send calculation method can be: hash_fold(d) % the number of partitions of t1 table. Other distribution methods include by range (if t1 is range partitioned) and by list (if t1 is list partitioned).

[0092] If thread 3# of subtask 1 finds that the hash table of Hash inner join is empty, that is, no data is included, the signal mark of Recv in thread 3# can be used as a signal mark to be modified, and its state can be changed to a non-receiving state. Then, the signal mark can be used to notify the three working threads of subtask 2. All records that need to be sent to thread 3# in subtask 1 after calculation can be directly discarded, and the relevant data on the other side of the connection can be sent to indicate early termination, thereby reducing unnecessary data transmission.

[0093] Example 2, Figure 4 Another example flow chart of a data distribution method provided in Embodiment 2 of the present invention is as follows: Figure 4 As shown, data distribution operators are inserted on both sides of the hash join operator.

[0094] Based on the tables t1 and t2 in the previous article, execute the following statements:

[0095] Select*from t1,t2 where c2=d2;

[0096] Compared with Example 1, data distribution is introduced on both sides of the hash inner join. The data distribution operator (send operator) in subtask 1 and subtask 2 performs hash value modulo operation according to the value of the connection column and the number of receiver threads 2 (the parallelism of subtask 3). The target line program number in send1 is calculated as follows: hash_fold(c2)%subtask 3 parallelism=i, that is, the data is sent to the corresponding thread No. i of subtask 3. Similarly, the target line program number calculation formula in send2 is: hash_fold(d2)%subtask 3 parallelism. The send sending method is by N, that is, the number of target receiver threads is used to calculate which thread of the corresponding data receiving operator the data belongs to, which is similar to by hash. In Example 2, if thread 1# of subtask 3 finds that no records have been collected by data receiving operator 1 (i.e. recv1), it can change the state of the signal mark to be modified of recv1 to the non-receiving state, and notify the two working threads of subtask 2 through the signal mark of recv1 to directly discard all data that need to be sent to thread 1# of subtask 3, so as to send a prompt of early termination of the relevant data on the other side of the connection, thereby reducing unnecessary data transmission.

[0097] Embodiment 3

[0098] Figure 5 This is a schematic diagram of the structure of a data distribution device provided in Embodiment 3 of the present invention. Figure 5 As shown, the device includes: an operator determination module 51, a state modification module 52, a tag acquisition module 53 and a data distribution module 54.

[0099] An operator determination module 51 is used to check whether there is a first data distribution operator on the data detection side in response to the hash table cache on the data cache side being finished and the total number of data caches being zero;

[0100] a state modification module 52, configured to modify the state of the to-be-modified signal flag of the thread to which the first data distribution operator belongs, if the first distribution type of the first data distribution operator exists and is the target type;

[0101] a tag acquisition module 53, configured to respond to the second data distribution operator of the data detection side, and if the second distribution type of the second data distribution operator is the target type, acquire the target signal tag of the target thread of the data receiver;

[0102] The data distribution module 54 is used to determine the distribution operation of the data sent to the target thread according to the state of the target signal mark.

[0103] The technical solution of the embodiment of the present invention is to check whether there is a first data distribution operator on the data detection side in response to the end of the hash table cache on the data cache side and the total number of data caches being zero; if there is and the first distribution type of the first data distribution operator is the target type, then modify the state of the signal mark to be modified of the thread to which the first data distribution operator belongs; in response to the second data distribution operator on the data detection side, if the second distribution type of the second data distribution operator is the target type, obtain the target signal mark of the target thread of the data receiver; and determine the distribution operation of the data sent to the target thread according to the state of the target signal mark. By modifying the signal mark state of the thread whose total number of data caches is zero, the signal mark is used to give advance prompts to other threads that target the target thread, thereby reducing unnecessary data transmission, reducing memory copying and transmission overhead, thereby improving resource utilization efficiency and improving system response time.

[0104] Furthermore, the state modification module 52 is specifically used for:

[0105] Determine the thread to which the first data distribution operator belongs;

[0106] Searching the signal tag to be modified which is the same as the thread program number of the thread in the signal tags of each data receiving operator;

[0107] The state of the signal mark to be modified is changed to a non-receiving state.

[0108] Furthermore, the tag acquisition module 53 is specifically used for:

[0109] Get cache records from child operators;

[0110] Determine, according to the second distribution type and the cache record, a target line program number and a target thread to which the data receiver belongs to receive the data;

[0111] According to the target thread program number, a target signal tag under the target thread is determined.

[0112] Furthermore, the data distribution module 54 is specifically used for:

[0113] If the state of the target signal mark is a non-receiving state, determining that the target thread is an empty set thread, and discarding the data sent to the target thread;

[0114] Otherwise, the matching data to be sent is obtained from the data detection side, and the data to be sent is sent to the target thread.

[0115] Optionally, the device further comprises:

[0116] A data sending module is used to send data to a target thread of a data receiver if the second distribution type of the second data distribution operator is not the target type.

[0117] Optionally, the device further comprises:

[0118] A state initialization module is used to modify the signal marks of all data receiving operators to an initial state before checking whether there is a first data distribution operator on the data detection side in response to the hash table cache on the data cache side ending and the total data cache being zero, wherein the initial state is a reception-allowed state.

[0119] The data distribution device provided in the embodiment of the present invention can execute the data distribution method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0120] Embodiment 4

[0121] Figure 6 A schematic diagram of the structure of an electronic device 60 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0122] like Figure 6 As shown, the electronic device 60 includes at least one processor 61, and a memory connected to the at least one processor 61, such as a read-only memory (ROM) 62, a random access memory (RAM) 63, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 61 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 62 or the computer program loaded from the storage unit 68 to the random access memory (RAM) 63. In the RAM 63, various programs and data required for the operation of the electronic device 60 can also be stored. The processor 61, the ROM 62, and the RAM 63 are connected to each other via a bus 64. An input / output (I / O) interface 65 is also connected to the bus 64.

[0123] A number of components in the electronic device 60 are connected to the I / O interface 65, including: an input unit 66, such as a keyboard, a mouse, etc.; an output unit 67, such as various types of displays, speakers, etc.; a storage unit 68, such as a disk, an optical disk, etc.; and a communication unit 69, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 69 allows the electronic device 60 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0124] The processor 61 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 61 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 61 performs the various methods and processes described above, such as the data distribution method.

[0125] In some embodiments, the data distribution method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 68. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 60 via the ROM 62 and / or the communication unit 69. When the computer program is loaded into the RAM 63 and executed by the processor 61, one or more steps of the data distribution method described above may be performed. Alternatively, in other embodiments, the processor 61 may be configured to perform the data distribution method in any other suitable manner (e.g., by means of firmware).

[0126] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0127] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0128] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0129] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0130] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0131] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0132] In one embodiment, the embodiment of the present invention further includes a computer program product, the computer program product includes a computer program, and the computer program implements the data distribution method of any embodiment of the present invention when executed by a processor.

[0133] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0134] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0135] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A data distribution method, characterized in that: include: In response to the hash table caching on the data caching side being finished and the total number of data caches being zero, checking whether there is a first data distribution operator on the data detection side; If it exists and the first distribution type of the first data distribution operator is the target type, modifying the state of the to-be-modified signal flag of the thread to which the first data distribution operator belongs; In response to the second data distribution operator of the data detection side, if the second distribution type of the second data distribution operator is the target type, obtaining a target signal flag of a target thread of a data receiver; A distribution operation of the data sent to the target thread is determined according to the state of the target signal flag.

2. The method according to claim 1, characterized in that The modifying the state of the to-be-modified signal mark of the thread to which the first data distribution operator belongs includes: Determine the thread to which the first data distribution operator belongs; Searching the signal tag to be modified which is the same as the thread program number of the thread in the signal tags of each data receiving operator; The state of the signal mark to be modified is changed to a non-receiving state.

3. The method according to claim 1, characterized in that The step of obtaining a target signal mark of a target thread of a data receiver includes: Get cache records from child operators; Determine, according to the second distribution type and the cache record, a target line program number and a target thread to which the data receiver belongs to receive the data; According to the target thread program number, a target signal tag under the target thread is determined.

4. The method according to claim 1, characterized in that: The step of determining the distribution operation of the data sent to the target thread according to the state of the target signal mark includes: If the state of the target signal mark is a non-receiving state, determining that the target thread is an empty set thread, and discarding the data sent to the target thread; Otherwise, the matching data to be sent is obtained from the data detection side, and the data to be sent is sent to the target thread.

5. The method according to claim 1, characterized in that Also includes: If the second distribution type of the second data distribution operator is not the target type, the data is sent to the target thread of the data receiver.

6. The method according to claim 1, characterized in that Before checking whether there is a first data distribution operator on the data detection side in response to the hash table cache on the data cache side being finished and the total number of data caches being zero, the method further includes: The signal flags of all data receiving operators are modified to an initial state, where the initial state is a reception-allowed state.

7. A data distribution device, characterized in that: include: An operator determination module, configured to check whether a first data distribution operator exists on the data detection side in response to the hash table cache on the data cache side ending and the total number of data caches being zero; a state modification module, configured to modify the state of the to-be-modified signal flag of the thread to which the first data distribution operator belongs, if the first distribution type of the first data distribution operator exists and is the target type; a tag acquisition module, configured to, in response to the second data distribution operator of the data detection side, acquire a target signal tag of a target thread of a data receiver if the second distribution type of the second data distribution operator is the target type; The data distribution module is used to determine the distribution operation of the data sent to the target thread according to the state of the target signal mark.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the data distribution method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data distribution method according to any one of claims 1 to 6 when executed.

10. A computer program product, characterized in that The computer program product comprises a computer program, which, when executed by a processor, implements the data distribution method according to any one of claims 1 to 6.

Images