Signal interaction implementation method, device, equipment and computer-readable storage medium

Abstract

The present invention relates to the field of financial technology, and discloses a method, device, equipment and computer-readable storage medium for implementing signal interaction. The method includes: receiving a signal sent by an upstream workflow, encapsulating the signal once to obtain first signal data, and determining a target signal processing cluster corresponding to the first signal data; performing a secondary encapsulation of the first signal data by the target signal processing cluster to obtain second signal data; determining a target storage partition corresponding to the second signal data based on the data range of each pre-constructed partition in the target signal processing cluster; writing the second signal data into the target storage partition, and generating corresponding metadata information; upon receiving a signal reception request sent by a downstream workflow based on a signal, querying the target storage location from the metadata information, and then obtaining the signal and returning it to the downstream workflow. The present invention can improve the flexibility of workflow signal interaction while ensuring the performance of signal interaction in high-concurrency scenarios.

Description

Technical Field

[0001] The present invention relates to the technical field of financial technology (Fintech), and in particular to a method, apparatus, device, and computer-readable storage medium for implementing signal interaction. Background Art

[0002] With the development of computer technology, more and more technologies are being applied in the financial field. The traditional financial industry is gradually transforming into financial technology (Fintech). However, due to the security and real-time requirements of the financial industry, higher requirements are also placed on technology.

[0003] In a scheduling system, signal exchange between two workflows requires establishing a persistent socket connection based on TCP (Transmission Control Protocol). Signals are sent and received between the upstream and downstream workflows via input and output streams. Specifically, if one process wants to send data to another process over the network, it writes to the output stream associated with the socket. The other process reads the data written by the other process from the input stream associated with the socket. In other words, for signal exchange between workflows in a process, one machine must run a process to wait for a connection, while the other machine must attempt to reach the first machine to establish a connection. Therefore, existing workflow signal exchange schemes have the following drawbacks: 1) The downstream workflow must start first and the upstream workflow must send a signal later for the downstream workflow to receive the signal, which limits the flexibility of scheduling. 2) Usage is limited, as sockets do not store data. If the downstream workflow needs to rerun, the upstream workflow must resend the signal. 3) Performance issues can arise in high-concurrency scenarios. If the downstream workflow receives data slower than the upstream workflow sends the signal, the signal may be lost. Therefore, the signal interaction solutions in the prior art have poor flexibility and performance issues. Summary of the Invention

[0004] The main purpose of the present invention is to provide a method, device, equipment and computer-readable storage medium for implementing signal interaction, aiming to improve the flexibility of workflow signal interaction while ensuring the performance of signal interaction in high-concurrency scenarios.

[0005] To achieve the above objectives, the present invention provides a method for implementing signal interaction, the method comprising:

[0006] Receive a signal sent by an upstream workflow, encapsulate the signal once to obtain first signal data, and determine a target signal processing cluster corresponding to the first signal data;

[0007] Performing secondary packaging on the first signal data by the target signal processing cluster to obtain second signal data;

[0008] determining a target storage partition corresponding to the second signal data according to a data range of each pre-constructed partition in the target signal processing cluster;

[0009] Writing the second signal data into the target storage partition and generating corresponding metadata information;

[0010] When a signal reception request is received from a downstream workflow based on the signal, a target storage location is obtained from the metadata information according to the signal reception request, the signal is obtained based on the target storage location, and returned to the downstream workflow.

[0011] Optionally, before the step of receiving a signal sent by an upstream workflow, the method further includes:

[0012] Periodically obtain historical signal data and hash values written in each signal processing cluster, and obtain an estimated data flow based on the historical signal data;

[0013] Calculating the number of partitions of each signal processing cluster based on the estimated data flow and the preset partition flow;

[0014] According to the number of partitions, a corresponding number of pre-constructed partitions are established in each signal processing cluster, and the hash value is divided according to the number of partitions. The data range of each pre-constructed partition is defined according to the division result, and the pre-constructed partitions after the data range is defined are mapped to each signal processing component of each signal processing cluster.

[0015] Optionally, the steps of establishing a corresponding number of pre-constructed partitions in each signal processing cluster according to the number of partitions, dividing the hash value according to the number of partitions, defining a data range of each pre-constructed partition according to the division result, and mapping the pre-constructed partitions after the data range is defined to each signal processing component of each signal processing cluster include:

[0016] Establishing a corresponding number of pre-constructed partitions in each signal processing cluster according to the number of partitions;

[0017] Sort the hash values of the historical signal data corresponding to each signal processing cluster respectively;

[0018] Dividing the hash value according to the sorting result and the number of partitions, and defining the data range of each pre-constructed partition according to the division result;

[0019] The number of components of each signal processing cluster is obtained respectively, a target value is determined according to the number of components, and the pre-constructed partitions after defining the data range are mapped to each signal processing component of each signal processing cluster according to the target value and the number of partitions.

[0020] Optionally, the step of receiving a signal sent by an upstream workflow, encapsulating the signal once to obtain first signal data, and determining a target signal processing cluster corresponding to the first signal data includes:

[0021] Receive the signal sent by the upstream workflow through the signal sending component, and obtain the user identification and subject information corresponding to the signal;

[0022] generating, by the signal sending component, a first identifier according to the user identifier and the subject information, and obtaining first signal data according to the first identifier and the signal;

[0023] A target signal processing cluster is determined according to the user identifier, and the first signal data is routed to the target signal processing cluster.

[0024] Optionally, the step of performing secondary packaging on the first signal data by the target signal processing cluster to obtain second signal data includes:

[0025] parsing the first signal data by the target signal processing cluster to obtain the first identifier;

[0026] Performing hash processing on the first identifier to obtain a target hash value;

[0027] Generate a second identifier according to the target hash value and the first identifier, and obtain second signal data according to the second identifier and the signal;

[0028] The step of determining the target storage partition corresponding to the second signal data according to the data range of each pre-configured partition in the target signal processing cluster includes:

[0029] Determine the data range of the target hash value according to the data range of each pre-constructed partition in the target signal processing cluster, and determine the target storage partition according to the data range;

[0030] The step of obtaining a target storage location from the metadata information according to the signal reception request when a signal reception request is received by the downstream workflow based on the signal comprises:

[0031] When a signal receiving request sent by a downstream workflow based on the signal is received through a signal receiving component, the target signal processing cluster and the target hash value are obtained according to the user identifier and subject information carried in the signal receiving request;

[0032] A target storage location is obtained by querying from the metadata information according to the target signal processing cluster and the target hash value.

[0033] Optionally, the step of writing the second signal data into the target storage partition and generating corresponding metadata information includes:

[0034] storing the second signal data in a buffer;

[0035] When the signal reception of the signal stream is completed, sorting the second signal data in the buffer;

[0036] The sorted second signal data is written into each warehouse file of each warehouse of the target storage partition in a preset order, and corresponding metadata information is generated.

[0037] Optionally, the signal interaction implementation method further includes:

[0038] Monitoring whether the signal data written by each signal processing component exceeds the data range of the signal processing cluster written by the signal data;

[0039] If so, a signal processing cluster to be adjusted is determined according to the monitoring result, and a new partition is added to the signal processing cluster to be adjusted.

[0040] Optionally, the signal interaction implementation method further includes:

[0041] Obtain the flow value of each signal processing component separately and calculate the flow mean of each signal processing cluster;

[0042] Compare the traffic value of each signal processing component with the traffic mean of the corresponding signal processing cluster and the corresponding preset network card load factor and the preset network card load factor;

[0043] The partitioning of each signal processing component in each signal processing cluster is adjusted according to the comparison results.

[0044] In addition, to achieve the above-mentioned purpose, the present invention further provides a signal interaction implementation device, the signal interaction implementation device comprising:

[0045] A signal receiving module, configured to receive a signal sent by an upstream workflow, encapsulate the signal once to obtain first signal data, and determine a target signal processing cluster corresponding to the first signal data;

[0046] a signal processing module, configured to perform secondary packaging on the first signal data through the target signal processing cluster to obtain second signal data;

[0047] a partition determination module, configured to determine a target storage partition corresponding to the second signal data according to a data range of each pre-constructed partition in the target signal processing cluster;

[0048] a signal writing module, configured to write the second signal data into the target storage partition and generate corresponding metadata information;

[0049] A signal sending module is used to, when receiving a signal receiving request sent by a downstream workflow based on the signal, query the target storage location from the metadata information according to the signal receiving request, obtain the signal based on the target storage location, and return it to the downstream workflow.

[0050] In addition, to achieve the above-mentioned purpose, the present invention also provides a signal interaction implementation device, which includes: a memory, a processor, and a signal interaction implementation program stored on the memory and runnable on the processor. When the signal interaction implementation program is executed by the processor, it implements the steps of the signal interaction implementation method described above.

[0051] In addition, to achieve the above-mentioned purpose, the present invention also provides a computer-readable storage medium, on which a signal interaction implementation program is stored. When the signal interaction implementation program is executed by a processor, the steps of the signal interaction implementation method described above are implemented.

[0052] The present invention provides a signal interaction implementation method, apparatus, device, and computer-readable storage medium. The method first receives a signal sent by an upstream workflow, encapsulates the signal once to obtain first signal data, and determines a target signal processing cluster corresponding to the first signal data. The target signal processing cluster then encapsulates the first signal data again to obtain second signal data. The target storage partition corresponding to the second signal data is determined based on the data range of each pre-constructed partition in the target signal processing cluster. The second signal data is then written into the target storage partition and corresponding metadata information is generated. Upon receiving a signal reception request sent by a downstream workflow based on a signal, the target storage location is queried from the metadata information based on the signal reception request, a signal is obtained based on the target storage location, and the signal is returned to the downstream workflow. In the present invention, the signal received from the upstream workflow is stored in the corresponding partition in the target signal processing cluster and corresponding metadata information is generated. Upon receiving a signal reception request sent by a downstream workflow, the target storage location is queried from the metadata information, and the information is obtained and returned. Through the above approach, signal data storage is achieved, allowing downstream workflows to initiate receive requests to receive signals at any time. Furthermore, when the downstream workflow needs to be rerun, the upstream workflow does not need to resend the signal, thereby improving the flexibility of signal interaction. Furthermore, in high-concurrency scenarios, signal loss will not occur, thus ensuring the performance of signal transmission in high-concurrency scenarios. Therefore, the present invention realizes a high-performance and flexible workflow signal interaction solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention;

[0054] Figure 2 Schematic diagram of the flow of the first embodiment of the signal interaction implementation method of the present invention;

[0055] Figure 3 A schematic diagram of the interaction of various components involved in the signal interaction implementation method of the present invention;

[0056] Figure 4 Schematic diagram of the architecture of the signal processing components involved in the signal interaction implementation method of the present invention;

[0057] Figure 5 This is a flow chart of a second embodiment of a method for implementing signal interaction according to the present invention;

[0058] Figure 6 This is a functional module diagram of the first embodiment of the signal interaction implementation device of the present invention.

[0059] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings and in conjunction with the embodiments. DETAILED DESCRIPTION

[0060] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0061] Reference Figure 1 , Figure 1 This is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present invention.

[0062] The device for implementing signal interaction in the embodiment of the present invention may be a smart phone, or a terminal device such as a PC (Personal Computer), a tablet computer, or a portable computer.

[0063] like Figure 1 As shown, the implementation device of the signal interaction may include: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may optionally be a storage device independent of the aforementioned processor 1001.

[0064] Those skilled in the art will understand that Figure 1 The signal interaction implementation device structure shown in the figure does not constitute a limitation on the signal interaction implementation device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0065] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating system, a network communication module, and a signal interaction implementation program.

[0066] exist Figure 1 In the terminal shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client and communicate data with the client; and the processor 1001 can be used to call the signal interaction implementation program stored in the memory 1005 and execute the various steps of the following signal interaction implementation method.

[0067] Based on the above hardware structure, various embodiments of the signal interaction implementation method of the present invention are proposed.

[0068] The present invention provides a method for realizing signal interaction.

[0069] Reference Figure 2 , Figure 2 This is a flow chart of the first embodiment of the signal interaction implementation method of the present invention.

[0070] In this embodiment, the signal interaction implementation method includes:

[0071] Step S10: receiving a signal sent by an upstream workflow, encapsulating the signal once to obtain first signal data, and determining a target signal processing cluster corresponding to the first signal data;

[0072] The signal interaction implementation method of this embodiment is implemented by a signal interaction implementation device, in which an EventQueue architecture is built to implement the EventQueue service (signal sending and receiving service) in the scheduling system. Figure 3 As shown, the EventQueue service includes three components: stateless EventServer (signal processing service), EventPro (signal sending component) and EventRec (signal receiving component). EventQueue is an architecture that separates storage and computing. The bottom layer is based on DB (DataBase) to store actual signals. The upper layer EventServer is a stateless and centerless computing node. The metadata information generated by EventServer is stored in DB. Among them, EventPro is used to receive signals sent by upstream workflows and encapsulate the signals once (i.e., redefine the data structure), and then write the encapsulated signals into the signal processing component; EventServer includes multiple signal processing clusters, each signal processing cluster corresponds to a corresponding Userid (user ID), and each signal processing cluster includes one or more signal processing components (entity EventServer, i.e., signal processing server), such as Figure 4 As shown, each signal processing component includes multiple partitions. EventServer is used to store the signal data sent by EventPro in the partition of the corresponding data range and generate corresponding metadata information, which is stored in the DB; EventRec is used to receive the signal information sent by the downstream workflow and then find the storage location of the signal from the metadata information based on the signal information, obtain the signal, and then return it to the downstream workflow.

[0073] In this embodiment, step S10 is implemented by the signal sending component EventPro. When the upstream workflow needs to send a signal, it can send a signal carrying the signal identifier Userid and the subject information topic to the signal sending component. Correspondingly, the signal sent by the upstream workflow is received by the signal sending component, and the signal is packaged once to obtain the first signal data. Specifically, when the signal is packaged once, the user identifier Userid and the subject information topic corresponding to the signal are first obtained, and then the first identifier is generated according to the user identifier and the subject information, recorded as desc, and desc is a string spliced by Userid+Topic. Then, based on the first identifier desc and the signal, the first signal data is obtained. The first signal data is the signal structure after one package: desc+signal. The specific acquisition process of the first signal data can refer to the third embodiment described below.

[0074] Then, the target signal processing cluster corresponding to the first signal data is determined based on the user identifier Userid in the first signal data, and the first signal data is routed to the target signal processing cluster. In this embodiment, by routing the first signal data to the corresponding target signal processing cluster based on the user identifier, multi-tenant partitioning is achieved, ensuring the isolation of signal data between different users.

[0075] Step S20, performing secondary packaging on the first signal data through the target signal processing cluster to obtain second signal data;

[0076] Then, the first signal data is secondary packaged through the target signal processing cluster to obtain the second signal data. Specifically, the target signal processing cluster will first parse the first signal data to obtain the first identifier desc, and then hash the first identifier to obtain the target hash value Hashkey. When hashing the first identifier desc, the first identifier (Userid+Topic) can be directly hashed to obtain the target hash value, or only the topic information Topic in the first identifier can be hashed to obtain the target hash value. In this embodiment, hashing the Topic in the first identifier to obtain the target hash value is used as an example for explanation.

[0077] Then, a second identifier is generated based on the target hash value and the first identifier. The second identifier is the string formed by concatenating the target hash value and the first identifier, which is Hashkey+Userid+Topic. The second signal data is generated based on the second identifier and the signal. The second signal data is the signal structure after secondary packaging: Hashkey+desc+signal.

[0078] Step S30, determining a target storage partition corresponding to the second signal data according to the data range of each pre-constructed partition in the target signal processing cluster;

[0079] The target storage partition corresponding to the second signal data is determined based on the data range of each pre-constructed partition in the target signal processing cluster. The target signal processing cluster includes one or more signal processing components, each of which includes one or more pre-constructed partitions. Each partition has a corresponding data range. The method for setting the data range of each pre-constructed partition can refer to the second embodiment described below. When determining the target storage partition corresponding to the second signal data, a target hash value in the second signal data can be obtained, and then the target storage partition can be determined by determining which pre-constructed partition's data range the target hash value falls within.

[0080] Step S40: writing the second signal data into the target storage partition and generating corresponding metadata information;

[0081] After determining the target storage partition corresponding to the second signal data, the second signal data is written into the target storage partition and corresponding metadata information is generated. The metadata information is data describing the storage location of the second signal data and is stored in the DB.

[0082] It should be noted that steps S20 to S40 are implemented by the target signal processing cluster.

[0083] In addition, it should be noted that in this embodiment, based on the data range of each pre-constructed partition in the target signal processing cluster, the target storage partition corresponding to the second signal data is determined, and metadata information is generated after writing. That is, a different data range is defined for each partition, and the signal is stored according to the range to which it belongs, and metadata information is generated, which can facilitate subsequent rapid retrieval of the storage location of the signal to be received within a smaller data range.

[0084] Step S50, upon receiving a signal reception request sent by a downstream workflow based on the signal, querying the target storage location from the metadata information according to the signal reception request, obtaining the signal based on the target storage location, and returning it to the downstream workflow.

[0085] In this embodiment, step S50 is implemented by the signal receiving component EventRec. When the downstream workflow obtains the signal, it will send a signal receiving request carrying the user identification and subject information of the signal to be received. At this time, when the signal receiving component receives the signal receiving request sent by the downstream workflow based on the signal (that is, the signal sent by the above-mentioned upstream workflow), it obtains the user identification and subject information of the signal to be received according to the signal receiving request, and then queries the metadata information based on the user identification of the signal to be received and the target hash value obtained based on the user identification and subject information or the target hash value obtained based on the subject information to obtain the target storage location of the signal, and then obtains the signal based on the target storage location, and returns it to the downstream workflow.

[0086] An embodiment of the present invention provides a method for implementing signal interaction. The method first receives a signal sent by an upstream workflow, encapsulates the signal once to obtain first signal data, and determines the target signal processing cluster corresponding to the first signal data. The target signal processing cluster then encapsulates the first signal data again to obtain second signal data. The target storage partition corresponding to the second signal data is determined based on the data range of each pre-configured partition in the target signal processing cluster. The second signal data is then written to the target storage partition and corresponding metadata information is generated. Upon receiving a signal reception request sent by a downstream workflow based on the signal, the target storage location is retrieved from the metadata information based on the signal reception request. The signal is then retrieved based on the target storage location and returned to the downstream workflow. In an embodiment of the present invention, the signal received from the upstream workflow is stored in the corresponding partition in the target signal processing cluster and the corresponding metadata information is generated. Upon receiving a signal reception request from the downstream workflow, the target storage location is retrieved from the metadata information, and the information is retrieved and returned. Through the above method, signal data storage is achieved, and the downstream workflow can initiate a reception request to receive the signal at any time. At the same time, when the downstream workflow needs to be rerun, the upstream workflow does not need to resend the signal, thereby improving the flexibility of signal interaction. Furthermore, in high-concurrency scenarios, there will be no signal loss, thereby ensuring the performance of signal transmission in high-concurrency scenarios. Therefore, the embodiment of the present invention implements a high-performance and flexible workflow signal interaction solution.

[0087] Furthermore, it's important to note that because the EventServer signal processing service is stateless, management and scheduling are simple, naturally supporting horizontal scalability and rapid expansion. Furthermore, it focuses on computational logic, not data synchronization and consistency. Therefore, in the event of a failure, failover is quick and simple, providing a resilient fault-tolerant mechanism. By implementing signal interaction through this solution, we can address the challenges of signal interaction within a comprehensive scheduling system and provide enterprise-level capabilities for workflow systems.

[0088] Further, refer to Figure 5 , Figure 5 This is a flow chart of the second embodiment of the signal interaction implementation method of the present invention.

[0089] based on Figure 2 In the first embodiment shown, before step S10, the signal interaction implementation method further includes:

[0090] Step S60: Periodically obtain historical signal data and hash values written in each signal processing cluster, and obtain an estimated data flow based on the historical signal data;

[0091] To ensure the storage of large amounts of signal data, sufficient partition patterns will be pre-built in each signal processing cluster to store the data, ensuring that EventPro can write data evenly. If too few partitions are set, there will be hot spots. When a partition is full of signals, it will split, causing certain IO (Input / Output) problems. If too many partitions are set, some partitions will be idle and consume system resources. To avoid the above situation, it is very important to build an appropriate number of partitions. To this end, an embodiment of the present invention is proposed to predict the appropriate number of partitions through a model.

[0092] In this embodiment, the historical signal data and its hash value written in each signal processing cluster are periodically obtained. In view of the periodic characteristics of the scheduling system, the periodicity can be set to every other day, that is, the historical signal data is obtained at a fixed time every day. Correspondingly, the historical signal data is the signal data received the previous day, and the hash value is the value carried in the historical signal data. Then, based on the historical signal data, an estimated data flow is obtained. The estimated data flow may be estimated in the following ways: 1) directly set as the data flow corresponding to the historical signal data, 2) calculated based on the data flow corresponding to the historical signal data and a preset change ratio (e.g., estimated data flow = data flow corresponding to the historical signal data * (1 + preset change ratio)).

[0093] Step S70, calculating the number of partitions of each signal processing cluster according to the estimated data flow and the preset partition flow;

[0094] The number of partitions for each signal processing cluster is calculated based on the estimated data traffic and the preset partition traffic. Number of partitions = estimated data traffic / preset partition traffic. For example, for a signal processing service, if the daily estimated data traffic is 3072MB (i.e., the estimated data traffic is 3072MB), and the traffic per partition (i.e., the preset partition traffic) is 256MB, then the number of partitions is calculated to be 3072MB / 256MB = 12.

[0095] Step S80: Establish a corresponding number of pre-constructed partitions in each signal processing cluster according to the number of partitions, divide the hash value according to the number of partitions, define the data range of each pre-constructed partition according to the division result, and map the pre-constructed partitions after the data range is defined to each signal processing component of each signal processing cluster.

[0096] Then, a corresponding number of pre-built partitions are created in each signal processing cluster based on the number of partitions, and the hash values are divided according to the number of partitions. When dividing the hash values, the hash values can be sorted first, and then divided into the corresponding number of shares based on the number of partitions. The data range of each pre-built partition is then defined based on the division results.

[0097] Next, the pre-constructed partitions after defining the data range are mapped to each signal processing component of each signal processing cluster.

[0098] As one mapping method, the pre-constructed partitions after the data range is defined can be randomly mapped to each signal processing component of each signal processing cluster, and the number of partitions and their data ranges in each signal processing component are random. As another mapping method, the pre-constructed partitions after the data range is defined can be mapped to each signal processing component of each signal processing cluster in sequence according to the average number (the number of partitions allocated to each signal processing component), that is, the number of partitions in each signal processing component is evenly distributed, and the data ranges of the partitions are continuous. For example, for a certain signal processing cluster, the calculated number of partitions is 12, respectively denoted as 1-12, and their data ranges increase successively. The number of signal processing components in this signal processing cluster is 5. Then, partitions 1-3 after the data range is defined can be mapped to signal processing component 1, partitions 4-6 after the data range is defined can be mapped to signal processing component 2, partitions 7-8 after the data range is defined can be mapped to signal processing component 3, partitions 9-10 after the data range is defined can be mapped to signal processing component 4, and partitions 11-12 after the data range is defined can be mapped to signal processing component 5.

[0099] Furthermore, step S80 includes:

[0100] Step a81 , establishing a corresponding number of pre-configured partitions in each signal processing cluster according to the number of partitions;

[0101] Step a82, sorting the hash values of the historical signal data corresponding to each signal processing cluster respectively;

[0102] Step a83, dividing the Hash value according to the sorting result and the number of partitions, and defining the data range of each pre-constructed partition according to the division result;

[0103] Step a84 , respectively obtain the number of components of each signal processing cluster, determine a target value based on the number of components, and map the pre-constructed partitions after defining the data range to each signal processing component of each signal processing cluster based on the target value and the number of partitions.

[0104] In this embodiment, as another mapping method, a corresponding number of pre-constructed partitions can be established in each signal processing cluster according to the number of partitions; then, the hash values of the historical signal data corresponding to each signal processing cluster are sorted respectively. When sorting, they can be sorted in order from small to large (i.e., ascending order) or from large to small (i.e., descending order). Optionally, an ascending sorting method is used. After the sorting is completed, the hash values are divided according to the sorting results and the number of partitions, and the data range of each pre-constructed partition is defined according to the division results. Since the data is sorted in ascending order, the hash values of the first and last data of each piece of data correspond to the StartKey (start value) and EndKey (end value) of each partition, and then the data range is formed by the StartKey and EndKey. For example, in Table 1 below, the corresponding Hashkey is calculated based on the topic, and then the StartKey and EndKey corresponding to each partition are obtained by division.

[0105] After defining the data range of each pre-built partition, the number of components of each signal processing cluster is obtained respectively, and the target value is determined based on the number of components, where the target value is set to the maximum prime number of the number of components. For example, if the number of components in a signal processing cluster is 5 and they are numbered EventServer0-4, then the target value M is 5. Then, based on the target value and the number of partitions, the pre-built partitions after defining the data range are mapped to each signal processing component of each signal processing cluster. The partition position = mod (partition sequence number * partition number, target value), that is, the product of the partition sequence and the number of partitions is first calculated, and then the remainder obtained by dividing the product by the target value is used as the partition position. For example, in Table 1 below, the partition with partition sequence number 1 has a partition position of mod (1 * 12, 5) = 12 mod 5 = 2, and the corresponding signal service component number is EventServer2. Through the above mapping method, the pre-built partitions can be evenly mapped to each signal processing component in the signal processing cluster. Based on the hashing property of Hashkey, a large number of signals can be evenly written under high concurrency, avoiding the write hotspot problem.

[0106]

[0107] Table 1

[0108] Through the above approach, an estimated data flow is obtained based on the historical signal data written to each signal processing cluster, and the number of partitions is then determined based on the estimated data flow. This achieves partition number prediction, ensures accurate partition number estimation, and avoids hotspot issues caused by too few partitions and waste of system resources caused by too many partitions. Furthermore, this embodiment implements a traffic smoothing hashing algorithm that evenly maps partitions to each signal processing component in the signal processing cluster. This achieves load balancing during data writing and ensures signal transmission performance under high concurrency.

[0109] Furthermore, based on the above-mentioned first embodiment, a third embodiment of a method for implementing signal interaction of the present invention is proposed.

[0110] In this embodiment, the above step S10 includes:

[0111] Step a11: receiving a signal sent by an upstream workflow through a signal sending component, and obtaining a user identifier and subject information corresponding to the signal;

[0112] Step a12: generating a first identifier according to the user identifier and the subject information by the signal sending component, and obtaining first signal data according to the first identifier and the signal;

[0113] Step a13: determining a target signal processing cluster according to the user identifier, and routing the first signal data to the target signal processing cluster.

[0114] In this embodiment, the signal sent by the upstream workflow is received through the signal sending component Eventpro, and the user identifier Userid and topic information Topic corresponding to the signal are obtained. Then, the first identifier is generated according to the user identifier and the topic information through the signal sending component. The first identifier is a string obtained by concatenating Userid+Topic, recorded as desc. Then, the first signal data is obtained according to the first identifier and the signal. The first signal data is the signal structure after one package: desc+signal.

[0115] After obtaining the first signal data, a target signal processing cluster is determined based on the user identifier, and the first signal data is routed to the target signal processing cluster to request a signal write. The target signal processing cluster can be determined based on a pre-set mapping relationship between the user identifier and the signal processing cluster.

[0116] In this embodiment, by routing the first signal data to the corresponding target signal processing cluster according to the user identifier, multi-tenant partitioning is achieved, thereby ensuring isolation of signal data between different users.

[0117] At this time, step S20 includes:

[0118] Step a21, parsing the first signal data by the target signal processing cluster to obtain the first identifier;

[0119] Step a22: performing hash processing on the first identifier to obtain a target hash value;

[0120] Step a23: generating a second identifier according to the target hash value and the first identifier, and obtaining second signal data according to the second identifier and the signal;

[0121] At this time, when the target signal processing cluster receives the first signal data requested to be written by Eventpro, it parses the first signal data to obtain the first identifier desc, and then hashes the first identifier to obtain the target hash value, and then generates a second identifier based on the target hash value Hashkey and the first identifier. The second identifier is the string obtained by concatenating the target hash value and the first identifier, which is Hashkey+dese, that is, Hashkey+Userid+Topic, and then obtains the second signal data based on the second identifier and the signal. The second signal data is the signal structure after secondary encapsulation: Hashkey+desc+signal.

[0122] At this time, step S30 includes:

[0123] Step a31, determining the data range of the target hash value according to the data range of each pre-constructed partition in the target signal processing cluster, and determining the target storage partition according to the data range;

[0124] After obtaining the second signal data, the data range of the target hash value is determined according to the data range of each pre-constructed partition in the target signal processing cluster, and the target storage partition is determined according to the data range.

[0125] After determining the target storage partition corresponding to the second signal data, step S40 is executed: writing the second signal data into the target storage partition and generating corresponding metadata information. The specific execution process can refer to the first and fourth embodiments.

[0126] The step of “when receiving a signal reception request sent by a downstream workflow based on the signal, querying and obtaining a target storage location from the metadata information according to the signal reception request” includes:

[0127] Step a51, when a signal receiving request sent by a downstream workflow based on the signal is received through the signal receiving component, the target signal processing cluster and the target hash value are obtained according to the user identifier and subject information carried in the signal receiving request;

[0128] Step a52: Obtain a target storage location from the metadata information according to the target signal processing cluster and the target hash value.

[0129] When a signal receiving request sent by the downstream workflow based on the signal (the signal sent by the upstream workflow) is received through the signal receiving component EventRec, the target signal processing cluster is determined according to the user identifier carried in the signal receiving request, and the user identifier and subject information carried in the signal receiving request are hashed or the subject information carried in the signal receiving request is hashed to obtain the target hash value, and then the target storage location is queried from the metadata information according to the target signal processing cluster and the target hash value.

[0130] After the target storage location is obtained through query, the signal is obtained based on the target storage location and returned to the downstream workflow.

[0131] Furthermore, after the signal is acquired, relevant reception information may be generated and written back to the DB to facilitate the acquisition of the signal next time.

[0132] This embodiment provides a high-performance and flexible workflow signal interaction solution through a signal sending component, a signal processing cluster, and a signal receiving component.

[0133] Furthermore, based on the above-mentioned first to third embodiments, a fourth embodiment of the method for implementing signal interaction of the present invention is proposed.

[0134] In this embodiment, the above step S40 includes:

[0135] Step a41, storing the second signal data in a buffer;

[0136] In this embodiment, since one signal stream includes multiple signals, when the target signal processing cluster receives the first signal data, it may first store the second signal data in the SignFlusher (buffer).

[0137] Step a42, when the signal reception of the signal stream is completed, sorting the second signal data in the buffer;

[0138] Step a43: writing the sorted second signal data into each warehouse file of each warehouse of the target storage partition in a preset order, and generating corresponding metadata information.

[0139] When the signal reception of the signal stream is completed, the second signal data in the buffer is sorted. Optionally, it can be sorted in ascending order, and then the sorted second signal data is written to each warehouse file (StoreFile) of each warehouse (Store) of the target storage partition in a preset order, and corresponding metadata information is generated. The metadata information is data that describes the storage location of the second signal data, and the metadata information is stored in the DB.

[0140] In this embodiment, the received second signal data is first cached. Once all signals in a signal stream have been received, the second signal data in the cache is sequentially written to the respective warehouse files in each warehouse of the target storage partition, and corresponding metadata information is generated. This allows the signals of a signal stream to be written sequentially, making subsequent retrieval and query operations more convenient, improving signal retrieval and query efficiency, and thus improving signal reception efficiency.

[0141] Furthermore, based on the above-mentioned first to third embodiments, a fifth embodiment of the method for implementing signal interaction of the present invention is proposed.

[0142] In this embodiment, the signal interaction implementation method further includes:

[0143] Step A, monitoring whether the signal data written by each signal processing component exceeds the data range of the signal processing cluster written by the signal data;

[0144] In actual operation, since the network cards and QPS (queries per second) of each signal processing component may vary to a certain extent, evenly mapping partitions to the signal processing components of the signal processing cluster may cause actual load imbalance. For example, it is obviously unreasonable to allocate the same number of partitions to the signal processing components of a gigabit network card and a 100M network card. Therefore, in this embodiment, the written signal data is monitored in real time based on the data monitoring strategy to dynamically allocate partitions, thereby ensuring load balancing of each signal processing component during actual operation and ensuring the timeliness of signal interaction in high-concurrency scenarios.

[0145] In this embodiment, the signal data written by each signal processing component is monitored in real time to determine whether it exceeds the data range of the signal processing cluster to which the signal data is written. Specifically, when the target signal processing cluster receives a signal write request from Eventpro, it obtains the corresponding signal data according to the signal write request, and then obtains the hash value corresponding to the signal data (the specific hash value acquisition process can refer to the target hash value acquisition process described above), and then detects whether the hash value of the signal data is within the data range of the signal processing cluster to which the signal data is written.

[0146] It should be noted that, in this embodiment, Figure 4 It is implemented by ParFlusher (dynamic partition service) in the signal processing component shown.

[0147] Step B: If it exists, determine the signal processing cluster to be adjusted according to the monitoring result, and add a new partition to the signal processing cluster to be adjusted.

[0148] If the signal data written by each signal processing component exceeds the data range of the signal processing cluster to which the signal data is written, the signal processing cluster to be adjusted is determined based on the monitoring results. The signal processing cluster with the above-mentioned situation is then identified as the signal processing cluster to be adjusted. A new partition is then added to the signal processing cluster to ensure the speed and stability of the current signal data writing. The data range of the new partition can be set based on the hash value of the signal data, and the new partition can be allocated according to the method described in the second embodiment.

[0149] The dynamic allocation of partitions is achieved through the above-mentioned data monitoring method, which can ensure the load balancing of each signal processing component during actual operation and the timeliness of signal interaction in high-concurrency scenarios. At the same time, it can also ensure the full utilization of signal processing components and partitions, thereby improving resource utilization.

[0150] Furthermore, based on the above-mentioned first to third embodiments, a sixth embodiment of the method for implementing signal interaction of the present invention is proposed.

[0151] In this embodiment, the signal interaction implementation method further includes:

[0152] Step C, respectively obtaining the flow value of each signal processing component and calculating the flow mean of each signal processing cluster;

[0153] This embodiment provides another monitoring strategy - traffic monitoring strategy, which dynamically adjusts the partitions based on the monitoring results, thereby ensuring the load balancing of each signal processing component and the timeliness of signal interaction in high-concurrency scenarios.

[0154] In this embodiment, the flow value of each signal processing component is obtained respectively, and the flow mean value of each signal processing cluster is calculated based on the obtained flow value.

[0155] It should be noted that this embodiment can be implemented through ParFlusher (dynamic partition service) in the signal processing component.

[0156] Step D: comparing the flow value of each signal processing component with the flow mean of the corresponding signal processing cluster and the corresponding preset network card load factor and the preset network card load factor;

[0157] Step E: adjusting the partition of each signal processing component in each signal processing cluster according to the comparison result.

[0158] The traffic value of each signal processing component is then compared with the traffic mean of the corresponding signal processing cluster and the corresponding preset network card load factor. Based on the comparison results, the partitions of each signal processing component in each signal processing cluster are adjusted. The adjustment principle is to adjust the partitions of signal processing nodes in the same signal processing cluster whose traffic value exceeds the traffic mean and is greater than the corresponding preset network card load factor to signal processing nodes whose traffic value does not exceed the traffic mean and is less than or equal to the corresponding preset network card load factor. The preset network card load factor is set based on the network card bandwidth of the signal processing component, for example, it can be set to 0.75 times the network card bandwidth.

[0159] For example, for signal processing cluster 1, which includes signal processing components 1-5, the flow values of each component are a1-a5 respectively. The flow mean of the signal processing cluster can be calculated as a=(a1+a2+a3+a4+a5) / 5. Then, the flow value a1 of signal processing component 1 is detected to be greater than the flow mean a, and a1 is detected to be greater than the preset network card load factor corresponding to signal processing component 1. Similarly, the flow values of other signal processing components 2-5 are detected to be greater than the flow mean and the corresponding preset network card load factor. If it is detected that a1 is greater than a, and a1 is greater than the preset network card load factor corresponding to signal processing component 1, then the number of partitions in the signal processing component needs to be reduced. If it is detected that a2 is less than or equal to a, and a2 is less than or equal to the preset network card load factor corresponding to signal processing component 2, then the number of partitions in the signal processing component needs to be increased. At this time, the partition with a large amount of data in signal processing component 1 is adjusted to signal processing component 2.

[0160] Through the above-mentioned traffic monitoring method, dynamic allocation of partitions is achieved, which can ensure the load balancing of each signal processing component during actual operation and the timeliness of signal interaction in high-concurrency scenarios. At the same time, it can also ensure the full utilization of signal processing components and partitions, thereby improving resource utilization.

[0161] The present invention also provides a device for implementing signal interaction.

[0162] Reference Figure 6 , Figure 6 This is a functional module diagram of the first embodiment of the signal interaction implementation device of the present invention.

[0163] like Figure 6 As shown, the signal interaction implementation device includes:

[0164] The signal receiving module 10 is configured to receive a signal sent by an upstream workflow, encapsulate the signal once to obtain first signal data, and determine a target signal processing cluster corresponding to the first signal data;

[0165] A signal processing module 20 is configured to perform secondary packaging on the first signal data through the target signal processing cluster to obtain second signal data;

[0166] a partition determination module 30, configured to determine a target storage partition corresponding to the second signal data according to a data range of each pre-constructed partition in the target signal processing cluster;

[0167] A signal writing module 40 is configured to write the second signal data into the target storage partition and generate corresponding metadata information;

[0168] The signal sending module 50 is used to, when receiving a signal receiving request sent by a downstream workflow based on the signal, query the target storage location from the metadata information according to the signal receiving request, obtain the signal based on the target storage location, and return it to the downstream workflow.

[0169] Furthermore, the signal interaction implementation device further includes:

[0170] A first acquisition module is configured to periodically acquire historical signal data and hash values written in each signal processing cluster, and obtain an estimated data flow based on the historical signal data;

[0171] A calculation module, configured to calculate the number of partitions of each signal processing cluster based on the estimated data flow and the preset partition flow;

[0172] A determination module is used to establish a corresponding number of pre-constructed partitions in each signal processing cluster according to the number of partitions, divide the hash value according to the number of partitions, define the data range of each pre-constructed partition according to the division result, and map the pre-constructed partitions after the data range is defined to each signal processing component of each signal processing cluster.

[0173] Furthermore, the determining module includes:

[0174] A partition construction unit, configured to establish a corresponding number of pre-constructed partitions in each signal processing cluster according to the number of partitions;

[0175] A first sorting unit is used to sort the hash values of the historical signal data corresponding to each signal processing cluster respectively;

[0176] a range partitioning unit, configured to partition the hash value according to the sorting result and the number of partitions, and define a data range of each pre-constructed partition according to the partitioning result;

[0177] The partition mapping unit is used to respectively obtain the number of components of each signal processing cluster, determine a target value according to the number of components, and map the pre-constructed partitions after defining the data range to each signal processing component of each signal processing cluster according to the target value and the number of partitions.

[0178] Furthermore, the signal receiving module 10 includes:

[0179] A signal receiving unit, configured to receive a signal sent by an upstream workflow through a signal sending component, and obtain a user identifier and subject information corresponding to the signal;

[0180] a first generating unit, configured to generate a first identifier according to the user identifier and the subject information through the signal sending component, and obtain first signal data according to the first identifier and the signal;

[0181] A determining unit is configured to determine a target signal processing cluster according to the user identifier, and route the first signal data to the target signal processing cluster.

[0182] Furthermore, the signal processing module 20 includes:

[0183] a parsing unit, configured to parse the first signal data through the target signal processing cluster to obtain the first identifier;

[0184] A first processing unit, configured to perform hash processing on the first identifier to obtain a target hash value;

[0185] a second generating unit, configured to generate a second identifier according to the target hash value and the first identifier, and obtain second signal data according to the second identifier and the signal;

[0186] The partition determination module 30 is specifically configured to: determine the data range of the target hash value according to the data range of each pre-constructed partition in the target signal processing cluster, and determine the target storage partition according to the data range;

[0187] The signal sending module 50 includes:

[0188] A second processing unit is configured to obtain the target signal processing cluster and the target hash value according to the user identifier and subject information carried in the signal reception request when receiving a signal reception request sent by a downstream workflow based on the signal through the signal reception component;

[0189] A location query unit is configured to query the metadata information and obtain a target storage location according to the target signal processing cluster and the target hash value.

[0190] Furthermore, the signal writing module 40 includes:

[0191] a data cache unit, configured to store the second signal data in a buffer;

[0192] a second sorting unit, configured to sort the second signal data in the buffer when signal reception of the signal stream is completed;

[0193] A signal writing unit is used to write the sorted second signal data into each warehouse file of each warehouse of the target storage partition in a preset order, and generate corresponding metadata information.

[0194] Furthermore, the signal interaction implementation device further includes:

[0195] A data monitoring module is used to monitor whether the signal data written by each signal processing component exceeds the data range of the signal processing cluster written by the signal data;

[0196] The first adjustment module is configured to determine a signal processing cluster to be adjusted according to a monitoring result, if any, and add a new partition to the signal processing cluster to be adjusted.

[0197] Furthermore, the signal interaction implementation device further includes:

[0198] The traffic monitoring module is used to obtain the traffic value of each signal processing component and calculate the traffic mean of each signal processing cluster;

[0199] a comparison module for comparing the flow value of each signal processing component with the flow mean of the corresponding signal processing cluster and the corresponding preset network card load factor and the preset network card load factor;

[0200] The second adjustment module is used to adjust the partition of each signal processing component in each signal processing cluster according to the comparison result.

[0201] Among them, the functional implementation of each module in the above-mentioned signal interaction implementation device corresponds to the various steps in the above-mentioned signal interaction implementation method embodiment, and its functions and implementation processes will not be repeated here one by one.

[0202] The present invention also provides a computer-readable storage medium, which stores a signal interaction implementation program. When the signal interaction implementation program is executed by a processor, it implements the steps of the signal interaction implementation method described in any of the above embodiments.

[0203] The specific embodiments of the computer-readable storage medium of the present invention are basically the same as the embodiments of the above-mentioned signal interaction implementation method, and are not described in detail here.

[0204] It should be noted that, in this document, the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, article, or system comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such process, method, article, or system. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or system comprising the element.

[0205] The serial numbers of the above embodiments of the present invention are for description only and do not represent the advantages or disadvantages of the embodiments.

[0206] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus the necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better embodiment. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for enabling a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present invention.

[0207] The above are only preferred embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention description and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A method for implementing signal interaction, characterized in that: The method is implemented by a signal interaction implementation device, in which an EventQueue architecture is built to implement a signal transceiver service in a scheduling system. The signal transceiver service includes three components: a stateless signal processing component, a signal sending component, and a signal receiving component. The signal interaction implementation method includes: The signal sending component receives a signal sent by an upstream workflow, encapsulates the signal once to obtain first signal data, and determines a target signal processing cluster corresponding to the first signal data; The signal processing component performs secondary packaging on the first signal data through the target signal processing cluster to obtain second signal data; The signal processing component determines a target storage partition corresponding to the second signal data according to a data range of each pre-constructed partition in the target signal processing cluster; The signal processing component writes the second signal data into the target storage partition and generates corresponding metadata information; When the signal receiving component receives a signal receiving request sent by a downstream workflow based on the signal, it queries the metadata information to obtain a target storage location according to the signal receiving request, obtains the signal based on the target storage location, and returns the signal to the downstream workflow; Before the step of receiving the signal sent by the upstream workflow, the method further includes: The signal processing component periodically obtains historical signal data and hash values written in each signal processing cluster, and obtains an estimated data flow based on the historical signal data; The signal processing component calculates the number of partitions of each signal processing cluster according to the estimated data flow and the preset partition flow; The signal processing component establishes a corresponding number of pre-constructed partitions in each signal processing cluster according to the number of partitions; The signal processing component sorts the hash values of the historical signal data corresponding to each signal processing cluster respectively; The signal processing component divides the hash value according to the sorting result and the number of partitions, and defines the data range of each pre-constructed partition according to the division result; The signal processing component obtains the number of components of each signal processing cluster respectively, determines a target value according to the number of components, and maps the pre-constructed partitions after defining the data range to each signal processing component of each signal processing cluster according to the target value and the number of partitions.

2. The method for implementing signal interaction according to claim 1, wherein: The steps of receiving a signal sent by an upstream workflow, encapsulating the signal once to obtain first signal data, and determining a target signal processing cluster corresponding to the first signal data include: Receive the signal sent by the upstream workflow through the signal sending component, and obtain the user identification and subject information corresponding to the signal; generating, by the signal sending component, a first identifier according to the user identifier and the subject information, and obtaining first signal data according to the first identifier and the signal; A target signal processing cluster is determined according to the user identifier, and the first signal data is routed to the target signal processing cluster.

3. The method for implementing signal interaction according to claim 2, wherein: The step of performing secondary packaging on the first signal data by the target signal processing cluster to obtain second signal data includes: parsing the first signal data by the target signal processing cluster to obtain the first identifier; Performing hash processing on the first identifier to obtain a target hash value; Generate a second identifier according to the target hash value and the first identifier, and obtain second signal data according to the second identifier and the signal; The step of determining the target storage partition corresponding to the second signal data according to the data range of each pre-configured partition in the target signal processing cluster includes: Determine the data range of the target hash value according to the data range of each pre-constructed partition in the target signal processing cluster, and determine the target storage partition according to the data range; The step of obtaining a target storage location from the metadata information according to the signal reception request when a signal reception request is received by the downstream workflow based on the signal comprises: When a signal receiving request sent by a downstream workflow based on the signal is received through a signal receiving component, the target signal processing cluster and the target hash value are obtained according to the user identifier and subject information carried in the signal receiving request; A target storage location is obtained by querying from the metadata information according to the target signal processing cluster and the target hash value.

4. The method for implementing signal interaction according to any one of claims 1 to 3, wherein: The step of writing the second signal data into the target storage partition and generating corresponding metadata information includes: storing the second signal data in a buffer; When the signal reception of the signal stream is completed, sorting the second signal data in the buffer; The sorted second signal data is written into each warehouse file of each warehouse of the target storage partition in a preset order, and corresponding metadata information is generated.

5. The method for implementing signal interaction according to any one of claims 1 to 3, characterized in that: The signal interaction implementation method further includes: Monitoring whether the signal data written by each signal processing component exceeds the data range of the signal processing cluster written by the signal data; If so, a signal processing cluster to be adjusted is determined according to the monitoring result, and a new partition is added to the signal processing cluster to be adjusted.

6. The method for implementing signal interaction according to any one of claims 1 to 3, characterized in that: The signal interaction implementation method further includes: Obtain the flow value of each signal processing component respectively, and calculate the flow mean of each signal processing cluster; Compare the traffic value of each signal processing component with the traffic mean of the corresponding signal processing cluster and the corresponding preset network card load factor and the preset network card load factor; The partitioning of each signal processing component in each signal processing cluster is adjusted according to the comparison results.

7. A signal interaction implementation device, characterized in that: The signal interaction implementation device is built with an EventQueue architecture to implement a signal transceiver service within the scheduling system. The signal transceiver service includes three components: a stateless signal processing component, a signal sending component, and a signal receiving component. The signal interaction implementation device includes: a signal receiving module, configured to instruct the signal receiving component to receive a signal sent by an upstream workflow, encapsulate the signal once to obtain first signal data, and determine a target signal processing cluster corresponding to the first signal data; A signal processing module, configured for the signal processing service to perform secondary packaging on the first signal data through the target signal processing cluster to obtain second signal data; a partition determination module, configured for the signal processing service to determine a target storage partition corresponding to the second signal data according to a data range of each pre-configured partition in the target signal processing cluster; a signal writing module, configured for the signal processing service to write the second signal data into the target storage partition and generate corresponding metadata information; a signal sending module, configured for the signal receiving component to, upon receiving a signal receiving request sent by a downstream workflow based on the signal, query the metadata information to obtain a target storage location according to the signal receiving request, obtain the signal based on the target storage location, and return the signal to the downstream workflow; The partition determination module is used to periodically obtain historical signal data and hash values written in each signal processing cluster, and obtain an estimated data flow based on the historical signal data; calculate the number of partitions of each signal processing cluster based on the estimated data flow and the preset partition flow; establish a corresponding number of pre-constructed partitions in each signal processing cluster based on the number of partitions; sort the hash values of the historical signal data corresponding to each signal processing cluster; divide the hash values according to the sorting results and the number of partitions, and define the data range of each pre-constructed partition according to the division results; obtain the number of components of each signal processing cluster, determine the target value according to the number of components, and map the pre-constructed partitions with the defined data range to each signal processing component of each signal processing cluster based on the target value and the number of partitions.

8. A device for implementing signal interaction, characterized in that: The signal interaction implementation device includes: a memory, a processor, and a signal interaction implementation program stored in the memory and runnable on the processor. When the signal interaction implementation program is executed by the processor, the steps of the signal interaction implementation method as described in any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a signal interaction implementation program, and when the signal interaction implementation program is executed by a processor, the steps of the signal interaction implementation method according to any one of claims 1 to 6 are implemented.

Images