Processing node determination method, apparatus, computer equipment, and storage medium

Abstract

The present application relates to a processing node determination method, device, computer equipment, storage medium and computer program product. The method comprises: obtaining the array number of each array unit corresponding to the next processing link of the data to be transmitted, the next processing link includes multiple processing nodes to be selected; from each array number, determining the target array number corresponding to the target parameter of the data to be transmitted; based on the correspondence between the node index of each processing node to be selected and each array number, determining the target node index corresponding to the target array number, the correspondence is determined according to the load of each processing node to be selected, and the target node index is used to uniquely indicate the target processing node for processing the data to be transmitted. The use of this method can avoid the problem of uneven load between different cores in the same processing link, thereby improving the throughput of the whole machine and improving the bandwidth utilization.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a method, device, computer equipment and storage medium for determining a processing node. Background Art

[0002] With the development of Internet technology, in the scenario of Network Function Virtualization (NFV), in the process of transmitting data based on the packet processing model of the multi-stage pipeline processing structure (pipeline) architecture, when the data completes the current processing link and is to be delivered to the next processing link, the five-tuple hash value of the data is directly used to modulo the number of central processing unit (CPU) cores of the next processing link. It should be understood that one CPU core corresponds to one instance, that is, the number of CPU cores is the number of instances. Based on this, the modulo result obtained is the index of the CPU core that is specifically responsible for processing the data in the next link. However, if the data rates between different streams are different, it will cause uneven loads between different cores in the same processing link. Therefore, how to avoid uneven loads between cores is a problem that needs to be solved urgently. Summary of the invention

[0003] Based on this, it is necessary to provide a processing node determination method, device, computer equipment and storage medium that can avoid uneven load between cores in order to address the above technical problems.

[0004] In a first aspect, the present application provides a method for determining a processing node. The method comprises:

[0005] Obtaining the array number of each array unit corresponding to the next processing link of the data to be transmitted, the next processing link including a plurality of processing nodes to be selected;

[0006] From each array number, determine the target array number corresponding to the target parameter of the data to be transmitted;

[0007] Based on the correspondence between the node index of each processing node to be selected and each array number, the target node index corresponding to the target array number is determined. The correspondence is determined according to the load of each processing node to be selected. The target node index is used to uniquely indicate the target processing node for processing the data to be transmitted.

[0008] In one embodiment, the method further comprises:

[0009] Obtaining the maximum processing load of each processing node to be selected, and calculating the maximum load ratio between the maximum processing loads of each processing node to be selected;

[0010] Based on the maximum load ratio, determine the number ratio of the node index number of each node to be selected for processing;

[0011] Generate the corresponding relationship between each node index and each array number based on the quantity ratio

[0012] In one embodiment, the method further comprises:

[0013] Obtaining the message processing rate of each processing node to be selected, and calculating the load ratio between the processing loads of each processing node to be selected based on each message processing rate, where the load ratio is used to describe the difference between the processing loads of each processing node to be selected;

[0014] Based on the load ratio, the correspondence between each array number and each node index is determined.

[0015] In one embodiment, determining the correspondence between each array number and each node index based on the load ratio includes:

[0016] If the load ratio is greater than or equal to the maximum ratio threshold, determine the first processing node to be selected and the second processing node to be selected based on the load ratio, the first processing node to be selected is the processing node to be selected with the largest processing load among all the processing nodes to be selected, and the second processing node to be selected is the processing node to be selected with the smallest processing load among all the processing nodes to be selected;

[0017] Reducing the number of node indexes of the first node to be selected for processing, and adjusting the correspondence between the node index of the first node to be selected for processing and each array number based on the reduced number of node indexes of the first node to be selected for processing;

[0018] The number of node indexes of the second to-be-selected processing nodes is increased, and based on the increased number of node indexes of the second to-be-selected processing nodes, the corresponding relationship between the node indexes of the second to-be-selected processing nodes and the array numbers is adjusted.

[0019] In one of the embodiments, the data to be transmitted is service data belonging to a target service, and the transmission order of the data to be transmitted in each service data of the target service is a preset transmission order;

[0020] The data to be transmitted also includes message sequence number information, which includes a first modulus value and a message sequence identifier. The first modulus value is obtained by taking the modulus of the number of array units corresponding to the first processing link of the target service based on the target parameter, and the message sequence identifier is obtained by increasing the first modulus value packet by packet.

[0021] In one embodiment, after adjusting the correspondence between each node index and each array number, the method further includes:

[0022] If the target array unit indicated by the target array number is an array unit whose corresponding relationship has been adjusted, the data to be transmitted is marked;

[0023] After the processing of the data to be transmitted is completed in the last processing link of the target service, the data to be transmitted after the marking processing is allocated to the first sorting domain corresponding to the first modulus value, and the first sorting domain includes each first service data carrying the first modulus value;

[0024] In the first sorting domain, reordering is performed based on the message sequence identifiers of each first service data, and the order of the data to be transmitted after the reordering is the preset transmission order in each first service data after the reordering.

[0025] In one of the embodiments, the target parameter is a five-tuple hash value;

[0026] Determine from each array number the target array number corresponding to the target parameter of the data to be transmitted, including:

[0027] Performing a modulo process on the five-tuple hash value to obtain a five-tuple hash value after modulo processing;

[0028] The target array number is determined from each array number by taking the modulo five-tuple hash value.

[0029] In a second aspect, the present application also provides a processing node determination device. The device comprises:

[0030] An acquisition module, used to acquire the array number of each array unit corresponding to the next processing link of the data to be transmitted, the next processing link including a plurality of processing nodes to be selected;

[0031] A determination module is used to determine the target array number corresponding to the target parameter of the data to be transmitted from each array number; and based on the correspondence between the node index of each processing node to be selected and each array number, determine the target node index corresponding to the target array number, the correspondence is determined according to the load of each processing node to be selected, and the target node index is used to uniquely indicate the target processing node for processing the data to be transmitted.

[0032] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0033] Obtaining the array number of each array unit corresponding to the next processing link of the data to be transmitted, the next processing link including a plurality of processing nodes to be selected;

[0034] From each array number, determine the target array number corresponding to the target parameter of the data to be transmitted;

[0035] Based on the correspondence between the node index of each processing node to be selected and each array number, the target node index corresponding to the target array number is determined. The correspondence is determined according to the load of each processing node to be selected. The target node index is used to uniquely indicate the target processing node for processing the data to be transmitted.

[0036] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0037] Obtaining the array number of each array unit corresponding to the next processing link of the data to be transmitted, the next processing link including a plurality of processing nodes to be selected;

[0038] From each array number, determine the target array number corresponding to the target parameter of the data to be transmitted;

[0039] Based on the correspondence between the node index of each processing node to be selected and each array number, the target node index corresponding to the target array number is determined. The correspondence is determined according to the load of each processing node to be selected. The target node index is used to uniquely indicate the target processing node for processing the data to be transmitted.

[0040] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0041] Obtaining the array number of each array unit corresponding to the next processing link of the data to be transmitted, the next processing link including a plurality of processing nodes to be selected;

[0042] From each array number, determine the target array number corresponding to the target parameter of the data to be transmitted;

[0043] Based on the correspondence between the node index of each processing node to be selected and each array number, the target node index corresponding to the target array number is determined. The correspondence is determined according to the load of each processing node to be selected. The target node index is used to uniquely indicate the target processing node for processing the data to be transmitted.

[0044] The above-mentioned processing node determination method, device, computer equipment, storage medium and computer program product first obtain the array number of each array unit corresponding to the next processing link of the data to be transmitted, and the next processing link includes multiple processing nodes to be selected, and then determine the target array number corresponding to the target parameter of the data to be transmitted from each array number, and determine the target node index corresponding to the target array number based on the correspondence between the node index of each processing node to be selected and each array number, the correspondence is determined according to the load of each processing node to be selected, and the target node index is used to uniquely indicate the target processing node for processing the data to be transmitted. Since the correspondence is determined according to the load of each processing node to be selected, that is, the relationship between each array number and multiple node indexes can be allocated based on the load of each processing node to be selected, so that the target array number corresponding to the target parameter is indexed and searched accordingly through the corresponding relationship, so that the data is transmitted to the corresponding service node for processing based on the load in the same processing link, so as to avoid uneven load between different cores in the same processing link, thereby improving the throughput of the whole machine and improving the bandwidth utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 It is a schematic diagram of transmitting data based on pipeline architecture in an NFV scenario in one embodiment;

[0046] Figure 2 An application environment diagram of a processing node determination method in an embodiment;

[0047] Figure 3 A schematic diagram of a process flow of a method for determining a processing node in an embodiment;

[0048] Figure 4 A schematic diagram of the correspondence between array units and array numbers in one embodiment;

[0049] Figure 5 A schematic diagram of the correspondence between node indexes and array numbers according to an embodiment;

[0050] Figure 6 A schematic diagram of a flow chart for determining a correspondence between each node index and each array number in an embodiment;

[0051] Figure 7 A schematic diagram of a flow chart for determining a corresponding relationship between each node index and each array number in another embodiment;

[0052] Figure 8 A schematic diagram of a flow chart for adjusting the correspondence between each node index and each array number in one embodiment;

[0053] Fig. 9 A schematic diagram of adjusting the corresponding relationship in one embodiment;

[0054] Fig.10 A schematic diagram of a data transmission process in an embodiment;

[0055] Fig.11 A schematic diagram of a process for determining a target array number in one embodiment;

[0056] Fig.12 A complete flowchart of a method for determining a processing node in one embodiment is shown;

[0057] Fig.13 is a structural block diagram of a processing node determination device in one embodiment;

[0058] Fig.14 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0060] Cloud technology refers to a hosting technology that unifies hardware, software, network and other resources in a wide area network or local area network to achieve data computing, storage, processing and sharing. Cloud technology is a general term for network technology, information technology, integration technology, management platform technology, application technology, etc. based on the cloud computing business model. It can form a resource pool that can be used on demand and is flexible and convenient. Cloud computing technology will become an important support. The background services of technical network systems require a large amount of computing and storage resources, such as video websites, picture websites and more portal websites. With the high development and application of the Internet industry, in the future, each item may have its own identification mark, and all need to be transmitted to the background system for logical processing. Data of different levels will be processed separately. All kinds of industry data require strong system backing support, which can only be achieved through cloud computing.

[0061] The solution provided in the embodiment of the present application relates to cloud computing in cloud technology. Cloud computing refers to the delivery and use mode of IT infrastructure, which refers to obtaining required resources through the network in an on-demand and easily scalable manner; in a broad sense, cloud computing refers to the delivery and use mode of services, which refers to obtaining required services through the network in an on-demand and easily scalable manner. Such services can be related to IT and software, the Internet, or other services. Cloud computing is the product of the development and integration of traditional computer and network technologies such as grid computing, distributed computing, parallel computing, utility computing, network storage technologies, virtualization, and load balancing. With the development of the Internet, real-time data streams, and the diversification of connected devices, as well as the promotion of search services, social networks, mobile commerce, and open collaboration, cloud computing has developed rapidly. Different from previous parallel distributed computing, the emergence of cloud computing will promote revolutionary changes in the entire Internet model and enterprise management model in terms of concept.

[0062] Currently, if Figure 1 As shown in the figure, in the NFV scenario, when the data is transmitted by the packet processing model based on the pipeline architecture, after the data is processed in the dispatch link, it needs to be sent to the next link of the dispatch link (i.e., the forward link). At this time, it is necessary to use the five-tuple hash value of the data to take the modulus of the number of CPU cores included in the forward link. Therefore, the modulus result is the index of the corresponding CPU core that specifically processes the data in the forward link. Figure 1 The method shown will result in uneven loads between different CPU cores in the same processing link if the data rates between different streams are different.

[0063] Based on this, in order to solve the aforementioned problem of uneven load between cores, the present application embodiment provides a method for determining a processing node. The processing node determination method provided by the present application embodiment can be applied to Figure 2In the application environment shown. Among them, server 202 communicates with server 204 through the network. The data storage system can store the data that server 202 and server 204 need to process. The data storage system can be integrated on server 202 and server 204, or it can be placed on the cloud or other servers. Among them, server 202 and server 204 can be independent physical servers, or they can be server clusters or distributed systems composed of multiple physical servers, or they can be cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0064] Specifically, in the embodiment of the present application, data transmission of the packet processing model of the pipeline architecture is performed on the server 202, so the server 202 includes multiple processing links, and the processing nodes included in each processing link are specifically the CPU cores in the server 202. It can be understood that the number of CPU cores included in each processing link can be different. Secondly, the processing node determination method provided in the embodiment of the present application can be applied to the cloud computing scenario in cloud technology, and specifically applied to the NFV scenario architecture of the cloud computing scenario, in the process of transmitting data based on the packet processing model of the pipeline architecture.

[0065] Based on this, in one embodiment, Figure 3 As shown, a processing node determination method is provided, and the method is applied to Figure 2 Taking the server 202 in the example as an example, the following steps are included:

[0066] Step 302, obtaining the array number of each array unit corresponding to the next processing link of the data to be transmitted, the next processing link including a plurality of processing nodes to be selected.

[0067] Among them, in the process of transmitting data based on the pipeline architecture packet processing model, that is, the process of processing the business data of the target business includes multiple processing links, such as Figure 1 As shown, the dispatch processing link and the forward processing link are both processing links in processing the target business.

[0068] Based on this, each processing link in the pipeline architecture will have a corresponding linear MAP table, which is specifically a one-dimensional array unit with a length of N, that is, each processing link has corresponding multiple array units, and each array unit also has a unique corresponding array number. For ease of understanding, such as Figure 4 As shown, array element 402 corresponds to array number 1, array element 404 corresponds to array number 7, and array element 406 corresponds to array number 16.

[0069] It should be understood that in practical applications, the length of the linear MAP table in each processing link in the pipeline architecture is the same, that is, each processing link has the same number of corresponding array units, for example, the number of array units corresponding to each processing link in the pipeline architecture is 1024. Therefore, the specific value and number of the array number can be flexibly determined based on actual conditions.

[0070] Specifically, after the server 202 completes the processing of the data to be transmitted in the current processing link, the server 202 first determines that the data to be transmitted should be transmitted to the next processing link. At this time, the linear MAP table corresponding to the next processing link will be obtained in real time, that is, each array unit corresponding to the next processing link will be obtained, and since each array unit has a unique corresponding array number, the server 202 can further obtain the array number of each array unit corresponding to the next processing link. Secondly, in the embodiment of the present application, an instance corresponds to a CPU core, and the CPU core is specifically a processing node belonging to a certain link. Based on this, the next processing link will include multiple processing nodes to be selected. For example, the next processing link includes processing node A1 to be selected, processing node A2 to be selected, and processing node A3 to be selected.

[0071] For ease of understanding, again Figure 1 For example, if the data to be transmitted needs to be sent to the next link of the dispatch link (i.e., the forward link) after data processing is completed in the dispatch link, it is necessary to determine to which selected processing node in the forward link the data to be transmitted needs to be transmitted. Therefore, the server 202 first obtains the array units corresponding to the forward link, as well as the array numbers of the array units corresponding to the forward link.

[0072] Step 304: Determine, from each array number, a target array number corresponding to the target parameter of the data to be transmitted.

[0073] The target parameter may be a five-tuple hash value or other numerical value.

[0074] Specifically, after completing the processing of the data to be transmitted in the current processing link, the server 202 can also determine the target parameters of the data to be transmitted, and then determine the corresponding target array number from each array number based on the target parameter. The server 202 can perform modulo processing on the target parameter, and then determine the array number that has a corresponding relationship with the target parameter after the modulo processing as the target array number. Secondly, the server 202 can also directly determine the array number that has a corresponding relationship with the target parameter as the target array number. The aforementioned corresponding relationship can be equal in value, or the difference between the values ​​is a preset value, etc., which is not limited here.

[0075] Step 306, based on the correspondence between the node index of each processing node to be selected and each array number, determine the target node index corresponding to the target array number. The correspondence is determined according to the load of each processing node to be selected. The target node index is used to uniquely indicate the target processing node used to process the data to be transmitted.

[0076] Among them, since each processing link will have a corresponding linear MAP table, the linear MAP table is specifically a one-dimensional array unit with a length of N, and each array unit also has a unique corresponding array number, so there is a corresponding relationship between the array number corresponding to each array unit and the node index of each processing node to be selected, and the node index can index a specific processing node, that is, the node index is used to uniquely indicate the corresponding processing node. Secondly, the corresponding relationship is determined according to the load of each processing node to be selected, for example, the corresponding relationship is determined based on the maximum processing load of each processing node to be selected, or the corresponding relationship is determined based on the real-time processing load of each processing node to be selected, which is not limited here.

[0077] For ease of understanding, Figure 5 As shown, array unit 5021 corresponds to array number 1, and array number 1 has a corresponding relationship with node index 0, and node index 0 is used to uniquely indicate the node to be selected for processing 5022. Similarly, array unit 5041 corresponds to array number 3, and array number 3 has a corresponding relationship with node index 2, and node index 2 is used to uniquely indicate the node to be selected for processing 5042. And array unit 5061 corresponds to array number 4, and array number 16 has a corresponding relationship with node index 4, and node index 4 is used to uniquely indicate the node to be selected for processing 5062.

[0078] Specifically, based on the correspondence between the node index of each to-be-selected processing node and each array number, the server 202 first determines the target node index corresponding to the target array number, and then the server 202 transmits the data to be transmitted to the target processing node uniquely indicated by the target node index, so that the target processing node performs corresponding data processing on the data to be transmitted. For example, the correspondence includes array number 1 and node index C1, array number 2 and node index C2, and array number 3 and node index C1. If the determined target array number is 3, based on the aforementioned correspondence, the node index C1 corresponding to array number 3 can be determined as the target node index, and thus, the server 202 transmits the data to be transmitted to the target processing node uniquely indicated by node index C1 for corresponding data processing.

[0079] In the above-mentioned processing node determination method, since the corresponding relationship is determined according to the load of each processing node to be selected, that is, the relationship between each array number and multiple node indexes can be allocated based on the load of each processing node to be selected, so that the target array number corresponding to the target parameter is indexed and searched accordingly through the corresponding relationship, so that data can be transmitted to the corresponding service node for processing based on the load in the same processing link, thereby avoiding the problem of uneven load between different cores in the same processing link, thereby improving the overall machine throughput and improving bandwidth utilization.

[0080] The following will introduce how to determine the corresponding relationship based on the load of each processing node to be selected. First, the method of determining the corresponding relationship based on the maximum processing load of each processing node to be selected is introduced:

[0081] In one embodiment, Figure 6 As shown, the method also includes:

[0082] Step 602: Obtain the maximum processing load of each processing node to be selected, and calculate the maximum load ratio between the maximum processing loads of each processing node to be selected.

[0083] The maximum processing load is the maximum load that the processing node to be selected can bear, and specifically may be a preset maximum load of the processing node to be selected, or a maximum load determined based on attribute information of the processing node to be selected, which is not limited here.

[0084] Specifically, the server 202 first obtains the maximum processing load of each processing node to be selected, and then calculates the ratio of the maximum processing load of each processing node to be selected to obtain the maximum load ratio. For example, the next processing link includes processing node A1 to be selected, processing node A2 to be selected, and processing node A3 to be selected, and the maximum processing load of processing node A1 to be selected is 100, the maximum processing load of processing node A2 to be selected is 80, and the maximum processing load of processing node A3 to be selected is 60, so the maximum load ratio can be obtained as 5:4:3 (100:80:60).

[0085] Step 604: Determine the quantity ratio of the node index quantity of each to-be-selected processing node based on the maximum load ratio.

[0086] The quantity ratio is specifically the quantity ratio of the node index quantity in the corresponding relationship of each to-be-selected processing node.

[0087] Specifically, the server 202 determines the number ratio of the number of node indexes of each to-be-selected processing node based on the maximum load ratio. The number ratio can be determined based on the maximum load ratio. Alternatively, the maximum load ratio is determined as the number ratio. For example, if the maximum load ratio between the to-be-selected processing node A1, the to-be-selected processing node A2, and the to-be-selected processing node A3 is 5:4:3, then the number ratio of the number of node indexes of the to-be-selected processing nodes is 5:4:3.

[0088] Step 606: Generate a correspondence between each node index and each array number based on the quantity ratio.

[0089] Among them, in the correspondence between each node index and each array number, the ratio between the number of each node index is the number ratio. For example, the determined number ratio is 5:4:3, based on which the number of node indexes of the node A1 to be selected for processing can be 20, the number of node indexes of the node A2 to be selected for processing can be 16, and the number of node indexes of the node A3 to be selected for processing can be 12.

[0090] Specifically, since each processing link in the pipeline architecture will have a corresponding linear MAP table, the linear MAP table is specifically a one-dimensional array unit with a length of N, that is, the next processing link has corresponding N array units, and each array unit also has a unique corresponding array number, so the next processing link has corresponding N array numbers.

[0091] Based on this, the server 202 first determines the number of node indexes of each node to be selected for processing based on the ratio of the number of node indexes of the nodes to be selected for processing calculated in step 604 and the specific number of each array number, and then maps and binds each node index with each array number based on the node index number of each node index, thereby generating a corresponding relationship between each node index and each array number.

[0092] For example, the ratio of the number of node indexes of the to-be-selected processing node A1, the to-be-selected processing node A2, and the to-be-selected processing node A3 is 2:1:1, and the to-be-selected processing node A1 corresponds to the node index C1, the to-be-selected processing node A2 corresponds to the node index C2, and the to-be-selected processing node A3 corresponds to the node index C3. And the next processing link corresponds to 8 array units, that is, the next processing link corresponds to 8 array numbers, if the array numbers are specifically 1 to 8, that is, the next processing link corresponds to array number 1, array number 2, array number 3 to array number 8.

[0093] Based on this, the ratio of the number of node indexes is 2:1:1, and the next processing link corresponds to 8 array units, so it can be determined that the number of node indexes C1 is 4, the number of node indexes C2 is 2, and the number of node indexes C2 is 2. Therefore, the generated correspondence includes the correspondence between 4 node indexes C1 and each array number, the correspondence between 2 node indexes C2 and each array number, and the correspondence between 2 node indexes C3 and each array number. For example: the correspondence between array number 1 and node index C1, the correspondence between array number 2 and node index C1, the correspondence between array number 3 and node index C2, the correspondence between array number 4 and node index C3, the correspondence between array number 5 and node index C2, the correspondence between array number 6 and node index C1, the correspondence between array number 7 and node index C3, and the correspondence between array number 8 and node index C1. It should be understood that the above examples are only used to understand the present application and should not be understood as limitations of the present application.

[0094] In this embodiment, by considering the maximum processing load of each processing node to be selected, the mapping relationship is bound based on the maximum processing load when generating the corresponding relationship. This avoids uneven load between different cores in the same processing link, further ensuring that each processing node to be selected can meet the data processing requirements, thereby improving the reliability of this solution and ensuring bandwidth utilization.

[0095] Next, we will introduce a method for determining the corresponding relationship based on the real-time processing load of each processing node to be selected:

[0096] In one embodiment, Figure 7 As shown, the method also includes:

[0097] Step 702, obtaining the message processing rate of each processing node to be selected, and calculating the load ratio between the processing loads of each processing node to be selected based on each message processing rate, the load ratio being used to describe the difference between the processing loads of each processing node to be selected.

[0098] The message processing rate is used to describe the real-time processing rate of the processing node to be selected.

[0099] Specifically, the server 202 first obtains the current cumulative amount of processed data of each processing node to be selected in real time, and then calculates the message processing rate based on the current cumulative amount of processed data of the processing node to be selected. Further, the load ratio between the processing loads of each processing node to be selected is calculated based on each message processing rate. The aforementioned load ratio is used to describe the difference between the processing loads of each processing node to be selected, that is, the load ratio reflects the degree of load balancing of each processing node to be selected.

[0100] For example, the next processing link includes the processing node A1 to be selected, the processing node A2 to be selected, and the processing node A3 to be selected, and the real-time processing rate of the processing node A1 to be selected is 60, the real-time processing rate of the processing node A2 to be selected is 30, and the real-time processing rate of the processing node A3 to be selected is 30, so the load ratio can be 2:1:1 (60:30:30). And it can be seen that the load of the processing node A1 to be selected is higher than that of the processing node A2 to be selected and the processing node A3 to be selected.

[0101] Step 704: Determine the correspondence between each array number and each node index based on the load ratio.

[0102] Among them, in the correspondence between each node index and each array number, the ratio between the number of each node index is the load ratio. For example, the determined load ratio is 2:1:1, based on which the number of node indexes of the node A1 to be selected for processing can be 20, the number of node indexes of the node A2 to be selected for processing can be 10, and the number of node indexes of the node A3 to be selected for processing can be 10.

[0103] Specifically, based on the load ratio calculated in step 702 and the specific number of each array number, the server 202 first determines the number of node indexes of each node to be selected for processing, and then maps and binds each node index to each array number based on the number of node indexes of each node index, thereby determining the corresponding relationship between each array number and each node index. The specific implementation is similar to step 606 and will not be repeated here.

[0104] In this embodiment, by considering the real-time processing load of each processing node to be selected, the mapping relationship is bound based on the real-time processing load when determining the corresponding relationship, so as to realize dynamic determination of the corresponding relationship, and further improve the flexibility and real-time performance of this solution on the basis of avoiding uneven load between different cores in the same processing link.

[0105] When the corresponding relationship is determined by considering the real-time processing load of each processing node to be selected, the corresponding relationship needs to be adjusted based on the real-time processing load of each processing node to be selected. The following specifically introduces a method for adjusting the corresponding relationship:

[0106] In one embodiment, Figure 8 As shown, step 704, based on the load ratio, determines the corresponding relationship between each array number and each node index, including:

[0107] Step 802, if the load ratio is greater than or equal to the maximum ratio threshold, determine the first processing node to be selected and the second processing node to be selected based on the load ratio, the first processing node to be selected is the processing node to be selected with the largest processing load among all the processing nodes to be selected, and the second processing node to be selected is the processing node to be selected with the smallest processing load among all the processing nodes to be selected.

[0108] The maximum ratio threshold may be determined based on the data type of the data to be transmitted, or may be determined based on actual business requirements, and is not limited here.

[0109] Specifically, when the load ratio is greater than or equal to the maximum ratio threshold, it indicates that there is a load imbalance among the processing nodes to be selected, and the adjustment of the corresponding relationship between the node index and each array number will be triggered. That is, based on the load ratio, the server 202 determines the processing node to be selected with the largest processing load among the processing nodes to be selected as the first processing node to be selected, and based on the load ratio, determines the processing node to be selected with the smallest processing load among the processing nodes to be selected as the second processing node to be selected.

[0110] For example, the maximum ratio threshold is 3:1:1, and the load ratio between the to-be-selected processing node A1, the to-be-selected processing node A2, and the to-be-selected processing node A3 is 5:1.5:1. At this time, the load ratio (5:1.5:1) is greater than the maximum ratio threshold (3:1:1), indicating that there is a load imbalance between the to-be-selected processing node A1, the to-be-selected processing node A2, and the to-be-selected processing node A3. At this time, based on the load ratio (5:1.5:1), it can be determined that the processing load of the to-be-selected processing node A1 is the largest, and the processing load of the to-be-selected processing node A3 is the smallest. Therefore, the to-be-selected processing node A1 is determined as the first to-be-selected processing node, and the to-be-selected processing node A3 is determined as the second to-be-selected processing node.

[0111] Step 804 , reducing the number of node indexes of the first node to be selected for processing, and adjusting the correspondence between the node index of the first node to be selected for processing and each array number based on the reduced number of node indexes of the first node to be selected for processing.

[0112] Since there is a load imbalance between the processing nodes to be selected, the processing load of the first processing node to be selected should be adjusted, that is, to avoid transmitting data to the first processing node to be selected for processing, so it is necessary to reduce the number of node indexes corresponding to the first processing node to be selected in the correspondence between the node index and each array number. Specifically, the server 202 reduces the number of node indexes of the first processing node to be selected, and adjusts the correspondence between the node index of the first processing node to be selected and each array number based on the reduced number of node indexes of the first processing node to be selected.

[0113] Step 806: increase the number of node indexes of the second node to be selected for processing, and adjust the correspondence between the node indexes of the second node to be selected for processing and each array number based on the increased number of node indexes of the second node to be selected for processing.

[0114] When adjusting the processing load of the first processing node to be selected, the processing load of the second processing node to be selected should also be adjusted. For the purpose of load balancing, the data should be transmitted to the second processing node to be selected for processing as much as possible. Therefore, it is necessary to increase the number of node indexes corresponding to the second processing node to be selected in the corresponding relationship between the node index and each array number. Specifically, the server 202 increases the number of node indexes of the second processing node to be selected, and adjusts the corresponding relationship between the node index of the second processing node to be selected and each array number based on the increased number of node indexes of the second processing node to be selected.

[0115] To facilitate understanding of step 804 and step 806, Fig. 9 As shown, the node 901 to be selected for processing corresponds to node index 0, the node 902 to be selected for processing corresponds to node index 1, and the node 903 to be selected for processing corresponds to node index 2, and the correspondence between the node index and each array number includes 4 correspondences between node index 0 and each array number, 2 correspondences between node index 1 and each array number, and 2 correspondences between node index 2 and each array number. And the specific correspondence is as follows Fig. 9 As shown in Figure (A): the correspondence between array number 1 and node index 0, the correspondence between array number 2 and node index 0, the correspondence between array number 3 and node index 1, the correspondence between array number 4 and node index 2, the correspondence between array number 5 and node index 1, the correspondence between array number 6 and node index 0, the correspondence between array number 7 and node index 2, and the correspondence between array number 8 and node index 0.

[0116] Based on this, if the node to be selected for processing 901 is determined as the first node to be selected for processing, and the node to be selected for processing 903 is determined as the second node to be selected for processing, it is necessary to reduce the number of node index 0 corresponding to the first node to be selected for processing (node ​​to be selected for processing 901), and increase the number of node index 2 corresponding to the second node to be selected for processing (node ​​to be selected for processing 903).

[0117] Therefore, server 202 can adjust the correspondence between the 4 node indexes 0 and the array numbers in the correspondence to 3 correspondences between node indexes 0 and the array numbers. And adjust the correspondence between the 2 node indexes 2 and the array numbers in the correspondence to 3 correspondences between node indexes 2 and the array numbers. For node index 0: the correspondence between array number 1 and node index 0, the correspondence between array number 2 and node index 0, the correspondence between array number 6 and node index 0, and the correspondence between array number 8 and node index 0. At this time, the correspondence between array number 8 and node index 0 can be untied. And for node index 2: the correspondence between array number 4 and node index 2 and the correspondence between array number 7 and node index 2. Since the correspondence between array number 8 and node index 0 has been untied, the correspondence between array number 8 and node index 2 can be established at this time.

[0118] Therefore, the corresponding relationship between the adjusted node index and each array number is as follows: Fig. 9 As shown in Figure (B), it specifically includes: the correspondence between array number 1 and node index 0, the correspondence between array number 2 and node index 0, the correspondence between array number 3 and node index 1, the correspondence between array number 4 and node index 2, the correspondence between array number 5 and node index 1, the correspondence between array number 6 and node index 0, the correspondence between array number 7 and node index 2, and the correspondence between array number 8 and node index 2.

[0119] In this embodiment, while determining the corresponding relationship by considering the real-time processing load of each processing node to be selected, the corresponding relationship is further adjusted based on the real-time processing load of each processing node to be selected to achieve dynamic adjustment of the corresponding relationship and ensure the real-time and reliability of the corresponding relationship.

[0120] In one embodiment, the data to be transmitted is service data belonging to the target service, and the transmission order of the data to be transmitted in each service data of the target service is a preset transmission order;

[0121] The data to be transmitted also includes message sequence number information, which includes a first modulus value and a message sequence identifier. The first modulus value is obtained by taking the modulus of the number of array units corresponding to the first processing link of the target service based on the target parameter, and the message sequence identifier is obtained by increasing the first modulus value packet by packet.

[0122] Specifically, the data to be transmitted is the business data of the target business, and the data to be transmitted has a corresponding transmission order in each business data of the target business, and the transmission order is a preset transmission order. For example, the target business includes business data D1, business data D2, business data D3 and business data D4, and business data D1 to business data D4 are transmitted in sequence. If the data to be transmitted is specifically business data D2 of the target business, then the preset transmission order of the data to be transmitted in each business data is the second.

[0123] Furthermore, processing the target business requires multiple processing steps. After the server 202 processes the data to be transmitted based on the first processing step of the target business, it is necessary to number the data to be transmitted, that is, to assign message sequence number information (m, index) to the data to be transmitted. The message sequence number information is specifically composed of two parts, including a first modulus value (m) and a message sequence identifier (index).

[0124] Among them, the first modulus value (m) is a value obtained by taking the modulus of the number of array units corresponding to the first processing link based on the target parameter of the data to be transmitted, specifically, a value obtained by taking the modulus of the number of array units corresponding to the first processing link based on the five-tuple hash value of the data to be transmitted. Secondly, the message sequence identifier (index) is a value that is incremented packet by packet on the premise that the first modulus value (m) is obtained after taking the modulus.

[0125] Furthermore, the message sequence number information (m, index) is specifically composed of the first modulus value (m) occupying the high-order bits and the message sequence identifier (index) occupying the low-order bits, and the message sequence number information can serve as the message sequence number of the data to be transmitted.

[0126] In this embodiment, since the message sequence number information can be used as the data to be transmitted, when multiple service data of the target service are transmitted, the orderliness of the transmission of each service data can be guaranteed based on the message sequence number information, thereby improving the reliability of this solution.

[0127] The above embodiment describes the process of adjusting the correspondence between each node index and each array number, which may cause changes in the transmission order between multiple service data of the target service during the transmission process, thereby affecting the orderliness of service data transmission. Therefore, the following will introduce how to ensure the orderliness of service data transmission:

[0128] In one embodiment, Fig.10 As shown, after adjusting the correspondence between each node index and each array number, the method further includes:

[0129] Step 1002: If the target array unit indicated by the target array number is an array unit whose corresponding relationship has been adjusted, mark the data to be transmitted.

[0130] Specifically, if the target array unit indicated by the target array number is an array unit whose corresponding relationship has been adjusted, the server 202 will mark the data to be transmitted. The marking process may be coloring the data to be transmitted. It should be understood that in actual applications, the array unit indicated by the array number corresponding to the business data in the target business after calculation is an array unit whose corresponding relationship has been adjusted, and all business data that meets this condition needs to be marked.

[0131] Step 1004, after the last processing link of the target service completes the processing of the data to be transmitted, the marked data to be transmitted is allocated to the first sorting domain corresponding to the first modulus value, and the first sorting domain includes each first service data carrying the first modulus value.

[0132] Among them, one first modulus value corresponds to one sorting domain, and business data of different sorting domains cannot be sorted together.

[0133] Specifically, after the server 202 completes the processing of the data to be transmitted in the last processing link of the target service, if the data to be transmitted has not been marked, the processing of the entire pipeline is completed normally. If the data to be transmitted has been marked, the data to be transmitted after the marking process needs to be allocated, and the first sorting field corresponding to the first modulus value in the included message sequence information includes each first service data carrying the first modulus value, and each first service data is the service data after the marking process.

[0134] Step 1006: in the first sorting domain, re-sorting is performed based on the message sequence identifiers of the first service data. The order of the re-sorted data to be transmitted in the re-sorted first service data is the preset transmission order.

[0135] Specifically, since the first sorting domain includes each first business data carrying a first modulus value, the server 202 performs reordering processing in the first sorting domain based on the message sequence identifier of each first business data, thereby obtaining the sorted data to be transmitted, and the order of each first business data after the reordering processing is the preset transmission order.

[0136] For example, the first sorting domain includes the first service data D1 and the second service data D2, and the message sequence identifier of the first service data D1 is (2, 4), and the message sequence identifier of the second service data D2 is (2, 5). Then, based on the message sequence identifier (4) of the first service data D1 and the message sequence identifier (5) of the second service data D2, re-sorting is performed, that is, the first service data D1 will be processed before the second service data D2.

[0137] In this embodiment, taking into account the situation that the adjustment of the corresponding relationship during the data transmission process may cause changes in the transmission order between multiple business data of the target business, the business data are sorted in the last processing link of the target business to ensure the orderliness of the transmission of each business data, thereby further improving the reliability of this solution.

[0138] In one embodiment, Fig.11 As shown, the target parameter is a five-tuple hash value;

[0139] Step 304, determining the target array number corresponding to the target parameter of the data to be transmitted from each array number, including:

[0140] Step 1102, performing a modulo process on the five-tuple hash value of the data to be transmitted to obtain a five-tuple hash value after the modulo process.

[0141] Specifically, since the target parameter of the data to be transmitted is a five-tuple hash value, the server 202 performs modulo processing on the number of array units corresponding to the next processing link based on the five-tuple hash value of the data to be transmitted to obtain the five-tuple hash value after modulo processing.

[0142] Step 1104, determine the target array number from each array number by taking the modulo five-tuple hash value.

[0143] Specifically, the server 202 determines the modulo five-tuple hash value as the target array number in each array number.

[0144] In this embodiment, a method for specifically determining the target array number in practical applications is obtained through the five-tuple hash value of the data to be transmitted, thereby ensuring the practicality of this solution in determining the target array number.

[0145] The following will introduce the overall implementation process of processing node determination, such as Fig.12 As shown, a method for processing node determination is provided, and the method is applied to Figure 2 Taking the server 202 in the example as an example, the following steps are included:

[0146] Step 1201, obtaining the array number of each array unit corresponding to the next processing link of the data to be transmitted.

[0147] Among them, in the process of transmitting data based on the packet processing model of the pipeline architecture, that is, the process of processing the business data of the target business includes multiple processing links. Based on this, each processing link in the pipeline architecture will have a corresponding linear MAP table, which is specifically a one-dimensional array unit with a length of N, that is, each processing link has corresponding multiple array units, and each array unit also has a unique corresponding array number.

[0148] Specifically, after the server 202 completes the processing of the data to be transmitted in the current processing link, the server 202 first determines that the data to be transmitted should be transmitted to the next processing link. At this time, the linear MAP table corresponding to the next processing link will be obtained in real time, that is, each array unit corresponding to the next processing link will be obtained, and since each array unit has a unique corresponding array number, the server 202 can further obtain the array number of each array unit corresponding to the next processing link.

[0149] Step 1202, performing a modulo process on the five-tuple hash value of the data to be transmitted to obtain a five-tuple hash value after the modulo process.

[0150] Step 1203, determine the target array number from each array number by taking the modulo five-tuple hash value.

[0151] Step 1204 , obtaining the maximum processing load of each processing node to be selected, and calculating the maximum load ratio between the maximum processing loads of each processing node to be selected.

[0152] The maximum processing load is the maximum load that the processing node to be selected can bear, and specifically may be a preset maximum load of the processing node to be selected, or a maximum load determined based on attribute information of the processing node to be selected, which is not limited here.

[0153] Specifically, the server 202 first obtains the maximum processing load of each processing node to be selected, and then calculates the ratio of the maximum processing load of each processing node to be selected to obtain the maximum load ratio.

[0154] Step 1205: Determine the quantity ratio of the node index quantity of each to-be-selected processing node based on the maximum load ratio.

[0155] The quantity ratio is specifically the quantity ratio of the node index quantity in the corresponding relationship of each to-be-selected processing node.

[0156] Specifically, the server 202 determines the quantity ratio of the node index quantity of each to-be-selected processing node based on the maximum load ratio. The quantity ratio may be determined based on the maximum load ratio.

[0157] Step 1206, based on the quantity ratio, generate a corresponding relationship between each node index and each array number.

[0158] Among them, in the correspondence between each node index and each array number, the ratio between the numbers of each node index is the quantity ratio.

[0159] Specifically, since each processing link in the pipeline architecture will have a corresponding linear MAP table, the linear MAP table is specifically a one-dimensional array unit with a length of N, that is, the next processing link has corresponding N array units, and each array unit also has a unique corresponding array number, so the next processing link has corresponding N array numbers.

[0160] Based on this, the server 202 first determines the number of node indexes of each node to be selected for processing based on the ratio of the number of node indexes of the nodes to be selected for processing calculated in step 604 and the specific number of each array number, and then maps and binds each node index with each array number based on the node index number of each node index, thereby generating a corresponding relationship between each node index and each array number.

[0161] Step 1207: Obtain the message processing rate of each processing node to be selected, and calculate the load ratio between the processing loads of each processing node to be selected based on each message processing rate.

[0162] The message processing rate is used to describe the real-time processing rate of the processing node to be selected.

[0163] Specifically, the server 202 first obtains the current cumulative amount of processed data of each processing node to be selected in real time, and then calculates the message processing rate based on the current cumulative amount of processed data of the processing node to be selected. Further, the load ratio between the processing loads of each processing node to be selected is calculated based on each message processing rate. The aforementioned load ratio is used to describe the difference between the processing loads of each processing node to be selected, that is, the load ratio reflects the degree of load balancing of each processing node to be selected.

[0164] Step 1208: If the load ratio is greater than or equal to the maximum ratio threshold, determine a first processing node to be selected and a second processing node to be selected based on the load ratio.

[0165] The maximum ratio threshold may be determined based on the data type of the data to be transmitted, or may be determined based on actual business requirements, and is not limited here.

[0166] Specifically, when the load ratio is greater than or equal to the maximum ratio threshold, it indicates that there is a load imbalance among the processing nodes to be selected, and the adjustment of the corresponding relationship between the node index and each array number will be triggered. That is, based on the load ratio, the server 202 determines the processing node to be selected with the largest processing load among the processing nodes to be selected as the first processing node to be selected, and based on the load ratio, determines the processing node to be selected with the smallest processing load among the processing nodes to be selected as the second processing node to be selected.

[0167] Step 1209 , reducing the number of node indexes of the first node to be selected for processing, and adjusting the correspondence between the node index of the first node to be selected for processing and each array number based on the reduced number of node indexes of the first node to be selected for processing.

[0168] Since there is a load imbalance between the processing nodes to be selected, the processing load of the first processing node to be selected should be adjusted, that is, to avoid transmitting data to the first processing node to be selected for processing, so it is necessary to reduce the number of node indexes corresponding to the first processing node to be selected in the correspondence between the node index and each array number. Specifically, the server 202 reduces the number of node indexes of the first processing node to be selected, and adjusts the correspondence between the node index of the first processing node to be selected and each array number based on the reduced number of node indexes of the first processing node to be selected.

[0169] Step 1210: increase the number of node indexes of the second to-be-selected processing nodes, and adjust the correspondence between the node indexes of the second to-be-selected processing nodes and the array numbers based on the increased number of node indexes of the second to-be-selected processing nodes.

[0170] When adjusting the processing load of the first processing node to be selected, the processing load of the second processing node to be selected should also be adjusted. For the purpose of load balancing, the data should be transmitted to the second processing node to be selected for processing as much as possible. Therefore, it is necessary to increase the number of node indexes corresponding to the second processing node to be selected in the corresponding relationship between the node index and each array number. Specifically, the server 202 increases the number of node indexes of the second processing node to be selected, and adjusts the corresponding relationship between the node index of the second processing node to be selected and each array number based on the increased number of node indexes of the second processing node to be selected.

[0171] Step 1211, based on the correspondence between the node index of each to-be-selected processing node and each array number, determine the target node index corresponding to the target array number.

[0172] Among them, since each processing link will have a corresponding linear MAP table, the linear MAP table is specifically a one-dimensional array unit with a length of N, and each array unit also has a unique corresponding array number, so there is a corresponding relationship between the array number corresponding to each array unit and the node index of each processing node to be selected, and the node index can index a specific processing node, that is, the node index is used to uniquely indicate the corresponding processing node. Secondly, the corresponding relationship is determined according to the load of each processing node to be selected, for example, the corresponding relationship is determined based on the maximum processing load of each processing node to be selected, or the corresponding relationship is determined based on the real-time processing load of each processing node to be selected, which is not limited here.

[0173] Specifically, based on the correspondence between the node index of each to-be-selected processing node and each array number, the server 202 first determines the target node index corresponding to the target array number, and then the server 202 transmits the data to be transmitted to the target processing node uniquely indicated by the target node index, so that the target processing node performs corresponding data processing on the data to be transmitted.

[0174] Step 1212: If the target array unit indicated by the target array number is an array unit whose corresponding relationship has been adjusted, mark the data to be transmitted.

[0175] Specifically, if the target array unit indicated by the target array number is an array unit whose corresponding relationship has been adjusted, the server 202 will mark the data to be transmitted. The marking process may be coloring the data to be transmitted. It should be understood that in actual applications, the array unit indicated by the array number corresponding to the business data in the target business after calculation is an array unit whose corresponding relationship has been adjusted, and all business data that meets this condition needs to be marked.

[0176] Step 1213, after the processing of the data to be transmitted is completed in the last processing link of the target service, the data to be transmitted after the marking processing is allocated to the first sorting field corresponding to the first modulus value.

[0177] Among them, one first modulus value corresponds to one sorting domain, and business data of different sorting domains cannot be sorted together.

[0178] Specifically, after the server 202 completes the processing of the data to be transmitted in the last processing link of the target service, if the data to be transmitted has not been marked, the processing of the entire pipeline is completed normally. If the data to be transmitted has been marked, the data to be transmitted after the marking process needs to be allocated, and the first sorting field corresponding to the first modulus value in the message sequence information included includes each first service data carrying the first modulus value, and each first service data is service data after the marking process.

[0179] Step 1214: In the first sorting domain, re-sort the data based on the message sequence identifiers of the first service data.

[0180] Specifically, since the first sorting domain includes each first business data carrying a first modulus value, the server 202 performs reordering processing in the first sorting domain based on the message sequence identifier of each first business data, thereby obtaining the sorted data to be transmitted, and the order of each first business data after the reordering processing is the preset transmission order.

[0181] It should be understood that the specific implementation of steps 1201 to 1214 is Figures 3 to 11 The embodiments described in are similar and will not be repeated here.

[0182] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0183] Based on the same inventive concept, the embodiment of the present application also provides a processing node determination device for implementing the processing node determination method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in the one or more processing node determination device embodiments provided below can refer to the limitations of the processing node determination method above, and will not be repeated here.

[0184] In one embodiment, Fig.13 As shown, a processing node determination device is provided, including: an acquisition module 1302 and a determination module 1304, wherein:

[0185] An acquisition module 1302 is used to acquire the array number of each array unit corresponding to the next processing link of the data to be transmitted, and the next processing link includes multiple processing nodes to be selected;

[0186] Determination module 1304 is used to determine the target array number corresponding to the target parameters of the data to be transmitted from each array number; and based on the correspondence between the node index of each processing node to be selected and each array number, determine the target node index corresponding to the target array number. The correspondence is determined according to the load of each processing node to be selected, and the target node index is used to uniquely indicate the target processing node used to process the data to be transmitted.

[0187] In one embodiment, the acquisition module 1302 is further used to acquire the maximum processing load of each processing node to be selected, and calculate the maximum load ratio between the maximum processing loads of each processing node to be selected;

[0188] The determination module 1304 is further configured to determine the quantity ratio of the node index quantity of each to-be-selected processing node based on the maximum load ratio; and generate a corresponding relationship between each node index and each array number based on the quantity ratio.

[0189] In one embodiment, the acquisition module 1302 is further used to acquire the message processing rate of each processing node to be selected, and calculate the load ratio between the processing loads of each processing node to be selected based on each message processing rate, and the load ratio is used to describe the difference between the processing loads of each processing node to be selected;

[0190] The determination module 1304 is further configured to determine the correspondence between each array number and each node index based on the load ratio.

[0191] In one embodiment, the determination module 1304 is specifically used to determine the first processing node to be selected and the second processing node to be selected based on the load ratio if the load ratio is greater than or equal to the maximum ratio threshold, the first processing node to be selected being the processing node to be selected with the largest processing load among all the processing nodes to be selected, and the second processing node to be selected being the processing node to be selected with the smallest processing load among all the processing nodes to be selected; and reduce the number of node indexes of the first processing node to be selected, and based on the reduced number of node indexes of the first processing node to be selected, adjust the correspondence between the node index of the first processing node to be selected and each array number; and increase the number of node indexes of the second processing node to be selected, and based on the increased number of node indexes of the second processing node to be selected, adjust the correspondence between the node index of the second processing node to be selected and each array number.

[0192] In one embodiment, the data to be transmitted is business data belonging to the target business, and the transmission order of the data to be transmitted among the business data of the target business is a preset transmission order; and the data to be transmitted also includes message sequence number information, the message sequence number information includes a first modulus value and a message sequence identifier, the first modulus value is obtained by taking the modulus of the number of array units corresponding to the first processing link of the target business based on the target parameter, and the message sequence identifier is obtained by performing packet-by-packet increment processing based on the first modulus value.

[0193] In one embodiment, the processing node determination device further includes: a marking processing module 1306, an allocation module 1308, and a sorting module 1310;

[0194] A processing module 1306 is used for marking the data to be transmitted if the target array unit indicated by the target array number is the array unit whose corresponding relationship has been adjusted after the determination module 1304 adjusts the corresponding relationship between each node index and each array number;

[0195] The allocation module 1308 is used to allocate the data to be transmitted after the marking process is completed in the last processing link of the target service to a first sorting domain corresponding to the first modulus value, wherein the first sorting domain includes each first service data carrying the first modulus value;

[0196] The sorting module 1310 is used to perform reordering processing in the first sorting domain based on the message sequence identifier of each first business data, and the order of the reordered data to be transmitted in each first business data after the reordering processing is the preset transmission order.

[0197] In one embodiment, the target parameter is a five-tuple hash value;

[0198] The determination module 1304 is specifically used to perform a modulo process on the five-tuple hash value to obtain the five-tuple hash value after the modulo process; and determine the target array number from each array number through the five-tuple hash value after the modulo process.

[0199] Each module in the above-mentioned processing node determination device can be implemented in whole or in part by software, hardware, or a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each of the above modules.

[0200] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.14As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the array number of each array unit, the corresponding relationship between its own node index and each array number, etc. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a processing node determination method is implemented.

[0201] Those skilled in the art will understand that Fig.14 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0202] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0203] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0204] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0205] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0206] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0207] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A method for determining a processing node, characterized in that: The method comprises: Obtaining array numbers of each array unit corresponding to a next processing link of the data to be transmitted, wherein the next processing link includes a plurality of processing nodes to be selected; Determine, from each of the array numbers, a target array number corresponding to the target parameter of the data to be transmitted; Acquire the message processing rate of each of the processing nodes to be selected, and calculate the load ratio between the processing loads of each of the processing nodes to be selected based on each of the message processing rates, wherein the load ratio is used to describe the difference between the processing loads of each of the processing nodes to be selected; If the load ratio is greater than or equal to the maximum ratio threshold, determine a first processing node to be selected and a second processing node to be selected based on the load ratio, wherein the first processing node to be selected is the processing node to be selected with the largest processing load among the processing nodes to be selected, and the second processing node to be selected is the processing node to be selected with the smallest processing load among the processing nodes to be selected; Reducing the number of node indexes of the first to-be-selected processing nodes, and adjusting the correspondence between the node indexes of the first to-be-selected processing nodes and the array numbers based on the reduced number of node indexes of the first to-be-selected processing nodes; Increasing the number of node indexes of the second nodes to be selected for processing, and adjusting the correspondence between the node indexes of the second nodes to be selected for processing and the array numbers based on the increased number of node indexes of the second nodes to be selected for processing; Based on the correspondence between the node index of each of the processing nodes to be selected and each of the array numbers, the target node index corresponding to the target array number is determined, the correspondence is determined according to the load of each of the processing nodes to be selected, and the target node index is used to uniquely indicate the target processing node for processing the data to be transmitted.

2. The method according to claim 1, characterized in that The method further comprises: Acquire the maximum processing load of each of the processing nodes to be selected, and calculate the maximum load ratio between the maximum processing loads of each of the processing nodes to be selected; Based on the maximum load ratio, determining a quantity ratio of the node index quantities of the nodes to be selected for processing; Based on the quantity ratio, a corresponding relationship between each of the node indexes and each of the array numbers is generated.

3. The method according to claim 1, characterized in that The data to be transmitted is service data belonging to a target service, and the transmission order of the data to be transmitted in each service data of the target service is a preset transmission order; The data to be transmitted also includes message sequence number information, which includes a first modulus value and a message sequence identifier. The first modulus value is obtained by taking the modulus of the number of array units corresponding to the first processing link of the target service based on the target parameter, and the message sequence identifier is obtained by increasing the first modulus value packet by packet.

4. The method according to claim 3, characterized in that After adjusting the correspondence between each of the node indexes and each of the array numbers, the method further includes: If the target array unit indicated by the target array number is an array unit whose corresponding relationship has been adjusted, marking the data to be transmitted; After the last processing link of the target service completes the processing of the data to be transmitted, the data to be transmitted after the marking processing is allocated to a first sorting domain corresponding to the first modulus value, where the first sorting domain includes each first service data carrying the first modulus value; In the first sorting domain, reordering is performed based on the message sequence identifier of each of the first service data, and the order of the data to be transmitted after the reordering in each of the first service data after the reordering is the preset transmission order.

5. The method according to claim 1, characterized in that The target parameter is a five-tuple hash value; The determining, from each of the array numbers, a target array number corresponding to the target parameter of the data to be transmitted includes: Performing a modulo process on the five-tuple hash value to obtain a five-tuple hash value after the modulo process; The target array number is determined from each of the array numbers using the modulo five-tuple hash value.

6. A processing node determination device, characterized in that: The device comprises: An acquisition module, used to acquire the array number of each array unit corresponding to the next processing link of the data to be transmitted, wherein the next processing link includes a plurality of processing nodes to be selected; The acquisition module is further used to acquire the message processing rate of each of the processing nodes to be selected, and calculate the load ratio between the processing loads of each of the processing nodes to be selected based on each of the message processing rates, wherein the load ratio is used to describe the difference between the processing loads of each of the processing nodes to be selected; A determination module, specifically used to determine a first processing node to be selected and a second processing node to be selected based on the load ratio if the load ratio is greater than or equal to a maximum ratio threshold, the first processing node to be selected being the processing node to be selected with the largest processing load among the processing nodes to be selected, and the second processing node to be selected being the processing node to be selected with the smallest processing load among the processing nodes to be selected; reduce the number of node indexes of the first processing node to be selected, and adjust the correspondence between the node index of the first processing node to be selected and each of the array numbers based on the reduced number of node indexes of the first processing node to be selected; increase the number of node indexes of the second processing node to be selected, and adjust the correspondence between the node index of the second processing node to be selected and each of the array numbers based on the increased number of node indexes of the second processing node to be selected; The determination module is also used to determine the target array number corresponding to the target parameter of the data to be transmitted from each of the array numbers; and based on the correspondence between the node index of each of the processing nodes to be selected and each of the array numbers, determine the target node index corresponding to the target array number, the correspondence is determined according to the load of each of the processing nodes to be selected, and the target node index is used to uniquely indicate the target processing node for processing the data to be transmitted.

7. The device according to claim 6, characterized in that The acquisition module is further used to acquire the maximum processing load of each of the processing nodes to be selected, and calculate the maximum load ratio between the maximum processing loads of each of the processing nodes to be selected; The determination module is further configured to determine a quantity ratio of node index quantities of each of the nodes to be selected for processing based on the maximum load ratio; Based on the quantity ratio, a corresponding relationship between each of the node indexes and each of the array numbers is generated.

8. The device according to claim 6, characterized in that The data to be transmitted is business data belonging to the target business, and the transmission order of the data to be transmitted among the business data of the target business is a preset transmission order; the data to be transmitted also includes message sequence number information, and the message sequence number information includes a first modulus value and a message sequence identifier, the first modulus value is obtained by taking the modulus of the number of array units corresponding to the first processing link of the target business based on the target parameter, and the message sequence identifier is obtained by increasing the first modulus value packet by packet.

9. The device according to claim 8, characterized in that The device also includes a marking processing module, an allocation module and a sorting module; The marking processing module is used for, after adjusting the correspondence between each of the node indexes and each of the array numbers, if the target array unit indicated by the target array number is the array unit with the adjusted correspondence, performing marking processing on the data to be transmitted; The allocation module is used to allocate the data to be transmitted after the marking process is completed on the data to be transmitted in the last processing link of the target service, to a first sorting domain corresponding to the first modulus value, wherein the first sorting domain includes each first service data carrying the first modulus value; The sorting module is used to perform reordering processing in the first sorting domain based on the message sequence identifier of each of the first business data, and the order of the data to be transmitted after the reordering processing in each of the first business data after the reordering processing is the preset transmission order.

10. The device according to claim 6, characterized in that The target parameter is a five-tuple hash value; The determination module is specifically used to perform a modulo process on the five-tuple hash value to obtain a five-tuple hash value after the modulo process; and determine the target array number from each of the array numbers through the five-tuple hash value after the modulo process.

11. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

13. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images