Message data processing method and device, computer device and storage medium

Abstract

The application relates to a message data processing method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a message stream containing multiple pieces of message data; the message data comprises a message type and a message identification number; according to the mapping relationship between the message types and the shunt identification, the shunt identification corresponding to each message type is determined; for each piece of message data, in the case that the shunt identification of the message data corresponds to multiple candidate processing systems, the identification number interval to which the message identification number belongs is determined; and a target processing system matched with the identification number interval is determined from the candidate processing systems. According to the above method, each piece of message data is divided according to the message type and the message identification number, the error probability of the subsequent candidate processing system is reduced, and the message data distribution accuracy is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of communication, in particular to a message data processing method and device, computer equipment, storage medium and computer program product. BACKGROUND

[0002] With the development of database technology, it is possible to perform server-based and domestic database-based small machine downward domestic reform, database migration and other solutions.

[0003] In the traditional technology, for mass message data, the mass message data is distributed according to the real-time processing capacity of different database systems. Although this distribution method can avoid database system overload to a certain extent, it still has database system error and low distribution accuracy. SUMMARY

[0004] Therefore, it is necessary to provide a message data processing method and device, computer equipment, computer readable storage medium and computer program product capable of message data distribution accuracy for the above technical problems.

[0005] In a first aspect, the present application provides a message data processing method. The method comprises:

[0006] Obtaining a message stream containing a plurality of message data; the message data includes message type and message identification number; determining the shunt identification corresponding to each message type according to each message type and the mapping relationship between message type and shunt identification; for each message data, if the shunt identification of the message data corresponds to a plurality of candidate processing systems, determining the identification number interval to which the message identification number belongs; determining the target processing system matched with the identification number interval from each candidate processing system, and sending the message data to the target processing system.

[0007] In one embodiment, the method further comprises: obtaining a plurality of preset intervals configured for identification numbers; each preset interval includes an identification number lower limit and an identification number upper limit; comparing the message identification number with the identification number lower limit and the identification number upper limit of each preset interval respectively, and determining the identification number interval to which the message identification number belongs from each preset interval.

[0008] In one embodiment, the method further comprises: for each candidate processing system, determining the data processing capacity of the candidate processing system based on the system information of the candidate processing system; determining the identification number interval matched with the candidate processing system according to the data processing capacity; the span of the message identification number contained in the identification number interval is positively correlated with the data processing capacity.

[0009] In one of the embodiments, the determining the identification number interval matched with the candidate processing system according to the data processing capability comprises: obtaining an identification number span range configured for the identification number interval; determining a target span matched with the data processing capability from the identification number span range; and selecting at least a part from a plurality of to-be-assigned identification numbers according to the target span to form the identification number interval matched with the candidate processing system.

[0010] In one of the embodiments, the determining the target processing system matched with the identification number interval from the candidate processing systems comprises: determining at least two to-be-selected processing systems matched with the identification number interval from the candidate processing systems; obtaining an identification number processing record of each of the to-be-selected processing systems and an original message identification number associated with the message identification number; and screening a target processing system containing the original message identification number in the identification number processing record from the candidate processing systems.

[0011] In one of the embodiments, the sending the message data to the target processing system comprises: obtaining a system real-time load of the target processing system; screening a target message sending mode matched with the system real-time load from a plurality of message sending modes; and sending the message data to the target processing system according to the target message sending mode.

[0012] In one of the embodiments, the target processing system comprises at least two target systems with different system database architectures; and the method further comprises: determining a sending order of the message data for each of the target systems respectively; sending the message data to a target system in front of the sending order; and the target system in front of the sending order is used to copy and forward the message data to a target system behind the sending order.

[0013] In a second aspect, the present application further provides a message data processing device. The device comprises:

[0014] The obtaining module is configured to obtain a message stream containing a plurality of message data; the message data comprises a message type and a message identification number; the shunt identification determining module is configured to determine a shunt identification corresponding to each of the message types according to the message types and a mapping relationship between the message types and the shunt identifications; the identification number interval determining module is configured to determine an identification number interval to which the message identification number belongs for each of the message data in a case that a shunt identification of the message data corresponds to a plurality of candidate processing systems; and the processing module is configured to determine a target processing system matched with the identification number interval from the candidate processing systems and send the message data to the target processing system.

[0015] In a third aspect, the present application provides a computer device. The computer device comprises a memory and a processor. The memory stores a computer program. The processor implements the following steps when executing the computer program:

[0016] A packet stream comprising a plurality of packet data is obtained. The packet data comprises a packet type and a packet identification number. A mapping relationship between the packet type and a shunt identification is determined according to each packet type. For each packet data, a target processing system is determined from a plurality of candidate processing systems according to a corresponding shunt identification of the packet data and a corresponding identification number interval of the packet identification number, and the packet data is sent to the target processing system.

[0017] In a fourth aspect, the present application provides a computer readable storage medium. The computer readable storage medium stores a computer program. The computer program is executed by a processor to implement the following steps:

[0018] A packet stream comprising a plurality of packet data is obtained. The packet data comprises a packet type and a packet identification number. A mapping relationship between the packet type and a shunt identification is determined according to each packet type. For each packet data, a target processing system is determined from a plurality of candidate processing systems according to a corresponding shunt identification of the packet data and a corresponding identification number interval of the packet identification number, and the packet data is sent to the target processing system.

[0019] In a fifth aspect, the present application provides a computer program product. The computer program product comprises a computer program. The computer program is executed by a processor to implement the following steps:

[0020] A packet stream comprising a plurality of packet data is obtained. The packet data comprises a packet type and a packet identification number. A mapping relationship between the packet type and a shunt identification is determined according to each packet type. For each packet data, a target processing system is determined from a plurality of candidate processing systems according to a corresponding shunt identification of the packet data and a corresponding identification number interval of the packet identification number, and the packet data is sent to the target processing system.

[0021] The message data processing method, device, computer device, storage medium and computer program product, obtain a message stream containing a plurality of message data; the message data includes a message type and a message identification number; according to the mapping relationship between each message type and a shunt identification, the shunt identification corresponding to each message type is determined, and each message data is preliminarily divided according to the message type, which is beneficial to reduce the number of message types corresponding to the message data borne by the subsequent candidate processing system, and reduce the error probability of the candidate processing system. For each message data, in the case that the shunt identification of the message data corresponds to a plurality of candidate processing systems, the identification number interval to which the message identification number belongs is determined, and the target processing system matched with the identification number interval is determined from each candidate processing system. By using the above method, on the one hand, each message data is preliminarily divided according to the message type, which reduces the error probability of the subsequent candidate processing system and improves the distribution accuracy of the message data; on the other hand, based on the message identification number, the target processing system to which the message data is to be sent is determined in each candidate system, the target processing system is accurately determined, the message data is sent to the target processing system, and the distribution accuracy of the message data is further improved. BRIEF DESCRIPTION OF DRAWINGS

[0022] Figure 1 An application environment diagram of a message data processing method in an embodiment;

[0023] Figure 2 A flowchart of a message data processing method in an embodiment;

[0024] Figure 3 A flowchart of a target processing system determination method in an embodiment;

[0025] Figure 4 A flowchart of a message shunting processing method in an embodiment;

[0026] Figure 5 A flowchart of a message shunting method in an embodiment;

[0027] Figure 6 A structural block diagram of a message data processing device in an embodiment;

[0028] Figure 7 An internal structure diagram of a computer device in an embodiment. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application is further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application.

[0030] It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or authorized by all parties. And the acquisition, storage, processing, transmission, etc. of the data comply with the relevant provisions of laws and regulations.

[0031] The message data processing method provided by the embodiments of the present application can be applied to an application environment as shown in Figure 1 . Among them, the processing server 102 is in communication connection with a plurality of candidate servers 104. The server data storage system can be integrated on the service, or placed on the cloud or other network servers.

[0032] The processing server 102 obtains a message stream containing a plurality of message data. The message data includes message type and message identification number. The processing server 102 determines the shunt identification corresponding to each message type according to the mapping relationship between each message type and the shunt identification. For each message data, the processing server 102 determines the identification number interval to which the message identification number belongs when the shunt identification corresponding to the message data corresponds to a plurality of candidate processing systems. The processing server 102 determines the target processing system matching the identification number interval from the candidate processing systems, and sends the message data to the target processing system.

[0033] Among them, the candidate processing system includes the processing system contained in the processing server 102 and each candidate server 104. Specifically, the processing server 102 determines the target processing system (the processing system contained in the target server in the processing server 102 and each candidate server 104) matching the identification number interval from the candidate processing systems, and sends the message data to the target processing system, that is, to the target server.

[0034] Among them, the processing server 102 and each candidate server 104 can be realized by an independent server or a server cluster composed of a plurality of servers.

[0035] In one embodiment, as shown in Figure 2 , a message data processing method is provided. Taking the processing server in Figure 1 as an example, the method includes:

[0036] S202, obtaining a message stream containing a plurality of message data.

[0037] The message flow is a process in which messages flow in the network according to certain orders and rules, and the process involves message data generation, encapsulation, transmission, reception, decryption and the like. The message flow can be a request message flow or a response message flow, which is not limited here.

[0038] The message flow can also be regarded as a collection of multiple message data.

[0039] The message data represents the specific information content of the message and is the actual data transmitted in the network communication process. The message data generally contains different contents according to different communication protocols, message sending parties, message receiving parties and the like. Generally, the message data contains two parts of a message header and a data body. The message header can include the message type and the message identification number of the message data, and the data body contains the actual transmitted data content, such as specific files, pictures, videos and the like.

[0040] The message type represents the type of the message data. In the transaction payment scenario, the message type can include ibps.101, ibps.103, Saps.601, ccms801 and the like.

[0041] The message identification number represents the identification number of the message data. In the transaction payment scenario, the message identification number can be a number containing 8 digits.

[0042] Specifically, the processing server can directly obtain the message flow containing multiple message data from other servers or other terminals. The processing server can also extract the message flow containing multiple message data from the data storage system.

[0043] In one embodiment, the processing server obtains the first message flow from other servers, obtains the second message flow from other terminals, and extracts the message flow containing multiple message data from the data storage system.

[0044] In one embodiment, the processing server combines the first message flow, the second message flow and the third message flow to obtain the combined message flow after obtaining the first message flow, the second message flow and the third message flow.

[0045] In one embodiment, the processing server combines part of the message flow in the first message flow, the second message flow and the third message flow to obtain the combined message flow according to the number of processing threads of the processing server.

[0046] S204, according to the mapping relationship between each message type and the split identification, determine the split identification corresponding to each message type.

[0047] The shunt identifier indicates an identifier of a shunt direction configured for the message data. For example, the shunt identifier 1 indicates that the message data can correspond to the shunt direction 1 and the shunt direction 2, and the shunt identifier 2 indicates that the message data can correspond to the shunt direction 1 and the shunt direction 3.

[0048] The mapping relationship between the message type and the shunt identifier can indicate that a plurality of message types correspond to one shunt identifier. For example, the message type ccms.801 corresponds to the shunt identifier 1.

[0049] It should be noted that the shunt identifier corresponding to the message type can indicate a shunt direction. For example, the shunt identifier 0 can indicate that the message data corresponds to only the shunt direction 1.

[0050] Specifically, the processing server obtains the mapping relationship between the message type and the shunt identifier from the data storage system, and determines the shunt identifier corresponding to each message type according to the message type and the mapping relationship.

[0051] S206, for each message data, in a case where the shunt identifier of the message data corresponds to a plurality of candidate processing systems, determining an identifier interval to which the message identifier belongs.

[0052] In a case where the shunt identifier of the message data corresponds to a plurality of candidate processing systems, it indicates that the message data can be distributed to a plurality of candidate processing systems. For example, the message data is sent to the candidate processing systems included in the processing server and the candidate processing systems included in the candidate server.

[0053] It should be noted that the shunt identifier of the message data can correspond to one candidate processing system, in which case it indicates that the message data can be distributed to one candidate processing system. For example, the message data is sent to the candidate processing system included in the processing server. For another example, the message data is sent to the candidate processing system included in the candidate server.

[0054] The candidate processing system indicates a system for carrying and processing the message data. More specifically, the candidate processing system can be a database system, for example, a domestic database system GoldenDB or a non-domestic database system Oracle.

[0055] The identifier interval indicates a set of identifiers. For example, the identifier interval can be [1, 10], [2, 15], [20, 100], etc. The identifier interval to which the message identifier belongs indicates which kind or kinds of identifier intervals the identifier belongs to.

[0056] The method for determining the interval of the identification number to which the message identification number belongs comprises the following steps: specifically, the processing server acquires a plurality of preset intervals configured for the identification number, and determines the preset interval to which the message identification number belongs as the interval of the identification number to which the message identification number belongs according to the interval boundary values contained in each of the preset intervals.

[0057] The interval of the identification number can be a preset interval, and the interval configuration can be performed in various ways.

[0058] In one of the embodiments, the processing server determines, for each candidate processing system, the data processing capability of the candidate processing system based on the system information of the candidate system, and determines the interval of the identification number matched with the candidate processing system according to the data processing capability, and the span of the message identification number contained in the interval of the identification number is positively correlated with the data processing capability.

[0059] S208, determining the target processing system matched with the interval of the identification number from the candidate processing systems, and sending the message data to the target processing system.

[0060] The target processing system can represent the candidate processing system corresponding to the interval of the identification number.

[0061] Specifically, before determining the target processing system, the processing server determines, for each candidate processing system, the data processing capability of the candidate processing system based on the system information of the candidate system, and determines the interval of the identification number matched with the candidate processing system according to the data processing capability, and the span of the message identification number contained in the interval of the identification number is positively correlated with the data processing capability.

[0062] Specifically, after obtaining the interval of the identification number matched with the candidate processing system, the processing server determines the target processing system matched with the interval of the identification number from the candidate processing systems according to the interval of the identification number matched with the candidate processing system.

[0063] The message data can be sent to the target processing system in various ways.

[0064] Specifically, the processing server sends the message data to the target processing system according to the target message sending mode matched with the target processing system.

[0065] In one of the embodiments, the processing server acquires the system real-time load of the target processing system, filters the target message sending mode matched with the system real-time load from the plurality of message sending modes, and sends the message data to the target processing system according to the target message sending mode.

[0066] In one of the embodiments, the processing server sends the message data to the target processing system according to the target message sending mode configured by the target processing system.

[0067] Specifically, after determining that the target processing system comprises at least two target systems with different system database architectures, the processing server determines a sending sequence of the message data for each target system respectively, and sends the message data to the target system with a higher sending sequence.

[0068] In one embodiment, after determining a sending sequence of the message data for each target system respectively, the processing server sends the message data to the target system with a higher sending sequence first, and then sends the message data to the target system with a lower sending sequence.

[0069] In the message data processing method, a message stream comprising a plurality of message data is obtained, the message data comprises a message type and a message identification number, a shunt identification corresponding to each message type is determined according to the message type and a mapping relationship between the message type and the shunt identification, and each message data is preliminarily classified according to the message type, so as to reduce the number of message types corresponding to the message data borne by the subsequent candidate processing system and reduce the error probability of the candidate processing system. For each message data, in a case where the shunt identification of the message data corresponds to a plurality of candidate processing systems, an identification number interval to which the message identification number belongs is determined, and a target processing system matched with the identification number interval is determined from the candidate processing systems. According to the above method, on the one hand, each message data is preliminarily classified according to the message type, so as to reduce the error probability of the subsequent candidate processing system and improve the distribution accuracy of the message data. On the other hand, based on the message identification number, the target processing system to which the message data is to be sent is determined in the candidate systems, the target processing system is accurately determined, the message data is sent to the target processing system, and the distribution accuracy of the message data is further improved.

[0070] In one embodiment, determining the identification number interval to which the message identification number belongs comprises: obtaining a plurality of preset intervals configured for the identification number, each preset interval comprises an identification number lower limit and an identification number upper limit, and the message identification number is compared with the identification number lower limit and the identification number upper limit of each preset interval respectively, and the identification number interval to which the message identification number belongs is determined from the preset intervals.

[0071] The preset interval represents an interval configured for the identification number, for example, [1, 10], [2, 15], [20, 100], etc.

[0072] The identification number lower limit represents an interval lower limit value of the preset interval, and the identification number upper limit represents an interval upper limit value of the preset interval. If the preset interval is [1, 10], the interval upper limit value is 10, and the interval lower limit value is 1.

[0073] The preset interval identifier lower limit and the preset interval identifier upper limit are used for comparison with the message identifier. If the message identifier is greater than or equal to the preset interval identifier lower limit and less than or equal to the preset interval identifier upper limit, the preset interval corresponding to the preset interval identifier lower limit and the preset interval identifier upper limit is determined as the identifier interval to which the message identifier belongs.

[0074] Specifically, the processing server can directly obtain a plurality of preset intervals configured for the identifier. The processing server can also configure a plurality of preset intervals for the identifier.

[0075] In one embodiment, the preset intervals can be matched with the candidate processing systems. The processing server can determine the preset intervals matched with the respective data processing capabilities of the candidate processing systems. The stronger the data processing capability, the larger the preset interval, and vice versa.

[0076] In this embodiment, by obtaining a plurality of preset intervals configured for the identifier, each preset interval includes an identifier lower limit and an identifier upper limit, the message identifier is compared with the identifier lower limit and the identifier upper limit of each preset interval, and the identifier interval to which the message identifier belongs is determined from each preset interval. The identifier interval of the message identifier is accurately determined, which helps to improve the accuracy of subsequent message data distribution.

[0077] In one embodiment, the message data processing method further includes: for each candidate processing system, determining the data processing capability of the candidate processing system based on the system information of the candidate processing system, and determining the identifier interval matched with the candidate processing system according to the data processing capability. The span of the message identifier contained in the identifier interval is positively correlated with the data processing capability.

[0078] The system information can represent the system attribute information of the candidate processing system. The system information of the candidate processing system includes the thread number of the candidate processing system, the IO load rate of the candidate processing system, the processing success rate of the candidate processing system for message data, etc.

[0079] The data processing capability can represent the processing capability of the candidate processing system for message data. The stronger the data processing capability, the more data the candidate processing system processes per unit time, and the higher the accuracy of processing message data.

[0080] The span of the message identifier can represent the size of the identifier interval to which the message identifier belongs. For example, when the identifier interval is [1, 10], the span is 10, and when the identifier interval is [1, 20], the span is 20.

[0081] The span of the identification number interval matched with the candidate processing system is positively correlated with the data processing capability of the candidate processing system.

[0082] Specifically, the processing server determines the data processing capability of each candidate processing system based on the system information of the candidate processing system, and configures an identification number interval matched with the data processing capability for each candidate processing system according to the data processing capability. The span of the identification number contained in the identification number interval is positively correlated with the data processing capability.

[0083] In one of the embodiments, the processing server sorts the candidate processing systems according to the data processing capability of each candidate processing system, obtains multiple candidate intervals with different spans of identification number intervals, and sorts the candidate intervals according to the span. The processing server determines the identification number interval matched with the candidate processing system according to the sorting result of each processing system and the sorting result of each candidate interval.

[0084] In this embodiment, the data processing capability of each candidate processing system is determined based on the system information of the candidate processing system, and the identification number interval matched with the candidate processing system is determined according to the data processing capability. The span of the identification number contained in the identification number interval is positively correlated with the data processing capability. The identification number interval matched with the candidate processing system is accurate, which is beneficial to improving the accuracy of subsequent message data distribution.

[0085] In one of the embodiments, the identification number interval matched with the candidate processing system is determined according to the data processing capability, which includes: obtaining an identification number span range configured for the identification number interval, determining a target span matched with the data processing capability from the identification number span range, and selecting at least a part of the multiple to-be-allocated identification numbers according to the target span to form the identification number interval matched with the candidate processing system.

[0086] The identification number span range can be a variable range representing the size of the identification number interval, and more specifically, an interval formed by the lower limit value and the upper limit value of the span. For example, when the identification number span range is 10-100, the identification number interval can be [1, 10], [2, 11], [1, 100], [2, 101], etc. However, the identification number interval [1, 9] does not belong to the identification number interval corresponding to the identification number span range 10-100 because the span is 9, which is smaller than the identification number span range.

[0087] The target span represents a span matching the data capability of the candidate processing system. For example, for a certain candidate processing system, the stronger the data processing capability of the candidate processing system is, the larger the target span selected from the span range of the identification number is, and the more the number of identification numbers in the identification number interval corresponding to the target span is. For example, the target span of the identification number interval corresponding to the candidate processing system with stronger data processing capability is larger. For example, the target span of the identification number interval corresponding to the candidate processing system with stronger data processing capability is 20, and the target span of the identification number interval corresponding to the candidate processing system with weaker data processing capability is 10.

[0088] The to-be-assigned identification number can be an identification number reserved for message data, for example, the to-be-assigned identification number can be any one of 1-99999999.

[0089] Specifically, the processing server acquires the identification number span range configured for the identification number interval, determines the target span matching the data processing capability from the identification number span range, selects at least a part from the plurality of to-be-assigned identification numbers according to the target span, and constitutes the identification number interval matching the candidate processing system.

[0090] In one of the embodiments, the processing server selects at least a part from the plurality of to-be-assigned identification numbers according to the selection strategy and the target span, and constitutes the identification number interval matching the candidate processing system.

[0091] The selection strategy can include selecting at least a part from the plurality of to-be-assigned identification numbers according to the target span in the order of small to large of each to-be-assigned identification number, and constituting the identification number interval matching the candidate processing system.

[0092] The selection strategy can also include preferentially assigning the identification number interval to the candidate processing system with larger target span, and selecting at least a part from the unassigned identification number to assign the identification number interval to other candidate processing systems.

[0093] In this embodiment, the identification number span range configured for the identification number interval is acquired, the target span matching the data processing capability is determined from the identification number span range, at least a part is selected from the plurality of to-be-assigned identification numbers according to the target span, and the identification number interval matching the candidate processing system is constituted, so as to accurately match the identification number interval for the candidate processing system, and improve the accuracy of message data distribution.

[0094] In one embodiment, as shown in Figure 3 the target processing system matching the identification number interval is determined from each candidate processing system, including:

[0095] S302, determining at least two candidate processing systems matching the identification number interval from each candidate processing system.

[0096] The candidate processing system represents a system for carrying and processing message data, and the at least two candidate processing systems matching the identification number interval represent systems for carrying and processing message data. In other words, the message data needs to be sent to at least two candidate processing systems.

[0097] The identification number processing record represents a processing record of each type of message data in the candidate processing system. The processing record records the sending and receiving processing record of each type of message data, and the sending and receiving processing record of each type of original message.

[0098] S304, obtaining the identification number processing record of each candidate processing system and the original message identification number associated with the message identification number.

[0099] The original message identification number associated with the message identification number represents the message identification corresponding to the original message data associated with the message data.

[0100] S306, in each candidate processing system, screening the target processing system containing the original message identification number in the identification number processing record.

[0101] If the identification number processing record of the candidate processing system contains the original message identification number, it means that the candidate processing system has sent the original message data, and the message data corresponding to the original message data needs to be sent to the candidate system, which can improve the processing efficiency of the message data and avoid misdelivery of the message data.

[0102] If the identification number processing record of the candidate processing system does not contain the original message identification number, it means that the candidate processing system has not sent the original message data, and the message data corresponding to the original message data does not need to be sent to the candidate system, avoiding misdelivery of the message data.

[0103] Specifically, the processing server determines at least two candidate processing systems matching the identification number interval from each candidate processing system, obtains the identification number processing record of each candidate processing system and the original message identification number associated with the message identification number, and screens the target processing system containing the original message identification number in the identification number processing record in each candidate processing system.

[0104] In one embodiment, the processing server is configured to determine whether the identification number processing record of the candidate processing system contains the original message identification number, and according to the determination, screen the target processing system containing the original message identification number in the identification number processing record in each candidate processing system.

[0105] In the embodiment, at least two candidate processing systems matching the identification number interval are determined from each candidate processing system, the identification number processing record of each candidate processing system and the original message identification number associated with the message identification number are obtained, and the target processing system containing the original message identification number in the identification number processing record is screened out in each candidate processing system, which can improve the processing efficiency of the message data and avoid misdelivery of the message data.

[0106] In one embodiment, sending the message data to the target processing system comprises: obtaining the system real-time load of the target processing system, screening the target message sending mode matching the system real-time load from multiple message sending modes, and sending the message data to the target processing system according to the target message sending mode.

[0107] The system real-time load represents the average load of the target processing system within a certain time interval. The system real-time load includes CPU utilization, IO load rate, etc.

[0108] The CPU utilization rate represents the proportion of the time that the CPU executes non-idle tasks to the total time. It reflects the busy degree of the CPU. High CPU utilization rate may mean that the system is efficiently running multiple tasks, but it may also indicate that there is resource contention or some tasks cannot get enough CPU time.

[0109] The IO load rate value represents the busy degree of the system when performing input and output operations. It includes disk read / write, network transmission, etc. High IO load rate may mean that the system is processing a large number of data read / write operations, or there is an I / O bottleneck.

[0110] The higher the system real-time load, the higher the CPU utilization rate and / or IO load rate, and vice versa. The lower the system real-time load, the lower the CPU utilization rate and / or IO load rate.

[0111] The message sending mode represents the specific way of sending the message data. The message sending mode at least includes a first message sending mode and a second message sending mode. The first message sending mode can be a parameter addressing transmission mode, and the second message sending mode can be a transmission mode through middleware MQ communication. The parameter addressing mode is real-time transmission, which is more efficient than the MQ transmission mode. The parameter addressing mode includes but is not limited to immediate addressing, register addressing, etc.

[0112] Specifically, the processing server obtains the system real-time load of the target processing system, screens the target message sending mode matching the system real-time load from multiple message sending modes, and sends the message data to the target processing system according to the target message sending mode.

[0113] In one of the embodiments, when the system real-time load is high, the target message sending mode can be a mode with lower transmission efficiency, so as to reduce the transmission data amount of the message data in a short time and avoid the error of the target processing system. Conversely, when the system real-time load is low, the target message sending mode can be a mode with higher transmission efficiency, so as to improve the transmission efficiency of the message data.

[0114] In the embodiment, the system real-time load of the target processing system is acquired, and in a plurality of message sending modes, the target message sending mode matched with the system real-time load is screened out. The message data is sent to the target processing system according to the target message sending mode, and the message data is sent according to the message sending mode matched with the system real-time load, so as to reduce the error probability of the target processing system.

[0115] In one of the embodiments, the target processing system includes at least two target systems with different system database architectures, and the method further includes: determining the sending order of the message data for each target system respectively, sending the message data to the target system with earlier sending order, and the target system with earlier sending order is used to copy and forward the message data to the target system with later sending order.

[0116] The sending order of the target system represents the sending order of the message data.

[0117] Specifically, the target system with smaller sending order receives the message data sent by the processing server first, and the target system with larger sending order receives the message data sent by the processing server later.

[0118] In one of the embodiments, the processing server sends the message data to the target system with earlier sending order, and controls the candidate server (target server) to which the target system with earlier sending order belongs to copy and forward the message data to the target system with later sending order. In this way, the distribution efficiency of the message data is greatly improved.

[0119] In one of the embodiments, the processing server determines the sending order of the message data for each target system according to the thread number of the processing server. For example, the thread number of the processing server is 3, and the number of the target systems is 9. The message data is distributed to the first three target processing servers at the same time, and the three target processing servers can copy and forward the message data to three target processing servers in the remaining six target processing servers. At the same time, the processing server sends the message data to the remaining three target processing servers.

[0120] In the embodiment, the target processing system includes at least two target systems with different system database architectures, the sending sequence of the message data to each target system is determined respectively, the message data is sent to the target system with the earlier sending sequence, and the target system with the earlier sending sequence is used to copy and forward the message data to the target system with the later sending sequence, so that the data transmission pressure of the processing server is reduced, and the overall sending efficiency of the message data is improved.

[0121] In one embodiment, a processing method for realizing message distribution by interval proportion step by step is provided as shown in Figure 4 The processing method comprises the steps of:

[0122] S402, acquiring a message stream containing a plurality of message data.

[0123] The message data includes a message type and a message identification number.

[0124] Specifically, taking the message data distribution between the Oracle system and the GoldenDB system as an example, a message distribution method is provided as shown in Figure 5 The message distribution method comprises the steps of:

[0125] Step 1, receiving a normal incoming message, and taking an original message distribution identifier in the original message type in the normal incoming message.

[0126] Step 2, if the original message distribution identifier is 0, it is judged whether to distribute according to the original message identification number, if yes, the message data temporarily stored in the Oracle system is sent to the GoldenDB system, and if no, the message data is sent to the Oracle system.

[0127] Step 3, if the original message distribution identifier is 1, it is unconditionally sent to the Oracle system, if the original message distribution identifier is 2, it is unconditionally sent to both the Oracle system and the GoldenDB system, and if the original message distribution identifier is 3, it is unconditionally sent to the GoldenDB system.

[0128] Step 4, if the original message identification is 4, it is judged whether there is a record in the Oracle system, if yes, the message data is sent to the Oracle system, and if no, the message data is sent to the GoldenDB system.

[0129] Step 5, judging the distribution by which way.

[0130] The parameter addressing and the middleware MQ communication are two ways of distributing from the Oracle end to the GoldenDB end.

[0131] The parameter addressing mode is used for real-time transmission, and is more efficient than the MQ transmission mode. In the later stage, the parameter addressing mode can be gradually improved.

[0132] S404, determine the shunt identification corresponding to each message type according to the mapping relationship between each message type and the shunt identification.

[0133] S406, for each message data, if the shunt identification of the message data corresponds to multiple candidate processing systems, obtain multiple preset intervals configured for the identification number.

[0134] Each preset interval includes an identification number lower limit and an identification number upper limit.

[0135] S408, compare the message identification number with the identification number lower limit and the identification number upper limit of each preset interval, and determine the identification number interval to which the message identification number belongs from each preset interval.

[0136] S410, for each candidate processing system, determine the data processing capacity of the candidate processing system based on the system information of the candidate processing system.

[0137] S412, obtain the identification number span range configured for the identification number interval.

[0138] S414, determine the target span matching the data processing capacity from the identification number span range.

[0139] S416, select at least a part of the multiple to-be-allocated identification numbers according to the target span to form an identification number interval matching the candidate processing system.

[0140] The span of the message identification number contained in the identification number interval is positively correlated with the data processing capacity.

[0141] S418, determine at least two candidate processing systems matching the identification number interval from the multiple candidate processing systems.

[0142] S420, obtain the identification number processing record of each candidate processing system and the original message identification number associated with the message identification number.

[0143] S422, in the multiple candidate processing systems, select a target processing system containing the original message identification number in the identification number processing record.

[0144] S424, obtain the system real-time load of the target processing system.

[0145] S426, in the multiple message sending modes, select a target message sending mode matching the system real-time load.

[0146] S428, sending the message data to the target processing system according to the target message sending mode.

[0147] S430, determining the sending order of the message data for each target system respectively.

[0148] The target processing system includes at least two target systems with different system database architectures.

[0149] S432, sending the message data to the target system with an earlier sending order.

[0150] The target system with an earlier sending order is used to copy and forward the message data to the target system with a later sending order.

[0151] In the embodiment, a message stream containing multiple message data is obtained. The message data includes message types and message identification numbers. According to the mapping relationship between the message types and the shunting identification, the shunting identification corresponding to each message type is determined, and each message data is preliminarily divided according to the message type, which is beneficial to reduce the number of message types corresponding to the message data carried by the subsequent candidate processing system and reduce the error probability of the candidate processing system. For each message data, in the case that the shunting identification of the message data corresponds to multiple candidate processing systems, the identification number interval to which the message identification number belongs is determined, and the target processing system matching the identification number interval is determined from each candidate processing system. By using the above method, on the one hand, each message data is preliminarily divided according to the message type, which reduces the error probability of the subsequent candidate processing system and improves the distribution accuracy of the message data. On the other hand, based on the message identification number, the target processing system to which the message data is to be sent is determined in each candidate system, the target processing system is accurately determined, the message data is sent to the target processing system, and the distribution accuracy of the message data is further improved. The method provides a message shunting mode, which is suitable for the scenario of applying the traffic of the original server to the localization application by gradually increasing the transaction volume of the localization environment, avoids the failure of large transaction volume, and is controllable and relatively safe.

[0152] It should be understood that although each step in the flowchart involved in each embodiment as described above is shown in sequence according to the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless explicitly stated herein, there is no strict order limitation for the execution of these steps, and these steps can be executed in other orders. Moreover, at least part of the steps in the flowchart involved in each embodiment as described above can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be alternately executed with at least part of other steps or steps or stages in other steps.

[0153] Based on the same inventive concept, the embodiments of the present application also provide a packet data processing apparatus for implementing the packet data processing method involved above. The implementation scheme for solving the problem provided by the apparatus is similar to the implementation scheme described in the above method, and therefore the specific limitations in one or more packet data processing apparatus embodiments provided below can refer to the limitations of the packet data processing method described above, which will not be repeated here.

[0154] In one embodiment, as shown in Figure 6 a packet data processing apparatus is provided, comprising: an acquisition module 602, a shunt identification determination module 604, an identification number interval determination module 606, and a processing module 608, wherein:

[0155] The acquisition module 602 is configured to acquire a packet stream containing a plurality of packet data; the packet data includes a packet type and a packet identification number;

[0156] The shunt identification determination module 604 is configured to determine the shunt identification corresponding to each packet type according to each packet type and the mapping relationship between the packet type and the shunt identification;

[0157] The identification number interval determination module 606 is configured to, for each packet data, determine the identification number interval to which the packet identification number belongs in the case that the shunt identification of the packet data corresponds to a plurality of candidate processing systems;

[0158] The processing module 608 is configured to determine a target processing system matching the identification number interval from the candidate processing systems, and send the packet data to the target processing system.

[0159] In one of the embodiments, the identification number range determining module 606 is further configured to acquire a plurality of preset ranges configured for the identification numbers; each preset range comprises a lower limit of identification number and an upper limit of identification number; and compare the message identification number in the message data with the lower limit of identification number and the upper limit of identification number of each preset range respectively, so as to determine the identification number range to which the message identification number belongs from the preset ranges.

[0160] In one of the embodiments, the device further comprises an information monitoring module configured to determine, for each candidate processing system, the data processing capacity of the candidate processing system based on the system information of the candidate processing system; determine the identification number range matched with the candidate processing system according to the data processing capacity; and the span of message identification numbers contained in the identification number range is positively correlated with the data processing capacity.

[0161] In one of the embodiments, the information monitoring module is further configured to acquire an identification number span range configured for the identification number range; determine a target span matched with the data processing capacity from the identification number span range; and select at least a part of the plurality of to-be-allocated identification numbers according to the target span to form the identification number range matched with the candidate processing system.

[0162] In one of the embodiments, the processing module 608 is further configured to determine at least two candidate processing systems matched with the identification number range from the candidate processing systems; acquire the identification number processing records of each candidate processing system and the original message identification number associated with the message identification number; and filter a target processing system containing the original message identification number in the identification number processing record from the candidate processing systems.

[0163] In one of the embodiments, the processing module 608 is further configured to acquire the system real-time load of the target processing system; filter a target message sending mode matched with the system real-time load from the plurality of message sending modes; and send the message data to the target processing system according to the target message sending mode.

[0164] In one of the embodiments, the target processing system comprises at least two target systems with different system database architectures; and the device further comprises a data sending module configured to determine the sending sequence of the message data for each target system respectively; send the message data to the target system in the front of the sending sequence; and the target system in the front of the sequence is configured to copy and forward the message data to the target system in the back of the sequence.

[0165] Each module in the message data processing device described above can be realized by software, hardware and combinations thereof in whole or in part. Each module described above can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0166] In one embodiment, a computer device is provided, which can be a server, and an internal structure diagram of the computer device can be as shown in FIG. 1. Figure 7 The computer device includes a processor, a memory, an input / output interface, and a communication interface. The processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for running the operating system and the computer program in the non-volatile storage medium. The database of the computer device is configured to store message data. The input / output interface of the computer device is configured to exchange information between the processor and external devices. The communication interface of the computer device is configured to communicate with terminals outside through a network connection. The computer program is executed by the processor to implement a message data processing method.

[0167] Those skilled in the art can understand that Figure 7 The structure shown in FIG. 1 is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0168] In one embodiment, a computer device is provided, which includes a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps in each of the method embodiments.

[0169] In one embodiment, a computer readable storage medium is provided, which stores a computer program. The computer program is executed by a processor to implement the steps in each of the method embodiments.

[0170] In one embodiment, a computer program product is provided, which includes a computer program. The computer program is executed by a processor to implement the steps in each of the method embodiments.

[0171] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer readable storage medium, and when the computer program is executed, the processes of the above-mentioned embodiments of the methods can be included. Any reference to memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration but not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc. The database involved in the embodiments provided in the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided in the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0172] Any combination of the technical features of the above embodiments can be made. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the present application.

[0173] The above embodiments only express several implementation manners of the present application, and the description is more specific and detailed, but it should not be understood as a limitation on the scope of the patent of the present application. It should be pointed out that for ordinary skilled in the art, without departing from the concept of the present application, a number of modifications and improvements can be made, which are within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.

Claims

1. A method of processing packet data, characterized by, The method comprises: acquiring a message stream containing a plurality of message data; the message data includes message types and message identification numbers; determining the shunt identification corresponding to each message type according to each message type and the mapping relationship between message types and shunt identification; for each message data, determining the identification number interval to which the message identification number belongs in the case that the shunt identification of the message data corresponds to a plurality of candidate processing systems; determining the target processing system matching the identification number interval from each candidate processing system and sending the message data to the target processing system, wherein the determination of the target processing system matching the identification number interval from each candidate processing system comprises: determining at least two candidate processing systems matching the identification number interval from each candidate processing system; acquiring the identification number processing record of each candidate processing system and the original message identification number associated with the message identification number; screening the target processing system containing the original message identification number in the identification number processing record from each candidate processing system.

2. The method of claim 1, wherein, The determination of the identification number interval to which the message identification number belongs comprises: acquiring a plurality of preset intervals configured for identification numbers; each preset interval includes an identification number lower limit and an identification number upper limit; comparing the message identification number with the identification number lower limit and the identification number upper limit of each preset interval respectively to determine the identification number interval to which the message identification number belongs from each preset interval.

3. The method of claim 1, wherein, The method further comprises: for each candidate processing system, determining the data processing capacity of the candidate processing system based on the system information of the candidate processing system; determining the identification number interval matching the candidate processing system according to the data processing capacity; the span of the message identification number contained in the identification number interval is positively correlated with the data processing capacity.

4. The method of claim 3, wherein, The determination of the identification number interval matching the candidate processing system according to the data processing capacity comprises: acquiring the identification number span range configured for the identification number interval; determining the target span matching the data processing capacity from the identification number span range; selecting at least a part of the plurality of to-be-allocated identification numbers according to the target span to constitute the identification number interval matching the candidate processing system.

5. The method according to any one of claims 1 to 4, characterized in that, The sending of the message data to the target processing system comprises: acquiring the system real-time load of the target processing system; screening the target message sending mode matching the system real-time load from a plurality of message sending modes; sending the message data to the target processing system according to the target message sending mode.

6. The method according to any one of claims 1 to 4, characterized in that, The target processing system at least includes two target systems with different system database architectures; The method further comprises: determining the sending order of the message data for each target system respectively; sending the message data to the target system in the front of the sending order; the target system in the front of the order is used to copy and forward the message data to the target system in the back of the order.

7. A packet data processing apparatus, characterized by comprising: The device comprises: The acquisition module is configured to acquire a message stream containing a plurality of message data; the message data includes a message type and a message identification number; The shunt identification determination module is configured to determine a shunt identification corresponding to each message type according to the message types and a mapping relationship between the message types and the shunt identifications; The identification number interval determination module is configured to, for each message data, determine an identification number interval to which the message identification number belongs in a case where the shunt identification of the message data corresponds to a plurality of candidate processing systems. The processing module is configured to determine a target processing system matching the identification number interval from the candidate processing systems and send the message data to the target processing system; the processing module is further configured to determine at least two candidate processing systems matching the identification number interval from the candidate processing systems, acquire an identification number processing record of each candidate processing system and an original message identification number associated with the message identification number, and filter a target processing system containing the original message identification number in the identification number processing record from the candidate processing systems.

8. The apparatus of claim 7, wherein, The identification number interval determination module is further configured to acquire a plurality of preset intervals configured for identification numbers; each preset interval includes a lower limit of an identification number and an upper limit of an identification number; compare the message identification number with the lower limit and the upper limit of the identification number of each preset interval, and determine the identification number interval to which the message identification number belongs from the preset intervals.

9. The apparatus of claim 7, wherein, The device further includes an information monitoring module configured to, for each candidate processing system, determine a data processing capacity of the candidate processing system based on system information of the candidate processing system, determine an identification number interval matching the candidate processing system according to the data processing capacity, and make the span of the message identification number contained in the identification number interval positively correlated with the data processing capacity.

10. The apparatus of claim 9, wherein, The information monitoring module is further configured to acquire an identification number span range configured for an identification number interval, determine a target span matching the data processing capacity from the identification number span range, and select at least a part of a plurality of to-be-allocated identification numbers according to the target span to form the identification number interval matching the candidate processing system.

11. The apparatus of any one of claims 7 to 10, wherein, The processing module is further configured to acquire a system real-time load of the target processing system, filter a target message sending mode matching the system real-time load from a plurality of message sending modes, and send the message data to the target processing system according to the target message sending mode.

12. The apparatus of any one of claims 7 to 10, wherein, The target processing system includes at least two target systems having different system database architectures; the device further includes a data sending module configured to determine a sending order of the message data for each target system, send the message data to a target system in an earlier sending order, and copy and forward the message data from the target system in the earlier sending order to a target system in a later sending order.

13. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that, The processor executes the computer program to implement the steps of the method in any one of claims 1 to 6.

14. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program, which when executed by a processor, implements the steps of the method of any one of claims 1 to 6.

15. A computer program product comprising a computer program, characterized in that, The computer program, which when executed by a processor, implements the steps of the method of any one of claims 1 to 6.

Images