Message data processing method and device, computer equipment and storage medium

Abstract

The invention relates to a message data processing method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a message flow containing a plurality of pieces of message data; the message data comprises message types and message identification numbers, according to the message types and the mapping relation between the message types and the shunting identifications, determining the shunting identifications corresponding to the message types respectively, and for each piece of message data, under the condition that the shunting identifications of the message data correspond to a plurality of candidate processing systems, determining the shunting identifications corresponding to the message types respectively. And determining the identification number interval to which the message identification number belongs, and determining the target processing system matched with the identification number interval from the candidate processing systems, and by adopting the method, dividing each message data according to the message type and the message identification number, the error reporting probability of the subsequent candidate processing systems is reduced, and the distribution accuracy of the message data is improved.

Description

Technical Field

[0001] The present application relates to the field of communication technology, and in particular to a message data processing method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] With the development of database technology, it has become possible to carry out localization transformation of small machines and database migration based on servers and domestic databases.

[0003] In traditional technology, massive message data is distributed according to the real-time data processing capabilities of different database systems. Although this distribution method can avoid database system overload to a certain extent, database system errors still exist and the distribution accuracy is low. Summary of the invention

[0004] Based on this, it is necessary to provide a message data processing method, apparatus, computer equipment, computer-readable storage medium and computer program product that can distribute message data accurately in response to the above technical problems.

[0005] In a first aspect, the present application provides a method for processing message data. The method comprises:

[0006] Acquire a message stream including a plurality of message data; the message data include a message type and a message identification number; determine the diversion identification corresponding to each of the message types according to the mapping relationship between the message type and the diversion identification; for each of the message data, when the diversion identification of the message data corresponds to a plurality of candidate processing systems, determine the identification number interval to which the message identification number belongs; determine a target processing system matching the identification number interval from among the candidate processing systems, and send the message data to the target processing system.

[0007] In one of the embodiments, determining the identification number interval to which the message identification number belongs includes: obtaining multiple preset intervals configured for the identification number; each of the preset intervals includes an identification number lower limit and an identification number upper limit; comparing the message identification number with the identification number lower limit and the identification number upper limit of each preset interval, respectively, and determining the identification number interval to which the message identification number belongs from each preset interval.

[0008] In one of the embodiments, the method further includes: for each of the candidate processing systems, determining the data processing capability of the candidate processing system based on the system information of the candidate processing system; determining the identification number interval that matches the candidate processing system according to the data processing capability; and the span of the message identification numbers contained in the identification number interval is positively correlated with the data processing capability.

[0009] In one of the embodiments, determining the identification number interval that matches the candidate processing system according to the data processing capability includes: obtaining an identification number span range configured for the identification number interval; determining a target span that matches the data processing capability from the identification number span range; and selecting at least a portion from a plurality of identification numbers to be allocated according to the target span to form an identification number interval that matches the candidate processing system.

[0010] In one of the embodiments, determining the target processing system that matches the identification number interval from each of the candidate processing systems includes: determining at least two candidate processing systems that match the identification number interval from each of the candidate processing systems; obtaining the identification number processing records of each of the candidate processing systems, and the original message identification number associated with the message identification number; and screening, among each of the candidate processing systems, the target processing system whose identification number processing record contains the original message identification number.

[0011] In one of the embodiments, sending the message data to the target processing system includes: obtaining the system real-time load of the target processing system; screening a target message sending method that matches the system real-time load among multiple message sending methods; and sending the message data to the target processing system according to the target message sending method.

[0012] In one of the embodiments, the target processing system includes at least two target systems with different system database architectures; the method also includes: determining the sending order of the message data to each of the target systems respectively; sending the message data to the target system with a higher sending order; the target system with a higher order is used to copy and forward the message data to the target system with a lower order.

[0013] In a second aspect, the present application also provides a message data processing device. The device comprises:

[0014] An acquisition module is used to acquire a message stream containing multiple message data; the message data includes a message type and a message identification number; a diversion identification determination module is used to determine the diversion identification corresponding to each of the message types according to the mapping relationship between the message type and the diversion identification; an identification number interval determination module is used to determine the identification number interval to which the message identification number belongs for each of the message data when the diversion identification of the message data corresponds to multiple candidate processing systems; a processing module determines a target processing system that matches the identification number interval from each of the candidate processing systems, and sends the message data to the target processing system.

[0015] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0016] Acquire a message stream including a plurality of message data; the message data include a message type and a message identification number; determine the diversion identification corresponding to each of the message types according to the mapping relationship between the message type and the diversion identification; for each of the message data, when the diversion identification of the message data corresponds to a plurality of candidate processing systems, determine the identification number interval to which the message identification number belongs; determine a target processing system matching the identification number interval from among the candidate processing systems, and send the message data to the target processing system.

[0017] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0018] Acquire a message stream including a plurality of message data; the message data include a message type and a message identification number; determine the diversion identification corresponding to each of the message types according to the mapping relationship between the message type and the diversion identification; for each of the message data, when the diversion identification of the message data corresponds to a plurality of candidate processing systems, determine the identification number interval to which the message identification number belongs; determine a target processing system matching the identification number interval from among the candidate processing systems, and send the message data to the target processing system.

[0019] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0020] Acquire a message stream including a plurality of message data; the message data include a message type and a message identification number; determine the diversion identification corresponding to each of the message types according to the mapping relationship between the message type and the diversion identification; for each of the message data, when the diversion identification of the message data corresponds to a plurality of candidate processing systems, determine the identification number interval to which the message identification number belongs; determine a target processing system matching the identification number interval from among the candidate processing systems, and send the message data to the target processing system.

[0021] The above-mentioned message data processing method, device, computer equipment, storage medium and computer program product obtain a message stream containing multiple message data; the message data includes a message type and a message identification number, and the shunt identification corresponding to each message type is determined according to each message type and the mapping relationship between the message type and the shunt identification, and each message data is preliminarily divided according to the message type, which is conducive to reducing the number of message types corresponding to the message data carried in the subsequent candidate processing system and reducing the probability of the candidate processing system reporting an error. For each message data, when the shunt identification of the message data corresponds to multiple candidate processing systems, the identification number interval to which the message identification number belongs is determined, and the target processing system matching the identification number interval is determined from each candidate processing system. Using the above-mentioned method, on the one hand, each message data is preliminarily divided according to the message type, which reduces the error probability of the subsequent candidate processing system and improves the distribution accuracy of the message data. On the other hand, based on the message identification number, the target processing system to which the message data is to be sent is determined in each candidate system, the target processing system is accurately determined, and the message data is sent to the target processing system, which further improves the distribution accuracy of the message data. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] FIG1 is an application environment diagram of a message data processing method according to an embodiment;

[0023] FIG2 is a schematic diagram of a flow chart of a message data processing method in one embodiment;

[0024] FIG3 is a schematic flow chart of a method for determining a target processing system in one embodiment;

[0025] FIG4 is a flow chart of a method for gradually implementing message diversion according to interval ratio in one embodiment;

[0026] FIG5 is a schematic diagram of a flow chart of a message diversion method in an embodiment;

[0027] FIG6 is a structural block diagram of a message data processing device in one embodiment;

[0028] FIG. 7 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0029] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0030] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. In addition, the acquisition, storage, processing, and transmission of data are in compliance with relevant laws and regulations.

[0031] The message data processing method provided in the embodiment of the present application can be applied in the application environment shown in Figure 1. The processing server 102 is respectively connected to multiple candidate servers 104. The data storage system for each server can be integrated on the service, or placed on the cloud or other network servers.

[0032] The processing server 102 obtains a message stream containing multiple message data; the message data includes a message type and a message identification number. The processing server 102 determines the diversion identifier corresponding to each message type according to each message type and the mapping relationship between the message type and the diversion identifier. For each message data, the processing server 102 determines the identification number interval to which the message identification number belongs when the diversion identifier of the message data corresponds to multiple candidate processing systems. The processing server 102 determines a target processing system that matches the identification number interval from each candidate processing system, and sends the message data to the target processing system.

[0033] The candidate processing systems include the processing server 102 and the processing systems included in each of the candidate servers 104. Specifically, the processing server 102 determines the target processing system (the processing server 102 and the processing system included in the target server in each of the candidate servers 104) that matches the identification number interval from the candidate processing systems, and sends the message data to the target processing system, that is, to the target server.

[0034] The processing server 102 and each candidate server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0035] In one embodiment, as shown in FIG. 2 , a message data processing method is provided, which is described by taking the method applied to the processing server in FIG. 1 as an example, including:

[0036] S202, obtaining a message flow including a plurality of message data.

[0037] Among them, the message flow is the process in which the message flows in the network according to a certain order and rules during the network communication process. This process involves the generation, encapsulation, transmission, reception, decryption and other links of the message data. The message flow can be a request message flow or a response message flow, depending on the specific situation, and is not limited here.

[0038] A message stream can also be viewed as a collection of multiple message data.

[0039] Message data means the specific information content of the message, which is the actual data transmitted during the network communication process. The content of the message data is generally different according to different communication protocols, message senders, message receivers, etc. Generally speaking, the message data consists of two parts: the message header and the data body. The message header can include the message type and message identification number of the message data, and the data body contains the actual transmitted data content, such as specific files, pictures, videos, etc.

[0040] The message type indicates the type of message data. In the transaction payment scenario, the message types may include: ibps.101, ibps.103, Saps.601, ccms801, etc.

[0041] The message identification number indicates the identification number of the message data. In the transaction payment scenario, the message identification number can be an 8-digit number.

[0042] Specifically, the processing server may directly obtain the message stream containing multiple message data from other servers or other terminals. The processing server may also extract the message stream containing multiple message data from the data storage system.

[0043] In one of the embodiments, the processing server obtains a first message stream from other servers, obtains a second message stream from other terminals, and extracts a message stream containing multiple message data from a data storage system.

[0044] In one of the embodiments, after acquiring the first message flow, the second message flow and the third message flow, the processing server combines the first message flow, the second message flow and the third message flow to obtain a combined message flow.

[0045] In one of the embodiments, the processing server combines partial message flows in the first message flow, the second message flow, and the third message flow to obtain a combined message flow according to the number of processing threads of the processing server itself.

[0046] S204: Determine the diversion identifier corresponding to each message type according to each message type and the mapping relationship between the message type and the diversion identifier.

[0047] Among them, the diversion identifier represents the identification number of the diversion direction configured for the message data. For example, the diversion identifier is 1, which means that the message data may correspond to diversion direction 1 and diversion direction 2. The diversion identifier is 2, which means that the message data may correspond to diversion direction 1 and diversion direction 3.

[0048] The mapping relationship between the message type and the diversion identifier can represent that multiple message types correspond to one diversion identifier. For example, the message type is ccms.801, and the corresponding diversion identifier is 1.

[0049] It should be noted that the diversion identifier corresponding to the message type can represent the diversion direction. For example, a diversion identifier of 0 can indicate that the message data corresponds to only diversion direction 1.

[0050] Specifically, the processing server obtains a mapping relationship between a message type and a diversion identifier from the data storage system, and determines a diversion identifier corresponding to each message type according to the message type and the mapping relationship.

[0051] S206 , for each message data, when the diversion identifier of the message data corresponds to multiple candidate processing systems, determine the identification number interval to which the message identification number belongs.

[0052] When the diversion identifier of the message data corresponds to multiple candidate processing systems, it means that the message data may be distributed to multiple candidate processing systems. For example, the message data is sent to the candidate processing systems included in the processing server and the candidate processing systems included in the candidate server.

[0053] It should be noted that the diversion identifier of the message data may correspond to a candidate processing system, in which case it means that the message data may be distributed to a candidate processing system. For example, the message data is sent to the candidate processing system included in the processing server. For another example, the message data is sent to the candidate processing system included in the candidate server.

[0054] The candidate processing system refers to a system for carrying and processing message data. More specifically, the candidate processing system may be a database system, for example, a domestic database system GoldenDB or a non-domestic database system Oracle.

[0055] The identification number interval represents a set of identification numbers, for example, the identification number interval may be [1,10], [2,15], [20,100], etc. The identification number interval to which the message identification number belongs represents which one or several identification number intervals the identification number belongs to.

[0056] Among them, the method for determining the interval of the identification number to which the message identification number belongs includes: specifically, the processing server obtains multiple preset intervals configured for the identification number, and determines the preset interval to which the message identification number belongs as the identification number interval to which the message identification number belongs according to the interval boundary values ​​contained in each preset interval.

[0057] The identification number interval may be a pre-configured interval, and there are multiple ways to configure the interval.

[0058] In one of the embodiments, the processing server determines the data processing capability of each candidate processing system based on the system information of the candidate system, and determines the identification number interval that matches the candidate processing system according to the data processing capability, and the span of the message identification numbers contained in the identification number interval is positively correlated with the data processing capability.

[0059] S208, determining a target processing system that matches the identification number interval from each candidate processing system, and sending the message data to the target processing system.

[0060] The target processing system may represent a candidate processing system corresponding to the identification number interval.

[0061] Specifically, before determining the target processing system, the processing server determines the data processing capability of the candidate processing system based on the system information of the candidate processing system for each candidate processing system, and determines the identification number interval that matches the candidate processing system according to the data processing capability. The span of the message identification numbers contained in the identification number interval is positively correlated with the data processing capability.

[0062] Specifically, after obtaining the identification number interval matched by the candidate processing system, the processing server determines the target processing system that matches the identification number interval from among the candidate processing systems according to the identification number interval matched by the candidate processing system.

[0063] There are many ways to send message data to the target processing system.

[0064] Specifically, the processing server sends the message data to the target processing system according to the target message sending mode matched by the target processing system.

[0065] In one of the embodiments, the processing server obtains the system real-time load of the target processing system; selects a target message sending method that matches the system real-time load from multiple message sending methods; and sends the message data to the target processing system according to the target message sending method.

[0066] In one of the embodiments, the processing server sends the message data to the target processing system according to the target message sending mode configured by the target processing system.

[0067] Specifically, after determining that the target processing system includes at least two target systems with different system database architectures, the processing server determines the sending order of message data for each target system respectively; sends the message data to the target system with an earlier sending order; the target system with an earlier order is used to copy and forward the message data to the target system with a later order.

[0068] In one embodiment, after the processing server determines the sending order of message data to each target system respectively, the processing server first sends the message data to the target system with an earlier sending order, and then sends the message data to the target system with a later sending order.

[0069] In the above message data processing method, a message stream containing multiple message data is obtained; the message data includes a message type and a message identification number, and the shunt identification corresponding to each message type is determined according to each message type and the mapping relationship between the message type and the shunt identification number, and each message data is preliminarily divided according to the message type, which is conducive to reducing the number of message types corresponding to the message data carried in the subsequent candidate processing system and reducing the probability of the candidate processing system reporting an error. For each message data, when the shunt identification number of the message data corresponds to multiple candidate processing systems, the identification number interval to which the message identification number belongs is determined, and the target processing system matching the identification number interval is determined from each candidate processing system. Using the above method, on the one hand, each message data is preliminarily divided according to the message type, which reduces the error probability of the subsequent candidate processing system and improves the distribution accuracy of the message data. On the other hand, based on the message identification number, the target processing system to which the message data is to be sent is determined in each candidate system, the target processing system is accurately determined, and the message data is sent to the target processing system, which further improves the distribution accuracy of the message data.

[0070] In one embodiment, determining the identification number interval to which a message identification number belongs includes: obtaining multiple preset intervals configured for the identification number, each preset interval including an identification number lower limit and an identification number upper limit, comparing the message identification number with the identification number lower limit and the identification number upper limit of each preset interval, and determining the identification number interval to which the message identification number belongs from each preset interval.

[0071] The preset interval refers to an interval configured for the identification number, such as [1,10], [2,15], [20,100], etc.

[0072] The lower limit of the identification number indicates the lower limit value of the preset interval, and the upper limit of the identification number indicates the upper limit value of the preset interval. If the preset interval is [1,10], the upper limit value of the interval is 10, and the lower limit value of the interval is 1.

[0073] The preset interval identification number lower limit and the preset interval identification number upper limit are used to compare with the message identification. If the message identification number is greater than or equal to the preset interval identification number lower limit, and the message identification number is less than or equal to the preset interval identification number upper limit, then the preset interval corresponding to the preset interval identification number lower limit and the preset interval identification number upper limit is determined as the identification number interval to which the message identification number belongs.

[0074] Specifically, the processing server may directly obtain multiple preset intervals configured for the identification number. The processing server may also configure multiple preset intervals for the identification number.

[0075] In one embodiment, the preset interval can be matched with the candidate processing system, and the processing server can determine the preset interval that matches each data processing capability according to the data processing capability of each candidate processing system. The stronger the data processing capability, the larger the preset interval, and vice versa.

[0076] In this embodiment, by obtaining multiple preset intervals configured for the identification number, each preset interval includes an identification number lower limit and an identification number upper limit, and the message identification number is compared with the identification number lower limit and the identification number upper limit of each preset interval respectively, and the identification number interval to which the message identification number belongs is determined from each preset interval, and the identification number interval described by the message identification number is accurately determined, which is conducive to improving the distribution accuracy of subsequent message data.

[0077] In one embodiment, the message data processing method also includes: for each candidate processing system, based on the system information of the candidate processing system, determining the data processing capability of the candidate processing system, and determining the identification number interval that matches the candidate processing system according to the data processing capability, and the span of the message identification numbers contained in the identification number interval is positively correlated with the data processing capability.

[0078] The system information may represent system attribute information of the candidate processing system, and the system information of the candidate processing system includes the number of threads of the candidate processing system, the IO load rate of the candidate processing system, the processing success rate of the candidate processing system for message data, and the like.

[0079] The data processing capability may indicate the processing capability of the candidate processing system for message data. If the data processing capability is stronger, it means that the candidate processing system processes more message data per unit time and has higher accuracy in processing message data.

[0080] The span of the message identification number may represent the size of the identification number interval to which the message identification number belongs. For example, when the identification number interval is [1,10], the span is 10; and when the identification number interval is [1,20], the span is 20.

[0081] The stronger the data processing capability of the candidate processing system, the larger the span of the identification number interval matching the candidate processing system. The span of the message identification numbers contained in the identification number interval is positively correlated with the data processing capability.

[0082] Specifically, the processing server determines the data processing capability of each candidate processing system based on the system information of the candidate processing system. The processing server then configures an identification number interval that matches the data processing capability for each candidate processing system according to the data processing capability. The span of the message identification numbers contained in the identification number interval is positively correlated with the data processing capability.

[0083] In one of the embodiments, the processing server sorts each candidate processing system according to their respective data processing capabilities, the processing server obtains multiple candidate intervals with different spans of identification number intervals, and sorts each candidate interval according to the span size, and the processing server determines the identification number interval that matches the candidate processing system based on the sorting results of each processing system and the sorting results of each candidate interval.

[0084] In this embodiment, for each candidate processing system, the data processing capability of the candidate processing system is determined based on the system information of the candidate processing system, and the identification number interval that matches the candidate processing system is determined according to the data processing capability. The span of the message identification numbers contained in the identification number interval is positively correlated with the data processing capability. This is an identification number interval that accurately matches the candidate processing system, which is beneficial to subsequently improving the distribution accuracy of the message data.

[0085] In one embodiment, according to the data processing capability, the identification number interval that matches the candidate processing system is determined, including: obtaining the identification number span range configured for the identification number interval, determining the target span that matches the data processing capability from the identification number span range, and according to the target span, selecting at least a part from a plurality of identification numbers to be allocated to form an identification number interval that matches the candidate processing system.

[0086] The identification number span range may be a variable range that represents the size of the identification number interval, more specifically, an interval formed by the lower limit value and the upper limit value of the span. For example, when the identification number span range is 10-100, the identification number interval may be [1,10], [2,11], [1,100], [2,101], etc. However, [1,9] is not within the identification number span range of 10-100 because the span is 9, which is less than 1. Therefore, the identification number interval [1,9] does not belong to the identification number interval corresponding to the identification number span range.

[0087] The target span represents the span that matches the data capability of the candidate processing system. For example, for a candidate processing system, if the data processing capability of the candidate processing system is stronger, then the target span selected from the identification number span range is correspondingly larger, and the number of identification numbers contained in the identification number interval corresponding to the target span is larger. For example, for a candidate processing system with stronger data processing capability, the target span of the identification number interval that matches the candidate system is larger. Exemplarily, the target span of the identification number interval corresponding to the candidate processing system with stronger data processing capability is 20, and the target span of the identification number interval corresponding to the candidate processing system with weaker data processing capability is 10.

[0088] The identification number to be allocated may be an identification number reserved for message data. For example, the identification number to be allocated may be any one of 1 to 99999999.

[0089] Specifically, the processing server obtains the identification number span range configured for the identification number interval, determines the target span that matches the data processing capability from the identification number span range, and selects at least a part from multiple identification numbers to be allocated according to the target span to form an identification number interval that matches the candidate processing system.

[0090] In one of the embodiments, the processing server selects at least a portion of the multiple identification numbers to be allocated according to the selection strategy and the target span to form an identification number interval that matches the candidate processing system.

[0091] The selection strategy may include selecting at least a portion of the multiple identification numbers to be allocated in order from the smallest to the largest according to the target span to form an identification number interval that matches the candidate processing system.

[0092] The selection strategy may further include preferentially allocating identification number intervals to candidate processing systems with larger target spans, and selecting at least one portion of unallocated identification numbers to allocate identification number intervals to other candidate processing systems.

[0093] In this embodiment, an identification number span range configured for an identification number interval is obtained, and a target span matching the data processing capability is determined from the identification number span range. According to the target span, at least a portion of multiple identification numbers to be allocated is selected to form an identification number interval matching the candidate processing system, thereby accurately matching the identification number interval for the candidate processing system, thereby improving the distribution accuracy of message data.

[0094] In one embodiment, as shown in FIG3 , determining a target processing system matching the identification number interval from each candidate processing system includes:

[0095] S302: Determine at least two candidate processing systems that match the identification number interval from among the candidate processing systems.

[0096] The processing system to be selected indicates a system for carrying and processing message data, and at least two processing systems to be selected match the identification number interval, indicating that both systems are used to carry and process message data. In other words, the message data needs to be sent to at least two processing systems to be selected.

[0097] The identification number processing record indicates the processing record for each type of message data in the processing system to be selected. The processing record includes the sending and receiving processing records of each type of message data and the sending and receiving processing records of each type of original message.

[0098] S304, obtaining the identification number processing record of each to-be-selected processing system and the original message identification number associated with the message identification number.

[0099] The original message identification number associated with the message identification number indicates the message identification corresponding to the original message data associated with the message data.

[0100] S306, screening the target processing system whose identification number processing record contains the original message identification number from among the candidate processing systems.

[0101] If the candidate processing system's identification number processing record contains the original message identification number, it means that the candidate processing system has sent out original message data. The message data corresponding to the original message data needs to be sent to the candidate system. This can improve the processing efficiency of the message data and avoid misdistribution of the message data.

[0102] If the candidate processing system's identification number processing record does not include the original message identification number, it means that the candidate processing system has not sent out the original message data, and the message data corresponding to the original message data does not need to be sent to the candidate system to avoid mis-distribution of the message data.

[0103] Specifically, the processing server determines at least two candidate processing systems that match the identification number interval from each candidate processing system, obtains the identification number processing record of each candidate processing system, and the original message identification number associated with the message identification number, and screens the target processing system whose identification number processing record contains the original message identification number among each candidate processing system.

[0104] In one embodiment, the processing server is used to determine whether the identification number processing record of the candidate processing system contains the original message identification number. Based on the determination, the target processing system whose identification number processing record contains the original message identification number is screened among the candidate processing systems.

[0105] In this embodiment, at least two candidate processing systems that match the identification number interval are determined from each candidate processing system, and the identification number processing records of each candidate processing system and the original message identification number associated with the message identification number are obtained. Among each candidate processing system, the target processing system whose identification number processing record contains the original message identification number is screened, which can improve the processing efficiency of the message data and avoid the misdistribution of the message data.

[0106] In one embodiment, sending message data to a target processing system includes: obtaining a system real-time load of the target processing system, screening a target message sending method that matches the system real-time load among multiple message sending methods, and sending the message data to the target processing system according to the target message sending method.

[0107] The system real-time load represents the average load of the target processing system in a specific time interval, including CPU utilization, IO load rate, etc.

[0108] Among them, CPU utilization indicates the ratio of the time the CPU executes non-idle tasks to the total time. It reflects how busy the CPU is. High CPU utilization may mean that the system is running multiple tasks efficiently, but it may also mean that there is resource contention or that some tasks do not get enough CPU time.

[0109] The IO load rate is a measure of how busy the system is during input and output operations, including disk reads and writes, network transmissions, etc. A high IO load rate may mean that the system is processing a large number of data read and write operations, or there is an I / O bottleneck.

[0110] A higher system real-time load represents a higher CPU utilization and / or IO load rate, and conversely, a lower system real-time load represents a lower CPU utilization and / or IO load rate.

[0111] The message sending mode indicates a specific mode of sending message data, and the message sending mode includes at least a first message sending mode and a second message sending mode. The first message sending mode may be a parameter addressing transmission mode, and the second message sending mode may be a transmission mode through the middleware MQ communication. The parameter addressing mode is real-time transmission, which is more efficient than the MQ transmission mode. Among them, the parameter addressing mode includes but is not limited to the addressing modes such as immediate addressing and register addressing.

[0112] Specifically, the processing server obtains the system real-time load of the target processing system, selects a target message sending method that matches the system real-time load from multiple message sending methods, and sends the message data to the target processing system according to the target message sending method.

[0113] In one embodiment, when the system real-time load is high, the target message sending mode can be a mode with low transmission efficiency, reducing the amount of message data transmitted in a short period of time and avoiding errors in the target processing system. Conversely, when the system real-time load is low, the target message sending mode can be a mode with high transmission efficiency, improving the transmission efficiency of message data.

[0114] In this embodiment, the system real-time load of the target processing system is obtained, and among multiple message sending modes, a target message sending mode that matches the system real-time load is screened, and message data is sent to the target processing system according to the target message sending mode. Message data is sent according to the message sending mode that matches the system real-time load, thereby reducing the error probability of the target processing system.

[0115] In one embodiment, the target processing system includes at least two target systems with different system database architectures, and the method also includes: determining the sending order of message data for each target system respectively, sending the message data to the target system with an earlier sending order, and the target system with an earlier order is used to copy and forward the message data to the target system with a later order.

[0116] The sending order of the target system represents the order in which the message data are sent.

[0117] Specifically, a target system with a smaller sending order receives the message data sent by the processing server first, and a target system with a larger sending order receives the message data sent by the processing server later.

[0118] In one embodiment, the processing server sends the message data to the target system with a higher order, and the processing server controls the candidate server (target server) to which the target system with a higher order belongs, and copies and forwards the message data to the target system with a lower order. This greatly improves the distribution efficiency of the message data.

[0119] In one embodiment, the processing server determines the order in which the message data is sent to each target system according to the number of threads of the processing server. For example, the number of threads of the processing server is 3, and the number of target systems is 9. The message data is first distributed to the first 3 target processing servers at the same time. The three target processing servers can copy and forward the message data to 3 of the remaining 6 target processing servers. At the same time, the processing server sends the message data to the remaining 3 target processing servers.

[0120] In this embodiment, the target processing system includes at least two target systems with different system database architectures, determines the sending order of message data for each target system respectively, and sends the message data to the target system with an earlier sending order. The target system with an earlier order is used to copy and forward the message data to the target system with a later order, thereby reducing the data transmission pressure of the processing server and improving the overall sending efficiency of the message data.

[0121] In one embodiment, as shown in FIG4 , a processing method for gradually implementing message diversion in interval proportion is provided, including:

[0122] S402, obtaining a message flow including a plurality of message data.

[0123] The message data includes the message type and the message identification number.

[0124] Specifically, taking the method applied to the message data distribution between the Oracle system and the GoldenDB system as an example, as shown in FIG5 , a message diversion method is provided, including:

[0125] Step 1: Receive a common incoming message, and obtain the original message diversion identifier from the original message type in the common incoming message.

[0126] Step 2: If the original message diversion flag is 0, determine whether to divert based on the original message identification number. If divert, send the message data temporarily stored in the Oracle system to the GoldenDB system. If not divert, send the message data to the Oracle system.

[0127] Step 3: If the original message diversion mark is 1, the Oracle system is unconditionally used. If the original message diversion mark is 2, the message data is sent to both the Oracle system and the GoldenDB system unconditionally. If the original message mark is 3, the GoldenDB system is unconditionally used.

[0128] Step 4: If the original message ID is 4, update the Oracle system to see if there is a record of the original message. If there is a record, send the message data to the Oracle system; if there is no record, send the message data to the GoldenDB system.

[0129] Step 5: Determine the method to be used for diversion.

[0130] Among them, parameter addressing and middleware MQ communication are two ways to divert traffic from the Oracle side to the GoldenDB side.

[0131] The real-time transmission method using parameter addressing is more efficient than the MQ transmission method. The parameter addressing method can be gradually improved in the future.

[0132] S404: Determine the diversion identifier corresponding to each message type according to each message type and the mapping relationship between the message type and the diversion identifier.

[0133] S406 , for each message data, when the diversion identifier of the message data corresponds to multiple candidate processing systems, obtain multiple preset intervals configured for the identification number.

[0134] Each preset interval includes a lower limit of the identification number and an upper limit of the identification number.

[0135] S408, comparing the message identification number with the identification number lower limit and identification number upper limit of each preset interval respectively, and determining the identification number interval to which the message identification number belongs from each preset interval.

[0136] S410 , for each candidate processing system, determining the data processing capability of the candidate processing system based on system information of the candidate processing system.

[0137] S412, obtaining the identification number span range configured for the identification number interval.

[0138] S414, determining a target span matching the data processing capability from the identification number span range.

[0139] S416, selecting at least a portion of the multiple identification numbers to be allocated according to the target span to form an identification number interval matching the candidate processing system.

[0140] Among them, the span of the message identification numbers contained in the identification number interval is positively correlated with the data processing capability.

[0141] S418, determining at least two candidate processing systems that match the identification number interval from among the candidate processing systems.

[0142] S420, obtaining the identification number processing record of each to-be-selected processing system and the original message identification number associated with the message identification number.

[0143] S422, among the candidate processing systems, screening the target processing system whose identification number processing record contains the original message identification number.

[0144] S424, obtaining the real-time system load of the target processing system.

[0145] S426, screening a target message sending method that matches the real-time load of the system from among multiple message sending methods.

[0146] S428, sending the message data to the target processing system according to the target message sending method.

[0147] S430, determining the order in which the message data is sent to each target system.

[0148] The target processing system includes at least two target systems with different system database architectures.

[0149] S432, sending the message data to the target system with a higher sending order.

[0150] Among them, the target system with a higher order is used to copy and forward the message data to the target system with a lower order.

[0151] In this embodiment, a message stream containing multiple message data is obtained; the message data includes a message type and a message identification number, and the shunt identification corresponding to each message type is determined according to each message type and the mapping relationship between the message type and the shunt identification number, and each message data is preliminarily divided according to the message type, which is conducive to reducing the number of message types corresponding to the message data carried in the subsequent candidate processing system and reducing the probability of the candidate processing system reporting an error. For each message data, when the shunt identification number of the message data corresponds to multiple candidate processing systems, the identification number interval to which the message identification number belongs is determined, and the target processing system matching the identification number interval is determined from each candidate processing system. The above method is adopted. On the one hand, each message data is preliminarily divided according to the message type, which reduces the probability of the subsequent candidate processing system reporting an error and improves the distribution accuracy of the message data. On the other hand, based on the message identification number, the target processing system to which the message data is to be sent is determined in each candidate system, the target processing system is accurately determined, and the message data is sent to the target processing system, which further improves the distribution accuracy of the message data. This method provides a message diversion mode. By gradually increasing the transaction volume in the localized environment, it is suitable for scenarios where localized applications take over the traffic of the original server, avoiding the failure of large-volume transactions. This message diversion method is controllable and relatively safe.

[0152] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0153] Based on the same inventive concept, the embodiment of the present application also provides a message data processing device for implementing the message data processing method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more message data processing device embodiments provided below can refer to the limitations on the message data processing method above, and will not be repeated here.

[0154] In one embodiment, as shown in FIG. 6 , a packet data processing device is provided, comprising: an acquisition module 602, a diversion identification determination module 604, an identification number interval determination module 606 and a processing module 608, wherein:

[0155] The acquisition module 602 is used to acquire a message stream containing multiple message data; the message data includes a message type and a message identification number;

[0156] A diversion identifier determination module 604, configured to determine a diversion identifier corresponding to each message type according to each message type and a mapping relationship between the message type and the diversion identifier;

[0157] The identification number interval determination module 606 is used to determine, for each message data, the identification number interval to which the message identification number belongs when the diversion identification of the message data corresponds to multiple candidate processing systems;

[0158] The processing module 608 determines a target processing system that matches the identification number interval from each candidate processing system, and sends the message data to the target processing system.

[0159] In one of the embodiments, the identification number interval determination module 606 is also used to obtain multiple preset intervals configured for the identification number; each preset interval includes a lower limit of the identification number and an upper limit of the identification number; the message identification number in the message data is compared with the lower limit of the identification number and the upper limit of the identification number of each preset interval, and the identification number interval to which the message identification number belongs is determined from each preset interval.

[0160] In one of the embodiments, the device also includes: an information monitoring module, which is used to determine the data processing capability of each candidate processing system based on the system information of the candidate processing system; determine the identification number interval matching the candidate processing system according to the data processing capability; the span of the message identification numbers contained in the identification number interval is positively correlated with the data processing capability.

[0161] In one of the embodiments, the information monitoring module is also used to obtain an identification number span range configured for an identification number interval; determine a target span that matches the data processing capability from the identification number span range; and select at least a portion from a plurality of identification numbers to be allocated according to the target span to form an identification number interval that matches the candidate processing system.

[0162] In one embodiment, the processing module 608 is further used to determine at least two candidate processing systems that match the identification number interval from each candidate processing system; obtain the identification number processing record of each candidate processing system, and the original message identification number associated with the message identification number; and in each candidate processing system, screen the target processing system whose identification number processing record contains the original message identification number.

[0163] In one embodiment, the processing module 608 is further used to obtain the system real-time load of the target processing system; select a target message sending method that matches the system real-time load among multiple message sending methods; and send the message data to the target processing system according to the target message sending method.

[0164] In one of the embodiments, the target processing system includes at least two target systems with different system database architectures; the device also includes: a data sending module, used to determine the sending order of message data for each target system respectively; sending the message data to the target system with a higher sending order; the target system with a higher order is used to copy and forward the message data to the target system with a lower order.

[0165] Each module in the above-mentioned message data processing device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0166] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be shown in FIG7. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store message data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a message data processing method is implemented.

[0167] Those skilled in the art will understand that the structure shown in FIG. 7 is merely a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0168] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program.

[0169] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0170] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0171] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0172] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0173] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A message data processing method, characterized in that: The method comprises: Acquire a message stream containing a plurality of message data; the message data includes a message type and a message identification number; Determine the diversion identifier corresponding to each of the message types according to the message types and the mapping relationship between the message types and the diversion identifiers; For each of the message data, when the diversion identifier of the message data corresponds to multiple candidate processing systems, determining the identification number interval to which the message identification number belongs; A target processing system matching the identification number interval is determined from each of the candidate processing systems, and the message data is sent to the target processing system.

2. The method according to claim 1, characterized in that The determining the identification number interval to which the message identification number belongs includes: Acquire multiple preset intervals configured for the identification number; each of the preset intervals includes a lower limit of the identification number and an upper limit of the identification number; The message identification number is compared with the identification number lower limit and the identification number upper limit of each preset interval respectively, and the identification number interval to which the message identification number belongs is determined from each preset interval.

3. The method according to claim 1, characterized in that The method further comprises: For each of the candidate processing systems, determining the data processing capability of the candidate processing system based on the system information of the candidate processing system; According to the data processing capability, an identification number interval matching the candidate processing system is determined; the span of the message identification numbers contained in the identification number interval is positively correlated with the data processing capability.

4. The method according to claim 3, characterized in that The determining, according to the data processing capability, an identification number interval matching the candidate processing system includes: Get the identification number span range configured for the identification number interval; Determining a target span matching the data processing capability from the identification number span range; According to the target span, at least a portion of the identification numbers to be allocated is selected to form an identification number interval matching the candidate processing system.

5. The method according to claim 1, characterized in that The step of determining a target processing system matching the identification number interval from among the candidate processing systems comprises: Determine at least two processing systems to be selected that match the identification number interval from among the candidate processing systems; Obtaining the identification number processing record of each of the to-be-selected processing systems and the original message identification number associated with the message identification number; In each of the candidate processing systems, a target processing system whose identification number processing record contains the original message identification number is screened.

6. The method according to any one of claims 1 to 5, characterized in that The sending of the message data to the target processing system comprises: Obtaining a real-time system load of the target processing system; Selecting a target message sending mode that matches the real-time load of the system from among multiple message sending modes; The message data is sent to the target processing system according to the target message sending mode.

7. The method according to any one of claims 1 to 5, characterized in that The target processing system includes at least two target systems with different system database architectures; The method further comprises: Determining the order in which the message data is sent to each of the target systems; The message data is sent to the target system with an earlier sending order; the target system with an earlier order is used to copy and forward the message data to the target system with a later order.

8. A message data processing device, characterized in that: The device comprises: An acquisition module, used to acquire a message stream containing a plurality of message data; the message data includes a message type and a message identification number; A shunt identifier determination module, configured to determine the shunt identifier corresponding to each of the message types according to the message types and the mapping relationship between the message types and the shunt identifiers; an identification number interval determination module, configured to determine, for each of the message data, an identification number interval to which the message identification number belongs when the diversion identification of the message data corresponds to a plurality of candidate processing systems; The processing module determines a target processing system matching the identification number interval from each of the candidate processing systems, and sends the message data to the target processing system.

9. The device according to claim 8, characterized in that The identification number interval determination module is also used to obtain multiple preset intervals configured for the identification number; each of the preset intervals includes a lower limit of the identification number and an upper limit of the identification number; the message identification number is compared with the lower limit of the identification number and the upper limit of the identification number of each preset interval respectively, and the identification number interval to which the message identification number belongs is determined from each preset interval.

10. The device according to claim 8, characterized in that The device also includes: an information monitoring module, which is used to determine the data processing capability of each candidate processing system based on the system information of the candidate processing system; determine the identification number interval matching the candidate processing system according to the data processing capability; the span of the message identification numbers contained in the identification number interval is positively correlated with the data processing capability.

11. The device according to claim 10, characterized in that The information monitoring module is also used to obtain the identification number span range configured for the identification number interval; determine the target span matching the data processing capability from the identification number span range; and select at least a part from a plurality of identification numbers to be allocated according to the target span to form an identification number interval matching the candidate processing system.

12. The device according to claim 8, characterized in that The processing module is further used to determine at least two candidate processing systems that match the identification number interval from each of the candidate processing systems; obtain the identification number processing records of each of the candidate processing systems, and the original message identification number associated with the message identification number; and in each of the candidate processing systems, screen the target processing system whose identification number processing record contains the original message identification number.

13. The device according to any one of claims 8 to 12, characterized in that The processing module is also used to obtain the system real-time load of the target processing system; select a target message sending method that matches the system real-time load among multiple message sending methods; and send the message data to the target processing system according to the target message sending method.

14. The device according to any one of claims 8 to 12, characterized in that The target processing system includes at least two target systems with different system database architectures; the device also includes: a data sending module, used to determine the sending order of the message data for each of the target systems respectively; sending the message data to the target system with a higher sending order; the target system with a higher order is used to copy and forward the message data to the target system with a lower order.

15. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

17. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images