A message queue-based data interaction method and system

Abstract

The application provides a kind of data interaction method and system based on message queue, it is related to computer technology field, wherein the method comprises the following steps: obtaining multiple first object data to be sent by sending end, and the sending parameter and original order corresponding to the first object data;Whether the original order of the first object data needs to be adjusted according to the sending parameter is determined, and second object data is generated according to the final order adjustment result;According to the second object data, the corresponding delivery message queue is generated;The data processing logic of multiple receiving ends is obtained, and is sent to the corresponding receiving end for processing according to the sending parameter of the second object data;The order of message is adjusted in advance to form a message queue, which can ensure that important messages are processed preferentially, can adapt to different business needs and scenarios, so as to improve the efficiency of overall data processing.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to a data interaction method and system based on message queues. Background Technology

[0002] Currently, a "message queue" is a container that stores messages during message transmission. It is a communication mechanism used to transfer data in distributed systems. It allows different applications or services to exchange information asynchronously and has a wide range of applications in distributed systems.

[0003] However, most message queues are not designed to directly adjust the order of existing messages in the queue. Therefore, messages need to be sorted before forming a message queue to be sent. However, existing technologies usually do not sort messages. Once messages are sent to the message queue, their order is fixed. The order of messages cannot be adjusted, but only the order of multiple message queues can be adjusted, which affects the efficiency of subsequent data processing. Furthermore, when there are multiple data processing methods for messages in the message queue, the most suitable data processing method cannot be selected for processing. This results in low processing efficiency and cannot solve the problem of high load.

[0004] Therefore, how to provide a message queue-based data interaction method that can solve the above problems is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] In view of this, the present invention provides a data interaction method and system based on message queues. By pre-adjusting the order of messages to form a message queue, it can ensure that important messages are processed first, adapt to different business needs and scenarios, and thus improve the overall data processing efficiency.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A data interaction method based on a message queue includes the following steps:

[0008] Obtain multiple first object data to be sent by the sending end, as well as the sending parameters and original order of the first object data;

[0009] Based on the sending parameters, determine whether the original order of the first object data needs to be adjusted, and generate the second object data based on the final order adjustment result;

[0010] Generate a corresponding delivery message queue based on the second object data;

[0011] The data processing logic of multiple receiving ends is obtained, and the data is sent to the corresponding receiving end for processing according to the sending parameters of the second object data.

[0012] Preferably, the specific process for determining whether the original order of the first object data needs to be adjusted based on the sending parameters includes:

[0013] Obtain the sending parameters of the first object data, wherein the sending parameters include message content, message type, priority, expiration time, and execution time;

[0014] When either the priority or the expiration time of the first object data meets the corresponding preset threshold requirements, the order of the first object data will not be adjusted.

[0015] When any one of the priority or expiration time in the first object data does not meet the corresponding preset threshold requirement, the sorting time is calculated based on the preset weight of priority, the preset weight of expiration time, and the preset weight of execution time, and the original order of the first object data is adjusted according to the sorting time.

[0016] Preferably, the specific process of generating the corresponding message delivery queue includes:

[0017] The original order of the first object data is adjusted according to the sorting time to generate the second object data, and the second object data constitutes the delivery message queue.

[0018] Preferably, the specific process of sending the message to the corresponding receiving end for processing according to the sending parameters of the delivery message queue includes:

[0019] Construct a first virtual processor and a second virtual processor, and each of the first virtual processor and the second virtual processor is pre-configured with multiple data processing logics;

[0020] The load space of the second object data is obtained. When the load space is greater than or equal to a preset space threshold, the second object data is divided according to a preset ratio to obtain a first queue data and a second queue data, which are then sent to the corresponding first virtual processor and second virtual processor respectively.

[0021] The first virtual processor and the second virtual processor process the corresponding first queue data and second queue data.

[0022] Preferably, the specific process of sending the message to the corresponding receiving end for processing according to the sending parameters of the delivery message queue further includes:

[0023] When the load space is less than the preset space threshold, the second object data is sent to the first virtual processor and the second virtual processor respectively, and the second object data is processed according to different data processing methods to determine the corresponding processing time;

[0024] Determine the shortest data processing method corresponding to the processing time;

[0025] The second object data is sent to the receiving end corresponding to the data processing method with the shortest processing time to complete the processing.

[0026] Preferred options also include:

[0027] The number of receiving terminals can be dynamically increased or decreased.

[0028] The present invention also provides a data interaction system based on a message queue, comprising:

[0029] The acquisition module is used to acquire the message queue sent by the sender, wherein the message queue includes multiple first object data;

[0030] The generation module is used to determine the sending parameters of the first object data, determine whether the order of multiple first object data needs to be adjusted according to the sending parameters, and generate a corresponding delivery message queue, wherein the delivery message queue includes multiple second object data.

[0031] The sending module is used to acquire the data processing logic of multiple receiving ends and send the data to the corresponding receiving end for processing according to the sending parameters of the second object data.

[0032] Preferred options also include:

[0033] The add / remove module is used to dynamically add or remove the receiving end.

[0034] As can be seen from the above technical solution, compared with the prior art, the present invention discloses a data interaction method and system based on a message queue. It acquires multiple first object data to be sent by the sending end, as well as the corresponding sending parameters and original order of the first object data. Based on the sending parameters, it determines whether the original order of the first object data needs adjustment. Based on the final order adjustment result, it generates second object data and a corresponding delivery message queue, and constructs multiple virtual machines. By using virtual machines to expand processing when the message load is large, the optimal receiving end is selected for processing. By pre-adjusting the message order to form a message queue, it is possible to ensure that important messages are processed first, adapting to different business needs and scenarios, thereby improving the overall data processing efficiency. Simultaneously, the present invention can select the optimal processing method for messages with multiple data processing methods, improving the system's flexibility, scalability, and response speed. Attached Figure Description

[0035] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0036] Figure 1 A flowchart of a data interaction method based on a message queue provided by the present invention;

[0037] Figure 2 The present invention provides a structural principle block diagram of a data interaction system based on a message queue. Detailed Implementation

[0038] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0039] The purpose of this invention is to provide a data interaction method and system based on a message queue. The method includes: acquiring multiple first object data items to be sent by a sender, along with corresponding sending parameters and their original order; determining whether the original order of the first object data needs adjustment based on the sending parameters, and generating second object data based on the final order adjustment result; generating a corresponding delivery message queue based on the second object data; acquiring data processing logic from multiple receivers, and sending the second object data to the corresponding receivers for processing according to the sending parameters of the second object data. This provides a solution to problems in existing technologies, such as the inability to pre-adjust the order of messages in the message queue while simultaneously failing to select the optimal data processing logic.

[0040] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0041] See Figure 1 As shown, this embodiment of the invention discloses a data interaction method based on a message queue, implemented by a sender and a receiver, including the following steps:

[0042] Obtain multiple first object data to be sent by the sender, as well as the sending parameters and original order of the first object data;

[0043] Determine whether the original order of the first object data needs to be adjusted based on the sending parameters, and generate the second object data based on the final order adjustment result;

[0044] Generate a corresponding delivery message queue based on the second object data;

[0045] The data processing logic of multiple receiving ends is obtained, and the data is sent to the corresponding receiving end for processing according to the sending parameters of the second object data. Each receiving end can have a data processing logic pre-set.

[0046] In a specific embodiment, the specific process of determining whether the original order of the first object data needs to be adjusted based on the transmission parameters includes:

[0047] Obtain the sending parameters for the first object data, including message content, message type, priority, expiration time, and execution time;

[0048] If either the priority or the expiration time of the first object data meets the corresponding preset threshold requirements, the order of the first object data will not be adjusted.

[0049] When any of the priority or expiration time in the first object data does not meet the corresponding preset threshold requirements, the sorting time is calculated based on the preset weight of priority, the preset weight of expiration time, and the preset weight of execution time, and the original order of the first object data is adjusted according to the sorting time.

[0050] Specifically, the calculation process for the sorting time can be obtained using the following expression:

[0051]

[0052] In the formula, α, β, and γ are the corresponding weights, and α+β+γ=1; t2 represents the expiration time, t3 represents the execution time, and b1 represents the priority.

[0053] In this embodiment of the invention, the above process determines in advance whether the order needs to be adjusted based on the sending parameters of the first object data. If so, the sorting time is calculated and the order is adjusted according to the sorting time before sending, thus ensuring the sequentiality of the first object data and meeting actual needs.

[0054] In a specific embodiment, the process of generating the corresponding delivery message queue includes:

[0055] The original order of the first object data is adjusted according to the sorting time to generate the second object data, and the second object data constitutes the message delivery queue.

[0056] In a specific embodiment, the process of sending the message according to the sending parameters of the delivery message queue to the corresponding receiving end for processing includes:

[0057] Construct a first virtual processor and a second virtual processor, and each of the first virtual processor and the second virtual processor is pre-configured with multiple data processing logics;

[0058] Obtain the load space of the delivery message queue. When the load space is greater than or equal to the preset space threshold, divide the second object data according to the preset ratio to obtain the first queue data and the second queue data, and send them to the corresponding first virtual processor and second virtual processor respectively.

[0059] The first virtual processor and the second virtual processor process the corresponding first queue data and second queue data.

[0060] In a specific embodiment, the process of sending the message to the corresponding receiving end for processing according to the sending parameters of the delivery message queue further includes:

[0061] When the load space is less than the preset space threshold, the second object data included in the delivery message queue is randomly sent to the first virtual processor and the second virtual processor, and the second object data is processed according to different data processing methods to obtain the corresponding processing time.

[0062] Select the data processing method with the shortest processing time;

[0063] The second object data is sent to the receiving end corresponding to the data processing method with the shortest processing time to complete the processing.

[0064] In one specific embodiment, it also includes:

[0065] The number of receivers can be dynamically increased or decreased.

[0066] Specifically, if a receiver is not used for an extended period after a certain running time, it can be deleted. In addition, when the load space of the second object data is greater than or equal to a preset space threshold and exceeds the processing space of the first and second virtual processors, the historical processing data of the receivers can be comprehensively analyzed through neural networks to identify the receivers with a higher probability of using the data processing method, and additional processing can be applied to the receivers to further improve the flexibility of the system.

[0067] In one specific embodiment, it also includes:

[0068] A corresponding delivery message queue is generated based on the second object data. At the same time, a corresponding hash value is generated for each second object data. When the delivery message queue is sent to the first virtual processor and the second virtual processor, the hash value is checked to determine whether the second object data has been tampered with. If it has changed, it is determined that the second object data has been attacked and tampered with. The second object data is then discarded and relevant prompts are sent to the sending and receiving ends, which can improve the security of the entire interaction and processing process.

[0069] See Figure 2 As shown, embodiments of the present invention also provide a system utilizing a message queue-based data interaction method as described in any of the above embodiments, comprising:

[0070] The acquisition module is used to acquire the message queue sent by the sender, wherein the message queue includes multiple first object data;

[0071] The generation module is used to determine the sending parameters of the first object data, determine whether the order of multiple first object data needs to be adjusted based on the sending parameters, and generate a corresponding delivery message queue, which includes multiple second object data.

[0072] The sending module is used to obtain the data processing logic of multiple receiving ends and send it to the corresponding receiving end for processing according to the sending parameters of the second object data.

[0073] In one specific embodiment, it further includes an addition / reduction module for dynamically adding or removing receivers.

[0074] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the message queue-based data interaction method as described in any of the above embodiments.

[0075] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0076] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A message queue based data interaction method, characterized in that, The method comprises the following steps: Obtaining a plurality of first object data to be sent by a sending end, and sending parameters and original orders corresponding to the first object data; Determining whether the original order of the first object data needs to be adjusted according to the sending parameters, and generating second object data according to the final order adjustment result, wherein the specific process of determining whether the original order of the first object data needs to be adjusted according to the sending parameters comprises: Obtaining the sending parameters of the first object data, wherein the sending parameters comprise message content, message type, priority, expiration time and execution time; When any one of the priority and the expiration time of the first object data meets the corresponding preset threshold requirement, the order of the first object data is not adjusted; When any one of the priority and the expiration time of the first object data does not meet the corresponding preset threshold requirement, the priority preset weight, the expiration time preset weight and the execution time preset weight are calculated to obtain a sorting time, and the original order of the first object data is adjusted according to the sorting time; Generating a corresponding delivery message queue according to the second object data; Obtaining data processing logic of a plurality of receiving ends, and sending to the corresponding receiving ends for processing according to the sending parameters of the delivery message queue, wherein the specific process comprises: Building a first virtual processor and a second virtual processor, and the first virtual processor and the second virtual processor are both pre-provided with a plurality of data processing logics; Obtaining a load space of the second object data, when the load space is greater than or equal to a preset space threshold, the second object data is divided according to a preset proportion to obtain first queue data and second queue data, and is respectively sent to the first virtual processor and the second virtual processor; Processing the corresponding first queue data and second queue data in the first virtual processor and the second virtual processor; When the load space is less than the preset space threshold, the second object data is respectively sent to the first virtual processor and the second virtual processor, and the second object data is processed according to different data processing methods to determine a corresponding processing time; Determining the shortest data processing method corresponding to the processing time; Sending the second object data to the receiving end corresponding to the data processing method with the shortest processing time for processing; Further comprising: dynamically increasing and decreasing the receiving end, wherein the specific process comprises: When a receiving end is not used for a long time after a running time is exceeded, the receiving end is deleted; In addition, when the load space of the second object data is greater than or equal to the preset space threshold and exceeds the processing space of the first virtual processor and the second virtual processor, the historical processing data of the receiving end is comprehensively analyzed by a neural network to obtain a receiving end with a higher data processing method use probability, and the receiving end is increased to further improve the flexibility of the system. Also comprising: generating a corresponding delivery message queue according to the second object data, and generating a corresponding hash value for each second object data; when the delivery message queue is sent to the first virtual processing machine and the second virtual processing machine, judging whether the second object data is tampered by judging whether the hash value is changed; if changed, discarding the second object data and sending a related prompt to the sending end and the receiving end.

2. The method of claim 1, wherein, The specific process of generating a corresponding delivery message queue comprises: Adjusting the original order of the first object data according to the sorting time, generating second object data, and constructing the delivery message queue by the second object data.

3. A system for data interaction based on message queue using the method according to any one of claims 1-2, characterized in that, Comprise: An acquisition module is used to acquire the message queue sent by the sending end, wherein the message queue comprises a plurality of first object data; A generation module is used to determine the sending parameter of the first object data, determine whether the order of the plurality of first object data needs to be adjusted according to the sending parameter, and generate a corresponding delivery message queue, wherein the delivery message queue comprises a plurality of second object data; A sending module is used to acquire the data processing logic of a plurality of receiving ends, and send to the corresponding receiving end for processing according to the sending parameter of the second object data.

4. The system of claim 3, wherein, Also comprising: An increase / decrease module is used to dynamically increase / decrease the receiving end.

Images