Message migration method, device, apparatus and storage medium

Abstract

The present disclosure provides a message migration method, device, equipment and storage medium, which can be applied to the field of financial technology. The method is applied to a message migration device. In response to a server switching request, each message content in a first server is pulled and sequentially written into a memory ring queue of the message migration device. The first server is a server before switching. An asynchronous thread is started to forward each message content written into the memory ring queue to a second server. The second server is a server after switching. In this technical solution, all messages in the old server are quickly read and pushed to the new server to realize fast and lossless migration of messages, and the dependence on business systems and manual monitoring in the traditional way is well solved.

Description

Technical Field

[0001] This disclosure relates to the field of financial technology, and in particular to a message migration method, apparatus, device, and storage medium. Background Technology

[0002] In situations where the server is under excessive load or undergoing maintenance, it may be necessary to migrate messages from one server to another for continued processing in order to avoid impacting the user experience.

[0003] In existing technologies, message queue (MQ) server failover primarily relies on the producer switching first, followed by the consumer, and is achieved through manual monitoring of message queue depth. Specifically, the message sender's business system switches over first, ceasing all message sending to the old MQ server. At this point, the message count in the old queue stops increasing. The consumer's business system processes all messages in its queue before switching over. During the failover process, operations personnel manually monitor the message queue depth to ensure it reaches zero, thus determining if all messages have been processed. Only after all messages have been read can the business system connect to the new MQ server.

[0004] However, existing solutions rely on the rate at which the consumer's business system pulls and processes messages. However, the business system is affected by real-time fluctuations in factors such as the operating status of upstream and downstream systems (i.e., the number of call requests), the system's own performance (e.g., memory, CPU, storage, etc.), and network conditions, which may result in a slow message processing rate and greatly slow down the server switchover process. Summary of the Invention

[0005] This disclosure provides a message migration method, apparatus, device, and storage medium to solve the technical problems of low message switching performance and high manual intervention in the prior art.

[0006] In a first aspect, embodiments of this disclosure provide a message migration method, including:

[0007] In response to a server switch request, each message content in the first server is retrieved and sequentially written into the memory circular queue of the message migration device. The first server is the server before the switch.

[0008] An asynchronous thread is started to forward the message content written in the memory circular queue to the second server, which is the server after the switch.

[0009] In one possible design of the first aspect, after the message content retrieved from the first server is sequentially written into the memory circular queue of the message migration device, the method further includes:

[0010] The asynchronous thread is started to write the message content in the memory circular queue to the local disk.

[0011] Optionally, the method further includes:

[0012] When it is detected that the second server has not received the target message content, the local disk is queried and the target message content is sent to the second server. The target message content is the message content written to the local disk that the second server has not received.

[0013] In another possible design of the first aspect, the second server and the first server are of different types.

[0014] Optionally, before starting the asynchronous thread to forward the message content written in the memory circular queue to the second server, the method further includes:

[0015] Perform format conversion operations on each message content to obtain message content of the same type as the second server.

[0016] In another possible design, both the second server and the first server are message queue servers.

[0017] In another possible design in the first aspect, the method further includes:

[0018] The process of writing each message content into the memory circular queue of the message migration device and forwarding each message content in the memory circular queue is visualized.

[0019] Secondly, embodiments of this disclosure provide a message migration apparatus, including:

[0020] The pull module is used to respond to a server switch request by pulling each message content from the first server and writing it sequentially into the memory circular queue of the message migration device. The first server is the server before the switch.

[0021] The forwarding module is used to start an asynchronous thread to forward the message content written in the memory circular queue to the second server, which is the server after the switch.

[0022] In one possible design of the second aspect, after the message content retrieved from the first server is sequentially written into the memory circular queue of the message migration device, the storage module is further configured to:

[0023] The asynchronous thread is started to write the message content in the memory circular queue to the local disk.

[0024] Optional, query module, used for:

[0025] When it is detected that the second server has not received the target message content, the local disk is queried and the target message content is sent to the second server. The target message content is the message content written to the local disk that the second server has not received.

[0026] In another possible design in the second aspect, the second server and the first server are of different types.

[0027] Optionally, before the asynchronous thread forwards the message content written into the memory circular queue to the second server, the processing module is configured to:

[0028] Perform format conversion operations on each message content to obtain message content of the same type as the second server.

[0029] In another possible design, both the second server and the first server are message queue servers.

[0030] In another possible design, the display module is used for:

[0031] The process of writing each message content into the memory circular queue of the message migration device and forwarding each message content in the memory circular queue is visualized.

[0032] Thirdly, embodiments of this disclosure provide a message migration device, including: a processor, and a memory and a transceiver communicatively connected to the processor;

[0033] The memory stores computer-executed instructions; the transceiver is used for sending and receiving data.

[0034] The processor executes computer execution instructions stored in the memory to implement the message migration method as described in the first aspect or any of the above.

[0035] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the message migration method described in the first aspect or any of the above embodiments.

[0036] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program, which is executed by a processor using the message migration method described in the first aspect or any of the above embodiments.

[0037] The message migration method, apparatus, device, and storage medium provided in this disclosure can be applied to the fintech field. The method, applied to a message migration device, responds to a server switchover request by retrieving message content from a first server and sequentially writing it into a circular queue in the message migration device's memory. The first server is the server before the switchover. An asynchronous thread is then initiated to forward the message content written into the circular queue to a second server, which is the server after the switchover. This technical solution quickly reads all messages from the old server and pushes them to the new server, achieving rapid and lossless message migration and effectively solving the reliance on business systems and manual monitoring in traditional methods. Attached Figure Description

[0038] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0039] Figure 1 This is an illustration of an application scenario provided by the embodiments of this disclosure. Figure 1 ;

[0040] Figure 2 Flowchart of the message migration method provided in the embodiments of this disclosure Figure 1 ;

[0041] Figure 3 This is an illustration of an application scenario provided by the embodiments of this disclosure. Figure 2 ;

[0042] Figure 4 A schematic diagram of a memory circular queue provided in an embodiment of this disclosure;

[0043] Figure 5 Flowchart of the message migration method provided in the embodiments of this disclosure Figure 2 ;

[0044] Figure 6 This is a schematic diagram of the structure of an embodiment of the message migration device provided in this disclosure.

[0045] Figure 7 This is a schematic diagram of the structure of a message migration device provided in an embodiment of this disclosure.

[0046] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0047] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0048] Before introducing the embodiments of this disclosure, the technical terms and technical background involved in the embodiments of this disclosure will be explained first:

[0049] In situations where the server is under excessive load or undergoing maintenance, it may be necessary to migrate messages from one server to another for continued processing in order to avoid impacting the user experience.

[0050] In existing technologies, message queue (MQ) server failover primarily relies on the producer switching first, followed by the consumer, and is achieved through manual monitoring of message queue depth. Specifically, the message sender's business system switches over first, ceasing all message sending to the old MQ server. At this point, the message count in the old queue stops increasing. The consumer's business system processes all messages in its queue before switching over. During the failover process, operations personnel manually monitor the message queue depth to ensure it reaches zero, thus determining if all messages have been processed. Only after all messages have been read can the business system connect to the new MQ server.

[0051] For example, Figure 1 This is an illustration of an application scenario provided by the embodiments of this disclosure. Figure 1 ,like Figure 1 As shown, the application scenario includes: producer business system 11, old server 12, new server 13, and consumer business system 14.

[0052] Optionally, in one example, the producer business system 11 continuously pushes messages to the old server 12, and after receiving the messages, the old server 12 needs to forward them to the consumer business system 14.

[0053] However, if the old server 12 malfunctions or is under maintenance, it is necessary to switch the old server 12 to the new server 13 in order to continue to perform the corresponding forwarding tasks.

[0054] In the existing technology, the producer business system 11 switches over first and stops sending messages to the old server 12. At this time, the number of messages in the old message queue no longer increases. The consumer business system 14 processes all messages in the message queue before switching over. During the switchover process, the operation and maintenance personnel manually monitor whether the message queue depth is 0 to determine whether all messages have been processed. Only after all messages have been read can the producer business system 11 and the consumer business system 14 be connected to the server 13.

[0055] Therefore, the present disclosure aims to address the problem of existing technologies: existing solutions rely on the rate at which the consumer's business system retrieves and processes messages. However, the business system is affected by real-time fluctuations in factors such as the operating status of upstream and downstream systems (i.e., the number of call requests), the system's own performance (e.g., memory, CPU, storage, etc.), and network conditions, which may result in a slow message processing rate and greatly slow down the server switching progress.

[0056] In view of the technical problems existing in the prior art, the inventors of this disclosure have the following idea: if a message lossless migration tool can be provided, which can configure the message receiving queue and sending queue through a visual interface and support dynamic adjustment of the number of worker threads, it can quickly read all messages from the old server and push them to the new server, ultimately achieving fast and lossless message migration, it can effectively solve the dependence on business systems and manual monitoring under the traditional method.

[0057] The technical solutions of this disclosure will now be described in detail through specific embodiments. It should be noted that the following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0058] It is worth noting that the application areas of the migration methods, devices, equipment and storage media disclosed herein are not limited and may involve the financial technology field or other related fields.

[0059] The execution entity of this disclosure is a message migration device, which can be a personal computer, a mobile device, a server, etc.

[0060] Figure 2 Flowchart of the message migration method provided in the embodiments of this disclosure Figure 1 ,like Figure 2 As shown, the message migration method may include the following steps:

[0061] Step 21: In response to the server switch request, retrieve each message content from the first server and write it sequentially into the memory circular queue of the message migration device. The first server is the server before the switch.

[0062] In this step, if the first server experiences a failure, maintenance, or upgrade, the technicians need to switch the first server to another server to replace it in order to ensure that the message processing flow is not affected.

[0063] At this point, there are still unprocessed messages in the first server, and it is necessary to retrieve each message from the first server, that is, the message migration device pulls each message from the first server.

[0064] Furthermore, each message content is sequentially written into the memory circular queue of the message migration device.

[0065] Optional, Figure 3 This is an illustration of an application scenario provided by the embodiments of this disclosure. Figure 2 ,like Figure 3 As shown, the application scenario includes: producer business system 11, old server 12, new server 13, consumer business system 14, and message migration device 31.

[0066] The production system 11 continuously pushes messages to the old server 12. After receiving the messages, the old server 12 needs to forward them to the consumer system 14. However, if the old server 12 malfunctions or is under maintenance, it needs to be switched to the new server 13 to continue performing the corresponding forwarding tasks.

[0067] At this time, the production business system 11 switches first and no longer sends messages to the old server 12. The message migration device 31 pulls the message content from the old server 12, and the production business system 11 and the consumer business system 14 connect to the new server 13.

[0068] It should be understood that: the old server 12 is the first server; the new server 13 is the second server.

[0069] As an example, Figure 4 A schematic diagram of a memory circular queue provided in an embodiment of this disclosure.

[0070] Taking a memory circular queue that occupies 9 storage positions as an example, as message content is written, for example, as the memory circular queue rotates, message content 1 is written to storage position 6, message content 2 is written to storage position 7, message content 3 is written to storage position 8, and so on, message content 5 is written to storage position 1.

[0071] Step 22: Start an asynchronous thread to forward each message content written in the memory circular queue to the second server, which is the server after the switch.

[0072] In this step, message migration device 31 starts an asynchronous thread, and as the message content on the memory circular queue increases, it forwards each written message content to the second server.

[0073] In this step, an asynchronous thread can also be started to store the written message content, as detailed in the following embodiments.

[0074] As an example, Figure 4 As shown, as the memory circular queue rotates, message content 1 in storage position 6 is forwarded to the second server, message content 2 in storage position 7 is forwarded to the second server, message content 3 in storage position 8 is forwarded to the second server, and so on, message content 5 in storage position 1 is forwarded to the second server. As the message content is forwarded, the messages in the corresponding storage positions are cleared to prepare for the writing and forwarding of subsequent message content.

[0075] Optionally, the second server can be of a different type than the first server.

[0076] Therefore, before step 21, the method can also perform format conversion operations on each message content to obtain message content of the same type as the second server.

[0077] In this embodiment, since the second server and the first server are of different types, the message content needs to be formatted. That is, after receiving the message content from the first server, the message migration device performs format conversion on each message content according to the type of the second server, and then forwards the formatted message content to the second server.

[0078] Optionally, both the second and first servers can be message queue (MQ) servers.

[0079] MQ is a "First-In, First-Out" (FIFO) data structure in basic data structures. It refers to placing data (messages) to be transmitted in a queue and using a queue mechanism to implement message delivery—producers generate messages and put them into the queue, which are then processed by consumers. Consumers can pull messages from a specified queue or subscribe to the corresponding queue, where the MQ server will push messages to them.

[0080] During the above implementation process, the process of writing each message content into the memory circular queue of the message migration device and forwarding each message content in the memory circular queue can be visualized.

[0081] Optionally, during the above two steps, the message migration device can visualize the execution process, specifically, but not limited to, displaying the number of successful and failed forwardings.

[0082] The message migration method provided in this disclosure can be applied to the fintech field. This method is applied to a message migration device. In response to a server switchover request, it retrieves message content from a first server and sequentially writes it into a circular queue in the message migration device's memory. The first server is the server before the switchover. An asynchronous thread is then started to forward the message content written into the circular queue to a second server, which is the server after the switchover. This technical solution quickly reads all messages from the old server and pushes them to the new server, achieving fast and lossless message migration and effectively solving the reliance on business systems and manual monitoring in traditional methods.

[0083] Based on the above embodiments, Figure 5 Flowchart of the message migration method provided in the embodiments of this disclosure Figure 2 ,like Figure 5 As shown, the message migration method may also include the following steps:

[0084] Step 51: Start an asynchronous thread to write the contents of each message in the memory circular queue to the local disk.

[0085] In this step, while writing each message content to the memory circular queue, the written message content can also be written to the local disk to ensure that even if the message content is lost due to extreme environmental factors such as network jitter during the migration process, it can be resent as a backup.

[0086] Writing to the local disk can be an asynchronous process, with a single node efficiency of up to 100 tps.

[0087] Step 52: When it is detected that the second server has not received the target message content, query the local disk and send the target message content to the second server. The target message content is the message content that has been written to the local disk and has not been received by the second server.

[0088] In this step, when the message migration device records the message content that failed to be forwarded in real time, i.e. the target message content, and detects that the second server has not received the target message content, after the impact of extreme environmental factors such as network jitter has disappeared, the target message content is resent to the second server to ensure the reliability and security of message content forwarding.

[0089] The message migration method provided in this disclosure starts an asynchronous thread to write each message content in the memory circular queue to the local disk. When it is detected that the second server has not received the target message content, the local disk is queried and the target message content is sent to the second server. The target message content is the message content written to the local disk that the second server has not received. In this technical solution, the message is stored during the message forwarding process to ensure that even if the message is lost due to extreme environmental factors such as network jitter during the migration process, it can be resent as a backup, thus ensuring the reliability and security of message forwarding.

[0090] The following are embodiments of the message migration apparatus of this disclosure, which can be used to execute embodiments of the message migration method of this disclosure. For details not disclosed in the embodiments of the message migration apparatus of this disclosure, please refer to the embodiments of the message migration method of this disclosure.

[0091] Figure 6 This is a schematic diagram illustrating the structure of an embodiment of the message migration apparatus provided in this disclosure. Figure 6 As shown, the message migration device includes:

[0092] The pull module 61 is used to respond to the server switching request, pull each message content in the first server and write it into the memory circular queue of the message migration device in sequence. The first server is the server before the switch.

[0093] Forwarding module 62 is used to start an asynchronous thread to forward the message content written in the memory circular queue to the second server, which is the server after the switch.

[0094] In one possible design of this embodiment, after the message content is retrieved from the first server and sequentially written into the memory circular queue of the message migration device, the storage module 63 is further configured to:

[0095] An asynchronous thread is started to write the contents of each message in the memory circular queue to the local disk.

[0096] Optional, query module, used for:

[0097] When it is detected that the second server has not received the target message content, the local disk is queried and the target message content is sent to the second server. The target message content is the message content that was written to the local disk and was not received by the second server.

[0098] In another possible design of this disclosure embodiment, the second server and the first server are of different types.

[0099] Optionally, before starting the asynchronous thread to forward the individual message contents written to the memory circular queue to the second server, the processing module is used for:

[0100] Perform format conversion operations on each message content to obtain message content of the same type as the second server.

[0101] In another possible design of this disclosure embodiment, both the second server and the first server are message queue servers.

[0102] In another possible design of this disclosure embodiment, the display module is used for:

[0103] The process of writing each message content into the memory circular queue of the message migration device and forwarding each message content in the memory circular queue is visualized.

[0104] The message migration apparatus provided in this disclosure can be used to execute the message migration method in any of the above embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0105] It should be noted that the division of the various modules in the above device is merely a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated. Furthermore, these modules can be implemented entirely in software via processing element calls; they can be fully implemented in hardware; or some modules can be implemented by processing element calls to software, while others are implemented in hardware. Additionally, these modules can be fully or partially integrated together, or implemented independently. The processing element mentioned here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each of the above modules can be completed through the integrated logic circuits in the hardware of the processor element or through software instructions.

[0106] Figure 7 This is a schematic diagram of the structure of the message migration device provided in the embodiments of this disclosure, such as... Figure 7 As shown, the message migration device may include: a processor 71, a memory 72, and computer program instructions stored in the memory 72 and executable on the processor 71. When the processor 71 executes the computer program instructions, it implements the message migration method provided in any of the foregoing embodiments.

[0107] Optionally, the various devices mentioned above in the message migration device can be connected via a system bus.

[0108] The memory 72 can be a separate memory unit or a memory unit integrated into the processor 71. The number of processors 71 can be one or more.

[0109] It should be understood that the processor 71 can be a Central Processing Unit (CPU), or other general-purpose processors 71, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor 71 can be a microprocessor 71, or any conventional processor 71. The steps of the method disclosed in this disclosure can be directly manifested as being executed by the hardware processor 71, or being executed by a combination of hardware and software modules within the processor 71.

[0110] The system bus can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The system bus can be divided into address bus, data bus, control bus, etc. For ease of representation, only one thick line is used in the diagram, but this does not indicate that there is only one bus or one type of bus. Memory 72 may include random access memory (RAM) 72, and may also include non-volatile memory (NVM) 72, such as at least one disk storage device 72.

[0111] All or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable memory 72. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned memory 72 (storage medium) includes: read-only memory 72 (ROM), RAM, flash memory 72, hard disk, solid-state hard disk, magnetic tape, floppy disk, optical disk, and any combination thereof.

[0112] The message migration device provided in this disclosure can be used to execute the message migration method provided in any of the above method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0113] This disclosure provides a computer-readable storage medium storing computer instructions that, when executed on a computer, cause the computer to perform the aforementioned message migration method.

[0114] The aforementioned computer-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory, electrically erasable programmable read-only memory, erasable programmable read-only memory, programmable read-only memory, read-only memory, magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0115] Optionally, a readable storage medium can be coupled to the processor, enabling the processor to read information from and write information to the readable storage medium. Alternatively, the readable storage medium can be an integral part of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0116] This disclosure also provides a computer program product, which includes a computer program stored in a computer-readable storage medium. At least one processor can read the computer program from the computer-readable storage medium, and the at least one processor can implement the above-described method when executing the computer program.

[0117] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A message migration method, characterized in that, Applied to message migration devices, the method includes: In response to a server switch request, each message content in the first server is retrieved and sequentially written into the memory circular queue of the message migration device. The first server is the server before the switch. An asynchronous thread is started to forward the message content written in the memory circular queue to the second server, which is the server after the switch. After the message content retrieved from the first server is sequentially written into the memory circular queue of the message migration device, the method further includes: The asynchronous thread is started to write the message content in the memory circular queue to the local disk; When it is detected that the second server has not received the target message content, the local disk is queried and the target message content is sent to the second server. The target message content is the message content that was written to the local disk and was not received by the second server. The second server and the first server are of different types.

2. The method according to claim 1, characterized in that, Before the asynchronous thread forwards the message content written to the memory circular queue to the second server, the method further includes: Perform format conversion operations on each message content to obtain message content of the same type as the second server.

3. The method according to claim 1 or 2, characterized in that, Both the second server and the first server are message queue servers.

4. The method according to claim 1 or 2, characterized in that, The method further includes: The process of writing each message content into the memory circular queue of the message migration device and forwarding each message content in the memory circular queue is visualized.

5. A message migration device, characterized in that, Applications in message migration devices, including: The pull module is used to respond to a server switch request by pulling each message content from the first server and writing it sequentially into the memory circular queue of the message migration device. The first server is the server before the switch. The forwarding module is used to start an asynchronous thread to forward the message content written in the memory circular queue to the second server, which is the server after the switch. After the message content retrieved from the first server is sequentially written into the memory circular queue of the message migration device, the storage module is further configured to: The asynchronous thread is started to write the message content in the memory circular queue to the local disk; The query module is used to query the local disk and send the target message content to the second server when it is detected that the second server has not received the target message content. The target message content is the message content written to the local disk and not received by the second server. The second server and the first server are of different types.

6. A message migration device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the message migration method as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the message migration method as described in any one of claims 1 to 4.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the message migration method as described in any one of claims 1 to 4.

Images