Information migration method, information generation method, device, equipment and readable medium
By automatically creating topics and new processors in the target message processing system and adjusting them according to the configuration information of the original processors, the problem of cumbersome steps and resource waste caused by manual operation in the existing technology is solved, and efficient and stable information migration and data transmission are achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-02-09
- Publication Date
- 2026-03-17
AI Technical Summary
Existing technologies rely on manual operation for topic switching and disaster recovery mechanisms between message processing systems, resulting in cumbersome procedures, high error rates, and wasted resources, which affects data stability.
An information migration method is provided, which automatically creates topics and new processors in the target message processing system, adjusts relevant information, including data synchronization and monitoring parameters, based on the original processor's configuration information, generates migration records, and stores them in a database.
It improves information migration efficiency, reduces human error, ensures data stability, enhances resource utilization, reduces manual operations, and ensures data integrity and consistency.
Smart Images

Figure CN116610652B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of this disclosure relate to the field of computer technology, and more specifically to information migration methods, information generation methods, apparatus, devices, and readable media. Background Technology
[0002] With the continuous development of internet technology, the trend of big data is becoming increasingly prominent. The timeliness, accuracy, and stability of data processing are becoming increasingly important. Under this trend, real-time big data computing platforms have emerged. To meet the business requirements for data stability, these platforms typically need to be able to switch topics between different message processing systems (such as Kafka, a distributed publish-subscribe messaging system) to achieve topic upgrades or disaster recovery.
[0003] Currently, version upgrades typically rely on manual operation. This is not only cumbersome but also prone to errors, impacting overall stability. Furthermore, existing technologies generally employ a dual-stream approach for disaster recovery, pre-purchasing a set of topics, publishers, and subscribers in another system. However, in the event of a system failure, users still need to manually modify the configurations in the publish and subscribe tasks. This mechanism wastes resources and still carries the risk of failure. Summary of the Invention
[0004] The summary portion of this disclosure is intended to provide a brief overview of concepts that will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions. Some embodiments of this disclosure provide information migration methods, information generation methods, apparatuses, electronic devices, and computer-readable media to address one or more of the technical problems mentioned in the background section above.
[0005] In a first aspect, some embodiments of this disclosure provide an information migration method, including: in response to receiving migration information, determining whether a target topic exists in a target message processing system, wherein the migration information includes the target topic, an identification information set of the original processor of the target topic, and identification information of the target message processing system; in response to determining that the target topic does not exist, creating a target topic in the target message processing system; determining whether information on a new processor exists in the target message processing system, wherein the new processor is used to process messages belonging to the target topic; in response to determining that information on a new processor does not exist, creating information on a new processor in the target message processing system; and adjusting relevant information in the target message processing system according to the configuration information of the original processor of the target topic.
[0006] In some embodiments, adjusting relevant information in the target message processing system based on the configuration information of the original processor of the target topic includes: determining whether the original processor of the target topic has a target data synchronization task, wherein the target data synchronization task is a data synchronization task related to the target topic; in response to determining that there is a target data synchronization task, constructing a new data synchronization task based on the target data synchronization task, wherein the new data synchronization task is used to indicate the target topic in the target message processing system.
[0007] In some embodiments, the relevant information in the target message processing system is adjusted according to the configuration information of the original processor of the target topic, including: determining whether the original processor of the target topic has a data interaction task; and in response to determining that there is a data interaction task, modifying the message processing system corresponding to the original processor in the data interaction task to the target message processing system.
[0008] In some embodiments, adjusting relevant information in the target message processing system based on the configuration information of the original processor of the target topic includes: modifying monitoring parameters in the target message processing system based on the monitoring configuration information of the original processor of the target topic.
[0009] In some embodiments, information for creating a new processor in the target message processing system includes: information for creating a new subscriber to a target topic in the target message processing system, and resetting the new subscriber's location to a preset position.
[0010] In some embodiments, the method further includes generating migration records and storing them in a database.
[0011] In some embodiments, the method further includes: determining whether a target record is contained in the database, wherein the target record is a record representing the creation of a target topic in the target message processing system; generating and storing a target record in response to determining that no target record is contained; and establishing an association between a new processor and the target topic; and generating and storing a modification record based on the association.
[0012] Secondly, some embodiments of this disclosure provide an information migration apparatus, including: a topic determination unit configured to determine whether a target topic exists in a target message processing system in response to receiving migration information, wherein the migration information includes the target topic, an identification information set of the original processor of the target topic, and identification information of the target message processing system; a topic creation unit configured to create a target topic in the target message processing system in response to determining that the target topic does not exist; a processor determination unit configured to determine whether information on a new processor exists in the target message processing system, wherein the new processor is used to process messages belonging to the target topic; a processor creation unit configured to create information on a new processor in the target message processing system in response to determining that information on a new processor does not exist; and an adjustment unit configured to adjust relevant information in the target message processing system according to the configuration information of the original processor of the target topic.
[0013] Thirdly, some embodiments of this disclosure provide an information generation method, including: receiving a migration request, wherein the migration request includes an identification information set of processors to be migrated and identification information of a target message processing system; determining a target topic based on the identification information in the identification information set to obtain a target topic set; selecting identification information groups from the identification information set based on the target topics in the target topic set to obtain a set of identification information groups; and generating migration information based on the identification information groups in the set of identification information groups to obtain a migration information set, wherein the migration information includes a target topic, an identification information set of processors of the target topic, and identification information of the target message processing system.
[0014] In some embodiments, determining a target topic and obtaining a target topic set based on the identifier information in the identifier information set includes: querying the original message processing system for topics corresponding to the publishers indicated by the identifier information in the identifier information set of the publishers to be migrated, and using these topics as candidate topics to obtain a candidate topic set, wherein the original message processing system is the message processing system from which the information is to be migrated; for each candidate topic in the candidate topic set, performing the following query steps: querying the original message processing system for data interaction tasks related to the candidate topic; querying the topics involved in the data interaction tasks to obtain a topic set; determining whether the topic set contains other topics, wherein other topics are topics corresponding to the publishers to be migrated and not included in the candidate topic set; and determining the candidate topic set as the target topic set in response to the determination that it does not contain other topics.
[0015] In some embodiments, selecting an identifier information group from an identifier information set based on the target topics in the target topic set to obtain an identifier information group set includes: querying the identifier information of subscribers of the target topics in the target topic set in the original message processing system to obtain a subscriber identifier information set; grouping the target topics in the target topic set according to the data interaction task to obtain multiple target topic subsets; and selecting an identifier information group from the publisher identifier information set and the subscriber identifier information set based on the target topics in each of the multiple target topic subsets to obtain an identifier information group set.
[0016] In some embodiments, the method further includes: performing migration processing on migration information in the migration information set using the information migration method described in any of the implementations of the first aspect above.
[0017] Fourthly, some embodiments of this disclosure provide an information generation apparatus, including: a receiving unit configured to receive a migration request, wherein the migration request includes an identification information set of processors to be migrated and identification information of a target message processing system; a determining unit configured to determine a target topic based on the identification information in the identification information set, thereby obtaining a target topic set; a selecting unit configured to select identification information groups from the identification information set based on the target topics in the target topic set, thereby obtaining a set of identification information groups; and a generating unit configured to generate migration information based on the identification information groups in the set of identification information groups, thereby obtaining a migration information set, wherein the migration information includes a target topic, an identification information set of processors of the target topic, and identification information of the target message processing system.
[0018] Fifthly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the methods described in any of the implementations of the first or third aspects above.
[0019] Sixthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first or third aspect above.
[0020] The various embodiments of this disclosure have the following beneficial effects: The information migration method of some embodiments of this disclosure provides an implementation method for automatically switching topics and topic processors (i.e., publishers and / or subscribers) between systems, which can improve migration efficiency and ensure data stability. Specifically, the reason for low information migration efficiency and data instability between message processing systems is that it mainly relies on manual operation. Based on this, the information migration method of some embodiments of this disclosure can automatically create topic and new processor information in the target (i.e., the message processing system to be migrated) system. Furthermore, it can automatically adjust relevant information in the target message processing system according to the original processor's configuration information. This can greatly reduce manual operation, thereby reducing the possibility of human error. It can improve migration efficiency and ensure data stability. Attached Figure Description
[0021] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.
[0022] Figure 1 This is an architecture diagram of an exemplary system to which some embodiments of this disclosure can be applied;
[0023] Figure 2 This is a flowchart of some embodiments of the information migration method according to this disclosure;
[0024] Figure 3 These are schematic diagrams illustrating the structure of some embodiments of the information migration apparatus according to this disclosure;
[0025] Figure 4 This is a flowchart of some embodiments of the information generation method according to this disclosure;
[0026] Figure 5 These are schematic diagrams illustrating the structure of some embodiments of the information generation apparatus according to this disclosure;
[0027] Figure 6A and Figure 6B This is a schematic diagram illustrating application scenarios of the information migration method and information generation method according to some embodiments of this disclosure;
[0028] Figure 7 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation
[0029] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.
[0030] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.
[0031] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.
[0032] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".
[0033] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
[0034] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.
[0035] Figure 1 An exemplary system architecture 100 is shown for an information migration method or information migration apparatus to which some embodiments of the present disclosure may be applied.
[0036] like Figure 1 As shown, system architecture 100 may include terminal device 101, networks 102 and 104, server 103, and server cluster 105 (servers 1051 and 1052). Network 102 can be used as a medium to provide a communication link between terminal device 101 and server 103. Network 104 can be used as a medium to provide a communication link between server 103 and server cluster 105. Networks 102 and 104 may include various connection types, such as wired or wireless communication links or fiber optic cables, etc.
[0037] Users can use terminal device 101 to interact with server 103 via network 102 to receive or send messages, etc. Various client applications can be installed on terminal device 101, such as big data real-time computing applications, message processing system management applications, web browsers, shopping applications, and instant messaging tools, etc.
[0038] The terminal device 101 here can be either hardware or software. When the terminal device 101 is hardware, it can be various electronic devices with a display screen, including but not limited to smartphones, tablets, e-book readers, laptops, and desktop computers. When the terminal device 101 is software, it can be installed in the electronic devices listed above. It can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or it can be implemented as a single software program or software module. No specific limitations are made here.
[0039] Servers 1051 and 1052 in server cluster 105 can be servers that provide various services, such as servers that support message processing systems, such as the distributed publish-subscribe messaging system (Kafka).
[0040] Server 103 can also be a server providing various services, such as a backend server supporting applications installed on terminal device 101. Upon receiving migration information, the backend server can determine whether the target topic and the new processor for that topic exist in the target message processing system (such as server 1051). Based on the determination, it can then perform the corresponding information migration processing. The processing result (such as adjustment results or adjusted information about the target message processing system) can be sent to the user's terminal device 101.
[0041] Server 103 and servers 1051 and 1052 in server cluster 105 can be either hardware or software. When servers 103 and 1051 and 1052 are hardware, they can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When servers 103 and 1051 and 1052 are software, they can be implemented as, for example, multiple software programs or software modules used to provide distributed services, or as a single software program or software module. No specific limitations are made here.
[0042] It should be noted that the methods provided in the embodiments of this disclosure can be executed by the server 103 or by the terminal device 101. Accordingly, the apparatus can be located in the server 103 or in the terminal device 101. No specific limitations are made here.
[0043] It should be noted that if the servers in server cluster 105 have the functions of server 103, server 103 may not need to be set up in system architecture 100.
[0044] It should be understood that Figure 1 The terminal devices, networks, servers, server clusters, and the number of servers within a server cluster shown are merely illustrative. Depending on actual needs, there can be any number of terminal devices, networks, servers, server clusters, and servers within a server cluster.
[0045] Continue to refer to Figure 2 The diagram illustrates a flow 200 of some embodiments of an information migration method according to the present disclosure. The method includes the following steps:
[0046] Step 201: In response to receiving migration information, determine whether the target topic exists in the target message processing system.
[0047] In some embodiments, the entity performing the information migration method (e.g. Figure 1 The server 103 shown, upon receiving migration information, can determine whether the target topic exists in the target message processing system. The target topic is generally the topic to be migrated. The target message processing system is generally the message processing system to which the target topic is to be migrated.
[0048] Here, the message processing system can be a distributed publish-subscribe messaging system (such as Kafka). It typically features high throughput, low latency, scalability, message persistence, and high concurrency. A message processing system usually contains one or more servers. These servers are generally called brokers. They receive messages from publishers, typically set offsets for the messages, and commit the messages to disk. Generally, one broker in a message processing system also acts as the broker cluster controller. Each message published to the message processing system has a topic (also called a category).
[0049] In some embodiments, the executing entity can receive migration information via a wired or wireless connection. The migration information may include a target topic, a set of identification information for the original processor of the target topic, and identification information for the target message processing system. This identification information is generally used to uniquely identify what it represents. The identification information can be at least one of numbers, letters, characters, or symbols. Furthermore, the original processor is primarily distinguished from subsequent new processors. The original processor typically processes messages belonging to the target topic in the message processing system from which the information is to be migrated (i.e., the original message processing system).
[0050] Here, processors typically include publishers and / or subscribers. Publishers (i.e., producers) generally publish messages to the message processing system. Before publishing a message, they determine the topic to which the message belongs. Subscribers (i.e., consumers) generally read messages from the message processing system for the topics they have subscribed to. Typically, one publisher corresponds to one topic, while one topic can correspond to multiple publishers and multiple subscribers.
[0051] It should be noted that when the executing entity receives the migration information, it can also lock the authentication operations of the involved processors to prevent data loss during subsequent migration processing. The locking method is not limited here; it could include tagging the relevant processors' identification information, or disconnecting the processors from the target topic in the message processing system to be migrated out.
[0052] In some embodiments, if the executing entity stores data of the target message processing system locally, such as when the executing entity and the target message processing system are located on the same server, it can be determined whether a topic matching (e.g., identical to) the target topic exists in the target message processing system. Conversely, the executing entity can obtain data from the target message processing system (e.g., via a wired or wireless connection) through a wired or wireless connection. Figure 1 The system retrieves topic data from server 1051 (as shown). It then determines whether the target topic exists within this topic data. Alternatively, the executing entity can send a query request containing the target topic to the target message processing system. Based on the returned query results, it can then be determined whether the target topic exists.
[0053] In some embodiments, if it is determined that the target topic does not exist in the target message processing system, step 202 can be executed. If it is determined that the target topic exists, step 203 can be executed.
[0054] Step 202: Create a target topic in the target message processing system.
[0055] In some embodiments, if it is determined that the target topic does not exist, the executing entity can create the target topic in the target message processing system. For example, the executing entity can send a creation instruction for the target topic to the target message processing system. As an example, the target topic can be created by calling the corresponding interface (such as JSF, Java Server Faces, a standard framework for building applications). Alternatively, the executing entity can send a message to the target message processing system, the topic of which is the target topic. This will ensure that the target topic exists in the target message processing system. The relevant configuration during the creation process can follow the configuration information in the original message processing system.
[0056] Step 203: Determine whether there is information about a new processor in the target message processing system.
[0057] In some embodiments, the executing entity may further determine whether there is information about a new processor in the target message processing system. This new processor is used to process messages belonging to the target topic in the target message processing system.
[0058] In some embodiments, the executing entity may determine whether there is information about a processor corresponding to the target topic in the target message processing system. This processor corresponds to the desired new processor. That is, if the new processor is a subscriber, then the processor here is also a subscriber. If so, it indicates that there is information about a new processor. Otherwise, it indicates that there is no such information.
[0059] Furthermore, to improve the accuracy of the determination, if information about a processor corresponding to the target topic is identified, the executing entity can further determine whether the processor's identification information matches the identification information of the original processor for the target topic, that is, whether it matches the identification information of the processor corresponding to the target topic in the original message processing system (equivalent to the processor to be migrated out). If they match (e.g., the same), it indicates that there is information about a new processor in the target message processing system. If they do not match (e.g., different), it indicates that there is no new processor.
[0060] Optionally, the executing entity can determine whether other topics exist in the target message processing system. These other topics can be topics with the same processor as the target topic, such as topics subscribed to by the same subscriber. If such other topics exist, it indicates that there is information about a new processor in the target message processing system. This is because the information for the new processor was already created when the other topic was created in the target message processing system, and therefore does not need to be created again here. This helps improve the efficiency of information migration.
[0061] Here, if it is determined that there is no information for a new processor, step 204 can be executed. If it is determined that there is information for a new processor, step 205 can be executed.
[0062] Step 204: Create information for a new processor in the target message processing system.
[0063] In some embodiments, if it is determined that no new processor information exists, the executing entity can create new processor information in the target message processing system. A series of configuration operations can then be performed on the new processor, such as authorization (authentication) and rate limiting (limiting processing speed or memory resources). This achieves balanced resource allocation, contributing to improved stability of the target message processing system. Furthermore, a relationship can be established between the new processor and the target topic. Thus, the new processor can publish messages on the target topic to the target message processing system, or read messages on the target topic from the target message processing system.
[0064] In some embodiments, if the identification information set of the migration information includes the identification information of subscribers to the target topic, it is explained that the information of the new processor to be created in the target message processing system may include the information of the new subscribers to the target topic. Furthermore, upon completion of creation, the executing entity can reset the new subscriber's position to a preset location. Here, the position is typically metadata, an incrementing integer value used to record the subscriber's position during rebalancing, for data recovery purposes. That is, the new subscriber can start reading messages from the target message processing system from the preset location. The preset location can be any location, such as the starting position for message reading or a user-defined position.
[0065] Optionally, the executing entity can also determine whether there is information about idle or useless processors in the target message processing system. If so, the executing entity can use that processor as a new processor and perform a series of configuration operations. Alternatively, it can also use processors whose identification information does not match the target topic as new processors. This can improve creation efficiency and help further improve the efficiency of information migration.
[0066] Understandably, if a topic was previously migrated from message processing system A to message processing system B, and later migrated back from message processing system B to message processing system A, then message processing system A would typically contain information about that topic and its handlers.
[0067] Step 205: Adjust the relevant information in the target message processing system according to the configuration information of the original processor of the target topic.
[0068] In some embodiments, the executing entity can obtain or locate the configuration information of the original processor based on the identifier information of the original processor of the target topic. Then, based on this configuration information, relevant information in the target message processing system can be adjusted.
[0069] As an example, the executing entity can modify the monitoring parameters of the target message processing system (or a new processor) based on the original processor's monitoring configuration information. For instance, the executing entity can store the monitoring program in the target message processing system. Alternatively, monitoring can typically be performed by a separate monitoring node (software program or hardware). In this case, the executing entity can call the interface provided by the monitoring node to monitor the target message processing system. This improves the efficiency of modification.
[0070] In some embodiments, the executing entity can also determine whether the original processor of the target topic has a target data synchronization task. The target data synchronization task is typically a data synchronization task related to the target topic. A data synchronization task generally refers to synchronizing message data from one message processing system (such as a broker cluster) to another message processing system (such as a broker cluster) in real time. For example, messages published by a publisher are stored in system A, while subscribers can read these messages from system B. Such data synchronization tasks often run in the background and are not easily noticed by users. If it is determined that a target data synchronization task exists, the executing entity can construct a new data synchronization task based on the target data synchronization task. This new data synchronization task can be used to indicate the target topic in the target message processing system. For example, the executing entity can copy the target data synchronization task and modify its target topic to the target topic in the target message processing system, thereby obtaining and starting a new data synchronization task. That is, modifying the data source or target source to the target topic in the target message processing system.
[0071] In some application scenarios, the executing entity can also determine whether the original processor has a data interaction task. Here, a data interaction task is generally a type of task that exchanges data between multiple data sources (such as a Fregat access / distribution task). It typically supports data interaction between various data sources, including relational database management systems (MySQL), non-relational databases (NoSQL), and topics, and can integrate data integration / synchronization. Such data interaction tasks are often user-specified. For example, messages are stored in system A by default, but the user can specify that they be stored in system B. If it is determined that a data interaction task exists, the executing entity can modify the message processing system corresponding to the original processor in the data interaction task to the target message processing system. To ensure the effectiveness of the modified data interaction task, the executing entity can restart the data interaction task.
[0072] Understandably, to further improve migration efficiency, if the migration information contains multiple target topics, the aforementioned data synchronization and data interaction tasks can be performed after migrating multiple or all target topics to the target message processing system. This reduces redundant processing. Furthermore, upon completion of the new processor's information creation or the overall migration, the executing entity can generate a notification message and send it to the corresponding processor's terminal device to restart the message processing (publish and / or subscribe) task. The notification message can be presented in various ways, including but not limited to images, text, audio, or video. The sending method can include but not limited to email, SMS, or telephone. In this case, the user can choose whether to restart immediately based on the actual situation. For example, the user can choose to restart after reading data from the original message processing system to read data from the target message processing system. If the user can accept a certain amount of data loss, they can choose to restart immediately.
[0073] As described above, the information migration methods provided in some embodiments of this disclosure can achieve automatic migration of topics and processors between message processing systems. This eliminates the need to pre-apply for a set of topics and processors in other message processing systems, and also eliminates the need for manual operation. This not only improves resource utilization and information migration efficiency but also avoids or reduces human error, thereby enhancing data stability.
[0074] In some embodiments, the executing entity can also generate migration records according to the migration process described above, and store these records in a database. This facilitates user tracking and querying. Furthermore, the executing entity can also send the migration process to a terminal device for display to the user. This allows the user to understand the migration progress and promptly identify problems or areas for improvement.
[0075] As an example, before creating information for a new processor in the target message processing system, the executing entity can determine whether the database contains a target record. This target record can be a record representing the creation of a target topic in the target message processing system. If it is determined that the target record is not present, the executing entity can generate and store the target record in the database. Alternatively, the executing entity can also generate and store a modification record based on the association between the new processor and the target topic. This involves modifying the target topic's ID (Identity Document) in the new processor record in the database.
[0076] It is understood that the database mentioned above can be any database (such as HBase). The migration records (or logs) can be stored in ways including, but not limited to, logging systems (such as Logbook), topic-based or distributed full-text search engines (such as Elasticsearch).
[0077] Further reference Figure 3 As a response to the above Figure 2 The present disclosure provides some embodiments of the information migration apparatus for implementing the method shown, and these apparatus embodiments are similar to... Figure 2 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.
[0078] like Figure 3 As shown, the information migration apparatus 300 in some embodiments may include: a topic determination unit 301, configured to determine whether a target topic exists in the target message processing system in response to receiving migration information, wherein the migration information includes the target topic, the identification information set of the original processor of the target topic, and the identification information of the target message processing system; a topic creation unit 302, configured to create a target topic in the target message processing system in response to determining that no target topic exists; a processor determination unit 303, configured to determine whether information on a new processor exists in the target message processing system, wherein the new processor is used to process messages belonging to the target topic; a processor creation unit 304, configured to create information on a new processor in the target message processing system in response to determining that no information on a new processor exists; and an adjustment unit 305, configured to adjust relevant information in the target message processing system according to the configuration information of the original processor of the target topic.
[0079] In some embodiments, the adjustment unit 305 may be further configured to determine whether the original processor of the target topic has a target data synchronization task, wherein the target data synchronization task is a data synchronization task related to the target topic; in response to determining that there is a target data synchronization task, a new data synchronization task is constructed based on the target data synchronization task, wherein the new data synchronization task is used to indicate the target topic in the target message processing system.
[0080] In some embodiments, the adjustment unit 305 may be further configured to determine whether the original processor of the target topic has a data interaction task; in response to determining that there is a data interaction task, modify the message processing system corresponding to the original processor in the data interaction task to the target message processing system.
[0081] In some embodiments, the adjustment unit 305 may be further configured to modify the monitoring parameters in the target message processing system according to the monitoring configuration information of the original processor of the target topic.
[0082] In some embodiments, the processor creation unit 304 may be configured to create information about new subscribers to a target topic in the target message processing system, and to reset the new subscribers’ positions to a preset location.
[0083] In some embodiments, the device 300 may further include a storage unit ( Figure 3 (Not shown in the image), it is configured to generate migration records and store them in the database.
[0084] In some embodiments, the storage unit may be further configured to determine whether the database contains a target record, wherein the target record is a record representing the creation of a target topic in the target message processing system; generate and store a target record in response to determining that the database does not contain a target record; establish an association between a new processor and the target topic; and generate and store a modification record based on the association.
[0085] It is understandable that the units described in the device 300 are related to the reference. Figure 2 The steps in the described method correspond accordingly. Therefore, the operations, features, and beneficial effects described above for the method also apply to the device 300 and the units contained therein, and will not be repeated here.
[0086] See also Figure 4 The diagram illustrates a flow 400 of some embodiments of the information generation method according to this disclosure. The method may further include the following steps:
[0087] Step 401: Receive migration request.
[0088] In some embodiments, the entity executing the information generation method can receive migration requests via a wired or wireless connection. The migration request may include a set of identifiers for the processor to be migrated and identifiers for the target message processing system. The processor may include publishers and / or subscribers. The target message processing system is generally the message processing system to be migrated.
[0089] It is understandable that the execution subject of this embodiment is... Figure 2 The execution entity in the embodiments can be the same execution entity (e.g. Figure 1 The server 103 shown may or may not be the one mentioned. The migration request here can be sent by the user or by another electronic device detecting a trigger signal. For example, message processing system A (e.g., Figure 1If the server 1051 shown detects that a certain operational indicator has reached a preset value, it can send the aforementioned migration request to the execution entity, thereby migrating out the information. This operational indicator is usually related to the system's operating status. When this indicator reaches a preset value, it indicates that the system may be experiencing a malfunction or its processing efficiency may be significantly reduced. In this case, the target message processing system can be pre-set by the user or selected based on the current operating status of other systems.
[0090] Step 402: Determine the target topic based on the identification information in the identification information set to obtain the target topic set.
[0091] In some embodiments, the executing entity can determine the target topics based on the identification information in the migration request, thereby obtaining a set of target topics. Here, the target topics are generally the topics to be migrated.
[0092] As an example, if the migration request includes a set of identifiers for subscribers to be migrated (i.e., subscriber information migration), the executing entity can query the original message processing system for topics related to the identifiers based on each or more identifiers in the identifier set. That is, the topics of the messages the subscribers indicate they want to read. Thus, the queried topics can be used as target topics to obtain the target topic set. Here, the original message processing system is typically the message processing system from which the information is to be migrated.
[0093] For example, if the migration request includes a set of identifiers for the publishers to be migrated (i.e., publisher information migration), the executing entity can use the identifiers in the set to query the original message processing system for the topics corresponding to the publishers indicated by the identifiers. That is, the topics of the messages published by the publishers indicated by the identifiers. The queried topics can then be used as candidate topics, resulting in a candidate topic set.
[0094] Furthermore, for each candidate topic in the candidate topic set, the following query steps can be performed: query the data interaction tasks related to the candidate topic in the original message processing system; query the topics involved in the data interaction tasks to obtain a topic set; determine whether the topic set contains other topics; in response to the determination that no other topics are included, the candidate topic set is determined as the target topic set. Here, other topics are typically topics outside the candidate topic set that correspond to the publisher to be migrated. The publisher to be migrated here is the publisher indicated in the migration request. If the topic set contains other topics, the executing entity can use these other topics as candidate topics and perform the above query steps again.
[0095] Step 403: Based on the target themes in the target theme set, select the identification information groups from the identification information set to obtain the identification information group set.
[0096] In some embodiments, the executing entity may select an identification information group from the identification information set based on one or more target topics in the target topic set to obtain an identification information group set.
[0097] As an example, if the migration request includes a set of identifiers for the subscribers to be migrated, the executing entity can select a group of identifiers from the identifier set based on each target topic in the target topic set. That is, it selects the identifiers of all subscribers who subscribe to that target topic. For instance, if subscriber A and subscriber B are both consumers of topic T1, then subscriber A and subscriber B can be grouped together. In other words, subscribers in the same identifier group have all subscribed to the same target topic.
[0098] Optionally, if the migration request includes a set of identifiers for the publishers to be migrated, the executing entity can first query the identifiers of the subscribers of the target topics in the target topic set within the original message processing system to obtain the subscriber identifier set. Next, based on the data interaction task, the target topics in the target topic set can be grouped to obtain multiple target topic subsets. Then, based on the target topics in each of the multiple target topic subsets, identifier groups are selected from the publisher identifier set and the subscriber identifier set to obtain an identifier group set. For example, data interaction task 1 involves topics T1 and T2. In this case, topics T1 and T2 can be grouped together (target topic subset). Thus, the publishers and subscribers of topics T1 and T2 can be grouped together to obtain identifier groups. That is, the identifier information of the involved processors is grouped according to the task + topic approach.
[0099] Understandably, for publisher information migration, the information generation method in this embodiment can link publishing and subscription. That is, upon receiving the identifier information of the publisher to be migrated, it can automatically determine the identifier information of the relevant subscribers (i.e., the subscribers to be migrated), thereby generating migration information. This enables one-click migration and switching of topics, publishers, and subscribers. This further reduces manual operations and improves migration efficiency. Moreover, it avoids situations where subscribers cannot consume data after the upstream publisher switches.
[0100] Step 404: Generate migration information based on the identification information groups in the identification information group set to obtain the migration information set.
[0101] In some embodiments, based on the set of identification information groups obtained in step 403, the executing entity can generate migration information, thereby a set of information to be migrated, according to each identification information group. The migration information may include a target topic, an identification information set (i.e., an identification information group) of the processor of the target topic, and identification information of the target message processing system. It should be noted that the target topic in the migration information is the same target topic selected when the identification information group was chosen.
[0102] As an example, if both subscriber A and subscriber B are consumers of topic T1, then a group of identification information containing subscribers A and B can be selected based on topic T1. The resulting migration information can include topic T1, the identification information of subscriber A, the identification information of subscriber B, and the identification information of the target message processing system.
[0103] For example, based on a subset of topics T1 and T2, a group of identifier information containing publisher A, publisher B, and subscriber C is selected. The resulting migration information can include topics T1 and T2, the identifier information of publisher A, the identifier information of publisher B, the identifier information of subscriber C, and the identifier information of the target message processing system.
[0104] In some application scenarios, the executing entity can also adopt the above-mentioned Figure 2 The information migration method described in any implementation of the embodiments performs migration processing on each migration piece of information in the migration information set. It is understood that, to improve migration processing efficiency, the executing entity can use multiple threads to process the information separately. For example, one thread can perform the above-described information migration method operation on one migration piece of information. Alternatively, the executing entity can send each migration piece of information in the migration information set to a separate server in the server cluster (e.g., ...). Figure 1 Servers 1051 and 1052 are shown in the diagram. Each server can execute the aforementioned information migration method to process the received migration information. In other words, by grouping, a migration request can be split into multiple migration requests. These multiple migration requests can then be processed separately by multiple threads on multiple machines, thus achieving optimized information migration processing.
[0105] As described above, the information generation methods provided in some embodiments of this disclosure can automatically acquire and generate migration information. This reduces manual operations, not only improving information generation efficiency but also ensuring the integrity and comprehensiveness of the information. This avoids incomplete information migration due to omissions, which could affect the use of the message processing system after migration. Furthermore, by grouping and splitting the information involved in the migration request, it is easier to process the migration information in batches, thereby improving processing efficiency.
[0106] Further reference Figure 5 As a response to the above Figure 4 The present disclosure provides some embodiments of the information generation apparatus for implementing the method shown, and these apparatus embodiments are similar to... Figure 4 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.
[0107] like Figure 5 As shown, the information generation apparatus 500 in some embodiments may include: a receiving unit 501 configured to receive a migration request, wherein the migration request includes an identification information set of the processor to be migrated and identification information of the target message processing system; a determining unit 502 configured to determine a target topic based on the identification information in the identification information set, thereby obtaining a target topic set; a selecting unit 503 configured to select identification information groups from the identification information set based on the target topics in the target topic set, thereby obtaining a set of identification information groups; and a generating unit 504 configured to generate migration information based on the identification information groups in the set of identification information groups, thereby obtaining a migration information set, wherein the migration information includes a target topic, an identification information set of the processor of the target topic, and identification information of the target message processing system.
[0108] In some embodiments, the determining unit 502 may be further configured to query the original message processing system for topics corresponding to the publishers indicated by the identification information in the identifier information set of the publishers to be migrated, as candidate topics, to obtain a candidate topic set, wherein the original message processing system is the message processing system from which the information is to be migrated; for each candidate topic in the candidate topic set, the following query steps are performed: query the original message processing system for data interaction tasks related to the candidate topic; query the topics involved in the data interaction tasks to obtain a topic set; determine whether the topic set contains other topics, wherein other topics are topics corresponding to the publishers to be migrated and outside the candidate topic set; in response to determining that they do not contain other topics, the candidate topic set is determined as the target topic set.
[0109] In some embodiments, the selection unit 503 may be further configured to query the identifier information of subscribers of target topics in the target topic set in the original message processing system to obtain a set of subscriber identifier information; group the target topics in the target topic set according to the data interaction task to obtain multiple target topic subsets; and select an identifier information group from the identifier information set of publishers and the identifier information set of subscribers according to the target topics in each of the multiple target topic subsets to obtain a set of identifier information groups.
[0110] In some embodiments, the device 500 may further include a processing unit ( Figure 5 (not shown in the image), configured to employ the above Figure 2The information migration method described in any of the embodiments performs migration processing on the migration information in the migration information set.
[0111] It is understandable that the units described in the device 500 are related to the reference. Figure 4 The steps in the described method correspond accordingly. Therefore, the operations, features, and beneficial effects described above for the method also apply to the device 500 and the units contained therein, and will not be repeated here.
[0112] Please see Figure 6A and Figure 6B The diagrams illustrate an application scenario of the information migration method and information generation method disclosed herein.
[0113] In this application scenario, such as Figure 6A As shown, on the producer management page, users can select the identifier information set of publishers to be migrated from message processing system A, and select the message processing system B to be migrated to (hereinafter referred to as System B). The main server 601 can call a task interface (such as the Fregata interface) to query the relevant topics and subscribers, obtaining the target topic set and the identifier information set of the subscribers to be migrated. Furthermore, it can group the identifier information of publishers and subscribers according to a task + topic approach, thereby generating migration information and obtaining a migration information set. The main server 601 can then send each migration information set to the respective sub-servers 602 and 603.
[0114] from Figure 6A As can be seen, after receiving the migration information, subserver 602 first locks the authentication of the publishers and subscribers involved. Next, it determines whether the target topic exists in system B. If it doesn't exist, it creates the target topic in system B. If it exists, it determines whether the target record (i.e., the record for creating the target topic in system B) exists in database 604. If it doesn't exist, it generates the target record and inserts it into database 604. Afterwards, it performs publisher information creation, authorization, rate limiting operations, etc., in system B, and generates corresponding modification records. Then, it performs subscriber information creation, authorization, rate limiting operations, etc., in system B, and generates corresponding modification records. It can also reset the subscriber's position to the starting position, notify subscriber users to restart the program, and modify the monitoring information configured for publishers and subscribers in system B. Afterwards, it determines whether the original publisher has a data synchronization task. If so, it creates a new data synchronization task and adds relevant records to database 604. If not, it further determines whether the original publisher has a data interaction task. If so, it modifies the data interaction task. If not, the migration ends.
[0115] In addition, such as Figure 6BAs shown, on the producer management page, users can select the set of subscriber identification information to be migrated from message processing system A, select the message processing system B to be migrated to (hereinafter referred to as system B), and the time to reset the location in system B. The master server 601 can query the relevant topics to obtain the target topic set. It can also group the subscriber identification information according to topics to generate migration information, thus obtaining a migration information set. The master server 601 can then send each migration information set to the respective sub-servers 602 and 603.
[0116] from Figure 6B As can be seen, after receiving the migration information, subserver 602 first locks the authentication of the affected subscribers. Next, it determines whether the target topic exists in system B. If it doesn't exist, it creates the target topic in system B. If it exists, it determines whether the target record (i.e., the record for creating the target topic in system B) exists in database 604. If it doesn't exist, it generates the target record and inserts it into database 604. Afterwards, it can perform subscriber information creation, authorization, rate limiting operations, etc., in system B, and generate corresponding modification records. It can also reset the subscriber's position to the user-set time and modify the subscriber's configured monitoring information in system B. Then, it determines whether the original subscriber has a data synchronization task. If so, it creates a new data synchronization task and adds a relevant record to database 604. If not, it further determines whether the original subscriber has a data interaction task. If so, it modifies the data interaction task. If not, the migration ends.
[0117] It should be noted that the above migration process can be modularized, for example... Figure 6A and Figure 6B In this diagram, the processing flow contained within a dashed box can be completed by a single node. This node can be a software program or hardware. This reduces the coupling between processing flows and increases the flexibility of processing flow expansion. This allows for the rapid integration of new modules and reduces the difficulty of method expansion.
[0118] The following is for reference. Figure 7 It illustrates electronic devices suitable for implementing some embodiments of this disclosure (e.g., Figure 1 The structural diagram of server 103)700 in the middle. Figure 7 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.
[0119] like Figure 7As shown, the electronic device 700 may include a processing unit (e.g., a central processing unit, a graphics processor, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. The RAM 703 also stores various programs and data required for the operation of the electronic device 700. The processing unit 701, ROM 702, and RAM 703 are interconnected via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.
[0120] Typically, the following devices can be connected to I / O interface 705: input devices 706 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 707 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 708 including, for example, magnetic tapes, hard disks, etc.; and communication devices 709. Communication device 709 allows electronic device 700 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 7 An electronic device 700 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 7 Each box shown can represent a device or multiple devices as needed.
[0121] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 709, or installed from storage device 708, or installed from ROM 702. When the computer program is executed by processing device 701, it performs the functions defined in the methods of some embodiments of this disclosure.
[0122] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.
[0123] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.
[0124] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: determine, in response to receiving migration information, whether a target topic exists in the target message processing system, wherein the migration information includes the target topic, the identification information set of the original processor of the target topic, and the identification information of the target message processing system; create a target topic in the target message processing system in response to determining that no target topic exists; determine whether information on a new processor exists in the target message processing system, wherein the new processor is used to process messages belonging to the target topic; create information on a new processor in the target message processing system in response to determining that no new processor exists; and adjust relevant information in the target message processing system according to the configuration information of the original processor of the target topic. Alternatively, a migration request may be received, wherein the migration request includes a set of identifiers for the processor to be migrated and identifiers for the target message processing system; based on the identifiers in the identifier set, a target topic is determined to obtain a target topic set; based on the target topics in the target topic set, identifier groups are selected from the identifier set to obtain a set of identifier groups; based on the identifier groups in the identifier group set, migration information is generated to obtain a migration information set, wherein the migration information includes the target topic, the identifier set of the processor for the target topic, and identifiers for the target message processing system.
[0125] Furthermore, computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages or a combination thereof. Programming languages include object-oriented programming languages—such as Java, Smalltalk, and C++—and conventional procedural programming languages—such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0126] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0127] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a topic determination unit, a topic creation unit, a processor determination unit, a processor creation unit, and an adjustment unit. Alternatively, it may be a receiving unit, a determining unit, a selecting unit, and a generating unit. The names of these units do not necessarily limit the unit itself; for example, a topic determination unit may also be described as "a unit that determines whether a target topic exists in the target message processing system."
[0128] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.
[0129] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.
Claims
1. A method for information migration, comprising: determining whether a target topic exists in a target message processing system in response to receiving migration information, wherein the migration information comprises the target topic, a set of identification information of an original handler of the target topic, and identification information of the target message processing system; creating the target topic in the target message processing system in response to determining that the target topic does not exist; determining whether information of a new handler exists in the target message processing system, wherein the new handler is used to handle messages belonging to the target topic; creating the information of the new handler in the target message processing system in response to determining that the information of the new handler does not exist; adjusting related information in the target message processing system according to configuration information of the original handler of the target topic, comprising: determining whether a target data synchronization task exists for the original handler of the target topic; and constructing a new data synchronization task according to the target data synchronization task in response to determining that the target data synchronization task exists.
2. The information migration method according to claim 1, wherein The target data synchronization task is a data synchronization task related to the target topic, and the new data synchronization task is used to indicate the target topic in the target message processing system.
3. The information migration method of claim 1, wherein, The adjusting related information in the target message processing system according to the configuration information of the original handler of the target topic comprises: determining whether a data interaction task exists for the original handler of the target topic; and modifying a message processing system corresponding to the original handler in the data interaction task to the target message processing system in response to determining that the data interaction task exists.
4. The information migration method of claim 1, wherein, The adjusting related information in the target message processing system according to the configuration information of the original handler of the target topic comprises: modifying a monitoring parameter in the target message processing system according to monitoring configuration information of the original handler of the target topic.
5. The information migration method of claim 1, wherein, The creating the information of the new handler in the target message processing system comprises: creating information of a new subscriber of the target topic in the target message processing system, and resetting a location of the new subscriber to a preset location.
6. The information migration method according to any one of claims 1 to 5, wherein The method further comprises: generating a migration record and storing the migration record in a database.
7. The information migration method of claim 6, wherein, The method further comprises: determining whether a target record exists in the database, wherein the target record is a record representing the creation of the target topic in the target message processing system; generating and storing the target record in response to determining that the target record does not exist; and establishing an association between the new handler and the target topic; generating and storing a modification record according to the association.
8. An apparatus for information migration, comprising: a topic determination unit configured to determine whether a target topic exists in a target message processing system in response to receiving migration information, wherein the migration information comprises the target topic, a set of identification information of an original handler of the target topic, and identification information of the target message processing system; a topic creation unit configured to create the target topic in the target message processing system in response to determining that the target topic does not exist; A processor determining unit configured to determine whether there is information of a new processor in the target message processing system, wherein the new processor is used to process messages belonging to the target topic; A processor creating unit configured to, in response to a determination that there is no information of the new processor, create information of the new processor in the target message processing system; An adjusting unit configured to adjust relevant information in the target message processing system according to configuration information of original processors of the target topic, including: determining whether there is a target data synchronization task for the original processors of the target topic; and in response to a determination that there is the target data synchronization task, constructing a new data synchronization task according to the target data synchronization task.
9. An information generation method, comprising: receiving a migration request, wherein the migration request comprises a set of identification information of a processor to be migrated and identification information of a target message processing system; determining target topics according to identification information in the set of identification information, to obtain a set of target topics, including: querying, according to identification information in the set of identification information of the publisher to be migrated, topics corresponding to the publisher indicated by the identification information in the original message processing system as candidate topics, to obtain a set of candidate topics; for each candidate topic in the set of candidate topics, performing the following query steps: querying, in the original message processing system, data interaction tasks related to the candidate topic; querying topics involved in the data interaction tasks, to obtain a set of topics; determining whether the set of topics contains other topics, wherein the other topics are topics corresponding to the publisher to be migrated and outside the set of candidate topics; in response to a determination that the set of topics does not contain the other topics, determining the set of candidate topics as the set of target topics; selecting, according to target topics in the set of target topics, groups of identification information from the set of identification information, to obtain a set of groups of identification information; generating migration information according to groups of identification information in the set of groups of identification information, to obtain a set of migration information, wherein the migration information comprises a target topic, a set of identification information of a processor of the target topic, and identification information of the target message processing system.
10. The information generation method according to claim 9, wherein The original message processing system is a message processing system from which information is to be migrated.
11. The information generation method according to claim 10, wherein The selecting, according to target topics in the set of target topics, groups of identification information from the set of identification information, to obtain a set of groups of identification information, comprises: querying, in the original message processing system, identification information of subscribers of target topics in the set of target topics, to obtain a set of identification information of subscribers; grouping target topics in the set of target topics according to data interaction tasks, to obtain a plurality of subsets of target topics; selecting, according to target topics in each subset of target topics in the plurality of subsets of target topics, groups of identification information from the set of identification information of the publisher and the set of identification information of the subscriber, to obtain the set of groups of identification information.
12. The information generation method according to one of claims 9-11, wherein, The method further comprises: migrating, using the information migration method of any one of claims 1-7, migration information in the set of migration information.
13. An information generation apparatus, comprising: A receiving unit configured to receive a migration request, wherein the migration request comprises a set of identification information of a handler to be migrated and identification information of a target message processing system; A determining unit configured to determine target topics according to identification information in the set of identification information, to obtain a set of target topics, comprising: querying topics corresponding to publishers indicated by identification information in the set of identification information of the publisher to be migrated as candidate topics in the original message processing system, to obtain a set of candidate topics; for each candidate topic in the set of candidate topics, performing the following query steps: querying data interaction tasks related to the candidate topic in the original message processing system; querying topics involved in the data interaction tasks, to obtain a set of topics; determining whether the set of topics contains other topics, wherein the other topics are topics corresponding to the publisher to be migrated and outside the set of candidate topics; in response to determining that it does not contain, determining the set of candidate topics as the set of target topics; A selecting unit configured to select a group of identification information from the set of identification information according to target topics in the set of target topics, to obtain a set of groups of identification information; A generating unit configured to generate migration information according to groups of identification information in the set of groups of identification information, to obtain a set of migration information, wherein the migration information comprises a target topic, a set of identification information of a handler of the target topic and identification information of the target message processing system. 14.An electronic device, comprising: one or more processors; storage having one or more programs stored thereon, when the one or more programs are executed by the one or more processors, the one or more processors implement a method according to any one of claims 1-7 or 9-12.
15. A computer readable medium having stored thereon a computer program, wherein, The program is executed by the processor to implement the method according to any one of claims 1-7 or 9-12.
Citation Information
Patent Citations
Cluster migration method and device and storage medium
CN112286904A
Topic user information processing method applied to artificial intelligence, and cloud server
CN113434638A