Method for automatically creating message queue

Abstract

The invention relates to a method for automatically creating a message queue, which comprises the following steps of: in response to received database information, extracting newly added information based on first analysis configuration and determining a corresponding topic; determining configuration information corresponding to the topic based on the second analysis configuration; and creating a message queue based on the configuration information. According to the method, the message queue is automatically created and configured through the database information, so that the flexibility and management efficiency of the system are remarkably improved, the risk of human errors is reduced, and the response speed and expandability of the system are enhanced.

Description

Technical Field

[0001] The invention relates to the technical field of message queue fusion, in particular to a method for automatically creating a message queue. Background Art

[0002] In modern complex and highly distributed systems, Message Queue (MQ) plays a vital role as a key component. It is not only used to realize asynchronous communication between producers and consumers, effectively decouples different parts of the system, but also greatly improves the reliability, scalability and fault tolerance of the system. Through the message queue, the system can handle high-concurrency requests more flexibly, optimize resource utilization, and ensure data consistency and integrity.

[0003] However, traditional message queue configuration methods often rely on manual operations, which is not only cumbersome and complex, but also prone to errors. Especially in large-scale systems, with the continuous expansion of business, the number of message queues that need to be managed increases dramatically, and each queue may also involve complex configurations such as different producer groups, consumer groups, message formats, and processing strategies. Manual configuration methods that rely on manual labor are not only inefficient, but also difficult to ensure the accuracy and consistency of configuration, further increasing the difficulty of system maintenance and operating costs.

[0004] The information disclosed in this background technology section is only intended to deepen the understanding of the overall background technology of the present invention, and should not be regarded as acknowledging or suggesting in any form that the information constitutes the prior art already known to those skilled in the art. Summary of the invention

[0005] In view of the defects existing in the prior art, the purpose of the present invention is to provide a method for automatically creating a message queue, which automatically creates and configures the message queue through database information, significantly improves the system flexibility and management efficiency, reduces the risk of human error, and enhances the system response speed and scalability.

[0006] In order to achieve the above purpose, the technical solution adopted by the present invention is:

[0007] A method for automatically creating a message queue, characterized by comprising:

[0008] In response to the received database information, extracting the newly added information based on the first parsing configuration and determining a corresponding topic;

[0009] Determining configuration information corresponding to the topic based on the second parsed configuration;

[0010] Create a message queue based on the configuration information.

[0011] On the basis of the above technical solution, the configuration information at least includes: topic information, producer group information, consumer group information;

[0012] The configuration information is stored in a database.

[0013] Based on the above technical solution, the first parsing configuration first parses the newly added information to obtain at least one topic corresponding to the newly added information;

[0014] If the topic already exists, the new information is directly bound to the topic and a unique identification tag is assigned;

[0015] If the topic does not exist, create it first, then bind the new information to the topic and assign a unique identification tag.

[0016] Based on the above technical solution, the first analysis configuration obtains the data characteristics and data categories of the newly added information based on the metadata or tags added to each piece of data when the database information is stored, and corresponds the data characteristics and data categories to at least one topic respectively;

[0017] If the topic already exists, the new information is directly bound to the topic and a unique identification tag is assigned;

[0018] If the topic does not exist, create it first, then bind the new information to the topic and assign a unique identification tag.

[0019] Based on the above technical solution, the first parsing configuration obtains the message attributes and message routing of the newly added information from the message middleware, and corresponds the message attributes to at least one topic based on the routing rules configured in the message middleware;

[0020] If the topic already exists, the new information is directly bound to the topic and a unique identification tag is assigned;

[0021] If the topic does not exist, create it first, then bind the new information to the topic and assign a unique identification tag.

[0022] Based on the above technical solution, the first parsing configuration responds to a database trigger, determines the message attribute of the newly added information according to the database trigger and corresponds the message attribute to at least one topic;

[0023] If the topic already exists, the new information is directly bound to the topic and a unique identification tag is assigned;

[0024] If the topic does not exist, create it first, then bind the new information to the topic and assign a unique identification tag.

[0025] Based on the above technical solution, the second parsing configuration adapts the relevant producer group and consumer group according to the topic information.

[0026] On the basis of the above technical solution, according to the analysis of user historical behavior and user topic subscription history, determine the producer group and consumer group related to topic information adaptation;

[0027] Alternatively, determine the producer group and consumer group to which the topic information is adapted according to the preset subscription rules;

[0028] Alternatively, the producer group and consumer group related to the topic information adaptation are determined according to the user's subscription instruction.

[0029] Based on the above technical solution, the message queue management module QMM reads the topic information, producer group information and consumer group information, and creates and configures the message queue accordingly;

[0030] The message queue system MQS is responsible for hosting and managing the components of the message queue, including the storage, transmission and processing functions of the message.

[0031] The method for automatically creating a message queue described in the present invention has the following beneficial effects:

[0032] 1. Automatically create and configure message queues through database information, significantly improving system flexibility and management efficiency, reducing the risk of human errors, and enhancing system response speed and scalability.

[0033] 2. Ensure the consistency of message queue configuration and database definition, improve the reliability and stability of the system. Automated management reduces operation and maintenance dependence, reduces maintenance costs, enables the system to better adapt to business changes, and achieve efficient and stable operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] The present invention has the following accompanying drawings:

[0035] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0036] Figure 1 A flowchart of a first embodiment of a method for automatically creating a message queue according to the present invention. DETAILED DESCRIPTION

[0037] The present invention is further described in detail below in conjunction with the accompanying drawings. The detailed description is an explanation made in conjunction with the exemplary embodiments of the present invention, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those skilled in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0038] like Figure 1 As shown, the method for automatically creating a message queue according to the present invention includes:

[0039] In response to the received database information, extracting the newly added information based on the first parsing configuration and determining a corresponding topic;

[0040] Determining configuration information corresponding to the topic based on the second parsed configuration;

[0041] Create a message queue based on the configuration information.

[0042] In this embodiment, by automatically creating message queues, the system can create message queues for topics corresponding to the newly added information in real time according to the dynamic changes of database information. This approach makes message processing more flexible and can quickly adapt to different business needs and data changes; by creating different message queues for different topics, message classification and refined management can be achieved.

[0043] In actual applications, business scenarios are often complex and may need to process multiple types of data and messages. This embodiment can flexibly create and manage message queues according to different business requirements and data characteristics, thereby supporting complex business scenarios.

[0044] This embodiment can automatically allocate resources according to actual needs, avoiding waste of resources and complexity of configuration. At the same time, this embodiment can also dynamically adjust the number and capacity of message queues according to load conditions, thereby optimizing system performance.

[0045] As business develops and data volume increases, the system needs to be able to expand quickly to meet new needs. This embodiment allows the system to quickly create a new message queue when facing new business needs, thereby improving the scalability of the system; in addition, since the message queue has the ability to buffer and asynchronously process, it can also improve the fault tolerance and high availability of the system, ensuring that messages can continue to be processed and data integrity can be guaranteed in the event of an abnormal situation.

[0046] On the basis of the above technical solution, the configuration information at least includes: topic information, producer group information, consumer group information;

[0047] The configuration information is stored in a database.

[0048] Exemplarily, a topic table is maintained in the database, and table items of the topic table include topic information, producer group information, and consumer group information.

[0049] This embodiment introduces configuration information (including topic information, producer group information, and consumer group information) and stores it in a database, so that the system can dynamically adjust the configuration as needed without hard coding or manually modifying the configuration file. Through the configuration information, the system can support a variety of different business scenarios and needs. For example, different producer groups and consumer groups can be configured according to actual business logic to achieve specific message production and consumption modes. The configuration information is centrally stored in the database, which facilitates unified management and maintenance, reduces the complexity of system maintenance, and improves maintenance efficiency. In addition, as the business develops, the system may need to add new topics, producer groups, or consumer groups. By storing the configuration information in the database, these configurations can be easily added and modified to achieve system scalability. In the event of a failure, the cause of the problem can be quickly located by querying the configuration information in the database, thereby improving the efficiency of troubleshooting.

[0050] For example, fine-grained permission management can be implemented through the producer group and consumer group information in the configuration information. For example, some producers can be restricted to send messages only to specific topics, or some consumers can be allowed to subscribe to specific topics only.

[0051] For example, the configuration information can support complex message routing and filtering logic. For example, messages can be routed to different processing paths based on topic information, or unnecessary messages can be filtered out based on the configuration of the consumer group.

[0052] Based on the above technical solution, the first parsing configuration first parses the newly added information to obtain at least one topic corresponding to the newly added information;

[0053] If the topic already exists, the new information is directly bound to the topic and a unique identification tag is assigned;

[0054] If the topic does not exist, create it first, then bind the new information to the topic and assign a unique identification tag.

[0055] This embodiment parses the newly added information through the first parsing configuration, and can accurately identify the topic corresponding to the information. When the topic already exists, directly binding the newly added information to the topic can ensure the accuracy and consistency of the information, avoid repeatedly creating the same topic, and thus save system resources. Direct binding can shorten the information processing time and improve the processing speed of the system. At the same time, assigning a unique identification mark helps to track and manage the information later. When the topic does not exist, the system can automatically create a new topic and bind the newly added information. This flexibility enables the system to adapt to changing information needs and ensure that all information can be properly processed.

[0056] This embodiment can support the situation where one newly added information corresponds to multiple topics. Through parsing configuration, the system can flexibly handle this complex relationship and meet diverse business needs. By assigning a unique identification tag to each piece of information and binding it to the corresponding topic, classified storage and fast retrieval of information can be achieved, which helps to improve work efficiency and user experience.

[0057] Based on the above technical solution, the first analysis configuration obtains the data characteristics and data categories of the newly added information based on the metadata or tags added to each piece of data when the database information is stored, and corresponds the data characteristics and data categories to at least one topic respectively;

[0058] If the topic already exists, the new information is directly bound to the topic and a unique identification tag is assigned;

[0059] If the topic does not exist, create it first, then bind the new information to the topic and assign a unique identification tag.

[0060] In this embodiment, the system can accurately identify the data characteristics and categories of each new information through metadata or tags, which is more accurate and efficient than traditional manual classification, and helps to reduce classification errors and omissions. After identifying the data characteristics and categories, the system can automatically correspond the information to at least one topic, greatly improving the efficiency of information processing and reducing the cost of manual intervention. The system can dynamically create and update topics based on the metadata or tags of the new information, so that the system can adapt to the ever-changing information needs and ensure that all information can be properly processed.

[0061] By classifying information into topics according to data characteristics and categories, the system can form an orderly information organization structure, making information easier to manage and retrieve, and improving the convenience and efficiency of information processing.

[0062] By collecting statistics and analyzing metadata or tags, the system can find correlations and trends between data, which helps support business decisions and discover potential business opportunities. Based on the user's historical behavior and interests, the system can make personalized recommendations based on topics and metadata or tags, which can improve user satisfaction and loyalty and promote business development.

[0063] Based on the above technical solution, the first parsing configuration obtains the message attributes and message routing of the newly added information from the message middleware, and corresponds the message attributes to at least one topic based on the routing rules configured in the message middleware;

[0064] If the topic already exists, the new information is directly bound to the topic and a unique identification tag is assigned;

[0065] If the topic does not exist, create it first, then bind the new information to the topic and assign a unique identification tag.

[0066] In this embodiment, the message middleware, as the core component of message transmission in a distributed system, can dynamically send messages to a specified destination according to the configured routing rules. The first parsing configuration uses this feature of the message middleware to implement dynamic routing of information based on message attributes, ensuring that new information can be accurately sent to the corresponding topic.

[0067] The routing rules can be flexibly configured and adjusted according to business needs without modifying the code or redeploying the system. The system can quickly adapt to business changes and reduce maintenance costs.

[0068] Message middleware provides a variety of reliability assurance mechanisms, such as message persistence and retry mechanisms, to ensure that messages are not lost or duplicated during transmission.

[0069] The message middleware supports multiple consumers to subscribe to and consume messages under the same topic at the same time, realizing parallel processing, so that the system can efficiently handle large amounts of new information and improve throughput.

[0070] Message middleware usually has good scalability and can increase or decrease message nodes as needed to meet business needs of different scales. This scalability enables the system to continue to grow with the development of the business and maintain long-term competitiveness.

[0071] Based on the above technical solution, the first parsing configuration responds to a database trigger, determines the message attribute of the newly added information according to the database trigger and corresponds the message attribute to at least one topic;

[0072] If the topic already exists, the new information is directly bound to the topic and a unique identification tag is assigned;

[0073] If the topic does not exist, create it first, then bind the new information to the topic and assign a unique identification tag.

[0074] Based on the above technical solution, the second parsing configuration adapts the relevant producer group and consumer group according to the topic information.

[0075] Exemplarily, the topic information includes id, name, and description fields. Each topic is usually assigned a unique id (identifier) ​​to distinguish different topics; each topic has a name (topic name), so that the purpose or meaning of the topic can be quickly identified by the name; the description of the topic is a detailed explanation of the topic, usually containing more information than the name. Producer groups and consumer groups also include similar fields.

[0076] Exemplarily, based on the user's historical behavior analysis and the user's topic subscription history, determine the producer group and consumer group related to the topic information adaptation;

[0077] Exemplarily, determining the producer group and consumer group related to topic information adaptation according to a preset subscription rule;

[0078] Exemplarily, the producer group and consumer group related to topic information adaptation are determined according to the user's subscription instruction.

[0079] Based on the above technical solution, the message queue management module QMM reads the topic information, producer group information and consumer group information, and creates and configures the message queue accordingly;

[0080] The message queue system MQS is responsible for hosting and managing the components of the message queue, including the storage, transmission and processing functions of the message.

[0081] The contents not described in detail in this specification belong to the prior art known to professional and technical personnel in this field.

[0082] The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiment. Any equivalent modifications or changes made by those skilled in the art based on the contents disclosed by the present invention should be included in the protection scope recorded in the claims.

Claims

1. A method for automatically creating a message queue, characterized in that: include: In response to the received database information, extracting the newly added information based on the first parsing configuration and determining a corresponding topic; Determining configuration information corresponding to the topic based on the second parsed configuration; Create a message queue based on the configuration information.

2. A method for automatically creating a message queue as claimed in claim 1, characterized in that: The configuration information includes at least: topic information, producer group information, and consumer group information; The configuration information is stored in a database.

3. A method for automatically creating a message queue as claimed in claim 1, characterized in that: The first parsing configuration first parses the newly added information to obtain at least one topic corresponding to the newly added information; If the topic already exists, the new information is directly bound to the topic and a unique identification tag is assigned; If the topic does not exist, create it first, then bind the new information to the topic and assign a unique identification tag.

4. A method for automatically creating a message queue as claimed in claim 1, characterized in that: The first analysis configuration obtains data characteristics and data categories of the newly added information based on metadata or tags added to each piece of data when the information is entered into the database, and corresponds the data characteristics and data categories to at least one topic respectively; If the topic already exists, the new information is directly bound to the topic and a unique identification tag is assigned; If the topic does not exist, create it first, then bind the new information to the topic and assign a unique identification tag.

5. A method for automatically creating a message queue as claimed in claim 1, characterized in that: The first parsing configuration obtains the message attributes and message routing of the newly added information from the message middleware, and corresponds the message attributes to at least one topic based on the routing rules configured in the message middleware; If the topic already exists, the new information is directly bound to the topic and a unique identification tag is assigned; If the topic does not exist, create it first, then bind the new information to the topic and assign a unique identification tag.

6. A method for automatically creating a message queue as claimed in claim 1, characterized in that: The first parsing configuration is responsive to a database trigger, and determines a message attribute of the newly added information according to the database trigger to correspond the message attribute to at least one topic; If the topic already exists, the new information is directly bound to the topic and a unique identification tag is assigned; If the topic does not exist, create it first, then bind the new information to the topic and assign a unique identification tag.

7. A method for automatically creating a message queue as claimed in claim 1, characterized in that: The second parsing configuration adapts the relevant producer group and consumer group according to the topic information.

8. A method for automatically creating a message queue as claimed in claim 7, characterized in that: According to the user's historical behavior analysis and user topic subscription history, determine the producer group and consumer group related to topic information adaptation; Alternatively, determine the producer group and consumer group to which the topic information is adapted according to the preset subscription rules; Alternatively, the producer group and consumer group related to the topic information adaptation are determined according to the user's subscription instruction.

9. A method for automatically creating a message queue as claimed in claim 1, characterized in that: The message queue management module QMM reads topic information, producer group information, and consumer group information, and creates and configures message queues accordingly; The message queue system MQS is responsible for hosting and managing the components of the message queue, including the storage, transmission and processing functions of the message.

Images