Dynamic message data management system and method thereof
Patent Information
- Application Number
- TW114107858
- Authority / Receiving Office
- TW · TW
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2025-02-18
- Filing Date
- 2025-03-04
- Publication Date
- 2026-09-01
- Estimated Expiration
- 2045-03-03
AI Technical Summary
Traditional message queues based on application scenarios or events lead to chaotic queue management, difficulty in data tracking, and troubleshooting challenges in complex business environments.
A dynamic message data management system employing a domain-driven design (DDD) approach to create a domain-layered architecture for message queues, utilizing a data delivery, dynamic message data management, and data receiving modules to manage message data efficiently, with metadata-driven queue adjustments and business logic processing.
The system enables efficient data monitoring and management by simplifying data tracking and troubleshooting, optimizing resource utilization, and reducing system costs through hierarchical queue management and dynamic adjustments based on business data changes.
Smart Images
Figure TWG2TA001074124_001 
Figure TWG2TA001074124_002 
Figure TWG2TA001074124_003
Abstract
Description
[Technical Field]
[0001] This invention relates to data management, and more particularly to a dynamic message data management system and method thereof. [Previous Technology]
[0002] As enterprise business complexity increases and business scope expands, the requirements for data consistency also become more stringent, thus increasing the difficulty of designing microservice architectures. Therefore, the architecture design driven by highly complex business models needs to flexibly support system decoupling to enhance system scalability and flexible maintainability. However, traditional message queues are based on application scenarios or events, which easily leads to a series of problems such as chaotic queue management, difficulty in data tracking, and difficulties in troubleshooting. [Summary of the Invention]
[0003] The present invention relates to a dynamic message data management system and method, which can effectively manage message data.
[0004] According to an embodiment of the present invention, the dynamic message data management system of the present invention includes a data delivery module, a dynamic message data management module, and a data receiving module. The data delivery module is used to provide message data. The dynamic message data management module is coupled to the data delivery module and is used to input message data to one of a plurality of message queues in a message queue cluster. The data receiving module is coupled to the dynamic message data management module and is used to receive message data sent by one of the plurality of message queues, and to perform corresponding business logic processing based on the message data. The dynamic message data management module creates a plurality of message queues in the message queue cluster based on metadata. The plurality of message queues belong to a domain, subdomain, or model, respectively.
[0005] According to an embodiment of the present invention, the dynamic message data management method of the present invention includes the following steps: creating multiple message queues of a message queue cluster based on metadata through a dynamic message data management module, wherein the multiple message queues belong to a domain, subdomain, or model respectively; providing message data through a data delivery module; inputting message data to one of the multiple message queues in the message queue cluster through the dynamic message data management module; and receiving message data sent by one of the multiple message queues through a data receiving module, and performing corresponding business logic processing based on the message data.
[0006] Based on the above, the dynamic message data management system and method of the present invention are based on Domain Driven Design (DDD) to design the domain layering architecture of the queue in order to achieve efficient data monitoring.
[0007] In order to make the above features and advantages of the present invention more apparent and understandable, specific embodiments are described below, and detailed descriptions are given in conjunction with the accompanying drawings.
Implementation Method
[0008] Reference will now be made in detail to exemplary embodiments of the invention, examples of which are illustrated in the accompanying drawings. Wherever possible, the same element symbols are used in the drawings and description to denote the same or similar parts.
[0009] Figure 1 is a schematic diagram of a dynamic message data management system according to an embodiment of the present invention. Referring to Figure 1, the dynamic message data management system 100 includes a dynamic message data management module 110, a data delivery module 120, a data receiving module 130, a metadata management module 140, and a system configuration module 150. The dynamic message data management module 110 is coupled to the data delivery module 120, the data receiving module 130, and the metadata management module 140. The metadata management module 140 is also coupled to the system configuration module 150. In this embodiment, the dynamic message data management system 100 may be a software platform implementing Software as a Service (SaaS) in the cloud, but the present invention is not limited thereto. In one embodiment, at least some modules of the dynamic message data management system 100 may also be deployed on a ground-based device.
[0010] In this embodiment, the dynamic message data management system 100 may be located on one or more servers in the cloud. The server includes a processor and a storage device. In this embodiment, the processor may be a system-on-a-chip (SOC), or may include, for example, a central processing unit (CPU) or other programmable general-purpose or special-purpose microprocessors, digital signal processors (DSPs), programmable controllers, application-specific integrated circuits (ASICs), programmable logic devices (PLDs), other similar processing devices, or combinations thereof. The storage device may include, for example, dynamic random access memory (DRAM), flash memory, or non-volatile random access memory (NVRAM).
[0011] In this embodiment, the dynamic message data management module 110, the data delivery module 120, the data receiving module 130, the metadata management module 140, and the system configuration module 150 may be implemented in programming languages such as JSON (JavaScript Object Notation), Extensible Markup Language (XML), or YAML, and stored in a storage device for the processor to execute in order to implement the processes, functions, operations, and effects described in the various embodiments of the present invention.
[0012] In this embodiment, the dynamic message data management system 100 can be used to dynamically set message data queues and can automatically detect changes in business data by listening to the message queues. The data delivery module 120 can, for example, be connected to the application programming interface (API) of multiple different business services of the producer. In this embodiment, when business data changes, the data delivery module 120 can obtain the corresponding message data and deliver it to the dynamic message data management module 110. The dynamic message data management module 110 can further provide the message data to the data receiving module 130 through message queues. The data receiving module 130 can provide the message data to the terminal device of the user (e.g., a consumer or manager), such as a computer device. Dynamic message data management module 110.
[0013] In this embodiment, the dynamic message data management system 100 can design the domain-layered architecture of the queue based on Domain Driven Design (DDD). Specifically, the system configuration module 150 can provide metadata to the metadata management module 140 based on domain construction information, subdomain construction information, and model construction information. The metadata management module 140 can provide the metadata to the dynamic message data management module 110. In this embodiment, the metadata includes domain metadata, subdomain metadata, and model metadata. Furthermore, the dynamic message data management system 100 can plan the domain blueprint based on Domain Driven Design and identify core domains and supporting domains.
[0014] For example, the domain may be, for instance, a Bill of Materials (BOM) domain, and the corresponding subdomains may include, for instance, a Bill of Materials subdomain, a Substitute Material subdomain, a Cost Simulation subdomain, and a Planned Bill of Materials subdomain. The domain may be, for instance, a Sales domain, and the corresponding subdomains may include, for instance, a Sales Order subdomain, a Customer subdomain, a Sales Price subdomain, and a Sales Contract subdomain. The domain may be, for instance, a Procurement domain, and the corresponding subdomains may include, for instance, a Purchase Order subdomain, a Purchase Contract subdomain, a Purchase Price subdomain, and a Supplier subdomain. The domain may be, for instance, a Work Order domain, and the corresponding subdomains may include, for instance, a Work Order Subdomain, a Work Order Material Issuance subdomain, a Work Order Warehousing subdomain, and a Work Order Material Requisition subdomain. The model may include, for instance, an Autonomous Forecasting Model, a Sales Order Model, a Sales / Change Order Model, a Supplier Model, a Supplier Financial Model, and a Supplier Business Model. The dynamic message data management system 100 can determine the core domain model, such as the sales order model and the supplier model, based on the core business (e.g., transactions, payments) and / or technical monitoring indicators (application interface type and / or development volume).
[0015] Figure 2 is a flowchart of a dynamic message data management method according to an embodiment of the present invention. Referring to Figures 1 and 2, the dynamic message data management system 100 can perform the following steps S210-S240. In step S210, the dynamic message data management module 110 creates multiple message queues of a message queue cluster based on metadata. The multiple message queues can be classified into domains, subdomains, or models according to the domain hierarchical architecture designed above. In step S220, the data delivery module 120 provides message data to the dynamic message data management module 110. The data delivery module 120 can encapsulate changing business data to generate the message data. In step S230, the dynamic message data management module 110 inputs the message data to one of the multiple message queues in the message queue cluster. In step S240, the data receiving module 130 receives the message data sent by one of the multiple message queues and performs corresponding business logic processing based on the message data. The data receiving module 130 can, for example, call the corresponding application interface to perform corresponding business logic processing, and output corresponding data change information and / or business logic processing results to the terminal device operated by the user. Therefore, the dynamic message data management system 100 and its method in this embodiment can effectively detect changing data.
[0016] Figure 3 is a flowchart of dynamic message data processing according to an embodiment of the present invention. Referring to Figures 1 and 3, in this embodiment, the dynamic message data management module 110 may include a queue management service module 111, a message queue cluster (MQ) 112, a (data) listener 113, a data receiving service module 114, a database 115, and a retry module 116. In this embodiment, the metadata management module 140 can provide metadata to the dynamic message data management module 110. The queue management service module 111 can analyze metadata to initialize multiple queues of the message queue cluster 112 and create listeners 113 (the number of listeners can be one or more), and also determine queue exchanges and binding relationship information based on the metadata, wherein the binding relationship information is used to determine the binding relationship between specific message data and specific message queues. In addition, in one embodiment, the metadata management module 140 can also provide a function to query model configuration.
[0017] Next, the dynamic message data management module 110 can perform the following steps S301 to S311. In step S301, the data receiving service module 114 can receive message data (changing business data) provided by the data delivery module 120. In step S302, the data receiving service module 114 can input the message data into the corresponding message queue according to the binding relationship information and the queue exchanger. In step S303, the listener 113 can periodically detect the corresponding message queue at preset time intervals and listen for message data. The listener 113 can receive message data. The listener 113 can be used to listen to at least one of multiple message queues. In step S304, the listener 113 can save the message data to a local associated database (e.g., MySQL).
[0018] In step S305, the listener 113 can send message data to the data receiving module 130. In step S306, the listener 113 can receive the sending result information returned by the data receiving module 130. In step S307, the listener 113 can store the sending result information of the message data in the associated database 115. In step S308, the retry module 116 can query the sending failure record through the associated database 115. In step S309, the retry module 116 can obtain the sending failure information from the associated database 115. In step S310, the retry module 116 can resend the message data to the data receiving module 130. In step S306, the retry module 116 can receive the sending result information returned by the data receiving module 130. If the sending still fails, the retry module 116 can notify the corresponding user, for example, by sending an email or SMS.
[0019] For example, the queue management service module 111 can perform queue hierarchical planning based on the metadata provided by the metadata management module 140. To this end, the queue management service module 111 can first design the queues, queue exchanges, and binding relationships as shown in Table 1 below, according to the domain dimension. Queues can, for example, satisfy the following encoding rules: bm.domain_number.queue; bm.domain_number.subdomain_number.queue; bm.domain_number.subdomain_number.model_number.queue. Queue exchanges can, for example, satisfy the following encoding rules: bm.domain_number; bm.domain_number.subdomain_number; bm.domain_number.subdomain_number.model_number. Binding relationships can, for example, satisfy the following encoding rules: bm.domain_number.#; bm.domain_number.subdomain_number.#; bm.domain_number.subdomain_number.model_number. field subdomain (Domain) Model Queue exchanger Binding relationship queue mosc mo mo bm.mosc bm.mosc.# bm.mosc.queue moChange moSplit molssue molssue Table 1
[0020] Furthermore, referring to FIG4, FIG4 is a schematic diagram of the data structure of message data according to an embodiment of the present invention. For example, as shown in FIG4, the data structure 400 of the message data may include a string type field "caller" for describing the data source, a string type field "id" for representing a unique data identifier, an integer type field "tenantId" for representing a tenant identity code, a string type field "applicationCode" for representing an application code, an array list type field "deliveryData" for representing the delivered data, a string type field "deliveryData.model" for representing the business model name, an arrays type field "deliveryData.dataKey" for representing the business model primary key, a string type field "deliveryData.changeType" for representing the change type, and a string type field "deliveryData.data" for representing the delivered data. However, the data structure of the message data of the present invention is not limited to that shown in FIG4.
[0021] In this embodiment, the metadata management module 140 can change the configuration of message queues and the ownership of message data according to user settings or automatic adjustments. Therefore, when metadata (i.e., at least one of domain metadata, subdomain metadata, and model metadata) changes, the metadata management module 140 can provide the changed metadata to the queue management service module 111 of the dynamic message data management module 110, so that the queue management service module 111 can dynamically adjust the ownership of multiple message queues in the message queue cluster 112 based on the changed metadata. For example, it can dynamically adjust the ownership of at least one of the multiple message queues in the message queue cluster 112.
[0022] In this embodiment, the dynamic message data management system 100 may further include a data footprint module (not shown). The data footprint module may be coupled to the dynamic message data management module 110 and is used to record data footprint information of message data. In this embodiment, the data footprint module can provide data footprint information to the system configuration module 150, and the system configuration module 150 can determine whether to update at least one of the domain construction information, subdomain construction information, and model construction information based on the data footprint information. Therefore, the queue management service module 111 can dynamically adjust the message queue based on the data footprint information. The data footprint information may be, for example, information related to the actual business volume such as the daily request volume of information data, the data throughput per second, or the message body size.
[0023] In this embodiment, the queue management service module 111 can analyze the message proportion (ratio) of each model in the corresponding queue, and split and / or merge queues according to standards. The queue management service module 111 can split models exceeding the standard into corresponding subdomains or independent model queues. Alternatively, the queue management service module 111 can upgrade queues with small proportions (ratios) back to the parent queue. In one embodiment, the queue management service module 111 can determine whether the daily request volume, data throughput per second, or message body size of specific information data provided by data footprint information exceeds a preset threshold to decide whether to split the queue. Alternatively, the queue management service module 111 can determine whether the daily request volume, data throughput per second, or message body size of specific information data provided by data footprint information is lower than another preset threshold to decide whether to merge the queues. Alternatively, the queue management service module 111 can bind core business message data to specific individual message queues.
[0024] To give another example, taking Table 1 above as an example, when the queue management service module 111 determines that the daily request volume of the subdomain "mo" is greater than the preset threshold, the queue management service module 111 can assign the domain models "mo", "moChange", and "moSplit" corresponding to the subdomain "mo" to the new exchange "bm.mosc.mo", the new binding relationship "bm.mosc.mo.#", and the new queue "bm.mosc.mo.queue". In this regard, the architecture of Table 1 above can be updated as shown in Table 2 below. field subdomain (Domain) Model Queue exchanger Binding relationship queue mosc mo mo bm.mosc.mo bm.mosc.mo# bm.mosc.mo.queue moChange moSplit molssue molssue bm.mosc bm.mosc.# bm.mosc.queue Table 2
[0025] Taking Table 2 above as an example again, when the queue management service module 111 determines that the daily request volume of the domain model "mo" is greater than the preset threshold, the queue management service module 111 can assign the domain model "mo" to a new exchange "bm.mosc.mo.mo", a new binding relationship "bm.mosc.mo.mo.#", and a new queue "bm.mosc.mo.mo.queue". The architecture of Table 2 above can be updated as shown in Table 3 below. field subdomain (Domain) Model Queue exchanger Binding relationship queue mosc mo mo bm.mosc.mo.mo bm.mosc.mo.mo# bm.mosc.mo.mo.queue moChange bm.mosc.mo bm.mosc.mo# bm.mosc.mo.queue moSplit molssue molssue bm.mosc bm.mosc.# bm.mosc.queue Table 3
[0026] For another example, referring to Figure 5, which is a schematic diagram of the queue hierarchy planning of an embodiment of the present invention, the queue management service module 111 can establish the queue hierarchy architecture shown in Figure 5. In this regard, domain 510 can correspond to two subdomains 521 and 522. Subdomain 521 can correspond to models 531-534, and subdomain 522 can correspond to model 535. In this example, model 531 can be, for example, the core domain model, and therefore belongs to a separate message queue 541. Models 532 and 533 can belong to message queue 542. Model 534 can have a throughput exceeding a preset value per second, and therefore can also belong to a separate message queue 543. Model 535 can belong to message queue 544 of domain 510.
[0027] For another example, referring to Figures 6 and 7, which are schematic diagrams of queue adjustment according to an embodiment of the present invention, consider two message data entries: work order data 601 and material requisition data 602. As shown in Figure 6, based on the original binding relationship, work order data 601 and material requisition data 602 can be input into the same message queue 621 via queue exchange 611. However, when the daily request volume, data throughput per second, or message body size of work order data 601 exceeds the standard, as shown in Figure 7, the message data of work order data 601 can be switched to belong to a new message queue 622 via a new queue exchange 612. In this way, the message data of work order data 601 and material requisition data 602 can be efficiently provided to the data receiving module through message queues 621 and 622.
[0028] In summary, the dynamic message data management system and method of the present invention can design a domain-layered architecture for message queues using Domain Driven Design (DDD) to effectively achieve hierarchical management of business data detection. The dynamic message data management system and method of the present invention, based on a consistent data delivery structure and hierarchical queue management, greatly simplifies the recording and tracking of data footprints. Furthermore, based on consistent business data footprint recording, it is convenient to track which queue data is sent to and the usage status of queue messages, thereby achieving timely business data monitoring and management. The dynamic message data management system and method of the present invention can dynamically adjust message queues according to usage and / or whether they belong to core business, thereby greatly improving the efficiency of message data monitoring. The dynamic message data management system and method of the present invention, through reasonable queue management, can fully utilize the refined management of data in different domains and fully utilize system resources to effectively reduce the overall system setup and operating costs.
[0029] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention. [Simplified Explanation of the Diagram]
[0030] Figure 1 is a schematic diagram of a dynamic message data management system according to an embodiment of the present invention. Figure 2 is a flowchart of a dynamic message data management method according to an embodiment of the present invention. Figure 3 is a flowchart of dynamic message data processing according to an embodiment of the present invention. Figure 4 is a schematic diagram of the data structure of message data according to an embodiment of the present invention. Figure 5 is a schematic diagram of queue hierarchical planning according to an embodiment of the present invention. Figures 6 and 7 are schematic diagrams of queue adjustment according to an embodiment of the present invention.
Claims
1. A dynamic message data management system, comprising: The data delivery module is used to provide message data; A dynamic message data management module, coupled to the data delivery module, is used to input the message data to one of multiple message queues in a message queue cluster; and a data receiving module, coupled to the dynamic message data management module, is used to receive the message data sent by one of the multiple message queues and perform corresponding business logic processing based on the message data. The dynamic message data management module creates the multiple message queues of the message queue cluster based on metadata, wherein the multiple message queues belong to a domain, subdomain, or model, respectively. The dynamic message data management module analyzes the metadata to initialize the multiple message queues and create listeners, wherein the listeners listen to one of the multiple message queues to receive the message data and send the message data to the data receiving module.
2. The dynamic message data management system as described in claim 1 further includes: A metadata management module is coupled to the dynamic message data management module and is used to provide the metadata to the dynamic message data management module, wherein the metadata includes domain metadata, subdomain metadata and model metadata.
3. The dynamic message data management system as described in claim 2 further includes: The system configuration module is coupled to the metadata management module and is used to provide domain metadata, subdomain metadata, and model metadata to the metadata management module based on domain construction information, subdomain construction information, and model construction information.
4. The dynamic message data management system as described in claim 3, wherein when at least one of the domain construction information, the subdomain construction information, and the model construction information is updated, the system configuration module provides the changed metadata to the metadata management module, and the metadata management module provides the changed metadata to the dynamic message data management module, so that the dynamic message data management module dynamically adjusts the affiliation of at least one of the plurality of message queues according to the changed metadata.
5. The dynamic message data management system as described in claim 3 further includes: A data footprint module is coupled to the dynamic message data management module and is used to record data footprint information of the message data. The data footprint module provides the data footprint information to the system configuration module, and the system configuration module determines whether to update at least one of the domain construction information, the subdomain construction information, and the model construction information based on the data footprint information.
6. The dynamic message data management system as described in claim 1, wherein the dynamic message data management module further determines the queue exchange and binding relationship information based on the metadata, wherein the binding relationship information is used to determine the binding relationship between specific message data and a specific message queue.
7. The dynamic message data management system as described in claim 1, wherein the listener is further configured to save the message data to an associated database.
8. The dynamic message data management system as described in claim 7, wherein the listener also stores the message data sending result information in the associated database, and the dynamic message data management module obtains sending failure information from the associated database to resend the message data.
9. The dynamic message data management system as described in claim 1, wherein the data delivery module is further used to encapsulate changing business data to generate the message data.
10. A method for managing dynamic message data, comprising: The dynamic message data management module creates multiple message queues in a message queue cluster based on metadata, wherein the multiple message queues belong to a domain, subdomain, or model, respectively; a data delivery module provides message data; the dynamic message data management module inputs the message data to one of the multiple message queues in the message queue cluster; a data receiving module receives the message data sent by one of the multiple message queues and performs corresponding business logic processing based on the message data; the dynamic message data management module analyzes the metadata to initialize the multiple message queues and create listeners; and the listeners listen to one of the multiple message queues to receive the message data and send the message data to the data receiving module.
11. The dynamic message data management method as described in request item 10 further includes: The metadata is provided to the dynamic message data management module through the metadata management module, wherein the metadata includes domain metadata, subdomain metadata and model metadata.
12. The dynamic message data management method as described in request item 11 further includes: The system configuration module provides domain metadata, subdomain metadata, and model metadata to the metadata management module based on domain construction information, subdomain construction information, and model construction information.
13. The dynamic message data management method as described in request item 12 further includes: When at least one of the domain construction information, the subdomain construction information, and the model construction information is updated, the changed metadata is provided to the metadata management module through the system configuration module; and the changed metadata is provided to the dynamic message data management module through the metadata management module, so that the dynamic message data management module dynamically adjusts the affiliation of at least one of the multiple message queues according to the changed metadata.
14. The dynamic message data management method as described in claim 12 further includes: The data footprint information of the message data is recorded through the data footprint module; The data footprint module provides the data footprint information to the system configuration module; and the system configuration module determines, based on the data footprint information, whether to update at least one of the domain construction information, the subdomain construction information, and the model construction information.
15. The dynamic message data management method as described in claim 10 further includes: determining a queue exchange and binding relationship information based on the metadata through the dynamic message data management module, wherein the binding relationship information is used to determine the binding relationship between specific message data and a specific message queue.
16. The dynamic message data management method as described in request item 10 further includes: saving the message data to an associated database through the listener.
17. The dynamic message data management method as described in claim 16 further includes: storing the message data sending result information to the associated database through the listener; and obtaining sending failure information from the associated database through the dynamic message data management module to resend the message data.
18. The dynamic message data management method as described in request item 10 further includes: The data delivery module encapsulates the changed business data to generate the message data.