Model data conversion method and apparatus for real-time message queue

Abstract

The application provides a model data conversion method and device for a real-time message queue, and comprises the following steps: receiving a data message from a message queue corresponding to a data table, wherein the data message comprises a first message body; determining a message type according to the first message body; and changing model data according to the message type, the attributes of the data table and a set configuration table; wherein the message type comprises a main table addition type, a main table update type, a slave table addition type and a slave table update type; the real-time change of the model can be realized, the conversion efficiency is improved, and the cost is reduced.

Description

Technical Field

[0001] This invention relates to the field of data information technology, and in particular to a model data conversion method and apparatus for real-time message queues. Background Technology

[0002] With the continuous development of information technology, more and more industries are establishing their own data services, such as data lakes, data platforms, and data warehouses. As business operations constantly change and upgrade, data needs to generate value, and therefore, corresponding services are offered externally. Besides standardized data service models, some vendors and internal systems also have their own data models. A key technical challenge is the real-time conversion of the data platform's standard model into the data models of other application systems, which involves the consistency issue between the master and slave tables. The current implementation involves Flink supporting multi-stream merging on windows, meaning that two input data streams are associated within a single window under the same conditions. This requires ensuring that the input data streams are built on the same window and use the same type of key as the association condition.

[0003] The scrolling window join operation associates elements from two data streams with the same key within the scrolling window, and applies a user-defined JoinFunction (which defines the specific processing logic for the matched data when the window function is called) to calculate the join result. The window has a fixed time setting, which is generally set relatively long. Data elements from two data streams are joined within a single window based on the same key, and the joined data may overlap. The join is also based on an inner join. If a window contains only one key from a data stream, no join result will be output.

[0004] The session window divides the dataset into different windows based on the session gap. The session window association performs window association operations on the data elements of two streams. If the window contains elements from two datasets and the elements have the same key, the association calculation result is output.

[0005] The above scheme uses data stream association within a certain time window. If two data streams contain data with the same key that arrives outside the set time within the window, the data may change and the association will fail. In addition, the above technology does not trigger real-time changes in the corresponding transformed model data in the main table when the data in the slave table changes.

[0006] In summary, the above process, due to its massive data volume and the computational aspects involved in the model transformation, impacts real-time performance due to data retrieval and processing. Furthermore, the data transformation process often requires multiple tables for data retrieval, involving updates to both the main and subsidiary tables, necessitating consistent data throughout the entire process. Therefore, the current challenge is to efficiently and cost-effectively complete the data model transformation while maintaining data consistency. Summary of the Invention

[0007] In view of this, the purpose of the present invention is to provide a model data conversion method and apparatus for real-time message queues, which can realize real-time changes of model data, improve conversion efficiency, and reduce costs.

[0008] In a first aspect, embodiments of the present invention provide a model data conversion method for real-time message queues, the method comprising:

[0009] Receive data messages from the message queue corresponding to the data table, wherein the data message includes a first message body;

[0010] Determine the message type based on the first message body;

[0011] The model is modified according to the message type, the attributes of the data table, and the set configuration table;

[0012] The message types include main table add, main table update, slave table add, and slave table update.

[0013] Furthermore, the model is modified according to the message type, the attributes of the data table, and the set configuration table, including:

[0014] When the message type is either "add to main table" or "update to main table", and the attribute of the data table is "main table", the identifier of the first model and the field content of the first model are obtained from the view corresponding to the main table based on the identifier of the main table.

[0015] Using the name of the main table as a condition, the name of the first model corresponding to the main table is found in the set configuration table;

[0016] Based on the name of the first model, the identifier of the first model and the field content of the first model are sent to the first model.

[0017] Furthermore, the model is modified according to the message type, the attributes of the data table, and the set configuration table, including:

[0018] When the message type is "add to main table" or "update to main table", and the data table attribute is "sub-table", the name and monitoring point information of the main table corresponding to the sub-table are queried from the set configuration table.

[0019] A second message body is constructed based on the identifier of the slave table, the name of the master table, the monitoring point information, and the current message type, wherein the current message type is the slave table addition and slave table update type;

[0020] The second message body is sent to the message queue corresponding to the main table.

[0021] Furthermore, the model is modified according to the message type, the attributes of the data table, and the set configuration table, including:

[0022] When the message type is the add or update type of the sub-table, and the attribute of the data table is the main table, the monitoring point is matched from the view corresponding to the main table based on the identifier of the sub-table to obtain the monitoring point information, the identifier of the second model and the field content of the second model.

[0023] Using the name of the main table as a condition, retrieve the name of the second model corresponding to the main table from the set configuration table;

[0024] Based on the name of the second model, send the identifier of the second model and the field content of the second model to the second model.

[0025] Furthermore, the attributes of the data table include a master table and at least one slave table; the master table includes an identifier and field names of the master table; the slave table includes an identifier and field names of the slave table.

[0026] The master table has a corresponding view, which includes the view identifier, the model content, and the monitoring point information of the slave table corresponding to the master table.

[0027] Secondly, embodiments of the present invention provide a model data conversion device for real-time message queues, the device comprising:

[0028] A receiving module is used to receive data messages from the message queue corresponding to the data table, wherein the data message includes a first message body;

[0029] The determination module is used to determine the message type based on the first message body;

[0030] The modification module is used to modify the model according to the message type, the attributes of the data table, and the set configuration table;

[0031] The message types include main table add, main table update, slave table add, and slave table update.

[0032] Furthermore, the change module is specifically used for:

[0033] When the message type is either "add to main table" or "update to main table", and the attribute of the data table is "main table", the identifier of the first model and the field content of the first model are obtained from the view corresponding to the main table based on the identifier of the main table.

[0034] Using the name of the main table as a condition, find the name of the first model data corresponding to the main table from the set configuration table;

[0035] Based on the name of the first model, the identifier of the first model and the field content of the first model are sent to the first model.

[0036] Furthermore, the change module is specifically used for:

[0037] When the message type is "add to main table" or "update to main table", and the data table attribute is "sub-table", the name and monitoring point information of the main table corresponding to the sub-table are queried from the set configuration table.

[0038] A second message body is constructed based on the identifier of the slave table, the name of the master table, the monitoring point information, and the current message type, wherein the current message type is the slave table addition and slave table update type;

[0039] The second message body is sent to the message queue corresponding to the main table.

[0040] Thirdly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the method described above.

[0041] Fourthly, embodiments of the present invention provide a computer-readable medium having processor-executable non-volatile program code that causes the processor to perform the method described above.

[0042] This invention provides a method and apparatus for model data conversion in a real-time message queue, comprising: receiving a data message from a message queue corresponding to a data table, the data message including a first message body; determining a message type based on the first message body; and modifying the model data according to the message type, the attributes of the data table, and a set configuration table; wherein the message type includes main table addition, main table update, slave table addition, and slave table update; this enables real-time model modification, improves conversion efficiency, and reduces costs.

[0043] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention are realized and obtained in accordance with the structures particularly pointed out in the description, claims and drawings.

[0044] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0045] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0046] Figure 1 This is a flowchart of the model data conversion method for a real-time message queue provided in Embodiment 1 of the present invention;

[0047] Figure 2 This is a schematic diagram illustrating the model data conversion process between the master table and the slave table provided in Embodiment 1 of the present invention;

[0048] Figure 3 This is a schematic diagram illustrating another model data conversion process between a master table and a slave table provided in Embodiment 1 of the present invention;

[0049] Figure 4 This is a schematic diagram of a model data conversion device for a real-time message queue provided in Embodiment 2 of the present invention;

[0050] Figure 5 This is a schematic diagram of the structure of the electronic device provided in Embodiment 3 of the present invention. Detailed Implementation

[0051] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] With the continuous development of information technology, more and more industries are establishing their own data services, such as data lakes, data platforms, and data warehouses. As business operations continue to change and upgrade, data needs to generate value, and therefore, corresponding services are provided externally. In addition to standardized data service models, some vendors and internal systems also have their own data models.

[0053] Flink supports multi-stream merging on windows, which means performing association operations on two input data streams in a single window according to the same conditions. This requires ensuring that the input data streams are built on the same window and use the same type of key as the association condition.

[0054] The scrolling window join operation associates elements from two data streams with the same key within the scrolling window, and applies a user-defined JoinFunction (which defines the specific processing logic for the matched data when the window function is called) to calculate the join result. The window has a fixed time setting, which is generally set relatively long. Data elements from two data streams are joined within a single window based on the same key, and the joined data may overlap. The join is also based on an inner join. If a window contains only one key from a data stream, no join result will be output.

[0055] The session window divides the dataset into different windows based on the session gap. The session window association performs window association operations on the data elements of two streams. If the window contains elements from two datasets and the elements have the same key, the association calculation result is output.

[0056] Among them, a data platform refers to the collection, computation, storage, and processing of large amounts of data through data technology, while unifying standards and definitions; model data is an abstraction of data features, i.e., tables in a database; data to model refers to the process of transforming one or more model data from one system into a model for another system; Flink is a distributed computing framework for both stream and batch processing; Kafka is a high-throughput distributed publish-subscribe messaging system that can handle all action stream data of consumers on a website; Session Gap refers to a data association mechanism in Flink, such as when browsing a website (clicking, browsing, searching, and purchasing), user behavior is segmented, and the behavior within each segment is continuous and compact, with the correlation between behaviors within a segment being much greater than the correlation between behaviors between segments. We call each segment of user behavior a "Session," and the gap between segments a "Session gap"; JoinFunction defines the specific processing logic of the window function on the matched data when it is called.

[0057] The above scheme uses data stream association within a certain time window. If two data streams contain data with the same key that arrives outside the set time within the window, the data may change and the association will fail. In addition, the above technology does not trigger real-time changes in the corresponding transformed model data in the main table when the data in the slave table changes.

[0058] In summary, the above process, due to its massive data volume and the computational aspects involved in model transformation, impacts real-time performance due to data retrieval and processing. Furthermore, during model transformation, data may require multiple tables for retrieval, involving updates to both master and slave tables, necessitating ensuring data consistency throughout the entire process. Therefore, the current challenge is to efficiently and cost-effectively complete data model transformation while maintaining data consistency.

[0059] To facilitate understanding of this embodiment, the embodiments of the present invention will be described in detail below.

[0060] Example 1:

[0061] Figure 1 This is a flowchart of a model data conversion method for a real-time message queue provided in Embodiment 1 of the present invention.

[0062] Reference Figure 1 The method includes the following steps:

[0063] Step S11: Receive a data message from the message queue corresponding to the data table. The data message includes a first message body.

[0064] Step S12: Determine the message type based on the first message body;

[0065] Step S13: Modify the model according to the message type, data table attributes, and the set configuration table;

[0066] The message types include main table add, main table update, slave table add, and slave table update.

[0067] Specifically, data platforms or data warehouses provide real-time data services to multiple vendors and different applications. This requires transforming the data into models for the data requesters. During this transformation, the consistency between the master and slave tables becomes crucial; changes to the slave table data often prevent the transformed model from being updated. This application addresses this issue by receiving data messages from the data table's message queue. Each data message includes a first message body. The message type is determined based on the first message body. The model is then modified according to the message type, the data table's attributes, and a configured table, ensuring model transformation is completed while maintaining data consistency.

[0068] Furthermore, step S13 includes the following steps:

[0069] Step S21: When the message type is a new addition to the main table or an update to the main table, and the attribute of the data table is the main table, the identifier of the first model and the field content of the first model are obtained from the view corresponding to the main table based on the identifier of the main table.

[0070] Step S22: Using the name of the main table as a condition, find the name of the first model corresponding to the main table from the set configuration table; where the first model is a table in the database.

[0071] Step S23: Based on the name of the first model, send the identifier of the first model and the field content of the first model to the first model.

[0072] Here, the main table refers to the primary constituent table corresponding to a transformed data model, based on business requirements; the secondary table refers to the table associated with the main table, i.e., the table related to the main table; data consistency refers to the consistency of data across multiple upstream and downstream links; a message queue, generally abbreviated as MQ (Message Queue), refers to a container that stores messages during message transmission, utilizing an efficient and reliable message passing mechanism for platform-independent data exchange and integrating distributed systems based on data communication. Common examples include Kafka, RocketMQ, and ActiveMQ. Kafka is a high-throughput distributed publish-subscribe messaging system capable of handling consumer action stream data on a website; RocketMQ is a queue-based message middleware with high availability, high real-time performance, and distributed characteristics; ActiveMQ is an open-source message middleware.

[0073] Specifically, the main table is table A, and the secondary table is table B. Data messages are received from the message queue Kafka of table A. The first message body of the data message is read. The first message body includes the message type, which is determined to be either a new addition to the main table or an update to the main table. The value of the main table AID (identifier of table A) and other information are obtained from the first message body. Using the main table AID as a condition, the identifier of the first model MA and the field content of the first model MA are queried from the view (VA) corresponding to the main table A. The name of the corresponding first model MA is queried from the set configuration table using the name of table A as a condition.

[0074] Based on the name of the first model MA, send the identifier of the first model MA and the field content of the first model MA to the first model MA.

[0075] The content of the first message body is shown in Table 1. The first message body includes information such as message type, field value, field name, table name, and message sending time.

[0076] Table 1

[0077]

[0078] Furthermore, step S13 includes the following steps:

[0079] Step S31: When the message type is "Add to main table" or "Update to main table", and the data table attribute is "Sub-table", query the name and monitoring point information of the main table corresponding to the sub-table from the set configuration table.

[0080] Step S32: Construct a second message body based on the identifier of the slave table, the name of the master table, the monitoring point information, and the current message type, wherein the current message type is slave table addition and slave table update.

[0081] Step S33: Send the second message body to the message queue corresponding to the main table.

[0082] Specifically, the master table is table C, and the slave table is table A. The system retrieves the master table name and monitoring point AID corresponding to table A from the configured table, using the name of table A as the condition. A second message body is constructed, containing the value of table A's AID, the master table name C, and the monitoring point AID (a field name), with the message type being slave table add or slave table update. This message body is then sent to the message queue corresponding to master table C. If there are multiple master tables C, multiple corresponding queues are sent. The system notifies master table C via a message that the data in its corresponding slave table A has changed.

[0083] Furthermore, step S13 includes the following steps:

[0084] Step S41: When the message type is a table add or table update type, and the data table attribute is the main table, the monitoring point is matched from the view corresponding to the main table based on the identifier of the table to obtain the monitoring point information, the identifier of the second model, and the field content of the second model.

[0085] Step S42: Using the name of the main table as a condition, query the name of the second model corresponding to the main table from the set configuration table;

[0086] Step S43: Based on the name of the second model, send the identifier of the second model and the field content of the second model to the second model.

[0087] Specifically, data messages are received in the message queue of table A. The message type is determined based on the first message body of the data message. The message type is either a new addition from the table or an update from the table. Information such as the BID of table B is obtained. The monitoring point information BID, the identifier of the second model MA data, and the field content of the second model MA are obtained from the view VA corresponding to table A based on the BID of table B.

[0088] Using the name of table A as a condition, query the name of the second model MA corresponding to the main table A from the configured table; based on the name of the second model MA, send the monitoring point information BID, the identifier of the second model MA data, and the field content of the second model MA to the second model MA.

[0089] The above process ensures that when data in a slave table is updated, the corresponding model data in the master table is updated in real time.

[0090] Because the relationship between the slave table and the master table, as well as the monitoring point information, are configured, if the data in the slave table is updated, the corresponding associated master table can be obtained through the information in the configuration table, thereby realizing the real-time change of the model corresponding to the master table; in addition, this method has no time range requirement for the updated data.

[0091] Furthermore, the attributes of the data table include a master table and at least one slave table; the master table includes the identifier of the master table and the field names of the master table; the slave table includes the identifier of the slave table and the field names of the slave table.

[0092] The master table has corresponding views. The view includes the view identifier, the model content, and the watchpoint information of the corresponding slave table of the master table.

[0093] Specifically, each table has a corresponding message queue, each view has a corresponding master table, and also includes at least one slave table, which can be one or more. Each view has a VID field, which is used to uniquely identify each row of data in the view. VID is a variable, and the specific design can be based on business requirements. In this application, the value is the ID (Identity) of the master table.

[0094] like Figure 2 As shown, the data-to-model process involves the external system model MA sourcing data from tables A and B in the data platform. Table A is the master table, and table B is the slave table, forming a view. Each field is transformed according to certain rules. These rules can be either directly retrieving the values ​​of the original table fields or summing the values ​​of multiple fields. For example, field A1 in table A can have the value of the original table field (i.e., the value before any additions or changes to table A), or it can be the sum of the values ​​of multiple fields.

[0095] exist Figure 2 In Table A, AID represents the ID (Identity) of Table A, and A1 and A2 are the field names of Table A. Similarly, B1 and B2 are field names in Table B. The View VA contains the content of the ModelMA model data, and its content is added according to business requirements. In the View VA, VID (view identifier) ​​takes the value of AID (identifier of Table A), BID takes the value of BID (identifier of Table B), MA1 takes the value of A1, MA2 takes the value of A2, and MA3 takes the value of B1. Furthermore, MA1, MA2, and MA3 are the field names of the ModelMA model data, and MAID represents the ID (Identity) of the ModelMA model data. The required field content in the ModelMA model data is first transformed in the View VA, then retrieved, and finally pushed into the ModelMA model data. (See reference...) Figure 3 Table C is the primary table, and table A is the secondary table.

[0096] This application uses views and configuration tables to ensure that whenever field B1 in table B changes, the value of field MA3 in the corresponding model MA in the main table A changes in real time. The model conversion process is divided into two parts: one part involves adding or updating data in the main table, and the other part involves adding or updating data in the slave table. Details are as follows:

[0097] Implementation of views; utilizing the view functionality provided by the database, such as... Figure 2 The diagram shows table A as the main table, related to other tables such as table B. The data required by model MA is integrated into a single view. Then, information from the Kafka message queue is retrieved, and data in the view is queried using the VID, pushing the data to external systems. If the main table data changes, the required data is retrieved using the VID as the condition; if the secondary table data changes, the required data is retrieved using the monitoring point BID.

[0098] Additionally, refer to the configuration settings shown in Table 2:

[0099] Table 2

[0100] 1 B A BID MA 2021-05-17 17:33:14 2021-09-07 10:48:45 2 A C AID MC 2021-05-18 17:33:15 2021-09-07 10:48:46

[0101] As shown in Table 2, the configuration table records the required information such as the slave table name, master table name, monitoring point, and the model name corresponding to the master table.

[0102] This application is highly flexible and has a wide range of applications; it enables real-time changes to the transformation model data through views and configuration monitoring points; and it reduces development and learning time and server resource costs.

[0103] Example 2:

[0104] Figure 4 This is a schematic diagram of a model data conversion device for a real-time message queue provided in Embodiment 2 of the present invention.

[0105] Reference Figure 4 The device includes:

[0106] Receiving module 1 is used to receive data messages from the message queue corresponding to the data table. The data message includes a first message body.

[0107] Module 2 is used to determine the message type based on the first message body;

[0108] Modification module 3 is used to modify model data according to message type, data table attributes, and set configuration table;

[0109] The message types include main table add, main table update, slave table add, and slave table update.

[0110] Furthermore, module 3 is specifically used for:

[0111] When the message type is a new addition to the main table or an update to the main table, and the attributes of the data table are in the main table, the identifier of the first model and the field content of the first model are obtained from the view corresponding to the main table based on the identifier of the main table.

[0112] Using the name of the main table as a condition, find the name of the first model corresponding to the main table from the set configuration table;

[0113] Based on the name of the first model, the identifier of the first model data and the field content of the first model are sent to the first model.

[0114] Furthermore, module 3 is specifically used for:

[0115] When the message type is "Add to main table" or "Update to main table", and the data table attribute is "sub-table", the name and monitoring point information of the corresponding main table are queried from the configured table.

[0116] The second message body is constructed based on the identifier of the slave table, the name of the master table, the monitoring point information, and the current message type, where the current message type is slave table addition and slave table update.

[0117] Send the second message body to the message queue corresponding to the main table.

[0118] Furthermore, module 3 is specifically used for:

[0119] When the message type is "add from table" or "update from table", and the data table attribute is the main table, the monitoring point is matched from the view corresponding to the main table based on the identifier of the secondary table to obtain the monitoring point information, the identifier of the second model data, and the field content of the second model.

[0120] Using the name of the main table as a condition, retrieve the name of the second model corresponding to the main table from the configured table;

[0121] Based on the name of the second model, send the identifier of the second model and the field content of the second model to the second model.

[0122] Furthermore, the attributes of the data table include a master table and at least one slave table; the master table includes the identifier of the master table and the field names of the master table; the slave table includes the identifier of the slave table and the field names of the slave table.

[0123] The main table has corresponding views. The views include the view identifier, the model content, and the monitoring point information of the corresponding slave table in the main table.

[0124] This invention provides a model data conversion method for a real-time message queue, comprising: receiving a data message from a message queue of a data table, the data message including a first message body; determining a message type based on the first message body; and modifying the model according to the message type, the attributes of the data table, and a set configuration table; wherein the message type includes main table addition, main table update, slave table addition, and slave table update; this method enables real-time model modification, improves conversion efficiency, and reduces costs.

[0125] Example 3:

[0126] Figure 5 This is a schematic diagram of the structure of the electronic device provided in Embodiment 3 of the present invention.

[0127] like Figure 5 The diagram shows the structure of an electronic device 100. The electronic device 100 includes one or more processors 102, one or more storage devices 104, input devices 106, output devices 108, and an image acquisition device 110. These components are interconnected via a bus system 112 and / or other forms of connection mechanisms (not shown). It should be noted that... Figure 5The components and structure of the electronic device 100 shown are merely exemplary and not limiting; the electronic device may also have other components and structures as needed.

[0128] The processor 102 may be implemented in at least one of the following hardware forms: digital signal processor (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 102 may be one or a combination of several of the following: central processing unit (CPU) or other processing units with data processing capability and / or instruction execution capability, and may control other components in the electronic device 100 to perform the desired function.

[0129] The storage device 104 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 102 may execute the program instructions to implement the client functions (implemented by the processor) in the embodiments of the present invention described below, and / or other desired functions. Various applications and various data may also be stored in the computer-readable storage medium, such as various data used and / or generated by the applications.

[0130] The input device 106 may be a device used by a user to input commands, and may include one or more of the following: keyboard, mouse, microphone, and touch screen.

[0131] The output device 108 can output various information (e.g., images or sounds) to the outside (e.g., a user) and may include one or more of a display, a speaker, etc.

[0132] The image acquisition device 110 can capture images (such as photos, videos, etc.) desired by the user and store the captured images in the storage device 104 for use by other components.

[0133] For example, the example electronic device used to implement the face recognition method, apparatus and system according to the embodiments of the present invention can be implemented as a smart terminal such as a smartphone, tablet computer, or computer.

[0134] This invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the real-time message queue model data conversion method provided in the above embodiments.

[0135] This invention also provides a computer-readable medium having processor-executable non-volatile program code, on which a computer program is stored, and which, when run by a processor, executes the steps of the real-time message queue model data conversion method described above.

[0136] The computer program product provided in the embodiments of the present invention includes a computer-readable storage medium storing program code. The instructions included in the program code can be used to execute the methods described in the preceding method embodiments. For specific implementation, please refer to the method embodiments, which will not be repeated here.

[0137] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system and apparatus described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0138] Furthermore, in the description of the embodiments of the present invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral connection; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; and they can refer to the internal connection of two components. Those skilled in the art can understand the specific meaning of the above terms in the present invention based on the specific circumstances.

[0139] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, essentially, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0140] In the description of this invention, it should be noted that the terms "center," "upper," "lower," "left," "right," "vertical," "horizontal," "inner," and "outer," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing the invention and for simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0141] Finally, it should be noted that the above-described embodiments are merely specific implementations of the present invention, used to illustrate the technical solutions of the present invention, and not to limit it. The scope of protection of the present invention is not limited thereto. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed in the present invention, or make equivalent substitutions for some of the technical features; and these modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for model data conversion of a real-time message queue, characterized in that, The method includes: Receive data messages from the message queue corresponding to the data table, wherein the data message includes a first message body; Determine the message type based on the first message body; The model is modified according to the message type, the attributes of the data table, and the set configuration table; The message types include main table add, main table update, slave table add, and slave table update. The main table refers to the main constituent table corresponding to a transformed data model according to business settings; the subordinate table refers to the table associated with the main table. Modifying the model based on the message type, the attributes of the data table, and the set configuration table includes: When the message type is either "add to main table" or "update to main table", and the attribute of the data table is "main table", the identifier of the first model and the field content of the first model are obtained from the view corresponding to the main table based on the identifier of the main table. Using the name of the main table as a condition, the name of the first model corresponding to the main table is found in the set configuration table; Based on the name of the first model, send the identifier of the first model and the field content of the first model to the first model; The data table attributes include a main table and at least one secondary table; the main table includes an identifier and field names; the secondary table includes an identifier and field names. The master table has a corresponding view, which includes the view identifier, the model content, and the monitoring point information of the slave table corresponding to the master table.

2. The model data conversion method for real-time message queues according to claim 1, characterized in that, Modifying the model based on the message type, the attributes of the data table, and the set configuration table includes: When the message type is "add to main table" or "update to main table", and the data table attribute is "slave table", the name and monitoring point information of the main table corresponding to the slave table are queried from the set configuration table. A second message body is constructed based on the identifier of the slave table, the name of the master table, the monitoring point information, and the current message type, wherein the current message type is the slave table addition and slave table update type; The second message body is sent to the message queue corresponding to the main table.

3. The model data conversion method for real-time message queues according to claim 1, characterized in that, Modifying the model based on the message type, the attributes of the data table, and the set configuration table includes: When the message type is the add or update type of the sub-table, and the attribute of the data table is the main table, the monitoring point is matched from the view corresponding to the main table based on the identifier of the sub-table to obtain the monitoring point information, the identifier of the second model and the field content of the second model. Using the name of the main table as a condition, retrieve the name of the second model corresponding to the main table from the set configuration table; Based on the name of the second model, send the identifier of the second model and the field content of the second model to the second model.

4. A model data conversion device for a real-time message queue, characterized in that, The device includes: A receiving module is used to receive data messages from the message queue corresponding to the data table, wherein the data message includes a first message body; The determination module is used to determine the message type based on the first message body; The modification module is used to modify the model according to the message type, the attributes of the data table, and the set configuration table; The message types include main table add, main table update, slave table add, and slave table update. The main table refers to the main constituent table corresponding to a transformed data model according to business settings; the subordinate table refers to the table associated with the main table. The change module is specifically used for: When the message type is either "add to main table" or "update to main table", and the attribute of the data table is "main table", the identifier of the first model and the field content of the first model are obtained from the view corresponding to the main table based on the identifier of the main table. Using the name of the main table as a condition, the name of the first model corresponding to the main table is found in the set configuration table; Based on the name of the first model, send the identifier of the first model and the field content of the first model to the first model; The data table attributes include a main table and at least one secondary table; the main table includes an identifier and field names; the secondary table includes an identifier and field names. The master table has a corresponding view, which includes the view identifier, the model content, and the monitoring point information of the slave table corresponding to the master table.

5. The model data conversion device for real-time message queues according to claim 4, characterized in that, The change module is specifically used for: When the message type is "add to main table" or "update to main table", and the data table attribute is "slave table", the name and monitoring point information of the main table corresponding to the slave table are queried from the set configuration table. A second message body is constructed based on the identifier of the slave table, the name of the master table, the monitoring point information, and the current message type, wherein the current message type is the slave table addition and slave table update type; The second message body is sent to the message queue corresponding to the main table.

6. An electronic device comprising a memory and a processor, wherein the memory stores a computer program executable on the processor, characterized in that, When the processor executes the computer program, it implements the method described in any one of claims 1 to 3.

7. A computer-readable medium having processor-executable non-volatile program code, characterized in that, The program code causes the processor to execute the method according to any one of claims 1 to 3.

Images