Flexible data pipeline for data storage
By introducing a combination of router, cache module and domain module in industrial control systems, the automatic identification and processing of general and customized messages is realized, and data loss and system interruption problems during data pipeline updates are solved, ensuring flexible data storage and High reliability of the system.
Patent Information
- Application Number
- CN202410888988.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-07-06
- Filing Date
- 2024-07-04
- Publication Date
- 2025-01-07
AI Technical Summary
When the data pipeline is updated, existing industrial control systems cannot flexibly process different data formats, resulting in data loss and system interruption, and cannot adapt to customer needs.
A device is designed, including a router, a cache module and a domain module. Through predefined general conversion rules and customized conversion rules, common messages and customized messages can be automatically identified and processed, ensuring that data is stored in the corresponding database without interruption during updates. middle.
It realizes that there is no message loss when the data pipeline is updated, and incoming communication is not interrupted. It supports hot updates, adapts to storage in different data formats, and improves the flexibility and reliability of the system.
Smart Images

Figure CN119276932A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure generally relates to processing data sent from a data source to a database through a data pipeline that converts data received from the data source into a format expected by clients of the database. Background Art
[0002] Industrial automation / control systems are used to control the operation of a variety of systems, including processes, machines, etc., and are typically adapted to different control applications through the configuration and interconnection of multiple control system components or devices, such as control modules, input / output (I / O) modules, I / O devices, etc. Existing industrial control systems typically include a processor that runs or executes a control program to interact with an I / O system (e.g., typically one or more I / O modules or devices) to receive system information in the form of analog and / or digital inputs from field sensors and provide outputs (analog and / or digital) to one or more actuators. Industrial control systems are increasingly interconnected with management information and other systems in manufacturing facilities and can be operably connected to any number of communication networks to facilitate various business management functions in addition to process / machine control functions, such as inventory control, accounting, manufacturing control, etc.
[0003] refer to Figure 1 , some edge devices ED are located between the control layer (including sensors, actuators and controllers) of the industrial system and the cloud. The edge device ED collects data from the control layer and pushes it to the cloud server CS through the telecommunication network TN for analysis. The cloud server and / or edge device take actions to react to the process and / or notify the good or bad behavior of the process.
[0004] The edge software runs on a dedicated edge device, isolating the control layer from the enterprise network. The edge device can be embedded in an edge gateway such as an IT / OT box, which refers to the interface between the OT network including industrial equipment and a telecommunication network such as the Internet. The edge software pushes data to the cloud server CS through the application server AS, which acts as a cloud entry point and is connected to the cloud provider network CPN, which connects the application server AS to the cloud server CS. Data can also be sent to the cloud server CS from other edge devices ED' connected to another telecommunication network TN' via the same entry point (application server AS).
[0005] The entry point of the cloud server forms a data pipeline. Once the data is received, the data pipeline can then format the payload to store the data in a dedicated database of the cloud server.
[0006] An application that can retrieve data stored in a private database via the cloud provider network CPN knows the schema of the private database (e.g. how to read / write data in the database). For example, the schema is based on the standard ISA-95, an international standard for developing automation interfaces between enterprises and control systems. This standard was developed for global manufacturers and is applicable to all industries and various processes, such as batch, continuous and repetitive processes.
[0007] The goals of ISA-95 are to provide consistent terminology as a basis for vendor and manufacturer communications, to provide a consistent information model, and to provide a consistent operational model as a basis for articulating application functionality and how information is used.
[0008] Typically, for schema ISA-95, a data pipeline receives data from a source (edge and / or cloud), understands a payload that may or may not be compressed, like a json file or any other serialized data. Once the payload is deserialized, the message is stored into a database according to the well-known schema ISA95.
[0009] If all data published by the data source can be deserialized by the application server and stored in a database, for example according to ISA-95, the system operates without problems. A problem arises when the data cannot be serialized in a well-known pattern (for example to increase throughput), or when the data cannot be stored according to ISA-95: for example, for a KPI from Overall Equipment Effectiveness OEE computed at the edge, the OEE value cannot be stored in a database compliant with the ISA-95 standard.
[0010] In this case, the evolution of the database and data pipeline needs to be planned. Therefore, the data pipeline needs to be stopped, the database stopped, the new data schema deployed, and the new data pipeline deployed. Unfortunately, there will still be mismatches with the new implementation. In addition, each time the evolution and deployment is completed, different devices and servers are stopped / started, and data may be lost.
[0011] Therefore, there is a need for a flexible data pipeline that is able to process any kind of data to be stored in a database to adapt to the needs of the clients of the data without interrupting the execution of the data pipeline. Summary of the invention
[0012] This summary is provided to introduce concepts related to the claimed subject matter. This summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used to determine or limit the scope of the claimed subject matter.
[0013] In one embodiment, there is provided an apparatus serving as an entry point to a cloud provider network, the cloud provider network being connected to a general database capable of storing general messages and a customized database capable of storing customized messages, the apparatus comprising:
[0014] a router configured to receive a generic message from a generic data source and a customized message from a customized data source, each of the generic message and the customized message including a message type and a payload,
[0015] a cache module, storing at least an identifier of a general mode associated with an identifier of a general database and an identifier of a customized mode associated with an identifier of a customized database,
[0016] a generic domain module capable of deserializing the payload of a generic message according to a generic schema and delivering the generic message in an expected format to a generic database using generic conversion rules, wherein the generic conversion rules are predefined before the generic domain module and the generic database are deployed,
[0017] A custom domain module that can deserialize the payload of a custom message according to a custom schema and deliver the custom message in the expected format to a custom database using custom transformation rules, where the custom transformation rules are defined after the deployment of the common domain module and the common database,
[0018] wherein, upon receiving the generic message, the router is configured to read the message type to extract an identifier of the generic pattern, query the cache module with the identifier of the generic pattern to determine an identifier of the generic database associated with the extracted identifier of the generic pattern, and transmit the generic message to the generic domain module,
[0019] wherein, upon receiving a customized message, the router is configured to read the message type to extract an identifier of the customized pattern, query the cache module with the identifier of the customized pattern to determine an identifier of a customized database associated with the extracted identifier of the customized pattern, and transmit the customized message to the customized domain module,
[0020] Wherein after the custom domain module is deployed, the cache module is updated with an identifier of the custom schema associated with an identifier of the custom database.
[0021] Advantageously, an automatic process is provided that generates new streams and storage without any message loss during hot updates and without the use of any load balancers and data reinjection. Incoming communications are not interrupted and no messages are lost while the data pipeline is being updated.
[0022] More specifically, without deploying the custom domain module, custom messages are not supported by the router and are forwarded to a service that manages errors, and after deploying the custom domain module and correspondingly updating the cache module, custom messages can be routed to the custom domain module to be stored in a custom database.
[0023] The database used by the application can be improved with new data without modifying the legacy database.
[0024] In an embodiment, the custom domain module is deployed with custom transformation rules that allow deserialization of custom messages with custom schemas and storage of custom messages in a custom database.
[0025] In an embodiment, after the custom domain module is deployed, the cache module is updated with a key comprising an identifier of the custom schema and an identifier of the custom database.
[0026] In an embodiment, the generic transformation rules provide for querying storage in a generic database, while the customized transformation rules provide for querying storage in a customized database.
[0027] In an embodiment, the custom domain module is linked to a custom buffer layer, which forms a buffer storage between the router and the custom domain module.
[0028] In an embodiment, the custom buffer layer implements a queue or cache mechanism suitable for real-time data, or a persistent storage suitable for batch data.
[0029] In an embodiment, the generic messages and the custom messages contain industrial telemetry data.
[0030] In an embodiment, the router is configured to, when the identifier of the customization database is not determined, transmit the customization message to the undetermined domain module to be handled by the service managing the error.
[0031] In another embodiment, a method implemented in an application server that acts as an entry point to a cloud provider network connected to a general database capable of storing general messages and a customized database capable of storing customized messages is provided, the application server comprising:
[0032] a router configured to receive a generic message from a generic data source and a customized message from a customized data source, each of the generic message and the customized message including a message type and a payload,
[0033] a cache module, storing at least an identifier of a general mode associated with an identifier of a general database and an identifier of a customized mode associated with an identifier of a customized database,
[0034] The method includes:
[0035] When receiving a generic message, the router reads the message type to extract the identifier of the generic pattern, queries the cache module with the identifier of the generic pattern to determine the identifier of the generic database associated with the extracted identifier of the generic pattern, and transmits the generic message to the generic domain module,
[0036] Upon receiving the generic message, the generic domain module deserializes the payload of the generic message according to the generic schema and transmits the generic message in the expected format to the generic database using generic conversion rules, wherein the generic conversion rules are predefined before the deployment of the generic domain module and the generic database.
[0037] Upon receiving the customization message, the router reads the message type to extract the identifier of the customization pattern, queries the cache module with the identifier of the customization pattern to determine the identifier of the customization database associated with the extracted identifier of the customization pattern, and transmits the customization message to the customization domain module,
[0038] Upon receiving a custom message, the custom domain module deserializes the payload of the custom message according to the custom schema and transmits the custom message in the expected format to the custom database using custom conversion rules, where the custom conversion rules are defined after the common domain module and the common database are deployed.
[0039] Wherein after the custom domain module is deployed, the cache module is updated with the custom schema associated with the custom database.
[0040] In another embodiment, a computer readable medium is provided, on which a computer program for executing the method of storing general messages in a general database and storing customized messages in a customized database is contained. The computer program includes instructions for executing the steps of the method according to the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0041] The detailed description is described with reference to the accompanying drawings. In the drawings, the left-most digit(s) of a reference number identifies the drawing in which the reference number first appears. The same numerals are used to refer to similar features and components throughout the drawings. Some embodiments of systems and / or methods according to embodiments of the present subject matter are now described, by way of example only, and with reference to the accompanying drawings, in which:
[0042] Figure 1 A schematic block diagram of a communication system capable of storing only general messages in a general database is shown;
[0043] Figure 2 A schematic block diagram of a communication system for storing general messages in a general database and for storing customized messages in a customized database according to one embodiment is shown;
[0044] Figure 3is a flow chart of a method for storing generic messages in a generic database and for storing customized messages in a customized database according to one embodiment.
[0045] The same reference number represents the same element or the same type of element on all drawings.
[0046] Those skilled in the art will appreciate that any block diagram herein represents a conceptual diagram of an illustrative system that embodies the principles of the subject matter. Similarly, it will be appreciated that any flow chart, flowchart, state transition diagram, pseudo code, etc., represent various processes that can be substantially represented in a computer-readable medium and executed by a computer or processor, whether or not such a computer or processor is explicitly shown. DETAILED DESCRIPTION
[0047] The accompanying drawings and the following description illustrate specific exemplary embodiments of the present invention. Therefore, it should be understood that those skilled in the art will be able to design various arrangements that, although not explicitly described or shown herein, embody the principles of the present invention and are included within the scope of the present invention. In addition, any examples described herein are intended to aid in understanding the principles of the present invention and should be interpreted as not being limited to these specifically enumerated examples and conditions. Therefore, the present invention is not limited to the specific embodiments or examples described below, but is defined by the claims and their equivalents.
[0048] refer to Figure 2 The communication system includes an application server AS, a general data source GDS, a customized data source CDS, a general database GDB and a customized database CDB.
[0049] The application server AS acts as a cloud entry point, connected to a cloud provider network CPN, which connects the application server AS to a general database GDB and a custom database CDB. Each of the general database GDB and the custom database can be connected to or implemented in a cloud server, which provides access to data stored in the database for applications managed by customers that utilize these data.
[0050] The cloud provider network CPN can be a wired or wireless network, or a combination of wired and wireless networks. The first and second telecommunication networks can be associated with packet networks, such as IP ("Internet Protocol") high-speed networks, such as the Internet or an intranet, or even a private network dedicated to a company.
[0051] The cloud provider network is for example a digital cellular wireless communication network of the GPRS (General Packet Radio Service), UMTS (Universal Mobile Telecommunications System), CDMA (Code Division Multiple Access) type, LTE (Long Term Evolution) or even 5G (Fifth Generation) type.
[0052] The general data source GDS or the customized data source CDS can be implemented in the edge device and is located between the control layer and the cloud of the industrial system (including sensors, actuators and controllers). Data similar to industrial telemetry data is collected by the edge device from the control layer and sent to the general database GDB or the customized database CDB via the application server through the corresponding telecommunication network.
[0053] The general data source GDS provides data using a general schema known to the application server and can be stored in a general database GDB. In the example, the schema is "senml+json", which means that the payload is serialized in "json" according to the RFC8428 specification. In another example, the schema is "senml+cbor", which means that the payload is serialized in "cbor" according to the RFC8428 specification.
[0054] The custom data source CDS provides data with a custom schema that is initially unknown to the application server. The application server is updated with the custom schema to be able to store the custom data in the custom database CDB. Thus, the application server can recognize the serialization of the payload to adapt the data to the expected format of the custom database.
[0055] The application server AS includes a router RT, a cache module CM, a general domain module GDM, a customized domain module CDM and an undetermined domain module UDM.
[0056] The router RT is configured to receive messages containing, for example, industrial telemetry data from different data sources, such as a general data source GDS and a customized data source CDS. The router RT is configured to transmit general messages to a general domain module GDM and to transmit customized messages to a customized domain module CDM. The router RT is also configured to transmit unidentified messages to an undetermined domain module UDM.
[0057] The cache module CM can be implemented as a cache database and stores the configuration of the router RT, allowing external configuration of the router without restarting the router itself. The router can work with the cache module in a publish / subscribe mode, a polling mode or a call for each request. In an embodiment with a publish / subscribe mode between the router RT and the cache module CM, when the cache module is updated with a new configuration, a notification is sent to the router RT to read and apply the new configuration (incremental or global).
[0058] The general domain module GDM is only able to deserialize data of general messages, i.e. messages with predefined general schemas before the general domain module and the general database are deployed. The general domain module GDM is linked to a general buffer layer GBL, which forms a buffer storage between the router RT and the general domain module GDM.
[0059] The custom domain module CDM is able to deserialize the data, i.e. the payload, of a custom message, i.e. a message with a custom schema defined after the custom domain module and the custom database are deployed. This custom schema has a new configuration set for the custom domain module CDM. The custom domain module CDM is linked to a custom buffer layer CBL, which forms a buffer storage between the router RT and the custom domain module CDM.
[0060] Each of the general buffer layer GBL and the custom buffer layer CBL allows to stop, start, delete or update the configuration of one or all components of the communication system, such as the router RT, the custom domain module CDM or the general domain module GDM. Each of the general buffer layer GBL and the custom buffer layer CBL can be implemented with a queue mechanism, a cache mechanism or a persistent storage. For example, a queue or a cache mechanism is very suitable for real-time data (telemetry data, events, alarms, errors), while a persistent storage is a privileged mechanism for batch data (files).
[0061] In one embodiment, depending on the configuration, the application server AS forms a data pipeline responsible for receiving, transforming and storing data according to an ETL process (extract, transform and load), especially for event-driven data, or forms a data pipeline responsible for receiving, storing and transforming data according to an ELT process (extract, load and transform), especially for batch data.
[0062] With the ETL process, each of the general buffer layer GBL and the custom buffer layer CBL is configured to use a queue buffer with or without a survival time to achieve zero frame loss for real-time delivery. Using the ELT process, each of the general buffer layer GBL and the custom buffer layer CBL is configured to extract later using a permanent storage (e.g., sql or file storage).
[0063] When the configuration stored in the cache module CM is updated, it provides parameters to store customized messages in the customization buffer layer CBL. The router can access the list of customized patterns, allowing the router to transmit the message to the correct destination. More specifically, the cache module stores the common patterns associated with the common database and at least one customized pattern associated with the customization database.
[0064] In one embodiment, different customization modes may be associated with the same customization database.
[0065] In one embodiment, different customization databases are provided, and different customization modes may be associated with different customization databases, respectively.
[0066] In one embodiment, the cache module stores a key including a schema identifier and a database identifier, wherein the schema identifier specifies either a generic schema or a customized schema, and the database identifier specifies either a generic database or a customized database. The database identifier gives an indication of the path followed by the received message to be stored in the correct database: the router may route the received message to either the generic domain module or the customized domain module.
[0067] For example, a key includes a schema identifier "senml+json" specifying a generic schema and a database identifier "generic domain endpoint" specifying a generic database GDB, and another key includes a schema identifier "logfmt" specifying a custom schema and a database identifier "custom domain endpoint" specifying a custom database CDB.
[0068] A message (custom or generic) consists of two parts that can be read by the interested party: the message type and the payload. The message type is a set of metadata that is used to understand the payload and, if necessary, the context of the payload.
[0069] The message type contains information about the payload so that the payload's data can be deserialized and converted according to the schema required by the database storage.
[0070] The router is capable of reading the message type, extracting the pattern identifier from the message type, and interrogating the cache module CM with the extracted pattern identifier to determine the database associated with the extracted pattern identifier.
[0071] The router is able to route the message to the Generic Domain Module GDM or the Custom Domain Module CDM based on the pattern identifier extracted from the payload of the message. If the pattern identifier is not recognized, i.e. not present in the cache module, the message is routed to the Undetermined Domain Module UDM to manage the error.
[0072] The general domain module GDM is capable of deserializing the payload of the general message according to the general schema and transmitting the general message in the expected format to the general database using the general conversion rules, wherein the general conversion rules are predefined before the general domain module and the general database are deployed.
[0073] The custom domain module CDM can deserialize the payload of the custom message according to the custom schema and transmit the custom message in the expected format to the custom database using the custom conversion rules, wherein the custom conversion rules are defined after the custom domain module and the custom database are deployed.
[0074] In one embodiment, the custom domain module CDM corresponds to an application, for example in the form of a container.
[0075] Once the custom database CDB is deployed, the custom domain module CDM is deployed with custom transformation rules, wherein the custom transformation rules are generated based on the custom schema used by the custom message so as to store the custom message in a format accepted by the custom database. Once the custom domain module CDM is deployed, the cache module is updated with a key including an identifier of the custom schema and an identifier of the custom database.
[0076] In one embodiment, the deployment of the custom domain module CDM is based on a CI / CD (Continuous Integration, Continuous Deployment) pipeline, where CI / CD is a process centered around continuous integration of new code, which meets the need to easily update applications while running.
[0077] In one example, a custom schema for custom messages is provided, and a format for storing custom messages in a custom database is derived. A rule generator can automatically generate transformation rules for a custom domain module CDM, such as in the form of a configuration file, to store custom messages in a desired format in a custom database. For example, the transformation rule provides a query of the form: "store("foo=bar a=14baz="\hello kitty\"")".
[0078] In one example, the custom schema indicates a json format with a corresponding sql schema of a custom database, and the rule generator generates SQL queries to be executed by the custom domain module CDM to ingest input data into the custom database CDB as a SQL database. In this example, the CI pipeline retrieves the SQL driver from the repository and generates (compiles, generates scripts, etc.) an application to be deployed as the custom domain module CDM (if not already deployed), or the updated transformation rules in the custom domain module CDM (if already deployed).
[0079] refer to Figure 3 According to one embodiment of the present invention, a method for managing generic messages from a generic data source and customized messages from a customized data source through an application server includes steps S1 to S12, wherein the application server acts as an entry point to a cloud provider network, and the cloud provider network is connected to a generic database capable of storing generic messages and a customized database capable of storing customized messages.
[0080] Initially, the application server AS is configured with a router RT, a cache module CM, a general domain module GDM and an undetermined domain module UDM. The general domain module GDM is configured with conversion rules for sending general messages to a general database GDB.
[0081] In step S1 , the router RT receives a generic message and reads the message type to extract the generic mode identifier.
[0082] In step S2, when the extracted universal pattern identifier matches the given universal pattern identifier, the router RT queries the cache module CM with the extracted universal pattern identifier to determine a key comprising the given universal pattern identifier and the given universal database identifier. The router RT retrieves the given universal database identifier, the identifier indicating the path followed by the received universal message.
[0083] In step S3, the router RT transmits the generic message to the general domain module GDM through the general buffer layer GBL.
[0084] In step S4, the general domain module GDM deserializes the payload of the general message according to the general schema and transmits the general message in the expected format to the general database using the general conversion rules. The general database GDB is then able to store the general message. For example, the general domain module GDM creates an "insert" type sql query applied to some data of the payload.
[0085] In step S5, the router RT receives the custom message and reads the message type to extract the custom mode identifier.
[0086] At step S6, the router RT queries the cache module CM with the extracted customized pattern identifier and is unable to determine any key comprising a given universal pattern identifier that matches the extracted universal pattern identifier.
[0087] In step S7, the router RT transmits the custom message to the undetermined domain module UDM to be processed by the service managing the error.
[0088] In step S8, a requirement to store custom messages with a custom schema is provided (e.g., from a customer). A custom database CDB is deployed, and a custom domain module CDM is deployed with custom transformation rules that allow deserialization of custom messages with a custom schema and storage of custom messages in the custom database CDB. Furthermore, a cache module is updated with a key that includes an identifier of the custom schema and an identifier of the custom database.
[0089] In step S9, the router RT again receives the customized message and reads the message type to extract the customized mode identifier.
[0090] In step S10, when the extracted customized pattern identifier matches the given customized pattern identifier, the router RT queries the cache module CM with the extracted customized pattern identifier to determine a key comprising the given customized pattern identifier and the given customized database identifier. The router RT retrieves the given customized database identifier indicating the path followed by the received customized message.
[0091] In step S11, the router RT transmits the customization message to the customization domain module CDM through the customization buffer layer CBL.
[0092] In step S12, the custom domain module CDM deserializes the payload of the custom message according to the custom schema and transmits the custom message in the expected format to the custom database using custom conversion rules. The custom database CDB is then able to store the custom message. For example, the custom domain module CDM creates an "insert" type sql query applied to some data of the payload.
[0093] Although the present invention has been described above with reference to specific embodiments, the present invention is not limited to the specific forms set forth herein. Instead, the present invention is limited only by the appended claims, and within the scope of these appended claims, other embodiments besides the specific embodiments described above are equally possible.
[0094] In addition, although exemplary embodiments have been described above in some exemplary combinations of components and / or functions, it should be understood that alternative embodiments may be provided by different combinations of components and / or functions without departing from the scope of the present disclosure. In addition, it is specifically contemplated that specific features described separately or as part of an embodiment may be combined with other separately described features or parts of other embodiments.
Claims
1. An apparatus serving as an entry point to a cloud provider network (CPN), the cloud provider network being connected to a general database (GDB) capable of storing general messages and a customized database (CDB) capable of storing customized messages, the apparatus comprising: a router (RT) configured to receive a general message from a general data source (GDS) and a customized message from a customized data source (CDS), each of the general message and the customized message comprising a message type and a payload, a cache module (MD) storing at least an identifier of a universal mode associated with an identifier of the universal database and an identifier of a customized mode associated with an identifier of the customized database, a general domain module (GDM) capable of deserializing the payload of a general message according to the general schema and transmitting the general message in an expected format to the general database using general conversion rules, wherein the general conversion rules are predefined prior to deployment of the general domain module and the general database, a custom domain module (CDM) capable of deserializing the payload of the custom message according to the custom schema and transmitting the custom message in an expected format to the custom database using custom transformation rules, wherein the custom transformation rules are defined after deployment of the general domain module and the general database, wherein upon receiving a generic message, the router (RT) is configured to read the message type to extract an identifier of a generic pattern, query a cache module with the identifier of the generic pattern to determine an identifier of a generic database (GDB) associated with the extracted identifier of the generic pattern, and transmit the generic message to the generic domain module, wherein upon receiving a custom message, the router (RT) is configured to read the message type to extract an identifier of a custom pattern, query a cache module with the identifier of the custom pattern to determine an identifier of a custom database (CDB) associated with the extracted identifier of the custom pattern, and transmit the custom message to the custom domain module, Wherein after deploying the custom domain module (CDM), the cache module (MD) is updated with an identifier of the custom schema associated with an identifier of the custom database (CDB).
2. The apparatus according to claim 1, wherein the custom domain module (CDM) is deployed with custom conversion rules, which allow deserialization of custom messages having the custom schema and allow storage of the custom messages in the custom database (CDB). 3 . The apparatus of claim 2 , wherein after the custom domain module is deployed, the cache module is updated with a key including an identifier of the custom schema and an identifier of the custom database.
4. The apparatus of any one of the preceding claims, wherein the generic transformation rules provide for queries stored in the generic database and the customized transformation rules provide for queries stored in the customized database.
5. The apparatus according to any of the preceding claims, wherein the Custom Domain Module (CDM) is linked to a Custom Buffer Layer (CBL) forming a buffer storage between the router (RT) and the Custom Domain Module (CDM).
6. The apparatus according to claim 4, wherein the custom buffer layer (CBL) implements a queue or cache mechanism suitable for real-time data, or a persistent storage suitable for batch data.
7. An apparatus according to any one of the preceding claims, wherein the generic message and the custom message contain industrial telemetry data.
8. The device according to any of the preceding claims, wherein the router (RT) is configured to, when the identifier of the customization database (CDB) is not determined, transmit a customization message to an undetermined domain module (UDM) to be handled by a service managing errors.
9. A method implemented in an application server (AS), the application server acting as an entry point to a cloud provider network (CPN), the cloud provider network connected to a general database (GDB) capable of storing general messages and a customized database (CDB) capable of storing customized messages, the application server comprising: a router (RT) configured to receive a general message from a general data source (GDS) and a customized message from a customized data source (CDS), each of the general message and the customized message comprising a message type and a payload, a cache module (MD) storing at least an identifier of a universal mode associated with an identifier of the universal database and an identifier of a customized mode associated with an identifier of the customized database, The method comprises: Upon receiving a generic message, the router (RT) reads (S1) the message type to extract an identifier of the generic pattern, interrogates (S2) the cache module with the identifier of the generic pattern to determine an identifier of a generic database (GDB) associated with the extracted identifier of the generic pattern, and transmits (S3) the generic message to the generic domain module, Upon receiving the generic message, the generic domain module (GDM) deserializes (S4) the payload of the generic message according to the generic schema and transmits the generic message in an expected format to the generic database using generic conversion rules, wherein the generic conversion rules are predefined before the deployment of the generic domain module and the generic database, Upon receiving the customized message, the router (RT) reads (S9) the message type to extract an identifier of the customized pattern, queries (S10) the cache module with the identifier of the customized pattern to determine an identifier of a customized database (CDB) associated with the extracted identifier of the customized pattern, and transmits (S11) the customized message to the customized domain module, Upon receiving the customized message, the customized domain module (CDM) deserializes (S12) the payload of the customized message according to the customized schema and transmits the customized message in the expected format to the customized database using customized conversion rules, wherein the customized conversion rules are defined after the deployment of the universal domain module and the universal database, Wherein after deploying the custom domain module (CDM), the cache module (MD) is updated (S8) with a custom schema associated with the custom database (CDB).
10. A computer readable medium having embodied thereon a computer program for executing the method according to claim 9.