Method, system and computer readable medium for data transformation using a representational state transfer (REST) application programming interface (API) - Patents.com
A metadata-based REST API framework addresses the incompatibility of legacy data formats by generating conversion logic at runtime, ensuring efficient and reliable data transformation between legacy and modern systems.
Patent Information
- Application Number
- JP2021544671
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2019-03-13
- Filing Date
- 2020-03-11
- Publication Date
- 2024-11-11
- Estimated Expiration
- 2040-03-11
AI Technical Summary
Legacy systems often use proprietary data formats that are not compatible with modern clients, requiring manual conversion logic that leads to duplicate code, excessive testing, and unpredictable performance.
A metadata-based REST API framework converts data formats using metadata and reflection programming to generate conversion logic at runtime, enabling efficient data transformation between legacy and modern systems.
Enables legacy systems to communicate with modern clients by efficiently converting data formats while maintaining performance and data integrity, reducing manual effort and errors.
Smart Images

Figure 0007580379000013 
Figure 0007580379000014 
Figure 0007580379000015
Abstract
Description
[Technical field]
[0001] Priority claim This application claims the benefit of priority to U.S. Patent Application Serial No. 16 / 352,738, filed March 13, 2019, the disclosure of which is incorporated herein by reference in its entirety.
[0002] Technical Field The subject matter described herein relates to network communications. More specifically, the subject matter relates to methods, systems, and computer-readable media for data transformation using a representational state transfer (REST) application programming interface (API). [Background technology]
[0003] background Some computer systems use multiple data formats and / or protocols to communicate. For example, components in a cloud-based computer system may communicate using internet protocol (IP), hypertext transfer protocol (HTTP), and / or other protocols. Additionally, to enable communication between various entities, one or more application programming interfaces (APIs) may be supported, thereby providing a standardized manner for communication.
[0004] A Representational State Transfer (REST) API is a type of API for providing web services. For example, a REST API may enable communication between two systems over HTTP. In this example, an HTTP request including a payload may be sent to a uniform resource identifier (URI), such as a uniform resource locator (URL), and in response, an HTTP response including a payload may be provided to the requester. However, some systems or components may not be able to process the payload provided via the REST API because the data may be stored in an unsupported format. Summary of the Invention [Means for solving the problem]
[0005] overview Methods, systems, and computer-readable media for data transformation using a Representational State Transfer (REST) Application Programming Interface (API) are disclosed. According to one method, the method includes receiving an input in a first format from a client via the REST API, converting the input in the first format to an input in a second format using predetermined metadata, sending the input in the second format to a legacy system to perform an operation using the input in the second format, and receiving an output in the second format from the legacy system, the output being based at least in part on the operation performed using the input in the second format, the method further includes converting the output in the second format to an output in a first format using the predetermined metadata, and sending the output in the first format to the client via the REST API.
[0006] According to one system, the system includes at least one processor and a server implemented using the at least one processor, the server configured to receive input in a first format from a client via a REST API, convert the input in the first format to an input in a second format using predetermined metadata, send the input in the second format to a legacy system to perform an operation using the input in the second format, and receive output in the second format from the legacy system, the output based at least in part on the operation performed using the input in the second format, the server further configured to convert the output in the second format to an output in the first format using the predetermined metadata, and send the output in the first format to the client via the REST API.
[0007] The subject matter described herein may be implemented in software in combination with hardware and / or firmware. For example, the subject matter described herein may be implemented in software executed by a processor. In some implementations, the subject matter described herein may be implemented using a non-transitory computer-readable medium that stores computer-executable instructions that, when executed by a processor of a computer, control a computer to perform steps. Exemplary computer-readable media suitable for implementing the subject matter described herein include non-transitory devices such as disk memory devices, chip memory devices, programmable logic devices, and application specific integrated circuits. Furthermore, the computer-readable medium implementing the subject matter described herein may be located on a single device or computing platform, or may be distributed across multiple devices or computing platforms.
[0008] As used herein, the term "node" refers to at least one physical computing platform that includes one or more processors and memory.
[0009] As used herein, the term "function" or "module" refers to hardware, firmware, or software in combination with hardware and / or firmware to implement the features described herein.
[0010] The subject matter described herein will now be described with reference to the accompanying drawings. [Brief description of the drawings]
[0011] [Figure 1] FIG. 1 illustrates an example environment for data transformation using a Representational State Transfer (REST) application programming interface (API). [Figure 2A] FIG. 1 illustrates exemplary metadata for validating and / or converting between data formats. [Figure 2B] FIG. 1 illustrates exemplary metadata for validating and / or converting between data formats. [Diagram 3] FIG. 1 illustrates exemplary default value data associated with converting between data formats. [Figure 4] FIG. 2 illustrates exemplary request data in a JavaScript object notation (JSON) data format. [Diagram 5] FIG. 13 is a diagram showing exemplary request data in a field list (FList) data format. [Figure 6] FIG. 13 illustrates an exemplary output of a data format for an FList. [Figure 7]FIG. 1 illustrates an example output of a JSON data format. [Figure 8] FIG. 1 illustrates an exemplary data transformation process. [Figure 9] FIG. 1 illustrates an example process for REST-based data conversion. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0012] Detailed Description The subject matter described herein relates to a method, system, and computer-readable medium for data transformation using a Representational State Transfer (REST) Application Programming Interface (API). Some systems, e.g., legacy or older systems, that provide multiple important services may communicate using one or more proprietary APIs or protocols. Such systems may be very expensive and time-consuming to replace, so that eventually newer systems will need to interact with the older systems, and it is highly likely that the newer systems may not be configured for such communication. For example, a financial institution or other enterprise may use a legacy system (e.g., a database, storage system, or backend) that was originally designed in the 1980s. In this example, the enterprise may also use a modern front-end or client (e.g., a web-based user interface) that needs to communicate with the legacy system or component. Continuing with this example, if the legacy system is a billing and revenue management (BRM) system that uses a legacy API and / or a data format, such as the Field List (FList) format, when processing information in various service domains (e.g., accounts, subscriptions, and billing), the FList format may not be usable by modern clients. Thus, the data needs to be converted to a compatible data format, such as the JSON format, in order to be usable by the modern client.
[0013] One approach for converting legacy formats to non-legacy formats may involve hand-coding conversion logic into the legacy system or an intermediate gateway node. For example, such an approach may involve manually coding functions to create, update, and retrieve multiple data objects of each type, e.g., account data objects, subscription data objects, payment profile data objects, and billing data objects. However, such an approach may lead to duplicate code, excessive testing effort, error-prone behavior, and may give unpredictable performance results.
[0014] According to some aspects of the subject matter described herein, techniques, methods, or mechanisms are disclosed for a metadata-based data conversion framework that can be used to convert one data format to another using metadata, such as, for example, a mapping template provided by an operator at system run-time or prior to system run-time. For example, an exemplary data conversion algorithm or associated framework can efficiently convert FList data to JSON data, or vice versa, by defining a simple metadata file that indicates how various data portions should be mapped or converted. Furthermore, such a framework can provide backward compatibility and improve extensibility by performing compatibility tests to determine whether metadata (e.g., a mapping template) needs to be modified, such that, for example, if a new field is added to a service domain, the new field can be added to the JSON / FList mapping template for correct conversion or transformation.
[0015] According to some aspects of the subject matter described herein, techniques, methods, or mechanisms for a REST-based data transformation framework are disclosed. For example, an exemplary REST-based data transformation framework may utilize metadata and REST APIs such that all REST calls (e.g., HTTP requests) are processed using the data framework or associated data transformation algorithms. In this example, as REST calls are processed by the framework, performance results and benefits can be realized for all service domain entities without compromising data integrity and performance.
[0016] According to some aspects of the subject matter described herein, techniques, methods, or mechanisms are disclosed for a data conversion framework using reflection programming techniques. For example, an exemplary data conversion system may generate conversion or transformation logic at or near run-time based on metadata. In this example, assuming that the logic is written in Java or another object-oriented programming language, the data conversion system may use the metadata to generate code (e.g., classes, objects, interfaces, and associated logic) for converting and / or marshalling data from one format to another and vice versa.
[0017] Advantageously, according to some aspects of the subject matter described herein, by using metadata and REST APIs, a data conversion framework can efficiently convert between data formats so that older systems can communicate with newer clients or components. Furthermore, by using the metadata to derive conversion or transformation logic via reflection programming techniques, code can be generated at run time to define logic (e.g., including classes and / or instances of classes or other programming structures) for converting data from one format to another.
[0018] Reference will now be made in detail to various embodiments of the subject matter described herein, examples of which are illustrated in the accompanying drawings, in which the same reference numbers are used, where possible, to refer to the same or like parts throughout the drawings.
[0019] 1 illustrates an example environment 100 for REST-based data transformation. Environment 100 may represent one or more communication networks. For example, environment 100 may represent an enterprise network and / or the Internet including various network nodes, computing platforms, servers, or devices. In this example, environment 100 may be usable to transmit and receive communications, for example, to provide content or services to nodes and / or end users.
[0020] With reference to FIG. 1, the environment 100 may include a computing system 102. The system 102 may represent one or more suitable computing platforms or devices for performing one or more functions or services, such as, for example, billing and revenue management (BRM) services or customer relationship management (CRM) services. In some embodiments, the system 102 may include components (e.g., server 104 or legacy system 110) that may be in a single distinct geographic location and communicate via an internal communications network, and / or that may be in diverse geographic locations and communicate via an external communications network. For example, the system 102 may be deployed in a proprietary cloud network managed by a service provider and accessible to subscribing customers via the Internet. In another example, the system 102 may be implemented using one or more servers in a server room maintained by an enterprise, or using cloud-based resources in one or more data centers maintained by a third party, such as, for example, a cloud service provider.
[0021] System 102 may include or interact with server 104 and legacy system 110. Server 104 may represent any suitable entity or entities for facilitating communications between various entities. For example, server 104 may include one or more processors, memory, and logic configured to facilitate communications between entities external to system 102 (e.g., client 112) and system 102 or associated entities (e.g., legacy system 110).
[0022] In some embodiments, the server 104 may include functionality for communicating with the legacy system 110 or another entity via a legacy or proprietary API (e.g., a portal communications module (PCM) API or an OPCODE API used by a BRM or cloud financial system) and / or other APIs or protocols. For example, a PCM API may involve a service or function call that utilizes data in the FList format. In this example, the server 104 or associated entity may send a request indicating an operation to be performed and the FList data to the legacy system 110 via the PCM API. Continuing with this example, after processing in the legacy system 110, the server 104 or associated entity may receive an output in the FList format from the legacy system 110 via the PCM API.
[0023] Legacy system 110 may represent any suitable entity or entities (e.g., virtual machines, software, or physical devices) for performing a service or function and may use protocols and / or data formats not supported by all clients, e.g., by client 112. For example, legacy system 110 may include software or related logic for storing, managing, or processing financial data and may communicate via APIs that are proprietary, legacy, or not supported by client 112. In some embodiments, legacy system 110 may include functionality for communicating with server 104 or other entities using PCM APIs or other APIs and protocols.
[0024] In some embodiments, the server 104 may include functionality for communicating with the client 112 or another entity via a REST API and / or other APIs or protocols. For example, a REST API may utilize requests and responses via HTTP messages. In this example, the server 104 or an associated entity may receive JSON-formatted data from the client 112 via the REST API that is not compatible with the legacy system 110 and may convert or facilitate the conversion of the JSON-formatted data to an FList format that is compatible with the legacy system 110. Continuing with the example, after receiving the FList-formatted output from the legacy system 110, the server 104 or an associated entity may convert or facilitate the conversion of the FList-formatted output to a JSON-formatted output that is compatible with the client 112 and then send a REST-based HTTP response to the client 112 that includes the JSON-formatted output.
[0025] The client 112 may represent any suitable entity or entities (e.g., virtual machines, software, or physical devices) for interacting with the system 102, the server 104, and / or associated entities. For example, the client 112 may include a user interface (UI) application (e.g., a web-based UI acting as a user front end or portal) or other software (e.g., financial software or business applications) that uses data stored or managed by the system 102 or associated services, such as, for example, the legacy system 110. In some embodiments, the client 112 may include functionality for communicating with the server 104 or other entities using REST APIs and / or other APIs and protocols.
[0026] The server 104 may interact with a data translation framework (DTF) 106. In some embodiments, the DTF 106 may include a component separate from the server 104 and / or a component integrated with the server 104. The DTF 106 may represent any suitable entity or entities (e.g., software running on a processor, an application-specific integrated circuit (ASIC), and / or a field-programmable gate array (FPGA)) for performing aspects related to data conversion, data transformation, or related functionality. For example, the server 104 may receive JSON data from a REST API message (e.g., originating from the client 112), and the DTF 106 may be configured to convert or translate the JSON data into corresponding FList data that can be processed by the system 102 or an associated entity, such as the legacy system 110. In another example, the server 104 may receive FList data originating from the legacy system 110 in response to a call or request from the client 112 to the system 102, and the DTF 106 may be configured to convert or translate the FList data into corresponding JSON data that can be processed by the client 112 or an associated service, such as a web UI or client application.
[0027] In some embodiments, the DTF 106 or associated modules may manage conversion rules and metadata used in performing data conversion or transformation. For example, an operator (e.g., a software or solution architect) or other entity (e.g., a management system) may communicate with the server 104 or DTF 106 and provide one or more metadata files, templates, or other information for performing data conversion or related data processing. In this example, the server 104 or DTF 106 may process and store the received data for subsequent use.
[0028] In some embodiments, the DTF 106 or associated modules may manage validation rules and metadata used in performing data validation. For example, an operator (e.g., a software or solution architect) or other entity (e.g., a management system) may communicate with the server 104 or DTF 106 and provide one or more metadata files, templates, or other information for performing data validation or related data processing. In this example, the server 104 or DTF 106 may process and store the received data for subsequent use.
[0029] Data storage 108 may represent any suitable entity (e.g., a non-transitory computer-readable medium, embedded memory, or memory device) for storing data related to handling communications and / or data conversion. Data storage 108 may store logic and / or data for utilizing various APIs and / or protocols for communication. For example, data storage 108 may store metadata and / or associated logic (e.g., default values, converter logic, format templates, etc.) for converting data from one format to another (e.g., from JSON or XML to FList) and vice versa (e.g., from FList to JSON or XML). In some embodiments, data storage 108 may be accessible by server 104, DTF 106, and other entities and may be usable for various purposes. In some embodiments, data storage 108 may be located on a single device or computing platform or distributed across multiple devices or computing platforms.
[0030] In some embodiments, the server 104, DTF 106, or legacy system 110 may be a separate message processing module of a distributed computing platform, a computing blade in a blade-based distributed computing platform, a processing core element associated with a single-core or multi-core computing device, or a virtual node or machine instantiated on one or more physical computing devices.
[0031] In some embodiments, the server 104, the DTF 106, and / or another entity may utilize one or more programming languages for performing data transformations or conversions. In such embodiments, the server 104, the DTF 106, and / or another entity may generate or configure a marshaller (e.g., a software interface or associated code) for marshalling data from a first format to a second format and unmarshalling data from the second format.
[0032] In some embodiments, the marshaler may include one or more functions that, when called, may convert entities, e.g., JSON data, to corresponding FList data and vice versa. In some embodiments, two types of marshal / unmarshal entities may be used: one type in the context of entities (e.g., create Account, get Account, update Account) and the other type in the context of searches (e.g., find Account, find Subscription).
[0033] In some embodiments, the server 104, the DTF 106, and / or another entity may create and / or call a marshaller to get, create, update, or modify an entity. For example, assuming a request to create an account, the DTF 106 may create a Marshaller object instance from a MarshallerFactory object instance. Example code for creating a Marshaller for creating an account is as follows:
[0034]
number
[0035] In the example code, the account is a Java object representation of a JSON document, and an ActionEnum object instance is defined as "CREATE" based on the requested create action.
[0036] In some embodiments, marshalling JSON data to FList data (and unmarshalling FList data to JSON data) may differ based on the action requested. In some embodiments, possible types for an ActionEnum object instance include CREATE, GET, UPDATE, and MODIFY, which may correspond to or be related to their respective HTTP methods: CREATE, GET, PUT, and PATCH.
[0037] In some embodiments, after initializing the Marshaller object instance, the server 104, the DTF 106, and / or another entity may call a marshal function to convert the JSON entities to FList data. Example code for calling the marshal function using the Marshaller object instance is as follows:
[0038]
number
[0039] In some embodiments, after marshaling the JSON entities into FList data, the server 104, the DTF 106, and / or another entity may invoke an operation in or by the legacy system 110. Example code for invoking a customer-created operation in or by the legacy system 110 is as follows:
[0040]
number
[0041] In some embodiments, after receiving the FList formatted output from legacy system 110 in response to a commit operation performed by or in legacy system 110, server 104, DTF 106, and / or another entity may invoke an unmarshal function to convert the FList formatted output into JSON entities. Example code for invoking the unmarshal function using a Marshaller object instance is as follows:
[0042]
number
[0043] In some embodiments, the server 104, the DTF 106, and / or another entity may create and / or invoke a marshaller to find one or more entities, such as, for example, a REST API search. Example code for creating a Marshaller object instance for a search is as follows:
[0044]
number
[0045] In the example code, the ActionEnum object instance is defined as "GET". In some embodiments, the ActionEnum object instance may not be significant to the search. In such embodiments, for consistency, ActionEnum.GET or another ActionEnum type may be used when requesting a search.
[0046] In some embodiments, after initializing a Marshaller object instance for a search, the server 104, the DTF 106, and / or another entity may create a JSON search entity (e.g., a rest.model.Search entity). Example code for constructing a search entity is as follows:
[0047]
number
[0048] In some embodiments, after constructing or creating a search entity, the server 104, the DTF 106, and / or another entity may call a marshal function to convert the JSON search entity (e.g., a rest.model.Search entity) to FList data. Example code for calling the marshal function using a Marshaller object instance is as follows:
[0049]
number
[0050] In some embodiments, after marshaling the JSON entities into FList data, the server 104, the DTF 106, and / or another entity may invoke an operation in or by the legacy system 110. Example code for invoking a search operation in or by the legacy system 110 is as follows:
[0051]
number
[0052] In some embodiments, after receiving the formatted output of an FList from legacy system 110 in response to a search operation being performed by or at legacy system 110, server 104, DTF 106, and / or another entity may invoke an unmarshal function to convert the FList data into a JSON entity. In some embodiments, for a search, the unmarshal function may require as input the entity type and the FList data to be unmarshalled. Exemplary code for invoking the unmarshal function using a Marshaller object instance to generate a BillingPreference entity is as follows:
[0053]
number
[0054] In some embodiments, the server 104, the DTF 106, and / or another entity may invoke and / or use converters (e.g., software and / or conversion rules) to convert fields, data, or types of data. For example, when the DTF 106 invokes a marshaler to marshal / unmarshal an entity, one or more converters may be used to convert various data portions (e.g., fields or FListIDs defined in metadata or mapping templates).
[0055] In some embodiments, a converter may convert data to a different type. For example, suppose a JSON entity includes a field "Status" with a value of "Active", but an FList requires a numeric status, DTF 106 may need to use a status-related converter to convert the status value of "Active" to "1001" when converting to the FList format.
[0056] In some embodiments, the converter may change the field name or associated field identifier (e.g., FListID) based on the action performed (GET vs. CREATE) or based on the value of the field. For example, depending on the type of product or entity, the JSON field "Name" may be converted to a different FListID, e.g., the field "Name" for a 'charge' product may be converted to the FListID "FLDProduct" and the field "Name" for a 'discount' product may be converted to the FListID "FLDDiscount".
[0057] In some embodiments, the server 104, the DTF 106, and / or another entity may create and use specialized or customized converters. In such embodiments, the server 104, the DTF 106, and / or another entity may use metadata (e.g., mapping templates) to create one or more converters. For example, if a field of a given JSON entity requires a specialized conversion (e.g., different than a field of another JSON entity), a specialized converter for the given JSON entity may be defined. In this example, the specialized converter may be a class created in a software package (e.g., the “REST.converter” package) and may be named with the class name “ <entity>Converter" <entity>is the name of a JSON entity.
[0058] For example, for an "AccountStatus" entity, DTF 106 may use a specialized converter to convert the numeric status value to an alpha status value. In this example, assuming the specialized converter is defined in an "AccountConverter" class, DTF 106 or another entity may create an "AccountConverter" class and define a "status" method on the "AccountConverter" object instance to perform the data conversion. Example code for the "status" method in the "AccountConverter" class (comments are lines beginning with " / / ") is as follows:
[0059]
number
[0060] In some embodiments, the server 104, the DTF 106, and / or another entity may use a base converter or a default converter if a specialized converter is not needed. For example, <entity>If a Converter is not available or required for a given entity, a base converter or default converter (e.g., BaseConverter) may be called. BaseConverter may handle conversions for fields that are common to most entities, e.g., ID may be a converter to Poid. In some embodiments, <entity>Fields defined in Converter override fields defined in BaseConverter, and the fields are: <entity>If not defined in either Converter or BaseConverter, no conversion may be required for that field.
[0061] In some embodiments, if multiple FListIDs exist, for example based on the action being performed (e.g., "UPDATE" vs. "CREATE") or based on the value of a field, the server 104, the DTF 106, and / or another entity may: <entity>A Converter may be defined to specify one or more fields on which a conversion should be performed. For example, for a field "Name" in an "Account" entity, if the HTTP method is "CREATE", the field "Name" may be converted to FListID "FldNameCreate", and if the HTTP method is "UPDATE", the field "Name" may be converted to FListID "FldNameUpdate". Example code for the method "name" in the "AccountConverter" class is as follows:
[0062]
number
[0063] In some embodiments, the converter may utilize various, e.g., one or more, inputs. For example, assuming Converter.convert(fieldName, value, action, className) is a method for invoking a converter, "fieldName" may be the name of the field to be converted (e.g., name, id, status, etc.), "value" may be the actual value of the field to be converted (e.g., for status, "1001" should be converted), "action" may be the current action being performed (e.g., CREATE, GET, UPDATE, MODIFY, etc.) or may identify the type or associated field, and "className" may be the entity class being worked on (e.g., account, subscription, etc.).
[0064] In some embodiments, the server 104, the DTF 106, and / or another entity may invoke and / or use a validator (e.g., software and / or validation rules) to validate fields, data, or types of data. For example, when the DTF 106 invokes a marshaler to marshal / unmarshal an entity, one or more validators may be used to validate various data portions (e.g., fields or FListIDs defined in the metadata or mapping templates).
[0065] In some embodiments, a validator may be invoked to marshal the data from JSON to an FList. In some embodiments, a validator may not be invoked to unmarshal the data from an FList to JSON because the values returned from the legacy system 110 may be assumed to be valid.
[0066] In some embodiments, the server 104, the DTF 106, and / or another entity may create and use specialized or customized validators. In such embodiments, the server 104, the DTF 106, and / or another entity may use metadata (e.g., a mapping template) to create one or more validators. For example, if a field of a given JSON entity requires specialized validation (e.g., different than a field of another JSON entity), a specialized validator for the given JSON entity may be defined. In this example, the specialized validator may be a class created in a software package (e.g., the “REST.validator” package) and may be named with the class name “ <entity>Validator" <entity>is the name of a JSON entity.
[0067] For example, for a "contacts" entity, DTF 106 may validate a number of field values associated with a contact. In this example, assume that a specialized validator is defined in an "AccountValidator" class, and that DTF 106 or another entity may create an "AccountValidator" class and define a "contacts" method on the "AccountValidator" object instance to perform the data conversion. Example code for the "contacts" method in the "AccountValidator" class (comments are lines beginning with " / / ") is as follows:
[0068]
number
[0069] In some embodiments, the server 104, the DTF 106, and / or another entity may use a base or default validator if a specialized validator is not required. For example, <entity>If a Converter is not available or required for a given entity, a base converter or default validator (e.g., BaseValidator) may be called. The BaseValidator may handle validation for fields that are common to most entities, e.g., ID may be a validator to Poid. In some embodiments, <entity>Fields defined in a Validator override fields defined in a BaseValidator, and the fields are: <entity>If it is not defined in either Validator or BaseValidator, validation may not be required for that field.
[0070] It should be understood that Figure 1 is for illustrative purposes, and that the various nodes, their locations, and / or their functions (e.g., modules) described above in connection with Figure 1 may be changed, modified, added, or removed. For example, some nodes and / or functions may be combined into a single entity. In another example, some nodes and / or functions may be distributed across multiple nodes and / or platforms.
[0071] 2A-2B illustrate example metadata 200-202 for validating and / or converting between data formats. The metadata 200-202 may include information for identifying related or corresponding data elements from different data formats, such as JSON and Flist formats. The metadata 200-202 may include information for handling data storage or data representation for one or both data formats. The metadata 200-202 may further include information, logic, or conversion rules related to data conversion.
[0072] 2A-2B, the tables representing metadata 200-202 include columns and / or fields for data entries and associated descriptions regarding the data entries. The data entry fields may store information to represent a data entity or a portion thereof (e.g., a data element or data field) for storing information in a JSON format (e.g., a JSON data object) or another format (e.g., Flist data). In some embodiments, the data entry fields may further include a data type or an example field name associated with or indicative of the data entity or associated type.
[0073] In some embodiments, metadata 200-202 may include a collection of related data entries that represent a data entity or related data. For example, as shown in the table of FIG. 2A, a collection of data entries may represent an "Account" data entity or related data. In this example, the various represented rows or data entries may represent one or more fields (e.g., type, account number, etc.) and one or more other data entities (e.g., contacts, phone, preferences, etc.) that are associated with the "Account" data entity.
[0074] In some embodiments, metadata 200-202 may include mapping information to associate related data elements between data formats. In some embodiments, DTF 106 or related entities may use the mapping information when converting or translating data between data formats. For example, as shown in the table of FIG. 2A, the JSON field name "accountNumber" may map to the FlistID "FldAccountNo."
[0075] In some embodiments, the metadata 200-202 may include information to identify a converter or associated logic to use during data conversion. For example, the server 104 or DTF 106 may use the metadata 200-202 to identify which logic, rules, or software modules to use, e.g., consumer user account data may need to be processed differently than business user account data.
[0076] In some embodiments, the metadata 200-202 may include information to identify validators or associated logic to use during data validation. For example, the server 104 or DTF 106 may use the metadata 200-202 to identify that certain fields should be validated before or during data conversion, such as before or during conversion from JSON to an FList. In this example, as shown in the table of FIG. 2B, the JSON “contacts” data entity may be associated with a “validation:[mandatory]” specification, such that the DTF 106 or associated validators may perform validation on field values associated with the “contacts” entity, for example, before the marshaler assigns the values to the FList data set.
[0077] A description field may store information logic, context, or examples for describing or defining a corresponding data entry. For example, a description field may define how the associated data is stored, converted, or used. In another example, a description field may define how the associated data is derived, obtained, or generated.
[0078] In some embodiments, various descriptive information may be stored and / or represented as programming logic, data variables, or other formats. For example, "if / else" logic may be used to represent when a default value is to be used or not.
[0079] It will be understood that the metadata 200-202 are for illustrative purposes and that different and / or additional data than that shown in Figures 2A-2B may be used to indicate conversion logic, format preferences, data format relationships, or other information. Additionally, the metadata 200-202 may be stored or managed (e.g., within data storage 108) using a variety of data structures and / or computer-readable media.
[0080] 3 illustrates example default value data 300 related to converting between data formats. The data 300 may include information for identifying or determining default values for various data types, data fields, and / or data entities. For example, the data 300 may include numbers, values, text, and / or data pointers or identifiers for functions or software for obtaining values. In this example, the default values may involve executing a method or function, such as code for generating pseudorandom numbers or code for extracting the Internet Protocol (IP) address of a computer.
[0081] 3, a table representing data 300 includes columns and / or fields for FList identifier (FListID) type, FListID type example, data entity mapping type, and data entity mapping example. The FListID type field may store information to represent the type of a data portion (e.g., data element or data field) in the FList format or an associated file. For example, example FListID types may include "IntField", "PoidField", "StrField", and / or "ArrayField".
[0082] The FListID type example field may store information for representing one or more FListIDs that are examples of a corresponding type of data portion (e.g., data element or data field) in the FList format or an associated file. For example, some FListID type examples may include "FldFlags" (an example of an "IntField" type), "FldPayType" (an example of an "IntField" type), "FldPoid" (an example of a "PoidField" type), "FldName" (an example of a "StrField" type), "FldContactType" (an example of a "StrField" type), "FldLocale" (an example of a "StrField" type), "FldAccountNo" (an example of a "StrField" type), "FldBillinfoId" (an example of a "StrField" type), and / or "FldBalInfo" (an example of an "ArrayField" type).
[0083] The data entity mapping type field may store information to represent the type of a data portion (e.g., a data element or data field) in a JSON format or an associated file. For example, exemplary data entity mapping types may include "int", "string", "method call", and / or "null". In some embodiments, a method or function call may take predefined values and / or other values or parameters as input. For example, the method "generateRandomNumber" may generate a 10-digit random number and may include optional inputs, such as a pre-existing string and a delimiter character. In this example, if no input is provided, the method may generate a 10-digit random number, e.g., 4827495827. In another example, if a pre-existing string "ABC" is input in this method, e.g., generateRandomNumber(ABC), the method may generate a 10-digit random number with a given string pre-existing, e.g., ABC4827495827. In another example, if the pre-existing string "ABC" and the delimiter "-" are input to the method, e.g., generateRandomNumber(ABC,-), the method may generate a 10-digit random number pre-existing the given string and delimiter, e.g., ABC-4827495827.
[0084] The data entity mapping example fields may store information for representing a data portion (e.g., a data element or a data field) or an associated file in JSON format. Some data entity mapping examples may indicate a JSON field name (e.g., "field: id") that corresponds to an FListID (e.g., "FListId: FldPoid"). Some data entity mapping examples may indicate that a default value exists for the corresponding FListID, e.g., "FListDefault: true". Some data entity mapping examples may indicate a default value for the corresponding FlistID or may invoke a method to obtain the corresponding FlistID, e.g., "FListDefaultValue: getName()".
[0085] It will further be understood that the data 300 is for illustrative purposes and that different and / or additional data than that shown in Figure 3 may be available to indicate default values for particular data portions or other information. Additionally, the data 300 may be stored or managed (e.g., within data storage 108) using a variety of data structures and / or computer readable media.
[0086] 4 illustrates an example request data 400 in a JSON data format. The request data 400 may be data generated or provided from the client 112. For example, the client 112 may represent a web-based UI that serves as a user front to a cloud-based BRM system (e.g., system 102). In this example, the UI may be programmed using JavaScript or another programming language and may store user input in JSON format. In this example, the client 112 may generate and send an HTTP POST message to the server 104 or DTF 106. The HTTP POST message may include user input (e.g., request data 400) for creating a new billing account.
[0087] As shown in FIG. 4, JSON-formatted request data 400 may generally represent entities or objects within curly brackets “{}”, which may also include additional objects and fields or parameters. Sub-objects and object parameters may be indented relative to their parent. A field may be represented by the field name in quotation marks, followed by a colon, followed by its value (e.g., “City”: “New York”). Text values in JSON format generally occur between quotation marks, and numeric values generally do not occur between quotation marks. Internet Engineering Task Force (IETF) Request for Comments (RFC) 8259, dated December 2017, provides additional details regarding example JSON formats, the disclosure of which is incorporated herein by reference.
[0088] It will also be understood that the request data 400 is for illustrative purposes and that different and / or additional data than that shown in Figure 4 may be available to indicate data or input for conversion or translation to another data format. Additionally, the request data 400 may be stored or managed using a variety of data structures and / or computer readable media.
[0089] 5 shows example request data 500 in FList data format. The request data 500 may be data that is converted from JSON format to FList format. For example, the server 104 or the DTF 106 may receive the request data 400 from the client 112 and convert or translate the request data 400 to the request data 500. In this example, the conversion or translation may be performed by a marshaler that is configured using the metadata 202-204 and / or other information.
[0090] In some embodiments, the DTF 106 or associated software (e.g., a marshaler) may be configured to convert or translate data from one format to valid data in another format. For example, using metadata 202-204 and / or associated conversion rules, the DTF 106 may modify some text to allow for special characters in a particular format, such as single or double quotes (e.g., by using a backslash or other "escape" sequence before the special character). In another example, using metadata 202-204 and / or associated conversion rules, the DTF 106 may modify a value to a different value, e.g., adjust a number to fit a valid range in the new format or change a value from text to a numeric equivalent.
[0091] 5, the request data 500 may include fields and subfields that represent billing accounts to be created by the system 102 or the legacy system 110. For example, the DTF 106 or associated software (e.g., a marshaller) may be configured using the metadata 202-204. In this example, the DTF 106 may generate the request data 500 by determining associated field names and field values from corresponding data in the request data 400 using the metadata 202-204 and / or associated conversion rules. Continuing with this example, the DTF 106 may further generate default values and / or modify values when performing the data conversion.
[0092] 5, a line of request data 500 in FList format may include, from left to right, a first value indicating a nesting level (e.g., 0=top level, 1=sub level, 2=sub-sub level, etc.), a second value indicating a field identifier (e.g., PIN_FLD_BILLINFO, PIN_FLD_POID, etc.), a third value indicating a field type (e.g., ARRAY, STR, INT, etc.), a fourth value indicating an element identifier (e.g., [0], [1], etc.), and a fifth value indicating a value for the field (e.g., text, memory pointer, or number). Oracle® Communications Billing and Revenue Management Developer's Guide (Release 7.5), dated August 2016, provides additional details regarding an exemplary FList format, the disclosure of which is incorporated herein by reference.
[0093] It will also be understood that the request data 500 is for illustrative purposes and that different and / or additional data than that shown in Figure 5 may be used to illustrate data or inputs for processing by the legacy system 110. Additionally, the request data 500 may be stored or managed using various data structures and / or computer readable media.
[0094] 6 illustrates an example output of the FList data format. Output 600 may be data generated or provided by legacy system 110. For example, legacy system 110 may receive a PCM request, also referred to as an OPCODE request, that includes request data 500 and process the PCM request. In this example, the PCM request may involve adding or creating a new billing account. Continuing with this example, after processing the PCM request, output 600 may be generated and sent to server 104 or DTF 106 therein.
[0095] In some embodiments, output 600 may indicate that the associated request was successful. For example, output 600 may indicate current or valid stored data (e.g., the data after processing a PCM request). In this example, client 112 or another entity may be able to determine that the associated request was successful by inspecting output 600 and detecting that output 600 includes the requested changes or updates.
[0096] In some embodiments, output 600 may indicate that an error occurred or that the associated request (e.g., an HTTP Post message or an HTTP GET message) was not successful. For example, output 600 may include an error message or error code, or may indicate current or valid stored data (e.g., the data prior to the PCM request). In this example, client 112 or another entity may be able to determine that the associated request was not successful by inspecting output 600 and detecting an error message or error code.
[0097] 6, output 600 may include fields and subfields that represent billing accounts created by system 102 or legacy system 110. For example, output 600 may include one or more billing account identifiers and information for deriving one or more URLs or links to billing account information associated with system 102 or legacy system 110.
[0098] In some embodiments, output 600 or an associated response message may be generated after an operation associated with a request is attempted to be performed. For example, legacy system 110 may process a PCM request by performing a database commit or update operation and then generate output 600 associated with the operation. In this example, if successful, legacy system 110 may send a copy of the stored billing account information to server 104 in an HTTP 201 message. However, if unsuccessful, legacy system 110 may send an error message or error code in an HTTP 409 message.
[0099] It will also be understood that output 600 is for illustrative purposes and that different and / or additional data than that shown in Figure 6 may be used to represent the data or output for conversion or translation to another data format. Additionally, output 600 may be stored or managed using a variety of data structures and / or computer readable media.
[0100] 7 illustrates an example output 700 in JSON data format. The output 700 may be data converted from FList format to JSON format. For example, the server 104 or the DTF 106 may receive the output 600 from the legacy system 110 and convert or translate the output 600 to the output 700. In this example, the conversion or translation may be performed by a marshaler configured using the metadata 202-204 and / or other information.
[0101] In some embodiments, the DTF 106 or associated software (e.g., a marshaler or converter) may be configured to convert or translate data from one format to valid data in another format. For example, using metadata 202-204 and / or associated conversion rules, the DTF 106 may derive a URL from the FList data and provide the URL in a valid JSON format.
[0102] 7, the output 700 may include a "billings preferences" data portion that includes URLs or links to various preferences stored in or associated with the system 102 or the legacy system 110. For example, the client 112 may receive the output 700 and use the links therein to display hyperlinks in an associated UI.
[0103] In some embodiments, the output 700 or an associated response message (e.g., an HTTP Post message) may be inspected to determine whether the associated request was successful. For example, the client 112 may receive the output 700 and then inspect the information from the provided URL to determine that the modification or addition was successfully performed. In another example, the determination of whether the associated request was successful may be based at least in part on identifying the response message type, e.g., HTTP 2XX messages=success and HTTP 4XX-5XX messages=failure.
[0104] It will also be understood that output 700 is for illustrative purposes and that different and / or additional data than that shown in Figure 7 may be used to represent the data or output for client 112. Additionally, output 700 may be stored or managed using various data structures and / or computer readable media.
[0105] Figure 8 illustrates an example data marshalling process 800. Referring to Figure 8, the process 800 may be performed in marshalling data from one format to another.
[0106] In step 802, a marshaler (e.g., software running on the server 104, the DTF 106, or an associated processor) may receive a data entity (e.g., a JSON object including multiple field values) for marshaling the data from JSON to an FList.
[0107] In step 804, a translator (e.g., software executing on the server 104, the DTF 106, or an associated processor) may receive the values for conversion from JSON to an FList. In some embodiments, the conversion may be performed according to metadata 200-202 (e.g., a mapping template) and may require data validation and / or data conversion.
[0108] In step 806, a validator and / or converter (validator / converter) (e.g., software executing on the server 104, the DTF 106, or an associated processor) may receive the data entity or associated values for processing. In some embodiments, the validator / converter may utilize or invoke a method based on the field or value being processed.
[0109] In step 808, a listener (e.g., software executing on the server 104, the DTF 106, or an associated processor) may determine whether to instantiate (e.g., create instances of) one or more classes or objects for processing based on the received values and the processing required. For example, the listener may invoke or instantiate base validators and specialized validators for processing the data entity or values therein.
[0110] In step 810, it may be determined whether a corresponding method in a specialized validator / converter exists to process a particular value associated with the received data entity. If a method exists, the specialized validator / converter may process the value.
[0111] In step 812, if the method does not exist in the specialized validator / converter, it may be determined whether a corresponding method exists in the base validator / converter to process a particular value associated with the received data entity, and if a method exists, the base validator / converter may process the value.
[0112] In some embodiments, after processing the value (or if no processing is required), subsequent values from the data entity may be processed until the data entity is completely marshaled from JSON to an FList.
[0113] It will also be understood that process 800 is for illustrative purposes and that different steps and / or entities and / or additional steps and / or entities may be used to perform data processing and / or other data processing than those shown in FIG.
[0114] 9 illustrates an example process 900 for REST-based data conversion. In some embodiments, the process 900 or portions thereof (e.g., steps 902, 904, 906, 908, 910, and / or 912) may be performed by or at the server 104, the DTF 106, and / or another node or module.
[0115] With reference to process 900, at step 902, input in a first format may be received. For example, an HTTP PUT request including a payload including data in JSON format may be sent from client 112 to server 104. In this example, the payload may include account information for adding or updating an account handled by legacy system 110.
[0116] In step 904, the input in the first format may be converted to input in the second format using the predetermined metadata. For example, the server 104 or an associated DTF 106 may convert or transform the JSON data into FList data using a marshaler configured using the predetermined metadata.
[0117] In some embodiments, the first format may be a JSON format or an extensible markup language (XML) format, and the second format may be an FList format or a PCM format. In some embodiments, the first format may be an FList format or a PCM format, and the second format may be a JSON format or an XML format.
[0118] In some embodiments, converting data in a first format to data in a second format using predefined metadata may include selecting a conversion rule based on the action to be performed (e.g., “ActionEnum”), or the HTTP method, or the request type (e.g., DELETE, POST, GET, PUT, or PATCH).
[0119] In some embodiments, the conversion rules may include rules for validating values in a first format and / or rules for changing values in the first format that are invalid to valid values in a second format.
[0120] In some embodiments, converting data in a first format to data in a second format using the predetermined metadata may include using the predetermined metadata and reflective programming to generate conversion logic. For example, reflective programming allows software to inspect code at run time and generate classes and / or instances of classes or other structures for converting data from one format to another based on the predetermined metadata.
[0121] In step 906, the input in the second format may be sent to the legacy system 110 to perform an operation using the input in the second format. For example, after the DTF 106 converts the JSON data to FList data, the server 104 may send the FList data to the legacy system 110 via the PCM API. In this example, In step 908, output in the second format may be received from legacy system 110, where the output may be based at least in part on operations performed using the input in the second format. For example, legacy system 110 may process FList data (e.g., by adding or updating account information for a particular account) and generate output in the FList format.
[0122] In step 910, the output in the second format may be converted to the output in the first format using the predetermined metadata. For example, the server 104 or an associated DTF 106 may convert or translate the FList data into JSON data using a marshaler configured using the predetermined metadata.
[0123] In step 912, the output in the first format may be sent via a REST API to the client 112. For example, after converting the output received from the legacy system 110 to JSON data, the server 104 may generate an HTTP 201 response that includes the JSON data and send the response to the client 112.
[0124] In some embodiments, one or more backward compatibility tests may be performed to test whether the data transformation and / or conversion is accurate, e.g., converted to valid data. For example, the server 104 or DTF 106 may perform tests to determine whether data converted from one format to another is valid. In this example, the server 104 or DTF 106 may be configured to determine which pieces are invalid or what data is missing and may provide the test data or an associated report to an end user or other associated entity.
[0125] In some embodiments, backward compatibility tests may be triggered in response to receiving additional metadata from an operator or client. For example, the DTF 106 may receive a new metadata file during run-time to expand or use transformation and / or conversion capabilities. In this example, after receiving the new metadata file, the DTF 106 or an associated entity may trigger one or more tests before using the metadata file to perform data transformation and / or conversion.
[0126] In some embodiments, the output in the first format may be sent to the client 112. For example, the client 112 may receive the output as one or more JSON objects and use the data to populate or update a web-based user interface.
[0127] In some embodiments, the client 112 may include a user interface application, an external system, a customer relationship management system, a cloud service, a web service, a REST API client, or financial software.
[0128] It will be understood that process 900 is for illustrative purposes and that different and / or additional actions may be used. It will also be understood that the various actions described herein may be performed in a different order or sequence.
[0129] It should be noted that the system 102, server 104, DTF 106, and / or functions described herein may constitute special purpose computing devices. Furthermore, the system 102, server 104, DTF 106, and / or functions described herein may improve the technical field of network communication by providing mechanisms and / or techniques for data conversion between different data formats. For example, by utilizing certain metadata and reflection programming principles, software or conversion logic may be generated that converts or transforms JSON data to FList data or vice versa in various service domains. In another example, by using the REST API and DTF 106, a first entity (e.g., a web user interface application, a front end, financial software, etc.) may communicate with a system or service (e.g., legacy system 110) that uses a data format (e.g., FList or PCM) not supported by the first entity.
[0130] It will be understood that various details of the subject matter described herein can be changed without departing from the scope of the subject matter described herein. Moreover, the above description is for purposes of illustration only and not for purposes of limitation, and the subject matter described herein is defined by the appended claims.< / entity> < / entity> < / entity> < / entity> < / entity> < / entity> < / entity> < / entity> < / entity> < / entity> < / entity>
Claims
1. 1. A method for data transformation using a Representational State Transfer (REST) application programming interface (API), comprising: A server including a metadata-based data transformation framework for providing data transformation using a REST API and facilitating interaction with a legacy system, receiving an input in a first format from a client via the REST API using a HyperText Transfer Protocol (HTTP) request message, the input including an identifier identifying a target for conversion; The method further comprises: generating, at run-time, conversion logic for converting between a first format and a second format using the HTTP request message, the conversion logic being generated using predetermined metadata and reflection programming; The method further comprises: using the conversion logic to convert the input in the first format to an input in a second format, where converting the input in the first format to an input in the second format includes verifying that a value of the input in the first format is valid in the second format, and in response to determining that the value in the first format is invalid in the second format, changing the value in the first format to a valid value in the second format; The method further comprises: transmitting the input in the second format to a legacy system to perform an operation using the input in the second format; receiving an output in the second format from the legacy system, the output being based at least in part on the operations performed using the input in the second format; The method further comprises: converting the output in the second format to an output in the first format using the conversion logic; sending the output in the first format to the client via the REST API; A method comprising:
2. 2. The method of claim 1, wherein the first format is a JavaScript Object Notation (JSON) format or an Extensible Markup Language (XML) format, and the second format is a Field List (FList) format or a Portal Communication Module (PCM) format.
3. 2. The method of claim 1, wherein the first format is a Field List (FList) format or a Portal Communication Module (PCM) format, and the second format is a JavaScript Object Notation (JSON) format or an Extensible Markup Language (XML) format.
4. The method of any of claims 1 to 3, wherein generating the conversion logic further comprises selecting a conversion rule based on an action to be performed or a HyperText Transfer Protocol (HTTP) request type.
5. 5. The method of claim 1, further comprising performing a backward compatibility test to determine whether the data in the second format is valid or whether the output in the first format is valid.
6. The method of claim 5 , wherein the backward compatibility testing is triggered in response to receiving additional metadata from an operator or the client.
7. The method of any of claims 1 to 6, wherein the client comprises a user interface application, an external system, a customer relationship management system, a cloud service, a web service, a REST API client, or financial software.
8. 8. The method of claim 1, wherein generating the conversion logic comprises instantiating, at run-time, a converter object, and converting the input in the first format to the input in the second format comprises using the converter object.
9. 8. The method of claim 1, wherein generating the conversion logic comprises: using the identifier to determine whether a specialized validator is required to validate the input in the first format; and in response to determining that the specialized validator is required, instantiating the specialized validator at run-time to validate whether at least some input in the first format is valid in the second format.
10. 1. A system for data transformation using a Representational State Transfer (REST) application programming interface (API), comprising: At least one processor; a server including a metadata-based data transformation framework for providing data transformation using a REST API and facilitating interaction with legacy systems, said server being implemented using said at least one processor; The server, configured to receive input in a first format from a client via the REST API using a HyperText Transfer Protocol (HTTP) request message, the input including an identifier identifying a target for conversion; The server further comprises: configured to generate, at runtime, conversion logic for converting between a first format and a second format using the HTTP request message, the conversion logic being generated using predetermined metadata and reflection programming; The server further comprises: and configured to convert, using the conversion logic, the input in the first format to an input in a second format, where converting the input in the first format to an input in the second format includes verifying that a value of the input in the first format is valid in the second format, and in response to determining that the value in the first format is invalid in the second format, changing the value in the first format to a value that is valid in the second format; The server further comprises: transmitting the input in the second format to a legacy system to perform an operation using the input in the second format; and receiving an output in the second format from the legacy system, the output being based at least in part on the operations performed using the input in the second format; The server further comprises: converting the output in the second format to an output in the first format using the conversion logic; sending the output in the first format to the client via the REST API; A system configured to:
11. 11. The system of claim 10, wherein the first format is a JavaScript Object Notation (JSON) format or an Extensible Markup Language (XML) format, and the second format is a Field List (FList) format or a Portal Communication Module (PCM) format.
12. 11. The system of claim 10, wherein the first format is a Field List (FList) format or a Portal Communication Module (PCM) format, and the second format is a JavaScript Object Notation (JSON) format or an Extensible Markup Language (XML) format.
13. The system of any of claims 10 to 12, wherein the server is configured to select a conversion rule based on an action to be performed or a HyperText Transfer Protocol (HTTP) request type.
14. 13. The system of claim 10, wherein the server is configured to perform backward compatibility tests to determine whether the data in the second format is valid or whether the output in the first format is valid.
15. The system of claim 14 , wherein the server is configured to trigger the backward compatibility test in response to receiving additional metadata from an operator or the client.
16. The system of any of claims 10 to 15, wherein the client comprises a user interface application, an external system, a customer relationship management system, a cloud service, a web service, a REST API client, or financial software.
17. 17. The system of claim 10, wherein the server is configured to instantiate a converter object at runtime, and converting the input in the first format to the input in the second format comprises using the converter object.
18. 17. The system of claim 10, wherein the server is configured to use the identifier to determine whether a specialized validator is required to validate the input in the first format, and in response to determining that a specialized validator is required, at run-time instantiate the specialized validator to validate whether at least some input in the first format is valid in the second format.
19. A program for causing a computer processor to execute the method according to any one of claims 1 to 9.
Citation Information
Patent Citations
Systems and methods for transforming business process policy data
JP2004535024A
Request proxy system and request proxy method
JP2011048559A
Discovery and exposure of transactional middleware server-based applications as consumable service endpoints
US20180253342A1
