6 results about "Source schema" patented technology

The invention discloses a data integration and fusion method and system based on a large model, and the method comprises the steps: taking the metadata information of initial multi-source data and representative sample data as the input context of a pre-trained LLM large language model, and carrying out the training through a Prompt prompt project guide model, thereby obtaining a plurality of enhanced data source mode descriptions; inputting a plurality of enhanced data source mode descriptions into an LLM large-scale language model, enabling the model to be a data integration engine through a Prompt prompt project, guiding the model to recognize semantic equivalence fields among different data source modes, and outputting a dynamic mode mapping scheme; analyzing a conflict reason and reasoning a conflict resolution strategy, and outputting a data cleaning suggestion and a data analysis result; and loading the fused data into the target data storage. The method can adapt to complex and changeable data structures, and the flexibility and efficiency of data integration are improved.

A system for converting a source data feed schema into a canonical data product including a memory for storing computer-executable instructions and a processor for executing the instructions stored on the memory. Execution of the instructions programs the processor to perform operations that include receiving a source data feed having a source schema, identifying a plurality of data fields of the source schema, assigning a data category from a plurality of predefined data categories to each data field of the plurality of data fields, modifying the source schema based on predefined parameters, wherein the source schema is modified to match a target schema, comparing the modified source schema to the target schema, and in response to a determination that the modified source schema matches the target schema, converting the source data feed to a canonical data product having the target schema based on the assigned data categories.

Systems, methods, and non-transitory, machine-readable media may facilitate testing within sub-environments with respect to resource capacity data corresponding to resources. Specifications of resource allocations for resources may be collected. Observation data, including resource metrics data, may be consolidated and used to develop resource data composites with mapped resource metrics data. A sub-environment interface may be configured to allow creation of sub-environments based on the resource data composites. Source schemas and data structures corresponding to the resource data composites may be selected. A migration plan to migrate the selected data structures to a destination schema for testing in a sub-environment may be created. A migration according to the migration plan may be executed. The destination schema may be created to match the selected source data structures and the selected source schemas. The selected data structures may be migrated to the destination schema to allow for testing in the sub-environment.

Methods, computer program products, and / or systems are provided that perform the following operations: obtaining source schema metadata, wherein the source schema metadata is associated with fields of a source schema; obtaining target schema metadata with a target schema, wherein the target schema metadata is associated with fields of a target schema; determining, for each field of the source schema and each field of the target schema, a representation for each field based, at least in part, on the source schema metadata or the target schema metadata associated with each field; and providing the representation for each field of the source schema and each field of the target schema for use in generating data mappings between the source schema and the target schema.

Systems, methods, and non-transitory, machine-readable media may facilitate testing within sub-environments with respect to resource capacity data corresponding to resources. Specifications of resource allocations for resources may be collected. Observation data, including resource metrics data, may be consolidated and used to develop resource data composites with mapped resource metrics data. A sub-environment interface may be configured to allow creation of sub-environments based on the resource data composites. Source schemas and data structures corresponding to the resource data composites may be selected. A migration plan to migrate the selected data structures to a destination schema for testing in a sub-environment may be created. A migration according to the migration plan may be executed. The destination schema may be created to match the selected source data structures and the selected source schemas. The selected data structures may be migrated to the destination schema to allow for testing in the sub-environment.

This enables data integration that is independent of the schema definitions and attribute information of the source data. [Solution] A data integration support system that assists in the process of integrating data from a group of source data files into a destination database comprises: a schema information generation device that generates source schema information defining a source database which is a database of the group of data files; a suggestion device that presents the user with suggestion information showing possible table options in the source database for each table in the destination database, based on the relationships between tables in the source schema information where tables refer to other tables and the relationships between tables in the destination schema information which defines the destination database, and prompts the user to select one of the options; and a data integration device that determines which tables in the source database will be integrated into each table in the destination database based on the user's selection of options.