Heterogeneous model conversion method, system and equipment based on source data and storage medium

Abstract

The invention discloses a heterogeneous model conversion method, system and device based on source data and a storage medium. The method comprises the steps that table and domain mapping rules of a source end and a destination end are established; establishing an attribute standardization rule of the mapping model; establishing a model data dynamic verification rule in the conversion process; data extraction, conversion and writing are completed; the system comprises a table and domain mapping module, a standardization module, a verification module and a data processing module. According to the method, the model heterogeneous characteristics of the source end and the target end are combined, a flow mechanism of heterogeneous model conversion is established, the table and domain mapping relation is established for the heterogeneous model, consistency maintenance of heterogeneous model data is achieved, and the efficiency and accuracy of conversion maintenance are greatly improved.

Description

Technical Field

[0001] The present invention relates to a smart grid source-end model heterogeneous conversion technology, and in particular to a source data-based heterogeneous model conversion method, system, device and storage medium. Background Art

[0002] The power grid model is uniformly modeled and managed at the source data end and used separately in each application system. At the same time, the modeling process at the source data end is structured modeling-oriented, and its model attributes focus more on the comprehensive description of power objects, and are distributed in parameter tables, attribute tables, and mapping tables according to the description type; while in each application system, it focuses more on monitoring, control or data processing, and its model structure is oriented towards production design. Therefore, it is necessary to complete the conversion from the source data end to the heterogeneous data model to achieve the design concept of unified maintenance of source data and on-demand use of application system data. For the conversion of heterogeneous models, the existing technology mainly uses manual confirmation for source data to convert the corresponding records of the corresponding table one by one into heterogeneous models and write them into heterogeneous databases; or uses database structured language to query the structure to be converted through database structured language, and uses multi-table association or name conversion to map it into a heterogeneous model and write it into a heterogeneous database. However, both approaches have drawbacks. Manual verification is prone to errors due to the large volume of data, high redundancy, and complex data calculations and conversions. The labor involved is also high, and the timeliness of data cannot be guaranteed. Using a database structured language significantly reduces the workload of manual conversion and calculations, but it requires high professional expertise from operations and maintenance personnel. Complex mapping and conversions can make the resulting database structured language cumbersome and difficult to manage. Furthermore, database structured languages ​​have limitations. Data quality cannot be verified synchronously, and conversion rules for databases without implemented conversion functions are difficult to implement, making it impossible to ensure the accuracy of heterogeneous models after conversion. Summary of the Invention

[0003] Purpose of the invention: The purpose of the present invention is to provide a heterogeneous model conversion method, system, device and storage medium based on source data, so as to provide an accurate and efficient heterogeneous model conversion solution and solve the data consistency problem between the source and target models.

[0004] Technical solution: The heterogeneous model conversion method based on source data described in the present invention includes the following steps:

[0005] (1) Establish table and domain mapping rules between the source and destination ends;

[0006] (2) Establish attribute normalization rules for the mapping model;

[0007] (3) Establish dynamic verification rules for model data during the conversion process;

[0008] (4) Complete data extraction, conversion and writing.

[0009] Furthermore, the step (1) includes:

[0010] First, determine the source table of the table required by the destination end. Second, determine whether the required domain of the destination table originates from the specified domain in the source table, and whether domain name conversion is required. Finally, construct table and domain mapping rules.

[0011] Furthermore, the step (1) includes:

[0012] Based on the heterogeneous characteristics of the source and target models, the mapping of tables and domains is completed, providing a configuration basis for conversion and data splicing based on heterogeneous models.

[0013] Furthermore, the attribute normalization rules in step (2) include:

[0014] Unit consistency conversion, name, date, case standardization conversion, numerical precision, attribute type data standardization conversion and attribute format conversion based on power business definition.

[0015] Furthermore, the step (3) includes:

[0016] Design dynamic verification rules to synchronously verify the legitimacy of source data during the conversion process and provide a legal power model for the destination.

[0017] Furthermore, the model data dynamic verification rules in step (3) include a numerical legitimacy verification operator, a numerical reasonable range verification operator, and a custom verification operator.

[0018] Furthermore, the step (4) includes:

[0019] First, according to the established table and field mapping rules, the required data is accurately extracted from the source database. Using the built transformation prototype, several fields in one or more tables are extracted from the source according to the definition.

[0020] Secondly, the attribute normalization structure is performed for the extracted domain.

[0021] Finally, when writing the data to the destination database, the data is dynamically verified synchronously to complete the conversion task from source data to heterogeneous data.

[0022] The heterogeneous model conversion system based on source data of the present invention includes:

[0023] Table and domain mapping module, used to establish table and domain mapping rules between the source and destination ends;

[0024] Normalization module, used to establish attribute normalization rules for mapping models;

[0025] Verification module, used to establish dynamic verification rules for model data during the conversion process;

[0026] The data processing module is used to complete data extraction, conversion and writing.

[0027] Furthermore, when the processor executes the program, a heterogeneous model conversion method based on source data is implemented.

[0028] Furthermore, the computer program is designed to implement a heterogeneous model conversion method based on source data during runtime.

[0029] Beneficial effects: Compared with the existing technology, the present invention has the following significant advantages: combining the heterogeneous characteristics of the models at the source and destination ends, establishing a process-based mechanism for heterogeneous model conversion, establishing a mapping relationship between tables and fields for heterogeneous models, and realizing consistent maintenance of heterogeneous model data, which greatly improves the efficiency and accuracy of conversion maintenance. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 is a flow chart of the present invention;

[0031] Figure 2 The following is a typical mapping diagram of source and destination table domains. DETAILED DESCRIPTION

[0032] The technical solution of the present invention will be further described below with reference to the accompanying drawings.

[0033] like Figure 1 As shown, the heterogeneous model conversion method based on source data of the present invention includes the following steps:

[0034] (1) Establish table and domain mapping rules between the source and destination ends;

[0035] Based on the heterogeneous characteristics of the source and target models, the table domain mapping is completed, providing a configuration basis for conversion and data splicing generation based on heterogeneous models. According to the user's analysis of the source and target model structures, the source table of the table required by the target end is first determined on the source side. Then, the required domain of the target table is disassembled from the specified domain in the source table on the source side, and whether domain name conversion is required is determined. In this way, table and domain mapping rules are constructed.

[0036] Taking the mapping rules of the transformer winding model as an example, the model is defined in two tables TRANSFMWD_B and TRANSFMWD_P on the source side, and its connection relationship is defined in the PWRGRID_TOPO table. On the target side, its domains and connection relationships are defined in the TRANSFWD table, and the several domains representing assets on the source side are not needed. Therefore, it is necessary to establish a many-to-one table mapping relationship between the source and the target side, and establish a corresponding relationship for the domains that need to be mapped on the source side, such as Figure 2 As shown. Establish a mapping relationship between the id, name, commissioning date, capacity, and operating status in TRANSFMWD_B and the id, name, commissioning date, capacity, and operating status in TRANSFWD. Note that in the source-side TRANSFMWD_B table, the operating status field is called running_state, while in the destination-side TRANSFWD, the operating status field is called operator_state. It is necessary to synchronously convert the field names, establish a mapping relationship between the operating status in TRANSFMWD_P and the operating status in TRANSFWD, and establish a mapping relationship between the head-end node and the terminal node in PWRGRID_TOPO and the head-end node and the terminal node in TRANSFWD.

[0037] (2) Establish attribute normalization rules for the mapping model;

[0038] For the completed conversion prototype, due to the heterogeneous data model and different data types and content definitions, it is necessary to define the normalized construction rules of its mapping attributes. The normalized construction rule definition table is shown in Table 1.

[0039] Table 1 Definition of attribute normalization rules for mapping models

[0040]

[0041]

[0042] Taking the mapping rules for a transformer winding model as an example, the commissioning date on the source side is formatted as "YYYY-MM-DD HH:MM:SS," while on the destination side, the date format is "YYYY-MM-DD" without a specific time. Therefore, the date format needs to be converted. Furthermore, the capacity field on the source side is defined as a floating-point number with 4 decimal places, while on the destination side it is defined as 6 decimal places. Therefore, the precision of the capacity field needs to be extended. From a business perspective, the encoding formats of the source and destination head and end nodes differ. Therefore, the data needs to be parsed based on the corresponding bits and reorganized into the destination head and end nodes.

[0043] (3) Establish dynamic verification rules for model data during the conversion process;

[0044] During the conversion process, dynamic validation verifies data that meets the target database's input format requirements but is not legal from a business perspective. Each designated conversion prototype can set its own targeted dynamic validation rules. These rules can pre-set some common validation rules as validation operators, such as phone number rules and latitude and longitude rules. Users can also define new rules using custom regular expressions and set them as validation operators for use in other conversion prototypes. Table 2 shows the rules for defining dynamic validation operators.

[0045] Table 2 Dynamic verification operator rule definition table

[0046]

[0047]

[0048] For example, a piece of electric power equipment belongs to a power plant or substation, but its latitude and longitude attributes are different from the power plant or substation to which it belongs, or the emergency contact field should be an 11-digit mobile phone number, but the stored phone number does not comply with the rules.

[0049] (4) Complete data extraction, conversion and writing.

[0050] According to the established table and domain mapping rules, the required data is accurately extracted from the source database. Using the conversion prototype constructed by the above steps, several fields in one or more tables are first extracted from the source according to the definition. Then, the attributes of these fields are normalized so that their data meets the data requirements of the destination database. Finally, when writing the data to the destination database, the data is dynamically verified synchronously to complete the conversion task from source data to heterogeneous data.

[0051] The heterogeneous model conversion system based on source data described in the present invention includes: a table and domain mapping module for establishing table and domain mapping rules for the source and destination ends; a normalization module for establishing attribute normalization rules for the mapping model; a verification module for establishing dynamic verification rules for model data during the conversion process; and a data processing module for completing data extraction, conversion, and writing.

[0052] In addition, based on the same inventive concept, an embodiment of the present invention also provides an electronic device, including a memory, a processor, and a program stored in the memory and runnable on the processor, wherein when the processor executes the program, the steps of the heterogeneous model conversion method based on source data are implemented.

[0053] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0054] In addition, based on the same inventive concept, an embodiment of the present invention further provides a storage medium storing a computer program, wherein the computer program is designed to implement the steps of the source data-based heterogeneous model conversion method when running.

[0055] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0056] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0057] The computer-readable storage medium mentioned above may be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0058] The above specific description further illustrates the purpose, technical solutions and beneficial effects of the invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A heterogeneous model conversion method based on source data, characterized in that: The steps include: (1) Establish table and domain mapping rules for the source and destination ends; (2) Establish attribute normalization rules for the mapping model; (3) Establish dynamic verification rules for model data during the conversion process; (4) Complete data extraction, conversion and writing.

2. The heterogeneous model conversion method based on source data according to claim 1 is characterized in that: The step (1) comprises: First, determine the source table of the table required by the destination end on the source end; second, disassemble the required domain of the destination table from the specified domain in the source table on the source end, and whether domain name conversion is required; finally, construct table and domain mapping rules.

3. The heterogeneous model conversion method based on source data according to claim 1 is characterized in that: The step (1) comprises: Based on the heterogeneous characteristics of the source and target models, the mapping of tables and domains is completed, providing a configuration basis for conversion and data splicing based on heterogeneous models.

4. The heterogeneous model conversion method based on source data according to claim 1 is characterized in that: The attribute normalization rules in step (2) include: Unit consistency conversion, name, date, case standardization conversion, numerical precision, attribute type data standardization conversion and attribute format conversion based on power business definition.

5. The heterogeneous model conversion method based on source data according to claim 1 is characterized in that: The step (3) comprises: Design dynamic verification rules to synchronously verify the legitimacy of source data during the conversion process and provide a legal power model for the destination.

6. The heterogeneous model conversion method based on source data according to claim 1 is characterized in that: The model data dynamic verification rules in step (3) include a numerical legitimacy verification operator, a numerical reasonable range verification operator, and a custom verification operator.

7. The heterogeneous model conversion method based on source data according to claim 1 is characterized in that: The step (4) comprises: First, according to the established table and domain mapping rules, the required data is accurately extracted from the source database, and several domains in one or more tables are extracted from the source end according to the definition using the constructed transformation prototype; Secondly, attribute normalization is performed on the extracted domain. Finally, when writing the data to the destination database, the data is dynamically verified synchronously to complete the conversion task from source data to heterogeneous data.

8. A heterogeneous model conversion system based on source data, characterized in that: include: Table and domain mapping module, used to establish table and domain mapping rules for source and destination ends; Normalization module, used to establish attribute normalization rules for mapping models; Verification module, used to establish dynamic verification rules for model data during conversion; The data processing module is used to complete data extraction, conversion and writing.

9. An electronic device comprising a memory, a processor, and a program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the heterogeneous model conversion method based on source data is implemented.

10. A storage medium storing a computer program, characterized in that: The computer program is designed to implement the source data-based heterogeneous model conversion method according to any one of claims 1 to 7 when running.

Images