Data synchronization method, apparatus, device, and product

Abstract

This application discloses a data synchronization method, apparatus, device, and product, relating to the field of data processing technology. The method is applied to a data management end and includes: receiving multi-source heterogeneous data sent by a power generation enterprise; converting the multi-source heterogeneous data into standard structured data; determining the synchronization mode of the standard structured data through a pre-built dynamic switching engine, wherein the dynamic switching engine is pre-configured with priority determination rules and a dual-mode synchronization strategy; and writing the standard structured data into a corresponding data warehouse according to the synchronization strategy. This solves the problems of delayed synchronization, inconsistent access, unreasonable storage architecture, and poor adaptability of multi-source heterogeneous data in power generation enterprises, thereby improving the efficiency of data synchronization.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data synchronization method, apparatus, device, and product. Background Technology

[0002] In the current process of digital management in power generation enterprises, various business systems such as production operation, dispatch management, mobile collaboration, and operation and maintenance logs have been widely deployed. These systems are developed by different vendors, and there are significant differences in data formats, storage structures, and communication protocols, resulting in a multi-source heterogeneous data pattern. Data synchronization in the industry generally adopts a batch timed synchronization mode, which has a long synchronization cycle and high data latency, making it difficult to support near real-time decision-making scenarios such as real-time monitoring of units, dynamic load dispatching, and rapid fault early warning. At the same time, existing general data synchronization solutions have not customized dedicated access architectures for the high reliability, high security, and strong real-time business characteristics of power generation enterprises. Data synchronization processes are prone to interruptions and packet loss, and the stability and data security protection capabilities are difficult to meet the stringent requirements of the energy industry.

[0003] Existing data synchronization technologies for power generation companies have significant shortcomings: First, the synchronization mode is singular, relying solely on batch timed synchronization, which cannot achieve near real-time and real-time hierarchical synchronization. Data lag directly affects the timeliness and accuracy of production and operation decisions. Second, a unified multi-type business data access architecture has not been built, with inconsistent access interface standards across systems, resulting in high adaptation and development costs and hindering the integrated aggregation of multi-source heterogeneous data. Third, the data storage architecture is not designed in accordance with the data characteristics of power generation companies, leading to low data access efficiency, a lack of security control mechanisms, and poor compatibility with domestically produced digital infrastructure, thus restricting the company's transformation towards information technology innovation and data value mining.

[0004] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0005] The main purpose of this application is to provide a data synchronization method, device, equipment and product, which aims to solve the technical problems of data synchronization lag, inconsistent access, unreasonable storage architecture and poor adaptability of power generation enterprises.

[0006] To achieve the above objectives, this application proposes a data synchronization method, which is applied to a data management terminal and includes: Receive multi-source heterogeneous data sent by power generation companies and convert the multi-source heterogeneous data into standard structured data; The synchronization mode of standard structured data is determined by a pre-built dynamic switching engine, which is pre-configured with priority determination rules and dual-mode synchronization strategies. The standard structured data is written into the corresponding data warehouse according to the synchronization strategy.

[0007] In one embodiment, the step of converting the multi-source heterogeneous data into standard structured data includes: Receive multi-source heterogeneous data generated by the power generation enterprise during its production operation, dispatch management, and business analysis; The multi-source heterogeneous data is subjected to protocol identification to obtain the data conversion protocol, and the multi-source heterogeneous data is subjected to format parsing and source marking to obtain the system to which the data belongs and the data type; Based on the interface specifications of the data management terminal, the system to which the data belongs, and the data type, the multi-source heterogeneous data is converted into standard structured data.

[0008] In one embodiment, prior to the step of determining the synchronization mode of standard structured data using a pre-built dynamic switching engine, the method further includes: A dual-link data synchronization channel with a real-time synchronization channel and a near-real-time synchronization channel is established in the enterprise service bus of the power generation enterprise. Data priority determination rules are set based on the business attributes of the power generation enterprises, and the determination criteria for high-priority real-time data and medium-priority near-real-time data are divided. Based on the judgment criteria, a real-time incremental capture strategy and a timed incremental capture strategy are configured in the enterprise service bus to obtain a dual-mode synchronization strategy. The dual-mode synchronization strategy, priority determination rules, and enterprise service bus are integrated to obtain a dynamic switching engine.

[0009] In one embodiment, the step of determining the synchronization mode of standard structured data through a pre-built dynamic switching engine includes: The dynamic switching engine reads data information from standard structured data. Based on the data information, and through the priority determination rules, the standard structured data is determined to be either high-priority real-time data or medium-priority near-real-time data, and a determination result is obtained. Based on the judgment result, the standard structured data is allocated to the synchronization channel corresponding to the enterprise service bus to obtain the allocation result; Based on the allocation results and the dual-mode synchronization strategy, the synchronization mode corresponding to the standard structured data is determined.

[0010] In one embodiment, the step of writing the standard structured data into the corresponding data warehouse according to the synchronization strategy includes: When the synchronization strategy is in real-time synchronization mode, incremental change information of the standard structured data is monitored through the Change Data Capture (CDC) mechanism. The incremental change information is streamed through the real-time synchronization channel of the enterprise service bus. The incremental changes to the streaming data are written to the corresponding relational database, and the data synchronization point and transmission status are updated.

[0011] In one embodiment, the step of writing the standard structured data into the corresponding data warehouse according to the synchronization strategy further includes: When the synchronization strategy is in near real-time synchronization mode, the initial data synchronization point is determined according to the configurable time window. Incremental data of standard structured data is extracted based on the initial synchronization point and then batch-packaged and transmitted through the near real-time synchronization channel of the enterprise service bus. The incremental data that is packaged and transmitted in batches is written into the corresponding relational database, and the initial data synchronization point and task timestamp are updated.

[0012] In one embodiment, after the step of writing the standard structured data into the corresponding data warehouse according to the synchronization strategy, the method further includes: The newly added data in the data warehouse is subjected to consistency verification using the standard structured data to obtain the consistency verification result; If the consistency verification result is successful, all data will be stored in the persistent data warehouse according to the hierarchical storage rules, and high-frequency business data will be stored in the business data mart.

[0013] Furthermore, to achieve the above objectives, this application also proposes a data synchronization device, wherein the data synchronization is applied to a data management terminal, and the data synchronization device includes: The receiving module is used to receive multi-source heterogeneous data sent by power generation companies and convert the multi-source heterogeneous data into standard structured data. The determination module is used to determine the synchronization mode of standard structured data through a pre-built dynamic switching engine, wherein the dynamic switching engine is pre-configured with priority determination rules and dual-mode synchronization strategies. The writing module is used to write the standard structured data into the corresponding data warehouse according to the synchronization strategy.

[0014] In addition, to achieve the above objectives, this application also proposes a data synchronization device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the data synchronization method as described above.

[0015] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and which, when executed by a processor, implements the steps of the data synchronization method described above.

[0016] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the data synchronization method described above.

[0017] One or more technical solutions proposed in this application have at least the following technical effects: This application proposes a data synchronization method, apparatus, device, and product. The method, applied to a data management terminal, receives multi-source heterogeneous data from power generation companies and converts it into standard structured data. A pre-built dynamic switching engine determines the synchronization mode of the standard structured data, pre-configured with priority rules and a dual-mode synchronization strategy. The standard structured data is then written to the corresponding data warehouse according to the synchronization strategy. This approach solves the problem of inconsistent access by uniformly converting multi-source heterogeneous data into standard structured data. Real-time / near-real-time dual-mode synchronization is then achieved using the dynamic switching engine, matching the data synchronization rhythm as needed to resolve synchronization lag issues. Finally, the data is written to the corresponding data warehouse according to the strategy, adapting to hierarchical storage requirements and addressing problems of unreasonable storage architecture and poor adaptability. This improves the efficiency of data synchronization by resolving issues such as synchronization lag, inconsistent access, unreasonable storage architecture, and poor adaptability of multi-source heterogeneous data from power generation companies. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0019] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a flowchart illustrating an embodiment of the data synchronization method of this application. Figure 2 This is a schematic diagram illustrating the dynamic engine switching involved in the data synchronization method of this application; Figure 3 This is a flowchart illustrating Embodiment 2 of the data synchronization method of this application; Figure 4 This is a schematic diagram illustrating the synchronization strategy involved in the data synchronization method of this application; Figure 5 This is a schematic diagram of the module structure of the data synchronization device according to an embodiment of this application; Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the data synchronization method in this application embodiment.

[0021] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0022] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0023] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0024] The main solution of this application embodiment is as follows: receiving multi-source heterogeneous data generated during the production operation, scheduling management, and business analysis of the power generation enterprise; identifying the protocol of the multi-source heterogeneous data to obtain a data conversion protocol; parsing the format and marking the source of the multi-source heterogeneous data to obtain the system and data type of the data; converting the multi-source heterogeneous data into standard structured data based on the interface specification of the data management terminal, the system and data type of the data. A dual-link data synchronization channel with real-time synchronization and near-real-time synchronization channels is established in the enterprise service bus of the power generation enterprise; data priority determination rules are set according to the business attributes of the power generation enterprise, and judgment criteria for high-priority real-time data and medium-priority near-real-time data are defined; according to the judgment criteria, a real-time incremental capture strategy and a timed incremental capture strategy are configured in the enterprise service bus to obtain a dual-mode synchronization strategy; the dual-mode synchronization strategy, priority determination rules, and enterprise service bus are correlated and integrated to obtain a dynamic switching engine. The system reads standard structured data information through the dynamic switching engine; based on the data information and the priority determination rules, it determines whether the standard structured data is high-priority real-time data or medium-priority near-real-time data, obtaining a determination result; based on the determination result, the standard structured data is allocated to the synchronization channel corresponding to the Enterprise Service Bus, obtaining an allocation result; based on the allocation result and the dual-mode synchronization strategy, the synchronization mode corresponding to the standard structured data is determined. When the synchronization strategy is real-time synchronization mode, incremental change information of the standard structured data is monitored through the Change Data Capture (CDC) mechanism; the incremental change information is streamed through the real-time synchronization channel of the Enterprise Service Bus; the streamed incremental change data is written to the corresponding relational database, and the data synchronization point and transmission status are updated. When the synchronization strategy is near-real-time synchronization mode, the initial data synchronization point is determined according to a configurable time window; incremental data of the standard structured data is extracted according to the initial synchronization point, and batch-packaged and transmitted through the near-real-time synchronization channel of the Enterprise Service Bus; the batch-packaged incremental data is written to the corresponding relational database, and the initial data synchronization point and task timestamp are updated. The newly added data in the data warehouse is subjected to consistency verification using the standard structured data to obtain the consistency verification result. If the consistency verification result is successful, the full amount of data is stored in the persistent data warehouse according to the hierarchical storage rules, and the high-frequency business data is stored in the business data mart. This solves the problems of delayed synchronization of multi-source heterogeneous data, inconsistent access, unreasonable storage architecture and poor adaptability in power generation enterprises, realizes data synchronization, and improves the accuracy of data synchronization.Based on the present invention, this invention addresses the problem that existing general data synchronization solutions do not have customized access architectures for the high reliability, high security, and strong real-time business characteristics of power generation enterprises. This leads to issues such as data synchronization being prone to interruptions and packet loss, and the stability and data security protection capabilities failing to meet the stringent requirements of the energy industry, resulting in low efficiency. The invention proposes a data synchronization method, and verifies its effectiveness during data synchronization. Finally, the efficiency of data synchronization using the present invention is significantly improved.

[0025] In this embodiment, for ease of description, the data synchronization device will be used as the execution subject in the following description.

[0026] Due to limitations in existing data synchronization technologies and architectures in power generation companies, the efficiency and stability of data synchronization need improvement. One issue is the problem of multi-source heterogeneity, where data formats, storage structures, and communication protocols vary significantly across different business systems. The lack of a unified access architecture and high adaptation development costs lead to decreased data synchronization efficiency. Another issue is the synchronization mode; the industry commonly uses batch timed synchronization, which cannot achieve hierarchical synchronization, resulting in significant data lag. If it doesn't meet the company's strong real-time requirements, data synchronization efficiency will also decrease significantly. Furthermore, there's the storage architecture issue. Accurate data synchronization requires storage designs tailored to the company's characteristics, but the existing storage architectures of power generation companies do not incorporate their own data characteristics, lack security controls, and have poor adaptability, which also affects data synchronization efficiency. Therefore, current digital management in power generation companies faces challenges in data synchronization adaptability. Given the high reliability, high security, and strong real-time business characteristics of power generation companies, as well as their need for domestic transformation, if the data synchronization solution lacks targeted architecture design and mode optimization, efficiency and stability will also decline.

[0027] This application provides a solution that solves the problem of inconsistent access by uniformly converting multi-source heterogeneous data into standard structured data. Then, it relies on a dynamic switching engine to achieve real-time / near real-time dual-mode synchronization, matching the data synchronization rhythm as needed to solve the problem of synchronization lag. Finally, the data is written to the corresponding data warehouse according to the strategy to adapt to the hierarchical storage requirements and solve the problems of unreasonable storage architecture and poor adaptability. This solves the problems of synchronization lag, inconsistent access, unreasonable storage architecture and poor adaptability of multi-source heterogeneous data in power generation enterprises, and improves the efficiency of data synchronization.

[0028] Based on this, embodiments of this application provide a data synchronization method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the data synchronization method of this application.

[0029] In this embodiment, the data synchronization method is applied to the data management terminal, and the method includes steps S01~S03: Step S01: Receive multi-source heterogeneous data sent by power generation companies and convert the multi-source heterogeneous data into standard structured data; Before the implementation of this embodiment, it should be clear that the current multi-business systems of power generation enterprises have multi-source heterogeneous data due to different manufacturers. Existing data synchronization mostly adopts batch timed mode, which has high latency and cannot support near real-time decision-making. The general synchronization solution is not adapted to the high reliability, high security and strong real-time requirements of power generation enterprises, and is prone to interruption and packet loss problems.

[0030] Therefore, to address the aforementioned issues, this embodiment first identifies the source and parses the protocol of the received multi-source heterogeneous data. For different data sources commonly found in power generation companies, such as real-time telemetry data from SCADA systems, process control data from DCS systems, business management data from ERP systems, and power prediction data from new energy power plants, the corresponding protocol adapters are invoked to extract the data. During data extraction, a metadata-driven dynamic mapping mechanism is employed to automatically identify the field semantics, data types, and encoding formats of each data source, eliminating data misunderstandings caused by differences in vendor proprietary protocols. Subsequently, the extracted raw data undergoes quality verification using preset data cleaning rules, including checking the reasonableness of numerical ranges, verifying the continuity of timestamps, and verifying the integrity of key fields. Data streams with abnormal verification are triggered with alarms and entered into a fault-tolerant cache queue to prevent dirty data from contaminating subsequent processing stages. After cleaning, the data undergoes structured transformation according to a unified standard data model. This standard data model encompasses a unified coding system for power generation equipment, standardized naming conventions for measurement data, and precision alignment rules for time-series data, ensuring semantic consistency of data from different business systems and forming a standard structured data stream suitable for subsequent processing. This provides a unified data input foundation for the dual-mode synchronization of the dynamic switching engine.

[0031] Step S02: The synchronization mode of standard structured data is determined by a pre-built dynamic switching engine, which is pre-configured with priority determination rules and dual-mode synchronization strategies. As the core decision-making component of the data synchronization architecture in this embodiment, the dynamic switching engine needs to take into account multiple factors such as the real-time level of the power generation company's data business, network transmission environment fluctuations, and target system load status during its construction process.

[0032] During the engine initialization phase, a data hierarchy system is first established based on typical business scenarios of power generation enterprises. Based on this hierarchy, priority determination rules are embedded in the dynamic switching engine. These rules use the business type identifier, time sensitivity coefficient, and data source confidence score from the data tags as input parameters, and output a real-time priority score through weighted calculation. The time sensitivity coefficient is a quantitative indicator measuring the urgency of data transmission timeliness requirements, ranging from 0.1 to 1.0. Its acquisition process combines typical business scenarios of power generation enterprises, first identifying core scenarios such as real-time unit monitoring, equipment emergency alarms, daily operation and maintenance, business analysis, and data archiving, and setting basic coefficients for different scenarios. The basic coefficients for core scenarios such as real-time unit monitoring and equipment emergency alarms are 0.9-1.0; for important scenarios such as daily operation and maintenance and dispatch instructions, the basic coefficients are 0.6-0.8; and for non-real-time scenarios such as business analysis and data archiving, the basic coefficients are 0.1-0.5. Then, the basic coefficients are fine-tuned based on the "time requirement field" marked in the data tags. The coefficient for "real-time transmission" is increased by 0.1, and the coefficient for "acceptable delay ≤30" is increased by 0.5. The coefficient for "seconds" remains unchanged, while the coefficient for "non-real-time archiving" is reduced by 0.1-0.2. The final determined value is the time sensitivity coefficient of the data. Data tags are entered synchronously for dynamic switching engine calls. The data source confidence score is a quantitative score that measures the reliability and accuracy of the data source, with a value range of 0-100. Its acquisition process is divided into two core dimensions: first, the data source type score, accounting for 40%, of which the data source of the group's unified system (finance, human resources, group management, etc.) scores 85-100 points, the data source of the regional self-built core system (production management, equipment monitoring, etc.) scores 70-84 points, and the data source of third-party interfaces, manual entry, etc. scores 40-69 points; second, the data governance quality score, accounting for 60%, is the data governance result based on the full-stack digital intelligence foundation, scored from three dimensions: data integrity, accuracy, and consistency. The average of the three is multiplied by 60%. The final data source confidence score is calculated by "data source type score * 40% + data governance quality score * 60%". The score result is bound to the data tag and serves as the core input parameter for priority determination.

[0033] It should be noted that the weighted calculation in this embodiment requires the first quantification and unification of the three input parameters to ensure consistent calculation methods. The business type identifier is quantified and assigned a value according to its core importance: 100 points for unit safety data, 80 points for operation and maintenance scheduling data, 60 points for business analysis data, and 40 points for archived data. The time sensitivity coefficient retains its original value (0.1-1.0) and is converted into a quantified score (coefficient * 100). The data source confidence score retains its original score (0-100). Then, based on the business priority requirements of the power generation enterprise, a reasonable weighting is set, where the business type... The weighting is 45% for the identification quantification score, 35% for the time sensitivity coefficient quantification score, and 20% for the data source confidence score, totaling 100%. The real-time priority score is calculated using a weighted summation formula: Real-time priority score = (Business type identification quantification score * 45%) + (Time sensitivity coefficient * 100 * 35%) + (Data source confidence score * 20%). The calculated score (0-100) is the real-time priority score. The dynamic switching engine determines the data priority based on this score and then matches the corresponding synchronization mode, ensuring the quantification, controllability, and accuracy of priority determination.

[0034] The dual-mode synchronization strategy includes real-time synchronization mode (CDC mechanism) and near-real-time synchronization mode (time window mechanism). Real-time synchronization mode is based on the Change Data Capture (CDC) mechanism, which continuously monitors the transaction logs or incremental identifiers of the data source and immediately triggers the synchronization process when a data change event is detected. It is suitable for millisecond-level latency-sensitive services such as scheduling instructions and protection action signals. Near-real-time synchronization mode adopts the time window mechanism, which aggregates change data in batches according to a preset period (such as seconds or minutes) and completes batch transmission based on the synchronization point. It is suitable for business scenarios that can tolerate second-level to minute-level latency, such as equipment status statistics and energy consumption analysis.

[0035] Step S03: Write the standard structured data into the corresponding data warehouse according to the synchronization strategy.

[0036] This embodiment analyzes the target warehouse identifier and write mode parameters in the synchronization strategy configuration. For real-time synchronization mode, a streaming write channel is used to directly push standard structured data to the in-memory database or hot data layer. During the write process, the data version timestamp and synchronization point marker are updated synchronously to ensure that downstream applications can detect data changes in real time. For near real-time synchronization mode, standard structured data is first temporarily stored in a buffer queue. When the preset time window threshold or batch accumulation threshold is reached, a batch write operation is triggered to load the data to the distributed file system or cold data layer. After the write is completed, the metadata index and synchronization point record are updated in batches. In addition, the physical storage structure of the data warehouse is partitioned according to the aforementioned data classification system. High-priority data is allocated to high-performance storage media and configured with multiple replicas for redundancy. Low-priority data is compressed and encoded and lifecycle management strategies are used to optimize storage costs. Ultimately, differentiated data assets are collected in an orderly manner and accessed efficiently under a unified warehouse architecture.

[0037] Specifically, step S01 above, which involves converting the multi-source heterogeneous data into standard structured data, includes: Step S011: Receive multi-source heterogeneous data generated during the production operation, dispatch management, and business analysis of the power generation enterprise; Step S012: Perform protocol identification on the multi-source heterogeneous data to obtain the data conversion protocol, perform format parsing and source marking on the multi-source heterogeneous data to obtain the system to which the data belongs and the data type; Step S013: Based on the interface specifications of the data management terminal, the system to which the data belongs, and the data type, the multi-source heterogeneous data is converted into standard structured data.

[0038] To convert multi-source heterogeneous data from power generation enterprises into standard structured data, this embodiment first establishes stable physical connections for various business systems of power generation enterprises, such as production operation, dispatch management, mobile collaboration, operation and maintenance logs, equipment alarms, and business analysis. It comprehensively receives multi-source heterogeneous data generated by each system, covering multiple protocol formats such as Modbus, OPC UA, MQTT, HTTP, and JDBC, as well as different types of data such as structured, semi-structured, and unstructured data.

[0039] Subsequently, the received multi-source heterogeneous data is individually protocol-identified, accurately matching the corresponding data conversion protocol. Simultaneously, the data format is parsed to clarify the meaning of each field and data encoding rules, and data source marking is completed to clearly distinguish the business system and specific data type of each data entry, ensuring data traceability. Next, based on the unified interface specifications preset by the data management terminal, and combined with the determined data system and data type, the parsed multi-source heterogeneous data is cleaned and normalized, filtering outliers, filling in missing fields, and unifying the data format. This transforms heterogeneous data from different systems and formats into standardized structured data, suitable for... To meet the needs of subsequent synchronous transmission and storage, while ensuring data integrity and accuracy, this embodiment lays the foundation for subsequent synchronization processes. Furthermore, it adds a format adaptation module compatible with domestically produced intelligent data platforms. For domestic databases commonly used by power generation companies, it pre-sets exclusive adaptation rules during data conversion to ensure that the converted standard structured data can be directly connected to the domestically produced intelligent data platforms without additional secondary format conversion. This addresses the poor compatibility between existing solutions and domestically produced platforms. Simultaneously, it embeds data anonymization steps to perform hierarchical anonymization of core unit parameters and sensitive operation and maintenance data, ensuring data security without affecting subsequent data use.

[0040] To illustrate the data processing process more intuitively, this embodiment uses production operation data as an example: Specifically, production operation data refers to the real-time unit operation data collected by the power generation company's DCS system. This data is transmitted using the Modbus protocol and is semi-structured, containing core parameters such as unit load, main steam pressure, feedwater temperature, and furnace temperature. The data format is a non-standard, custom format with inconsistent field naming. After clarifying this information, the system prioritizes connecting to the DCS production operation system to establish a stable physical connection. It then receives the raw unit operation data pushed by this system in real time. Protocol identification is performed on this raw data to determine the appropriate Modbus protocol conversion protocol. Simultaneously, the data format is parsed to clarify the specific meaning and data encoding rules of each field, such as unit load and main steam pressure. Data source marking is also completed, confirming that the data originates from the DCS production operation system and is of the type of real-time unit operation data. Furthermore, an additional data redundancy backup mark is added for this real-time unit operation data. During the conversion process, a data checksum is generated to bind the correspondence between the raw data and the converted data. Subsequent synchronization and storage processes can use the checksum to quickly verify data integrity and prevent data distortion during conversion.

[0041] Subsequently, based on the unified interface specifications preset by the data management terminal, and combined with the DCS production operation system to which the data belongs and the real-time operation data type of the unit, the parsed raw data is cleaned, abnormal fluctuation values ​​of furnace temperature are filtered, missing fields of feedwater temperature are supplemented, parameters such as unit load and main steam pressure are normalized, and field naming and data format are unified. The original non-standard semi-structured Modbus protocol data is uniformly converted into standard structured data containing standard fields such as data source, data type, acquisition time, unit load, and main steam pressure, ensuring that the production operation data can be directly used for subsequent synchronous transmission and storage.

[0042] More specifically, prior to step S02 above, which determines the synchronization mode of standard structured data using a pre-built dynamic switching engine, the method further includes: Step S0201: Establish a dual-link data synchronization channel with a real-time synchronization channel and a near-real-time synchronization channel in the enterprise service bus of the power generation enterprise; Step S0202: Set data priority determination rules based on the business attributes of the power generation enterprise, and divide the determination criteria for high-priority real-time data and medium-priority near-real-time data. Step S0203: Based on the judgment criteria, configure a real-time incremental capture strategy and a timed incremental capture strategy in the enterprise service bus to obtain a dual-mode synchronization strategy. Step S0204: The dual-mode synchronization strategy, priority determination rules, and enterprise service bus are associated and integrated to obtain a dynamic switching engine.

[0043] Based on the existing Enterprise Service Bus (ESB) architecture of the power generation company, two independent and dynamically switchable dual-link data synchronization channels are first established. The real-time synchronization channel is used to carry high-speed transmission of high-priority data, while the near-real-time synchronization channel is used to carry batch transmission of medium-priority data. This avoids data of different priorities competing for transmission resources and prevents data synchronization interruption caused by a single channel failure.

[0044] Subsequently, based on the core business attributes of power generation enterprises, such as production, dispatch, operation and maintenance, and management, the transmission timeliness requirements of various business data were sorted out, exclusive data priority judgment rules were set, and the judgment criteria for high-priority real-time data (such as DCS / SCADA system production real-time data and equipment alarm data) and medium-priority near-real-time data (such as operation and maintenance logs, dispatch instructions, and business analysis data) were clearly defined to ensure that the priority division fits the actual business scenarios of enterprises.

[0045] Based on this criterion, two synchronization strategies are configured in the enterprise service bus. The real-time incremental capture strategy is built on the CDC change data capture mechanism and is used for second-level synchronization of high-priority data. The timed incremental capture strategy is built on a configurable time window and is used for batch synchronization of medium-priority data, forming a dual-mode synchronization strategy that adapts to data of different priorities.

[0046] The configured dual-mode synchronization strategy and priority determination rules are deeply integrated with the dual-link channels of the enterprise service bus to build a dynamic switching engine with automatic determination and dynamic switching functions. This enables intelligent matching of synchronization mode and data priority, and the switching of synchronization channels and strategies can be completed without manual intervention. On this basis, this embodiment also adds a load adaptive adjustment module to the dynamic switching engine. It collects indicators such as bandwidth utilization and data transmission latency of the two synchronization channels in real time. When the load of a certain channel exceeds a preset threshold, it automatically switches some non-core high-priority data to another idle channel. After the load is restored, it automatically switches back to the original channel. This adaptive adjustment mechanism solves the synchronization latency problem caused by unbalanced channel load in the existing solution. At the same time, a domestic algorithm module is embedded in the engine to replace the traditional open source switching algorithm, improving the compatibility of the engine with the domestic digital intelligence base.

[0047] Ultimately, one can obtain the following: Figure 2 The dynamic switching engine shown is built on the enterprise service bus and integrates dual-link synchronization channels, priority weighting judgment rules, dual-mode synchronization strategies and load adaptive adjustment modules. It can automatically complete the priority quantification and scoring of multi-source standardized data, synchronization channel matching and dynamic switching of synchronization mode. At the same time, it is compatible with the domestic digital intelligence base operating environment and effectively solves the technical defects of traditional data synchronization architecture such as uneven channel load, fixed mode and poor adaptability.

[0048] Furthermore, step S02 above, which involves determining the synchronization mode of standard structured data using a pre-built dynamic switching engine, includes: Step S021: Read the data information of standard structured data through the dynamic switching engine; Step S022: Based on the data information, and through the priority determination rule, determine whether the standard structured data is high-priority real-time data or medium-priority near-real-time data, and obtain the determination result; Step S023: Based on the judgment result, allocate the standard structured data to the synchronization channel corresponding to the enterprise service bus to obtain the allocation result; Step S024: Based on the allocation result and the dual-mode synchronization strategy, determine the synchronization mode corresponding to the standard structured data.

[0049] After multi-source heterogeneous data is converted into standard structured data, the dynamic switching engine reads the data information of the standard structured data. The data information includes core parameters such as the business domain to which the data belongs, data type, transmission timeliness requirements, and data volume, providing a comprehensive basis for determining the synchronization mode.

[0050] The retrieved data is then compared one by one with pre-set priority determination rules to extract the corresponding business attributes and timeliness requirements. This accurately determines whether the standard structured data is high-priority real-time data or medium-priority near-real-time data, generating a clear judgment result to ensure accuracy. Specifically, the priority determination rules are implemented using weighted calculations. Guided by the core business needs of power generation companies, three judgment dimensions are set and assigned corresponding weights: transmission timeliness requirement (40%), business impact degree (35%), and data update frequency (25%). A weighted summation is used to calculate the data priority score (out of 100 points). The specific calculation method and judgment criteria are as follows: First, each dimension parameter is quantified and assigned a value: transmission timeliness requirement ≤ 1 second is assigned 100 points, 1-30 seconds is assigned 70 points, and the business impact degree is "core" (e.g., affecting unit operation safety, equipment alarms). Data is assigned a score of 100 for "important" and 70 for "important" (such as daily maintenance and scheduling instructions). Data with an update frequency of ≥1 time / second is assigned 100 points, and data with an update frequency of 1-1 time / minute is assigned 70 points. Then, the total priority score is calculated according to the weight of each dimension, i.e., priority score = (transmission timeliness score × 40%) + (business impact score × 35%) + (data update frequency score × 25%). When the priority score is ≥90, it is judged as high-priority real-time data; when the priority score is between 70 and 89, it is judged as medium-priority near-real-time data. At the same time, an exception rule is set. When the data involves special scenarios such as emergency shutdown of the unit or major fault alarm, it is automatically judged as high-priority real-time data regardless of the weighted calculation score. This ensures that core emergency data is synchronized first. The weighted calculation realizes the quantitative controllability of priority determination and improves the accuracy and objectivity of the determination results.

[0051] Based on the judgment results, the dynamic switching engine automatically allocates high-priority real-time data to the real-time synchronization channel of the enterprise service bus, and allocates medium-priority near-real-time data to the near-real-time synchronization channel, so as to achieve accurate matching between data and synchronization channels and avoid link congestion.

[0052] After allocation, the pre-configured dual-mode synchronization strategy in the enterprise service bus is invoked to match the real-time incremental capture strategy for the real-time synchronization channel and the timed incremental capture strategy for the quasi-real-time synchronization channel. Finally, the synchronization mode corresponding to each standard structured data is determined. At the same time, the data load and link status are dynamically monitored to support flexible adjustment of the synchronization mode during operation. In addition, this embodiment also adds a priority dynamic adjustment mechanism, which allows the engine to automatically adjust the priority judgment threshold of various types of data according to changes in the business scenario of the power generation enterprise (such as unit maintenance, load peak). For example, during the unit load peak, the dispatch instruction data is upgraded to high priority and the real-time synchronization mode is enabled to ensure that the dispatch instructions are quickly synchronized, thereby breaking through the limitation of the fixed priority of the existing solution and further improving the flexibility and adaptability of the solution. At the same time, the priority adjustment log is recorded for easy follow-up and optimization.

[0053] This embodiment, through the above-described scheme, specifically receives multi-source heterogeneous data sent by power generation enterprises and converts it into standard structured data. A pre-built dynamic switching engine determines the synchronization mode of the standard structured data; this engine is pre-configured with priority determination rules and a dual-mode synchronization strategy. The standard structured data is then written to the corresponding data warehouse according to the synchronization strategy. Thus, by uniformly converting multi-source heterogeneous data into standard structured data, the problem of inconsistent access is solved. Subsequently, relying on the dynamic switching engine, real-time / near-real-time dual-mode synchronization is achieved, matching the data synchronization rhythm as needed to solve the synchronization lag problem. Finally, the data is written to the corresponding data warehouse according to the strategy, adapting to hierarchical storage requirements and solving the problems of unreasonable storage architecture and poor adaptability. This addresses the issues of synchronization lag, inconsistent access, unreasonable storage architecture, and poor adaptability of multi-source heterogeneous data from power generation enterprises, thereby improving data synchronization efficiency.

[0054] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3 In step S03, the step of writing the standard structured data into the corresponding data warehouse according to the synchronization strategy, the data synchronization method further includes steps S031 to S033: Step S031: When the synchronization strategy is in real-time synchronization mode, the incremental change information of the standard structured data is monitored through the Change Data Capture (CDC) mechanism. Step S032: Stream the incremental change information through the real-time synchronization channel of the enterprise service bus; Step S033: Write the incremental change data of the streaming transmission into the corresponding relational database, and update the data synchronization point and transmission status.

[0055] When the synchronization strategy is in real-time synchronization mode, the Change Data Capture (CDC) mechanism is immediately activated. This mechanism does not require scanning the full amount of standard structured data, but only listens for incremental changes in the standard structured data in real time, including changes such as data addition, modification, and deletion. It accurately captures incremental data and significantly reduces the resource consumption of data transmission and processing.

[0056] The captured incremental change information is then streamed through the real-time synchronization channel of the enterprise service bus. The streaming method enables data to be pushed in seconds, ensuring that incremental change data can be quickly transmitted to the target storage node, thus meeting the high-speed synchronization needs of power generation companies for real-time production data and equipment alarm data.

[0057] After the incremental change data is transmitted in a streaming manner, it is written to the corresponding relational database in real time. This relational database serves as the underlying storage carrier of the data warehouse, adapting to the storage needs of power generation enterprises' structured data. At the same time, it updates the data synchronization point and transmission status in real time, recording the current synchronization progress and data transmission status in detail. This ensures that subsequent data synchronization can continue based on this point, avoiding data duplication or loss, and achieving traceability and controllability of the synchronization process. To avoid synchronizing abnormal data during real-time synchronization, this embodiment adds an abnormal data interception module to the CDC mechanism. This module identifies abnormal incremental data (such as data format errors or parameter exceeding limits) in real time during the capture process, immediately suspends the data transmission of that data, and triggers a partial retry mechanism without interrupting the entire synchronization link. If the retry fails, the abnormal information is recorded and pushed to the operation and maintenance end. This abnormal interception and partial retry mechanism solves the problem of link interruption caused by abnormal data during the existing CDC capture process, further improving the stability of real-time synchronization. In addition, the synchronization point adopts a dual backup mechanism, stored on the local node and the backup node respectively, to avoid the loss of synchronization point due to single point failure and ensure the reliability of interrupted transmission.

[0058] Specifically, step S03 above, the step of writing the standard structured data into the corresponding data warehouse according to the synchronization strategy, further includes: Step S034: When the synchronization strategy is in near real-time synchronization mode, the initial data synchronization point is determined according to the configurable time window; Step S035: Extract incremental data of standard structured data based on the initial synchronization point, and batch package and transmit it through the near real-time synchronization channel of the enterprise service bus. Step S036: Write the incremental data to be transmitted in batches into the corresponding relational database, and update the initial data synchronization point and task timestamp.

[0059] When the synchronization strategy is in near real-time synchronization mode, the initial data synchronization point for this near real-time synchronization is determined based on the time window pre-configured according to the business needs of the power generation enterprise. This initial synchronization point is set based on the termination point of the previous synchronization to ensure the continuity and accuracy of incremental data extraction.

[0060] Subsequently, based on the determined initial synchronization point, incremental data is accurately extracted from the standard structured data, and synchronized historical data is removed to avoid duplicate transmission of all data and reduce system resource consumption.

[0061] The extracted incremental data is batch packaged to compress data volume and optimize transmission efficiency. It is then transmitted in batches through the near real-time synchronization channel of the enterprise service bus, balancing synchronization efficiency and resource utilization.

[0062] After the incremental data transmitted in batches arrives at the target node, it is completely written into the corresponding relational database. The initial data synchronization point is updated synchronously to the termination point of this synchronization. At the same time, the task timestamp is recorded, and key information such as the completion time and data volume of this near real-time synchronization are marked in detail. This facilitates subsequent anomaly investigation, breakpoint resumption, and traceability management of synchronization tasks. In addition, the newly added time window in this embodiment is also equipped with an adaptive adjustment function, which can automatically adjust the size of the time window according to the incremental volume of near real-time data. When the incremental data volume is large, the time window is automatically shortened to avoid transmission delays caused by excessive data volume in a single transmission. When the incremental data volume is small, the time window is automatically extended to reduce transmission frequency and reduce resource consumption. This adaptive adjustment function breaks through the limitation of the fixed time window of the existing near real-time synchronization, further improving the efficiency and flexibility of near real-time synchronization. At the same time, a domestic encryption algorithm is used to encrypt data during the batch transmission process to ensure the security of data transmission and adapt to the data security management requirements of the energy industry.

[0063] In summary, the two synchronization strategies described above are as follows: Figure 4 As shown, the CDC real-time incremental synchronization strategy and the configurable time window quasi-real-time batch synchronization strategy configured in this application are respectively. The two synchronization strategies cooperate with each other and operate independently. They can intelligently adapt to the synchronization needs of different business data according to the data priority weighted scoring results. At the same time, they integrate optimization mechanisms such as anomaly interception, site backup, time window adaptation, and domestic encryption, which greatly improves the stability, timeliness and security of multi-source heterogeneous data synchronization in power generation enterprises.

[0064] More specifically, after step S03 above, which involves writing the standard structured data into the corresponding data warehouse according to the synchronization strategy, the method further includes: Step S04: Perform consistency verification on the newly added data in the data warehouse using the standard structured data to obtain the consistency verification result; Step S05: If the consistency verification result is passed, store all data into the persistent data warehouse and store high-frequency business data into the business data mart according to the hierarchical storage rules.

[0065] After writing the standard structured data into the corresponding data warehouse according to the synchronization strategy, the newly added synchronized data in the data warehouse is comprehensively checked for consistency based on the standard structured data of the source end. The method of combining hash check and integrity comparison is adopted to compare the content, format and field parameters of the source end and the target end data one by one, and to check for data missing, error, duplication and other problems, and generate clear consistency check results.

[0066] If the consistency verification result passes, storage operations are executed according to the preset hierarchical storage rules. Based on the hierarchical storage architecture built on the "Ruisi" platform, all standard structured data is stored in a persistent data warehouse for long-term archiving and traceability of enterprise data, meeting the large-scale data storage needs of power generation enterprises. Simultaneously, frequently accessed business data such as production monitoring and dispatch instructions from power generation enterprises are stored in corresponding business data marts, divided into dedicated business partitions, shortening data query response time and improving data access efficiency. During the storage process, data encryption, access control, and operation auditing mechanisms are simultaneously enabled to ensure data security, adapt to the domestically produced digital infrastructure, and provide feedback on the storage status to the synchronization monitoring module, completing a closed loop of the entire data synchronization process.

[0067] It should be noted that this embodiment adds an automatic error correction module during the consistency verification process. For minor data format deviations, missing fields, and other correctable errors, it automatically calls preset correction rules to perform the correction. After the correction, the verification is performed again without manual intervention, which improves the verification efficiency. For errors that cannot be corrected, a detailed error report is generated, which clarifies the error location, error type and handling suggestions. At the same time, a data recovery mechanism is triggered to resynchronize the error data.

[0068] Furthermore, a new data lifecycle management module has been added to the tiered storage process. This module automatically sets the storage period based on data type and access frequency, automatically archives and compresses historical data that exceeds the storage period, freeing up storage resources while ensuring that archived data can be accessed at any time. This lifecycle management function solves the problem of resource waste caused by data accumulation in the existing storage architecture, further improving the rationality and efficiency of the storage architecture.

[0069] This embodiment, through the above-described scheme, specifically by using a Change Data Capture (CDC) mechanism to monitor incremental changes in the standard structured data when the synchronization strategy is in real-time synchronization mode; streaming the incremental change information through the real-time synchronization channel of the enterprise service bus; writing the streaming incremental change data into the corresponding relational database; and updating the data synchronization point and transmission status. Thus, by uniformly converting multi-source heterogeneous data into standard structured data, the problem of inconsistent access is solved. Subsequently, relying on a dynamic switching engine, real-time / near-real-time dual-mode synchronization is achieved, matching the data synchronization rhythm as needed to solve the synchronization lag problem. Finally, the data is written into the corresponding data warehouse according to the strategy, adapting to hierarchical storage requirements and solving the problems of unreasonable storage architecture and poor adaptability. This solves the problems of synchronization lag, inconsistent access, unreasonable storage architecture, and poor adaptability of multi-source heterogeneous data in power generation enterprises, improving the efficiency of data synchronization.

[0070] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the data synchronization method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0071] This application also provides a data synchronization device, please refer to... Figure 5 The data synchronization device is applied to the data management terminal, and the data synchronization device includes: The receiving module 10 is used to receive multi-source heterogeneous data sent by the power generation company and convert the multi-source heterogeneous data into standard structured data. The determination module 20 is used to determine the synchronization mode of standard structured data through a pre-built dynamic switching engine, wherein the dynamic switching engine is pre-configured with priority determination rules and dual-mode synchronization strategies. The writing module 30 is used to write the standard structured data into the corresponding data warehouse according to the synchronization strategy.

[0072] The data synchronization device provided in this application, employing the data synchronization method described in the above embodiments, can solve the technical problems of data synchronization lag, inconsistent access, unreasonable storage architecture, and poor adaptability in power generation enterprises. Compared with the prior art, the beneficial effects of the data synchronization device provided in this application are the same as those of the data synchronization method provided in the above embodiments, and other technical features in the data synchronization device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0073] This application provides a data synchronization device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the data synchronization method in Embodiment 1 above.

[0074] The following is for reference. Figure 6 The diagram illustrates a structural schematic of a data synchronization device suitable for implementing embodiments of this application. The data synchronization device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The data synchronization device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0075] like Figure 6 As shown, the data synchronization device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in the read-only memory 1002 or a program loaded from the storage device 1003 into the random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the data synchronization device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the data synchronization device to communicate wirelessly or wiredly with other devices to exchange data. Although the figures show data synchronization devices with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0076] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0077] The data synchronization device provided in this application, employing the data synchronization method described in the above embodiments, can solve the technical problems of data synchronization lag, inconsistent access, unreasonable storage architecture, and poor adaptability in power generation enterprises. Compared with the prior art, the beneficial effects of the data synchronization device provided in this application are the same as those of the data synchronization method provided in the above embodiments, and other technical features of this data synchronization device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0078] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0079] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0080] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the data synchronization method described in the above embodiments.

[0081] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0082] The aforementioned computer-readable storage medium may be included in the data synchronization device; or it may exist independently and not assembled into the data synchronization device.

[0083] The aforementioned computer-readable storage medium carries one or more programs. When the aforementioned one or more programs are executed by the data synchronization device, the data synchronization device: receives multi-source heterogeneous data sent by the power generation enterprise and converts the multi-source heterogeneous data into standard structured data; determines the synchronization mode of the standard structured data through a pre-built dynamic switching engine, wherein the dynamic switching engine is pre-configured with priority determination rules and a dual-mode synchronization strategy; and writes the standard structured data into the corresponding data warehouse according to the synchronization strategy.

[0084] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0085] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0086] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0087] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described data synchronization method. This solves the technical problems of delayed synchronization of multi-source heterogeneous data in power generation enterprises, inconsistent access, unreasonable storage architecture, and poor adaptability. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the data synchronization method provided in the above embodiments, and will not be repeated here.

[0088] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the data synchronization method described above.

[0089] The computer program product provided in this application can solve the technical problems of delayed synchronization of multi-source heterogeneous data, inconsistent access, unreasonable storage architecture, and poor adaptability in power generation enterprises. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the data synchronization method provided in the above embodiments, and will not be repeated here.

[0090] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A data synchronization method, characterized in that, The data synchronization method is applied to the data management terminal, and the data synchronization method includes: Receive multi-source heterogeneous data sent by power generation companies and convert the multi-source heterogeneous data into standard structured data; The synchronization mode of standard structured data is determined by a pre-built dynamic switching engine, which is pre-configured with priority determination rules and dual-mode synchronization strategies. The standard structured data is written into the corresponding data warehouse according to the synchronization strategy.

2. The data synchronization method as described in claim 1, characterized in that, The step of converting the multi-source heterogeneous data into standard structured data includes: Receive multi-source heterogeneous data generated by the power generation enterprise during its production operation, dispatch management, and business analysis; The multi-source heterogeneous data is subjected to protocol identification to obtain the data conversion protocol, and the multi-source heterogeneous data is subjected to format parsing and source marking to obtain the system to which the data belongs and the data type; Based on the interface specifications of the data management terminal, the system to which the data belongs, and the data type, the multi-source heterogeneous data is converted into standard structured data.

3. The data synchronization method as described in claim 1, characterized in that, Prior to the step of determining the synchronization mode of standard structured data through a pre-built dynamic switching engine, the method further includes: A dual-link data synchronization channel with a real-time synchronization channel and a near-real-time synchronization channel is established in the enterprise service bus of the power generation enterprise. Data priority determination rules are set based on the business attributes of the power generation enterprises, and the determination criteria for high-priority real-time data and medium-priority near-real-time data are divided. Based on the judgment criteria, a real-time incremental capture strategy and a timed incremental capture strategy are configured in the enterprise service bus to obtain a dual-mode synchronization strategy. The dual-mode synchronization strategy, priority determination rules, and enterprise service bus are integrated to obtain a dynamic switching engine.

4. The data synchronization method as described in claim 3, characterized in that, The step of determining the synchronization mode of standard structured data through a pre-built dynamic switching engine includes: The dynamic switching engine reads data information from standard structured data. Based on the data information, and through the priority determination rules, the standard structured data is determined to be either high-priority real-time data or medium-priority near-real-time data, and a determination result is obtained. Based on the judgment result, the standard structured data is allocated to the synchronization channel corresponding to the enterprise service bus to obtain the allocation result; Based on the allocation results and the dual-mode synchronization strategy, the synchronization mode corresponding to the standard structured data is determined.

5. The data synchronization method as described in claim 4, characterized in that, The step of writing the standard structured data into the corresponding data warehouse according to the synchronization strategy includes: When the synchronization strategy is in real-time synchronization mode, incremental change information of the standard structured data is monitored through the Change Data Capture (CDC) mechanism. The incremental change information is streamed through the real-time synchronization channel of the enterprise service bus. The incremental changes to the streaming data are written to the corresponding relational database, and the data synchronization point and transmission status are updated.

6. The data synchronization method as described in claim 4, characterized in that, The step of writing the standard structured data into the corresponding data warehouse according to the synchronization strategy further includes: When the synchronization strategy is in near real-time synchronization mode, the initial data synchronization point is determined according to the configurable time window. Incremental data of standard structured data is extracted based on the initial synchronization point and then batch-packaged and transmitted through the near real-time synchronization channel of the enterprise service bus. The incremental data that is packaged and transmitted in batches is written into the corresponding relational database, and the initial data synchronization point and task timestamp are updated.

7. The data synchronization method as described in claim 1, characterized in that, After the step of writing the standard structured data into the corresponding data warehouse according to the synchronization strategy, the method further includes: The newly added data in the data warehouse is subjected to consistency verification using the standard structured data to obtain the consistency verification result; If the consistency verification result is successful, all data will be stored in the persistent data warehouse according to the hierarchical storage rules, and high-frequency business data will be stored in the business data mart.

8. A data synchronization device, characterized in that, The data synchronization device is used in the data management terminal, and the data synchronization device includes: The receiving module is used to receive multi-source heterogeneous data sent by power generation companies and convert the multi-source heterogeneous data into standard structured data. The determination module is used to determine the synchronization mode of standard structured data through a pre-built dynamic switching engine, wherein the dynamic switching engine is pre-configured with priority determination rules and dual-mode synchronization strategies. The writing module is used to write the standard structured data into the corresponding data warehouse according to the synchronization strategy.

9. A data synchronization device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the data synchronization method as described in any one of claims 1 to 7.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the data synchronization method as described in any one of claims 1 to 7.

Images