Method and system for fast sharing of power data with third party databases

Abstract

This application provides a method and system for rapid sharing of power data with third-party databases. The method includes: accessing at least one type of power data source to obtain raw power data; performing semantic parsing and standardization transformation based on a power data semantic dictionary to produce standardized intermediate data; connecting to a third-party target database, automatically identifying its table structure, and adapting the intermediate data to a target format matching the database table structure using a format conversion rule library; configuring data sharing strategies and access permission rules to clarify the sharing scope, transmission frequency, and priority; establishing field mapping relationships and dynamically scheduling data transmission based on system load; and monitoring the entire process, conducting anomaly detection and tiered handling based on preset anomaly thresholds to ensure stable and standardized cross-database sharing and transmission of power data. This invention enables efficient, secure, and intelligent sharing of power data with heterogeneous third-party databases, reduces development and maintenance costs, and ensures the real-time performance, reliability, and security of data sharing.

Description

Technical Field

[0001] This invention belongs to the field of power system data processing technology, and in particular relates to a method and system for rapid sharing of power data with third-party databases. Background Technology

[0002] In the process of digital transformation of the power system, power data exhibits characteristics such as massive volume, multi-source heterogeneity, and high real-time requirements, covering multi-dimensional information including generating unit operation data, transmission line monitoring data, distribution equipment status data, and user energy consumption data. Third-party databases (such as energy management platform databases, government regulatory databases, and scientific research analysis databases) have an increasingly urgent need to share power data to support cross-domain data analysis, collaborative decision-making, and regulatory auditing scenarios.

[0003] There are many defects in the existing data sharing methods between power data and third-party databases: (1) Poor compatibility: The data storage formats within the power system are diverse (such as time-series data in real-time databases, structured data in relational databases, and unstructured log data), and the types of third-party databases are different (relational, non-relational, time-series, etc.). The existing sharing schemes lack a unified data adaptation mechanism and require the development of dedicated interfaces for different database types, resulting in high development costs and poor scalability. (2) Imbalance between real-time performance and reliability: Power data contains a large amount of real-time monitoring data (such as line load, equipment voltage and current, etc.), which requires millisecond-level sharing response. However, the existing schemes mostly adopt timed batch synchronization or simple trigger-based transmission, which either cannot meet the real-time requirements or are prone to data loss and transmission delays in high-concurrency scenarios. (3) Insufficient data security and access control: Power data belongs to the core data of critical infrastructure, and some data involves trade secrets and public safety. The existing sharing schemes lack fine-grained access control mechanisms, making it difficult to achieve differentiated access control based on data type, access subject, and usage scenario. Moreover, there is a lack of end-to-end encryption protection during data transmission, which poses a risk of data leakage. (4) Lack of intelligent adaptation and fault tolerance: Power system data may have quality problems such as outliers and missing values. The table structure of third-party databases may also be dynamically adjusted according to business needs. Existing solutions cannot automatically identify and repair data quality problems, nor can they adapt to changes in data table structure. Manual intervention is required for adjustment, resulting in high maintenance costs.

[0004] Therefore, there is an urgent need for a power data sharing solution with third-party databases that combines high compatibility, high real-time performance, high security, and intelligent adaptability to address the pain points of existing technologies. Summary of the Invention

[0005] The method and system for rapid sharing of power data with third-party databases provided in this application can achieve efficient, secure, and intelligent sharing of power data with heterogeneous third-party databases, reduce development and maintenance costs, and ensure the real-time performance, reliability, and security of data sharing.

[0006] In a first aspect, embodiments of this application provide a method for rapid sharing of power data with a third-party database, including:

[0007] S110. Access raw power data from at least one power data source, construct a power data semantic dictionary, and perform semantic parsing and standardization transformation on the raw power data based on the power data semantic dictionary to generate standardized intermediate data.

[0008] S120. Connect to the target third-party database, automatically identify the table structure of the target third-party database, and adapt the standardized intermediate data to the target data format that conforms to the table structure of the target third-party database based on the data format conversion rule base.

[0009] S130. Configure the sharing policy and data access permission rules for the target third-party database; the sharing policy shall include at least the data sharing scope, sharing frequency and transmission priority;

[0010] S140. Based on the sharing strategy, establish a mapping relationship between standardized intermediate data and target third-party database table fields, and dynamically schedule the data transmission process according to the system load status.

[0011] S150: Monitor the entire process from data access to data writing to the target third-party database, and perform anomaly detection and graded processing based on preset anomaly thresholds.

[0012] In one optional implementation, S110 includes:

[0013] S111. Configure the connection parameters of the power data source and test the connection status through the visual interface provided by the unified access layer.

[0014] S112. Through the general protocol adaptation component in the unified access layer, identify the communication protocol or data format of the raw power data and extract data features;

[0015] S113. Construct a power data semantic dictionary, match the extracted data features with the power data semantic dictionary, and obtain the corresponding semantic tags and standard format specifications; wherein, the power data semantic dictionary is constructed in the following way: classify power data according to the business scenarios of the power system, and store the mapping relationship between data features, semantic tags and standard format specifications; the business scenarios include at least one of power generation, power transmission, power distribution and power consumption;

[0016] S114. Convert the original power data according to the standard format specification to generate standardized intermediate data containing data identifiers, data values, timestamps and data quality identifiers;

[0017] S115. Perform data quality inspection on the raw power data, including outlier detection and missing value detection; repair the data according to the inspection results, and mark the corresponding quality level in the data quality identifier field of the standardized intermediate data according to the repair results.

[0018] In one alternative implementation, S120 includes:

[0019] S121. Configure the connection parameters and permission information of the target third-party database through the visual interface provided by the adaptive layer, and test the connection and permissions.

[0020] S122. Load the database adapter plugin corresponding to the target third-party database type, and manage the database connection pool through the database adapter plugin;

[0021] S123. Through the adaptive adaptation layer, a probe command is sent to the target third-party database to automatically identify its table structure and store the identified table structure information in the structure cache library.

[0022] S124. Based on the type mapping relationship stored in the data format conversion rule base, convert the data type of the standardized intermediate data into the data type corresponding to the field of the target third-party database table.

[0023] In an optional implementation, S120 further includes:

[0024] The table structure identification operation in S123 is executed periodically, and the newly identified table structure is compared with the table structure stored in the structure cache. When a change in the table structure is detected, the structure cache is updated and the data format conversion rule base is updated.

[0025] In one alternative implementation, S130 includes:

[0026] S131. Configure the sharing policy through the visual interface provided by the policy configuration module; wherein, configuring the data sharing scope includes filtering by data identifier, data type or time range; configuring the sharing frequency includes selecting real-time trigger, timed period or on-demand call mode; configuring the transmission priority includes assigning high, medium or low priority to the data.

[0027] S132. Through the policy configuration module, configure a data access whitelist and operation permissions for third-party entities that access the target third-party database; the operation permissions include at least read-only and read-write permissions.

[0028] In one alternative implementation, S140 includes:

[0029] S141. Calculate the semantic similarity between the data identifier of the standardized intermediate data and the field name of the target third-party database table, determine the compatibility of the data types of both parties, and automatically generate candidate field mapping relationships for user confirmation.

[0030] S142. Customize the mapping relationship between standardized intermediate data fields and target third-party database table fields through visual drag-and-drop configuration;

[0031] S143. Monitor the bandwidth utilization of the data transmission channel and the load status of the target third-party database in real time.

[0032] S144. Based on the transmission priority in the sharing strategy and the load status monitored in S143, dynamically allocate transmission channel resources and control the start and stop of data transmission.

[0033] In one optional implementation, S144 specifically includes:

[0034] When standardized intermediate data to be transmitted is marked as high priority, the transmission of medium and low priority data is paused, and a dedicated transmission channel is allocated for the high priority data for transmission.

[0035] When bandwidth utilization is below the first load threshold, start the transmission of batch low-priority data; when bandwidth utilization is above the second load threshold, pause the transmission of low-priority data.

[0036] When the CPU utilization or memory utilization of the target third-party database is higher than the third load threshold, all data transmission is suspended, and the load status is periodically checked. Transmission is resumed after the load drops below the third load threshold.

[0037] In one alternative implementation, S150 includes:

[0038] S151. Set up monitoring points at key nodes of data access, data parsing, data mapping, data transmission and data writing, and collect status data of each node;

[0039] S152. Based on the preset anomaly classification threshold, the collected status data is detected and the anomalies are classified into different severity levels.

[0040] S153. For different levels of anomalies, execute the corresponding processing procedures; the processing procedures include automatic reconnection, data retry, storing abnormal data in the cache pool, and sending alarm notifications through multiple channels.

[0041] In an optional implementation, in S153, for an exception caused by connection interruption, an automatic reconnection operation is performed and the corresponding sharing task is paused after the reconnection fails; for an exception caused by data parsing or writing failure, a limited number of automatic retry operations are performed, and the data that failed to be retried is marked and stored in the exception data cache pool.

[0042] Secondly, embodiments of this application provide a system for rapid sharing of power data with third-party databases, including:

[0043] The unified access module is used to access raw power data from at least one power data source, construct a power data semantic dictionary, and perform semantic parsing and standardization transformation on the raw power data based on the power data semantic dictionary to generate standardized intermediate data.

[0044] The adaptive adaptation module is used to connect to the target third-party database, automatically identify the table structure of the target third-party database, and adapt the standardized intermediate data into the target data format that conforms to the table structure of the target third-party database based on the data format conversion rule base.

[0045] The policy configuration module is used to configure the sharing policy and data access permission rules for the target third-party database; the sharing policy includes at least the data sharing scope, sharing frequency and transmission priority;

[0046] The intelligent mapping and scheduling module is used to establish a mapping relationship between standardized intermediate data and target third-party database table fields based on a sharing strategy, and to dynamically schedule the data transmission process according to the system load status.

[0047] The monitoring and anomaly handling module is used to monitor the entire process from data access to data writing to the target third-party database, and to perform anomaly detection and graded processing based on preset anomaly thresholds.

[0048] Thirdly, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the method provided in embodiments of this application.

[0049] Fourthly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method provided in embodiments of this application.

[0050] The technical solution provided in this application, based on a unified semantic dictionary and protocol adaptation components, parses raw power data of varying sources and formats into standard intermediate data. Combined with automatic database table structure identification and a dynamic format conversion rule base, it achieves seamless integration between heterogeneous data sources and heterogeneous target databases, greatly improving system compatibility and scalability. Through configurable fine-grained sharing strategies and hierarchical permission control, it achieves precise control over the scope, frequency, priority, and access subjects of shared data. Combined with two-factor authentication and permission verification, it ensures the security and compliance of the sharing process from both policy and authentication dimensions. Through intelligent mapping and a dynamic scheduling mechanism based on real-time load, it automatically establishes field associations before data writing and intelligently allocates channels during transmission based on data priority and system resource status, prioritizing the transmission of data with high real-time requirements, thus balancing real-time performance and overall reliability in complex scenarios. Through full-process status monitoring and hierarchical anomaly handling mechanisms, it can quickly detect and automatically respond to various faults such as connection interruptions and processing failures, achieving the system's self-awareness and self-healing capabilities, significantly improving the stability and maintainability of the sharing process. Attached Figure Description

[0051] Figure 1 This is a schematic diagram illustrating the steps of the method for rapidly sharing power data with a third-party database provided in an embodiment of this application;

[0052] Figure 2 This is a flowchart illustrating the method for rapidly sharing power data with a third-party database provided in an embodiment of this application;

[0053] Figure 3 This is a schematic diagram of the structure of the power data and third-party database rapid sharing system provided in the embodiments of this application. Detailed Implementation

[0054] The present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0055] Figure 1 This is a schematic diagram illustrating the steps of the method for rapidly sharing power data with a third-party database provided in an embodiment of this application. Figure 2 This is a flowchart illustrating a method for rapid sharing of power data with a third-party database provided in an embodiment of this application. The method can be executed by a system for rapid sharing of power data with a third-party database. The system can be implemented by software and / or hardware and can be configured in electronic devices such as computers.

[0056] like Figure 1 and Figure 2 As shown, the technical solution provided in this application includes the following steps:

[0057] S110. Access raw power data from at least one power data source, construct a power data semantic dictionary, and perform semantic parsing and standardization transformation on the raw power data based on the power data semantic dictionary to generate standardized intermediate data.

[0058] In some embodiments, the first step is to address the challenge of accessing multi-source, heterogeneous power data. This involves converting raw data from different sources and with varying formats into intermediate data with standardized semantics and formats, laying the foundation for subsequent sharing. Specifically, this includes the following sub-steps:

[0059] S111. Configure the connection parameters of the power data source and test the connection status through the visual interface provided by the unified access layer.

[0060] Specifically, the system supports access to various types of power data sources. Raw power data mainly includes, but is not limited to, the following types: time-series data from real-time power databases (such as PI, InTouch, etc.), structured data from power relational databases (such as SQL Server, Oracle, etc.), data reported by power equipment acquisition terminals (such as smart meters, sensors, etc.) through specific communication protocols, and unstructured log files (such as TXT, CSV, etc.) generated during power system operation. Users fill in the necessary connection parameters for the data source to be accessed through the system's visual configuration interface. These parameters vary depending on the data source type. For example, for databases, the database type, IP address, port number, login username, and password must be filled in; for device terminals, the communication protocol type, IP address, port number, and communication baud rate must be filled in; and for log files, their storage path must be specified.

[0061] After configuration, the system automatically attempts to establish a connection with the data source to verify the correctness of the parameters. If the connection is successful, the configuration information is saved to the local configuration library; if the connection fails, a specific error message is returned (such as port being in use, incorrect username or password, etc.) to facilitate user troubleshooting.

[0062] S112. Through the general protocol adaptation component in the unified access layer, identify the communication protocol or data format of the raw power data and extract data features.

[0063] Specifically, firstly, the system makes an initial determination based on the user-configured data source type (such as a PI database or the DL / T 645 protocol). Then, it performs a second, more precise identification by further analyzing the characteristics of the data itself (such as whether the data frame header is 0x68 or whether the data identifier starts with P), ensuring the accuracy of the protocol determination.

[0064] Based on the identified protocol or file format specifications, extract core data features from the raw data stream or file. For example, parse specified data fields from a protocol frame, retrieve the values ​​of specific fields from database query results, or match the content of key fields from log text.

[0065] S113. Construct a semantic dictionary for power data, match the extracted data features with the semantic dictionary for power data, and obtain the corresponding semantic tags and standard format specifications.

[0066] Specifically, the data features extracted in the previous step are matched with the entries stored in the power data semantic dictionary. The system uses a fuzzy matching algorithm (such as similarity calculation, with a matching threshold set to ≥0.8) to find the most suitable semantic labels and their corresponding standard format specifications for the extracted features.

[0067] The power data semantic dictionary is a key knowledge base for semantic parsing. It categorizes power data according to business scenarios such as power generation, transmission, distribution, and consumption, and stores the mapping relationship between data features, semantic tags, and standard format specifications. Data features are identifiers that uniquely identify or characterize the source and meaning of a piece of original data. For example, a specific point number in the PI database (e.g., P1001), or a specific frame segment in the DL / T 645 communication protocol (e.g., 0x00-0x07). Semantic tags are common and consistent textual descriptions of the meaning represented by data features. For example, the semantic tag corresponding to point number P1001 might be "#1 unit speed". Standard format specifications define the standard format that the data corresponding to this semantic tag should have in the system, including numerical precision, units, and timestamp precision. For example, numerical value (retaining two decimal places) + unit (r / min) + timestamp (milliseconds).

[0068] For example, a semantic dictionary for power data is constructed: it is classified into four major scenarios: power generation, transmission, distribution, and consumption, and stores the mapping relationship of "data features - semantic tags - standard format", as shown in Table 1.

[0069] Table 1. Semantic dictionary mapping relationship table for power data

[0070]

[0071] In some embodiments, users can maintain the semantic dictionary through a visual interface provided by the system, including adding, modifying, or deleting "data feature-semantic tag-standard format" mapping entries, and batch importing and exporting are also supported. New entries are synchronized to the parsing component and take effect without restarting the system, ensuring the scalability of the system.

[0072] S114. Convert the original power data according to the standard format specification to generate standardized intermediate data containing data identifiers, data values, timestamps and data quality identifiers.

[0073] Specifically, based on the matched standard format specifications, the original data is cleaned, reorganized, and formatted to generate standardized intermediate data with a uniform structure. This intermediate data typically contains the following core fields:

[0074] Data Identifier: This refers to the matched semantic tags (such as "#1 Unit Speed"), which give the data a clear business meaning.

[0075] Data value: The actual numerical value or content after cleaning and transformation.

[0076] Timestamp: The point in time when the data was generated or collected; the precision is determined according to the specifications.

[0077] Data type: Indicates whether the data is numeric, string, etc.

[0078] Data quality identifier: Used to record the quality status of the data (the assignment of this field will be completed in the next step of data quality processing).

[0079] S115. Perform data quality inspection on the raw power data, including outlier detection and missing value detection; repair the data according to the inspection results, and mark the corresponding quality level in the data quality identifier field of the standardized intermediate data according to the repair results.

[0080] Specifically, data quality inspection can include:

[0081] Outlier detection: Statistical methods, such as the 3σ rule, are employed. The system calculates the mean (μ) and standard deviation (σ) of recent historical data (e.g., the most recent 100 records) for a given data point. If the current data value is outside the range [μ-3σ, μ+3σ], it is marked as an outlier.

[0082] Missing value detection: This is determined based on the known data acquisition cycle. If the system fails to receive the expected data within twice the acquisition cycle, the data is marked as missing.

[0083] Data repair and labeling based on the test results include:

[0084] Outlier repair: For values ​​marked as outliers, replace them with the average of recent normal data (e.g., the last 10).

[0085] Missing value repair: Different strategies are adopted according to the duration of the missing value. Short-term missing values ​​(e.g., ≤3 collection cycles) are estimated and filled using linear interpolation; long-term missing values ​​(>3 collection cycles) are marked as unavailable and an alarm is triggered to notify relevant personnel.

[0086] Quality level label: Based on the results of detection and repair, the final data quality level is marked in the data quality label field of the standardized intermediate data, such as excellent (no anomalies), usable (repaired) or unusable (cannot be repaired).

[0087] S120. Connect to the target third-party database, automatically identify the table structure of the target third-party database, and adapt the standardized intermediate data to the target data format that conforms to the table structure of the target third-party database based on the data format conversion rule base.

[0088] Specifically, it includes the following sub-steps:

[0089] S121. Configure the connection parameters and permission information of the target third-party database through the visual interface provided by the adaptive layer, and test the connection and permissions.

[0090] Specifically, users can configure connection parameters for the target third-party database through another visual interface provided by the system. The required configuration information includes: database type (e.g., MySQL, Oracle, MongoDB, etc.), server IP address, port number, login username and password, target database / collection name (for NoSQL databases), and target table name. The system automatically attempts to connect to the target database using the user-provided configuration information and verifies whether the configured account has the necessary permissions (e.g., query and write permissions to the target table). Upon successful verification, the system saves the configuration and generates a unique identifier ID for the database connection for subsequent management.

[0091] S122. Load the database adapter plugin corresponding to the target third-party database type, and manage the database connection pool through the database adapter plugin.

[0092] Specifically, the system automatically loads the corresponding database adapter plugin based on the configured database type. This plugin encapsulates dedicated drivers and commands for interacting with that specific database type. Through the adapter plugin, the system establishes and manages a database connection pool. The connection pool has initialization parameters, such as: maximum number of connections (e.g., 20), minimum number of idle connections (e.g., 5), connection timeout (e.g., 30 seconds), and idle connection recycling timeout (e.g., 60 seconds). The connection pool management unit dynamically adjusts the number of active connections based on the current data transmission load. During periods of high-concurrency data transmission, it automatically expands the number of connections to increase throughput; during periods of low load, it automatically reduces the number of connections to release resources.

[0093] S123. Through the adaptive adaptation layer, a probe command is sent to the target third-party database to automatically identify its table structure and store the identified table structure information in the structure cache library.

[0094] Specifically, the system sends targeted probe commands or query statements to the target database through the established connection to obtain the structural information of the target table. The probe method varies depending on the database type; for example:

[0095] For relational databases (such as MySQL), you might execute the DESCRIBE [table_name] or SHOW COLUMNS FROM [table_name] statement.

[0096] For document-oriented databases (such as MongoDB), you might execute db.[collection name].find().limit(1) to retrieve a sample document and analyze its field structure.

[0097] For time-series databases (such as InfluxDB), field information can be obtained using commands such as SHOW FIELD KEYS FROM [table name].

[0098] The identified field information (including field name, field data type, field length, whether it is a primary key, and whether null values ​​are allowed) is stored in a local structure cache. Subsequent data format conversions will be performed directly based on this cached information, eliminating the need to query the database again before each write, thus improving efficiency.

[0099] S124. Based on the type mapping relationship stored in the data format conversion rule base, convert the data type of the standardized intermediate data into the data type corresponding to the field of the target third-party database table.

[0100] Specifically, the system has a built-in data format conversion rule library, which pre-stores basic mapping rules between intermediate power data and various types of database fields, as shown in Table 2:

[0101] Table 2 Data Format Conversion Rule Base Comparison Table

[0102]

[0103] Before writing data, the system queries the data format conversion rule library based on the field types of the target table in the structure cache library to find the corresponding conversion rules. Then, the value of each field in the standardized intermediate data is converted according to the type requirements of its target field. For example, a string type "timestamp" is converted into a DATETIME or Date object required by the target database.

[0104] S125. Periodically execute the table structure identification operation in S123, compare the newly identified table structure with the table structure stored in the structure cache library; when a change in the table structure is detected, update the structure cache library and trigger the update of the data format conversion rule library.

[0105] Specifically, the system periodically (e.g., every 5 minutes) re-executes the above automatic table structure identification operation to obtain the latest structure of the database tables. The newly identified table structure is then compared with the old structure stored in the structure cache. If changes are detected (e.g., a field has been added, a field's data type has changed, or a field has been deleted), the following operations are performed:

[0106] Update structure cache: Update the structure cache library with the new table structure information.

[0107] Triggering rule updates: The notification format adaptation unit will automatically match the basic mapping relationship from the rule base and generate new conversion rules for newly added fields based on their data types. If a special type (such as JSON) not predefined in the rule base is encountered, the system will trigger a prompt to guide the user to customize the conversion rules for the field through a visual interface (e.g., how to convert a complex intermediate data object into a JSON string).

[0108] S130. Configure the sharing policy and data access permission rules for the target third-party database; the sharing policy shall include at least the data sharing scope, sharing frequency and transmission priority.

[0109] Specifically, it includes the following sub-steps:

[0110] S131. Configure the sharing policy through the visual interface provided by the policy configuration module.

[0111] The configuration options include filtering by data identifier, data type, or time range; configuring the sharing frequency includes selecting real-time trigger, timed period, or on-demand calling mode; and configuring the transmission priority includes assigning high, medium, and low priorities to the data.

[0112] Specifically, when configuring the data sharing scope, users can filter the subset of data to be shared through multiple dimensions, including:

[0113] Filtering by data identifier (semantic tag): This is the core filtering method. Users can directly select semantic tags generated in S110 that have clear business meanings (such as "#1 unit speed" and "A line current") to precisely specify which business data to share.

[0114] Filter by data type: Users can filter by data format type, such as sharing only numerical monitoring data or only string-type status description data.

[0115] Filter by time range: Users can specify to share data within a specific time period, such as data from the last hour, data from the last 24 hours, or a custom start and end time range.

[0116] Strategy Templates: Users can save commonly used filter combinations as strategy templates for quick reuse later. Template import and export are also supported, making it easy to migrate configurations between different projects or environments.

[0117] When configuring the sharing frequency, users need to specify the mode in which data is pushed or retrieved from a third-party database, including:

[0118] Real-time triggered: Sharing is triggered immediately when the source data changes or specific state conditions are met (such as data updates or device status changes to "fault"). This mode has the lowest latency and is suitable for scenarios such as alarms and real-time monitoring.

[0119] Periodic timing: Data sharing is performed according to a fixed time period (such as every 1 second, every 5 minutes, or every hour). The period can be customized (e.g., from 1 millisecond to 1 hour), with a minimum step size of 1 millisecond to meet different real-time requirements.

[0120] On-demand delivery: Data is not actively pushed; it is only provided when a third-party database initiates a query request. This model consumes the least amount of resources.

[0121] When configuring transmission priorities, in order to ensure that critical data is delivered first when system resources are scarce, users need to assign transmission priorities to the data, including:

[0122] Priority Classification: The system supports three priority levels: high, medium, and low. The rules for prioritizing these levels can be preset, and the system will automatically prioritize data according to these rules. Users can also assign priorities to individual data identifiers or batches of data.

[0123] Scheduling rules: High-priority data will occupy a dedicated transmission channel and receive priority bandwidth guarantees during transmission; medium-priority and low-priority data will share other transmission channels and will be time-division multiplexed and transmitted in the order of "medium priority before low priority". This provides a clear strategic basis for the subsequent dynamic scheduling of S140.

[0124] S132. Through the policy configuration module, configure a data access whitelist and operation permissions for third-party entities that access the target third-party database; the operation permissions include at least read-only and read-write permissions.

[0125] Specifically, data access permissions are configured in the access control configuration unit:

[0126] Subject Grouping: The system supports grouping access subjects according to the user or purpose of the third-party database (such as energy management platform group, government supervision group, scientific research and analysis group), which facilitates batch authorization.

[0127] Data access whitelist: Configures the range of data that each accessing subject or group can access. This is typically a "allowed access" list, which can be precise down to specific data identifiers (semantic tags) and time ranges. The system also supports reverse configuration of "deny access" for more complex access control logic.

[0128] Operation permission configuration includes:

[0129] Operation type: Permissions can be further divided into different levels such as read-only, read-write (can read and write), modify (can update existing data), and delete.

[0130] Permission Assignment and Approval: The system assigns operational permissions to each subject or group. By default, new subjects are granted only the lowest-risk "read-only" permission. If higher-level permissions (such as read, write, modify, or delete) are required, a secondary approval process by the administrator is necessary to ensure the rigor of permission granting.

[0131] Permission logging: The system records all permission assignments and changes, including the operator, operation time, specific changes, and approver. These audit logs are retained for a period of time (e.g., 90 days) to meet security audit and compliance requirements.

[0132] S140. Based on the sharing strategy, establish a mapping relationship between standardized intermediate data and target third-party database table fields, and dynamically schedule the data transmission process according to the system load status.

[0133] Specifically, it includes the following sub-steps:

[0134] S141. Calculate the semantic similarity between the data identifier of the standardized intermediate data and the field name of the target third-party database table, determine the compatibility of the data types of both parties, and automatically generate candidate field mapping relationships for user confirmation.

[0135] Specifically, the system uses a cosine similarity algorithm to automatically calculate the semantic similarity between the data identifiers (i.e., semantic tags, such as "#1 unit speed") of the standardized intermediate data and the field names (such as generator_speed) of the target third-party database table. The calculation formula is as follows:

[0136] ;

[0137] Where cosθ represents cosine similarity, and A and B are the word vectors of the data identifier and the table field name after vectorization, respectively. The dot product of vectors is represented by |A| and |B|, where |A| and |B| represent the magnitudes (lengths) of the vectors. The calculated value of cosθ is between -1 and 1; the closer the value is to 1, the more semantically similar the two words are.

[0138] While calculating semantic similarity, the system references the data format conversion rule library established and used in S120 to determine whether the data types of the standardized intermediate data are compatible with the data types of the target table fields (e.g., whether numeric types can be converted to string types, whether timestamp formats match). The system will filter out field pairs with semantic similarity reaching a preset threshold (e.g., similarity ≥ 0.7) and compatible data types, automatically generate mapping rules for them, and mark them as automatically generated. These candidate rules will be presented to the user in list form for one-click confirmation or modification.

[0139] S142. Customize the mapping relationship between standardized intermediate data fields and target third-party database table fields through visual drag-and-drop.

[0140] Specifically, for scenarios where automatic mapping fails to cover certain areas or where users have specific mapping needs, the system provides a visual manual configuration interface.

[0141] The mapping method supports three modes:

[0142] One-to-one mapping: An intermediate data field is mapped to a field in the target table.

[0143] One-to-many mapping: An intermediate data field (such as a string containing comma-separated values) is split into multiple parts, which are then mapped to multiple fields in the target table.

[0144] Many-to-one mapping: Multiple intermediate data fields are merged (e.g., concatenated) and mapped to a single field in the target table.

[0145] After configuration, users can input test data, preview the mapping results, and ensure the rules are correct. Once the test is successful, the rules are saved to the mapping rule library and categorized according to the target database, with export and backup support supported.

[0146] S143. Monitor the bandwidth utilization of the data transmission channel and the load status of the target third-party database in real time.

[0147] Specifically, the load monitoring unit enables real-time monitoring of system load:

[0148] Transmission channel monitoring: The load monitoring unit collects the bandwidth utilization of the data transmission channel at a high frequency (e.g., every 100 milliseconds) and records the maximum, minimum, and average utilization.

[0149] Database load monitoring: At the same time, the load monitoring unit will collect the load status of the target third-party database at a slightly lower frequency (e.g., every 1 second), including key indicators such as CPU utilization, memory utilization, and disk I / O utilization.

[0150] This monitoring data will be recorded and stored for a certain period of time (e.g., 7 days) for analyzing load trends.

[0151] S144. Based on the transmission priority in the sharing strategy and the load status monitored in S143, dynamically allocate transmission channel resources and control the start and stop of data transmission.

[0152] Specifically, based on the aforementioned monitoring data and preset strategies, the system automatically makes scheduling decisions:

[0153] High-priority data scheduling: When data to be transmitted is marked as high priority (such as equipment fault alarm data), the dynamic scheduling unit immediately suspends the transmission of all medium-priority and low-priority data, releases transmission channel resources, and allocates a dedicated transmission channel for this batch of high-priority data to ensure that it is delivered with minimal delay. After the transmission is completed, the transmission of medium- and low-priority data is resumed.

[0154] Medium and low priority data scheduling: Flow control is performed based on channel bandwidth utilization.

[0155] When the bandwidth utilization rate is below a low threshold (first load threshold, such as 30%), it indicates that the channel is idle, and the system will initiate batch low-priority data transmission to make full use of the idle bandwidth.

[0156] When bandwidth utilization exceeds a high threshold (a second load threshold, such as 70%), it indicates that the channel is approaching congestion. The system will then suspend the transmission of low-priority data and only allow medium-priority data to pass through, in order to prevent network congestion from affecting overall efficiency.

[0157] Database load balancing: Protective scheduling is performed based on the load of the target database.

[0158] When the CPU utilization or memory utilization of the target database exceeds a safe threshold (a third load threshold, such as CPU > 80% or memory > 85%), it indicates that the database processing capacity has reached a bottleneck. At this time, the system will suspend all data write operations and check the database load status at regular intervals (e.g., every 10 seconds).

[0159] Once the database load has decreased below a safe threshold, data transmission should resume, and a notification to the system administrator indicating that the load has decreased should be sent. This effectively prevents the third-party database from being overwhelmed by excessively rapid data writes.

[0160] Scheduling log recording: The load monitoring unit records information such as the time of each scheduling, data priority, channel allocation, and load status. The log is retained for 30 days and supports retrieval by time and priority.

[0161] This solution addresses the accuracy of data semantic alignment through intelligent mapping and the efficiency and stability of data transmission through dynamic scheduling. By tightly integrating pre-configured business policies (sharing scope, frequency, priority) with real-time system status (network bandwidth, database load), it achieves a context-aware resource allocation and traffic control mechanism. This intelligently ensures the real-time performance of high-priority data in complex and ever-changing environments and guarantees the stable and reliable operation of the entire shared system.

[0162] S150: Monitor the entire process from data access to data writing to the target third-party database, and perform anomaly detection and graded processing based on preset anomaly thresholds.

[0163] Specifically, it includes the following sub-steps:

[0164] S151. Set up monitoring points at key nodes of data access, data parsing, data mapping, data transmission and data writing, and collect status data of each node.

[0165] Specifically, the system sets up status monitoring points in five key stages of data sharing, forming a complete monitoring chain:

[0166] Data source access node: Monitors the connection between the monitoring system and the original power data source;

[0167] Data parsing node: Monitors the standardized conversion process based on semantic dictionaries;

[0168] Data mapping node: Monitors the mapping process between standardized intermediate data and target database fields;

[0169] Data transmission node: Monitors the data transmission process in the network channel;

[0170] Data write node: Monitors the process of writing data to the target third-party database.

[0171] The system collects a series of key performance indicators (KPIs) and status information at each monitoring point at a high frequency (e.g., every 50 milliseconds), including:

[0172] Access status: Connection status (normal / disconnected), data reception rate (scans / second);

[0173] Parsing status: parsing success rate, percentage of abnormal data parsed, average parsing time per data record (milliseconds / record);

[0174] Mapping status: Field mapping success rate, number of target fields that failed to match, average mapping time per data record (milliseconds / record);

[0175] Transmission status: data transmission rate (KB / s), transmission success rate, current channel bandwidth utilization;

[0176] Write status: data write success rate, average write time per record (milliseconds / record), and CPU and memory load status of the target database.

[0177] All collected status data is centrally displayed through a system dashboard. The system uses different colors (e.g., green for normal, yellow for warning, and red for abnormal) to intuitively identify the health level of each node, and supports viewing real-time data streams and historical trend curves for the past 1 hour and 24 hours, making it easy for operations and maintenance personnel to quickly locate performance bottlenecks or anomalies.

[0178] S152. Based on the preset anomaly classification threshold, the collected status data is detected and the anomalies are classified into different severity levels.

[0179] Specifically, the anomaly detection unit compares real-time collected status data with pre-configured thresholds. Users can customize these thresholds, and the system also provides a set of standard threshold configurations. For example, a parsing success rate below 90% may be defined as an anomaly.

[0180] For example, the system classifies anomalies into three levels according to their severity and scope of impact, and executes corresponding automated processing procedures to form a closed-loop processing system.

[0181] Level 1 anomaly (fatal anomaly): refers to a failure that causes the entire data flow to be interrupted, such as the interruption of the data source connection or the interruption of the target database connection.

[0182] For exceptions caused by connection interruption, perform the following operations:

[0183] The exception handling unit immediately attempts to automatically re-establish the connection, retrying 3 times by default, with a 1-second interval between each attempt. If the reconnection fails, a detailed exception log is recorded (including time, cause, and scope of impact), and the system administrator is immediately notified via high-priority alarm units such as SMS and email. All data sharing tasks related to this connection are suspended to avoid generating a large number of error logs and wasting resources.

[0184] Level 2 anomaly (serious anomaly): refers to a failure that causes partial data loss or a serious deterioration in quality, such as a parsing success rate of less than 90%, a mapping success rate of less than 90%, or a data transmission timeout (failure to complete the data transmission within 30 seconds).

[0185] The processing flow is as follows:

[0186] The system marks failed data packets and skips them to continue processing subsequent data, preventing a single error from blocking the entire process. An exception log is recorded; if the abnormal state persists for a period of time (e.g., 5 minutes) without recovery, the alarm unit sends a warning notification to the administrator. The system attempts to activate backup parsing, mapping, or transmission channels to process the data, bypassing the current point of failure.

[0187] Level 3 anomalies (minor anomalies): These refer to occasional problems that have a minor impact on the overall process and can be automatically repaired, such as individual data parsing failures or excessively long single data write times (more than 1 second).

[0188] For exceptions caused by data parsing or writing failures, perform the following operations:

[0189] For data that fails to be parsed or written, the system automatically retryes a limited number of times (e.g., 2 retries for parsing, 3 retries for writing). If the process still fails after retries, the data is marked as failed and stored in a dedicated exception data cache pool with a configurable cache duration (e.g., 24 hours). A concise log is also recorded. Data stored in the cache pool is not lost; administrators can manually view it and trigger reprocessing, and the system will automatically reprocess it in subsequent attempts according to priority.

[0190] S153. For different levels of anomalies, execute the corresponding processing procedures; the processing procedures include automatic reconnection, data retry, storing abnormal data in the cache pool, and sending alarm notifications through multiple channels.

[0191] Specifically, after the fault is repaired or automatically handled, the system will execute a recovery process to ensure business continuity.

[0192] Once the status of the faulty node is monitored to be restored (such as network recovery or database service restart), the status monitoring unit will automatically perform an integrity check on the node and its upstream and downstream components to confirm that the entire data path has been restored to normal.

[0193] After successful verification, the exception handling unit will automatically resume the data sharing task that was previously suspended due to the fault. Simultaneously, the system will check the exception data cache pool and, according to the data's priority (high, medium, low), retrieve the cached data from the pool for reprocessing, compensating for any data loss that may have occurred during the exception and ensuring data integrity. After processing is complete, the status of the relevant exception logs will be updated.

[0194] This invention constructs a three-layer architecture: a unified access layer, an adaptive adaptation layer, and an intelligent scheduling and access control layer. First, the unified access layer, based on a power data semantic dictionary and a general protocol adaptation component, parses multi-source heterogeneous power data into standardized intermediate data and completes the processing. Next, the adaptive adaptation layer automatically identifies the structure of third-party database tables and dynamically adapts the data format, combining fine-grained sharing strategies (range, frequency, priority) with a two-factor authentication + hierarchical access control mechanism to ensure data security. Finally, the intelligent mapping and dynamic scheduling engine realizes automatic / custom mapping between intermediate data and database fields, and differentiated transmission scheduling based on system load, achieving efficient, secure, and intelligent sharing of power data with heterogeneous third-party databases.

[0195] Figure 3 This is a schematic diagram of the structure of the power data and third-party database rapid sharing system provided in the embodiments of this application, as shown below. Figure 3 As shown, the system includes:

[0196] The unified access module is used to access raw power data from at least one power data source, construct a power data semantic dictionary, and perform semantic parsing and standardization transformation on the raw power data based on the power data semantic dictionary to generate standardized intermediate data. The power data semantic dictionary is constructed in the following way: power data is classified according to the business scenarios of the power system, and the mapping relationship between data features, semantic tags and standard format specifications is stored. The business scenarios include at least one of power generation, transmission, distribution and consumption.

[0197] The adaptive adaptation module is used to connect to the target third-party database, automatically identify the table structure of the target third-party database, and adapt the standardized intermediate data into the target data format that conforms to the table structure of the target third-party database based on the data format conversion rule base.

[0198] The policy configuration module is used to configure the sharing policy and data access permission rules for the target third-party database; the sharing policy includes at least the data sharing scope, sharing frequency and transmission priority;

[0199] The intelligent mapping and scheduling module is used to establish a mapping relationship between standardized intermediate data and target third-party database table fields based on a sharing strategy, and to dynamically schedule the data transmission process according to the system load status.

[0200] The monitoring and anomaly handling module is used to monitor the entire process from data access to data writing to the target third-party database, and to perform anomaly detection and graded processing based on preset anomaly thresholds.

[0201] Specifically, the unified access module is used to realize the access, parsing, and data quality detection and repair of multi-source power data, and includes the following sub-units:

[0202] The access configuration unit is used to provide a visual configuration interface for data source access, perform connection tests, and store configurations.

[0203] Protocol identification unit, used to accurately identify the communication protocol or data format type of the data source;

[0204] Semantic dictionary units are used to store and manage the mapping relationship between "data features - semantic tags - standard format", and support dynamic expansion;

[0205] The semantic matching unit is used to match the extracted data features with the semantic dictionary to complete the standardized format conversion;

[0206] The data quality processing unit is used to detect and repair outliers, missing values, and inconsistent data, and to mark the quality level.

[0207] The adaptive adaptation module is used to achieve interface with heterogeneous third-party databases, table structure recognition, and data format adaptation. It specifically includes the following sub-units:

[0208] The database access unit is used to provide a configuration interface for third-party database access, and to perform connection tests and permission verifications.

[0209] The connection pool management unit is used to manage the database connection pool, enabling efficient reuse and dynamic expansion of connections.

[0210] The table structure processing unit is used to automatically identify the table structure and monitor changes in real time, and update the structure cache.

[0211] The format adaptation unit is used to convert the formats of intermediate power data and database fields based on the conversion rule base.

[0212] The policy configuration module provides a visual configuration of shared policies and access control rules, and includes the following sub-units:

[0213] The shared policy configuration unit supports fine-grained configuration of data range, sharing frequency, and transmission priority, and provides policy template management.

[0214] The access control configuration unit is used to configure data access permissions and operation permissions, and supports subject group management;

[0215] The rule storage unit is used to store shared policies and permission rules in a local database, supporting rule synchronization and backup;

[0216] The intelligent mapping and scheduling module is used to realize intelligent mapping between power data and third-party database table fields and dynamic scheduling of data transmission. It specifically includes the following sub-units:

[0217] The mapping rule configuration unit is used to support the automatic generation and customization of mapping rules, and provides rule testing and management.

[0218] The load monitoring unit is used to monitor the transmission channel bandwidth and database load status in real time and record the monitoring data.

[0219] The dynamic scheduling unit is used to perform differentiated data transmission scheduling based on shared policies and load status;

[0220] The monitoring and anomaly handling module is used to monitor the entire data sharing process, handle various anomalies, and generate alarm information. It specifically includes the following sub-units:

[0221] The status monitoring unit is used to collect status data of each key node and provide a visual display.

[0222] Anomaly detection unit is used to detect various anomalies based on configured thresholds and classify them according to severity.

[0223] The exception handling unit is used to execute corresponding processing procedures for exceptions of different levels, including reconnection, caching, and retry;

[0224] The alarm unit is used to send alarm notifications through multiple channels such as SMS, email, and system pop-ups, and to record alarm logs.

[0225] The execution process of the system part of this application embodiment is the same as that of the method part of the embodiment described above, and will not be repeated here.

[0226] This application also provides an electronic device, including: at least one processor; a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform any of the following methods for rapidly sharing power data with a third-party database.

[0227] This application also proposes a computer storage medium storing a computer program, which, when executed by a processor, implements a method for quickly sharing any power data with a third-party database.

[0228] Computer storage media may be simply referred to as media. Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Dual Data SDRAM (DDRSDRAM), Enhanced SDRAM (ESDRAM), Synchlink DRAM (SLDRAM), Rambus Direct RAM (RDRAM), Direct Memory Bus Dynamic RAM (DRDRAM), and Memory Bus Dynamic RAM (RDRAM). The various embodiments described in this specification are presented in a progressive manner, with reference allowed to each other for similar or identical parts. Each embodiment focuses on describing the differences from other embodiments. In particular, the embodiments for apparatuses, devices, and non-volatile computer storage media are described simply because they are substantially similar to the method embodiments; relevant details can be found in the descriptions of the method embodiments.

[0229] The above embodiments are merely illustrative examples and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations. However, obvious variations or modifications derived therefrom are still within the scope of protection of this application.

Claims

1. A method for rapid sharing of power data with third-party databases, characterized in that, include: S110. Access raw power data from at least one power data source, construct a power data semantic dictionary, and perform semantic parsing and standardization transformation on the raw power data based on the power data semantic dictionary to generate standardized intermediate data. S120. Connect to the target third-party database, automatically identify the table structure of the target third-party database, and adapt the standardized intermediate data to the target data format that conforms to the table structure of the target third-party database based on the data format conversion rule base. S130. Configure the sharing policy and data access permission rules for the target third-party database; the sharing policy includes at least the data sharing scope, sharing frequency, and transmission priority; S140. Based on the sharing strategy, establish a mapping relationship between the standardized intermediate data and the target third-party database table fields, and dynamically schedule the data transmission process according to the system load status. S150. Monitor the entire process from data access to data writing to the target third-party database, and perform anomaly detection and graded processing based on preset anomaly thresholds.

2. The method according to claim 1, characterized in that, S110 includes: S111. Configure the connection parameters of the power data source and test the connection status through the visual interface provided by the unified access layer. S112. Using the general protocol adaptation component in the unified access layer, identify the communication protocol or data format of the original power data and extract data features; S113. Construct a power data semantic dictionary, match the extracted data features with the power data semantic dictionary, and obtain the corresponding semantic tags and standard format specifications; wherein, the power data semantic dictionary is constructed in the following way: classify power data according to the business scenarios of the power system, and store the mapping relationship between data features, semantic tags and standard format specifications; the business scenarios include at least one of power generation, power transmission, power distribution and power consumption; S114. The original power data is converted according to the standard format specification to generate the standardized intermediate data containing data identifier, data value, timestamp and data quality identifier; S115. Perform data quality detection on the original power data, including outlier detection and missing value detection; repair the data according to the detection results, and mark the corresponding quality level in the data quality identifier field of the standardized intermediate data according to the repair results.

3. The method according to claim 1, characterized in that, S120 includes: S121. Configure the connection parameters and permission information of the target third-party database through the visual interface provided by the adaptive layer, and test the connection and permissions. S122. Load the database adapter plugin corresponding to the target third-party database type, and manage the database connection pool through the database adapter plugin; S123. Through the adaptive adaptation layer, a probe command is sent to the target third-party database to automatically identify its table structure and store the identified table structure information in the structure cache library. S124. Based on the type mapping relationship pre-stored in the data format conversion rule base, convert the data type of the standardized intermediate data into the data type corresponding to the field of the target third-party database table.

4. The method according to claim 3, characterized in that, The S120 further includes: The table structure identification operation in S123 is executed periodically, and the newly identified table structure is compared with the table structure stored in the structure cache library. When a change in the table structure is detected, the structure cache library is updated and the data format conversion rule library is updated.

5. The method according to claim 1, characterized in that, S130 includes: S131. Configure the sharing strategy through the visual interface provided by the strategy configuration module; wherein, configuring the data sharing range includes filtering by data identifier, data type, or time range; configuring the sharing frequency includes selecting real-time trigger, timed period, or on-demand calling mode; configuring the transmission priority includes assigning high, medium, and low priorities to the data. S132. Through the policy configuration module, configure a data access whitelist and operation permissions for third-party entities that access the target third-party database; the operation permissions include at least read-only and read-write permissions.

6. The method according to claim 1, characterized in that, S140 includes: S141. Calculate the semantic similarity between the data identifier of the standardized intermediate data and the field name of the target third-party database table, determine the compatibility of the data types of both parties, and automatically generate candidate field mapping relationships for user confirmation. S142. By visually dragging and dropping, customize the mapping relationship between the standardized intermediate data fields and the target third-party database table fields; S143. Monitor the bandwidth utilization of the data transmission channel and the load status of the target third-party database in real time; S144. Based on the transmission priority in the sharing strategy and the load status monitored in S143, dynamically allocate transmission channel resources and control the start and stop of data transmission.

7. The method according to claim 6, characterized in that, S144 specifically includes: When the standardized intermediate data to be transmitted is marked as high priority, the transmission of medium and low priority data is paused, and a dedicated transmission channel is allocated for the high priority data for transmission. When the bandwidth utilization rate is lower than the first load threshold, the transmission of batch low-priority data is initiated; when the bandwidth utilization rate is higher than the second load threshold, the transmission of low-priority data is suspended. When the CPU utilization or memory utilization of the target third-party database is higher than the third load threshold, all data transmission is suspended, and the load status is periodically checked. Transmission is resumed after the load drops below the third load threshold.

8. The method according to claim 6, characterized in that, S150 includes: S151. Set up monitoring points at key nodes of data access, data parsing, data mapping, data transmission and data writing, and collect status data of each node; S152. Based on a preset anomaly classification threshold, the collected state data is detected, and the anomalies are classified into different severity levels. S153. For different levels of anomalies, execute the corresponding processing procedures; the processing procedures include automatic reconnection, data retry, storing abnormal data in a cache pool, and sending alarm notifications through multiple channels.

9. The method according to claim 8, characterized in that, In step S153, for exceptions caused by connection interruption, an automatic reconnection operation is performed and the corresponding sharing task is paused after the reconnection fails; for exceptions caused by data parsing or writing failure, a limited number of automatic retry operations are performed, and the data that failed to be retried is marked and stored in the exception data cache pool.

10. A system for rapid sharing of power data with third-party databases, characterized in that: include: A unified access module is used to access raw power data from at least one power data source, construct a power data semantic dictionary, and perform semantic parsing and standardization transformation on the raw power data based on the power data semantic dictionary to generate standardized intermediate data. An adaptive adaptation module is used to connect to a target third-party database, automatically identify the table structure of the target third-party database, and adapt the standardized intermediate data to a target data format that conforms to the table structure of the target third-party database based on a data format conversion rule base. The policy configuration module is used to configure the sharing policy and data access permission rules for the target third-party database; the sharing policy includes at least the data sharing scope, sharing frequency and transmission priority; The intelligent mapping and scheduling module is used to establish a mapping relationship between the standardized intermediate data and the target third-party database table fields based on the sharing strategy, and to dynamically schedule the data transmission process according to the system load status. The monitoring and anomaly handling module is used to monitor the entire process from data access to data writing to the target third-party database, and to perform anomaly detection and graded processing based on preset anomaly thresholds.

Images