A digital operation and management platform based on shared service operation and management

Abstract

This invention provides a digital operation and management platform based on shared service operation and management, including a data acquisition module, a data management module, and a resource allocation module. The data acquisition module collects business data in real time from different business scenarios and standardizes the acquired data. The data management module builds a shared database based on the standardized business data and manages the association and identification of business data within the shared database. The resource allocation module receives data call requests initiated by requesters based on business scenarios, responds to these requests according to dynamic resource allocation rules, obtains the priority level of the data call requests, allocates corresponding processing nodes to retrieve the relevant data based on the priority level, and the processing nodes return the data processing results to the requester. This invention helps improve the intelligence level of data resource allocation in shared service processes, thereby improving the quality and effectiveness of shared services.

Description

Technical Field

[0001] This invention relates to the field of digital operation technology for power grids, and in particular to a digital operation and control platform based on shared service operation management. Background Technology

[0002] As the digital transformation of power grid operations accelerates, the demand for data sharing in power grid business scenarios is growing exponentially. Traditional digital operation and management models for power grids face the following core problems: static and singular resource allocation models are unable to cope with the dynamic load demands during peak business periods, often leading to service response delays or even service non-response, resulting in the inability to meet the shared data acquisition needs of shared services in practical applications. Summary of the Invention

[0003] To address the aforementioned issues, this invention aims to provide a digital operation and management platform based on shared service operation and management.

[0004] The objective of this invention is achieved through the following technical solution:

[0005] This invention proposes a digital operation and management platform based on shared service operation and management, comprising a data acquisition module, a data management module, and a resource allocation module; wherein,

[0006] The data acquisition module is used to access heterogeneous data sources from different business scenarios, collect business data in real time under different business scenarios, and standardize the acquired business data to obtain standardized business data.

[0007] The data management module is used to build a shared database based on standardized business data and to manage the association and identification of business data in the shared database.

[0008] The resource allocation module is used to obtain data call requests initiated by the requester based on the business scenario, respond to the data call requests according to the dynamic resource allocation rules, obtain the priority level of the data call requests, allocate the corresponding processing nodes according to the priority level to retrieve the corresponding data and complete the data processing, and return the data processing results to the requester.

[0009] Preferably, the system also includes a permission management module; wherein,

[0010] The permission management module is used to verify the identity information of the requester who initiates the data retrieval request, and to obtain the corresponding data retrieval permission of the requester based on the verified identity information.

[0011] Preferably, the data acquisition module includes a data access unit, a metadata extraction unit, and a standardization processing unit; wherein,

[0012] The data access unit is used to connect to the data interfaces of various business scenarios and obtain the raw data streams of each business scenario;

[0013] The metadata extraction unit is used to extract metadata from the acquired raw data stream and form a feature vector corresponding to the raw data stream. The extracted metadata includes data type, update frequency, single data volume, transmission latency, etc.

[0014] The standardization processing unit is used to call the corresponding standardization processing standard to standardize the original data stream based on the obtained feature vector, so as to obtain standardized business data.

[0015] Preferably, the data management module includes a database unit and a relationship unit;

[0016] The database unit is used to build a shared database based on standardized business data, and to classify, store and manage the standardized business data and corresponding metadata.

[0017] The association unit is used to extract association features from business data in the shared database, and to identify the associations of related business data based on the extracted features.

[0018] Preferably, the resource allocation module includes a request acquisition unit, an allocation response unit, and a dynamic allocation unit; wherein,

[0019] The request acquisition unit is used to acquire data call requests initiated by the requester based on the business scenario. The data call request includes requester information, business scenario description, and the range of requested data.

[0020] The allocation response unit is used to analyze the requester information, business scenario description and requested data range carried in the data call request to obtain the data call priority intensity;

[0021] The dynamic allocation unit is used to allocate data processing nodes of the corresponding level according to the obtained priority intensity to complete the corresponding data retrieval tasks and further data processing tasks. The data processing nodes retrieve the corresponding business data memory image from the shared database and return it to the requester.

[0022] Preferably, the dynamic allocation unit specifically includes:

[0023] Based on the analysis of the data call priority intensity of the data call request, when the data call priority intensity is within the set first priority range, a first-level data processing node is allocated to respond to the data call request, so that the first-level data processing node can directly access the shared database according to the data call request, directly obtain the mirror of the required business data from the shared database and transmit it to the requester.

[0024] When the data call priority is within the set second priority range, a secondary data processing node is allocated to respond to the data call request. This allows the secondary data processing node to utilize the load balancer to allocate idle secondary data processing nodes to respond to the data call request and complete the forwarding of the required data.

[0025] When the data call priority is within the set third priority range, a third-level data processing node is assigned to complete the response to the data call request. This allows the third data processing node to asynchronously obtain the required data through edge processing and then transmit the obtained data to the requester via mirror image.

[0026] The beneficial effects of this invention are as follows: Firstly, the data acquisition module collects and preprocesses heterogeneous source data from different business scenarios, ensuring data quality during data collection. For the standardized business data, a shared database is built for centralized storage and management, while related data within the business data is identified for easy subsequent retrieval. Secondly, the resource allocation module responds to data retrieval requests, allocating appropriate data processing nodes based on the characteristics of the data retrieval request to complete data extraction and retrieval. This facilitates real-time adjustment based on the priority of data retrieval requests and the current load, improving the intelligence level of data resource allocation during shared services and enhancing the quality and effectiveness of shared service provision. Attached Figure Description

[0027] The present invention will be further described with reference to the accompanying drawings, but the embodiments in the drawings do not constitute any limitation on the present invention. For those skilled in the art, other drawings can be obtained based on the following drawings without creative effort.

[0028] Figure 1 This is a framework diagram of a digital operation and control platform based on shared service operation and management, as shown in an embodiment of the present invention.

[0029] Figure 2 for Figure 1 A schematic diagram illustrating the specific configuration framework of each functional module in the embodiment. Detailed Implementation

[0030] The present invention will be further described in conjunction with the following application scenarios.

[0031] See Figure 1 It demonstrates a digital operation and control platform based on shared service operation and management, including a data acquisition module, a data management module, and a resource allocation module; among which,

[0032] The data acquisition module is used to access heterogeneous data sources from different business scenarios, collect business data in real time under different business scenarios, and standardize the acquired business data to obtain standardized business data.

[0033] The data management module is used to build a shared database based on standardized business data and to manage the association and identification of business data in the shared database.

[0034] The resource allocation module is used to obtain data call requests initiated by the requester based on the business scenario, respond to the data call requests according to the dynamic resource allocation rules, obtain the priority level of the data call requests, allocate the corresponding processing nodes according to the priority level to retrieve the corresponding data and complete the data processing, and return the data processing results to the requester.

[0035] In the above embodiments of the present invention, the data acquisition module first collects and preprocesses heterogeneous source data from different business scenarios, ensuring data quality during data collection. For the standardized business data, a shared database is built for centralized storage and management, and related data within the business data is identified for easy subsequent retrieval. When a data retrieval request is initiated by a requester based on a business scenario, a resource allocation module responds to the request. This module allocates appropriate data processing nodes based on the characteristics of the data retrieval request to complete data extraction and retrieval. This facilitates real-time adjustment based on the priority of the data retrieval request and the current load, improving the intelligence level of data resource allocation during the shared service process and enhancing the quality and effectiveness of the shared service.

[0036] In one scenario, the digital operation and management platform is built on a shared database server (server), which can acquire and store data in the shared database, respond to requests for business data in the shared database, and complete the transmission and scheduling of the required data.

[0037] Different data processing nodes have different performance and data scheduling methods when completing shared data. In one scenario, data processing nodes can be divided into dedicated data nodes, general data nodes, and edge data nodes. The three types of nodes use three different data scheduling methods to complete the data scheduling task.

[0038] Establishing a physical channel (such as through an RDMA network connection) between a dedicated data node and a shared database enables the dedicated data node to directly access the data mirror within the shared database and, with the assistance of a "dedicated" computing node, complete the corresponding data scheduling tasks, thereby obtaining the shared data resources required by the requester in the highest priority and lowest latency manner.

[0039] The general data nodes first establish a connection with the load balancing module. The load balancing module controls the performance of the general data nodes in obtaining shared data from the shared database. Based on the current access and data acquisition load of the shared database, the load balancing module coordinates the access priority and performance (data transmission rate and data transmission resources) of each general data node to maximize the satisfaction of the shared data scheduling requests of each general data node.

[0040] Edge data nodes retrieve relevant business data from the shared database using low-priority methods (such as asynchronous scheduling). Based on the load of the shared server or the set idle time period, they retrieve data mirrors from the shared database and store them in the local edge node. They complete the corresponding data scheduling tasks based on the asynchronous data. In other words, edge data nodes can handle real-time, low-priority data scheduling tasks, and additional edge node settings can share the computing power and data transmission pressure of the shared database.

[0041] In one scenario, the business scenarios include: power plant load monitoring, substation operation monitoring, line monitoring, load forecasting center, new energy power generation forecasting, meteorological monitoring and forecasting, equipment asset records, fault repair, emergency command, customer service, electricity management, power grid planning, safety and operation management, integrated energy services, and other data sources corresponding to different business scenarios. The shared data acquired includes real-time / near real-time SCADA measurement data, PMU data, equipment condition monitoring data (online monitoring), power generation plans, load forecasting, new energy power forecasting, power grid models, topology information, and equipment management departments need to comprehensively assess equipment health status, predict fault risks, and optimize maintenance plans by integrating equipment ledger information, historical defect / fault records, inspection reports, online monitoring data (such as oil chromatography, partial discharge, temperature, vibration), test data, environmental data, etc., based on equipment basic information (ledger), historical operation and maintenance records (defects, faults, maintenance), real-time / historical online monitoring data, inspection results, preventive test reports, environmental data (meteorology, pollution), and GIS spatial information. The data includes: fault trip signals, protection action information, SCADA remote signaling changes, fault indicator signals, power outage range analysis results (based on topology), affected customer information (customer-transformer relationship), GIS geographic information, emergency repair resource status and trajectory, emergency repair progress, power restoration notice, customer basic files, real-time / historical metered electricity (AMI), electricity bill information, power outage plans and fault outage information, application process status, service work order status, adjustable load resource information (capacity, status), electricity price information, long-term historical load data, load density data, load forecast results, new energy development plans and output characteristics, power grid equipment ledger and capacity, power grid topology model, GIS spatial data and land information, regional economic development data, work plans, work permit / operation ticket information, personnel qualification information, real-time personnel location and trajectory, on-site video images, equipment energized / de-energized status, GIS geographic information and risk labeling, user electricity consumption data (refined), power grid operation status (affecting access), distributed energy output data, meteorological data, and other user energy data (if any), etc.

[0042] Preferably, the system also includes a permission management module; wherein,

[0043] The permission management module is used to verify the identity information of the requester who initiates the data retrieval request, and to obtain the corresponding data retrieval permission of the requester based on the verified identity information.

[0044] By setting up an access control module, the identity information of the requester can be verified first, thereby improving the security of data scheduling and data resource distribution in the shared database.

[0045] Preferred, see Figure 2 The data acquisition module includes a data access unit, a metadata extraction unit, and a standardization processing unit; among them,

[0046] The data access unit is used to connect to the data interfaces of various business scenarios and obtain the raw data streams of each business scenario;

[0047] The metadata extraction unit is used to extract metadata from the acquired raw data stream and form a feature vector corresponding to the raw data stream. The extracted metadata includes data type, update frequency, single data volume, transmission latency, etc.

[0048] The standardization processing unit is used to call the corresponding standardization processing standard to standardize the original data stream based on the obtained feature vector, so as to obtain standardized business data.

[0049] For raw data streams obtained from different business scenarios, the raw data streams are first standardized and the data resources are labeled to facilitate subsequent classification and storage management in the shared database and subsequent data resource access, thereby improving data quality.

[0050] Given the massive volume of business data acquired from various scenarios, standardizing the raw data streams using the same standard (e.g., performing data cleaning and error correction on all streams) would consume significant data processing resources, increasing the system's workload during the data acquisition phase. Therefore, this standardization preprocessing approach proposes a method that first assesses the quality weights based on the characteristics of the raw data stream, then adaptively applies different standards to complete the standardization process, thereby improving system performance.

[0051] Preferably, in the standardization preprocessing unit, the corresponding standardization processing standard is invoked to perform standardization processing on the raw data stream, specifically including:

[0052] The quality weight of the current raw data stream is determined based on the historical data quality of the data source to which the raw data stream belongs. The quality weight acquisition function used is as follows:

[0053] In the formula, This represents the quality weight of the i-th original data stream. This represents the percentage of historical missing fields in the i-th original data stream. This represents the historical reception delay of the i-th raw data stream, obtained from the difference between the timestamp when the data was generated and the timestamp when the data was received; This indicates the maximum allowed reception delay. This represents the historical data error rate of the i-th raw data stream, obtained by statistically analyzing the proportion of data in the historical raw data streams that exceeds the corresponding standard value range. This indicates the maximum allowable data error rate. , , These represent the set weighting factors;

[0054] Based on the quality weight of the original data stream, if the quality weight exceeds the preset weight standard, then only the current original data stream is format converted to the preset standard format to obtain standardized business data; otherwise, if the quality weight does not exceed the preset weight standard, the original data stream is further cleaned by data difference and outlier removal, and then the cleaned data is format converted to obtain standardized business data.

[0055] In one scenario, based on the historical data quality of the data source to which the original data stream belongs, where the historical data from the same data source is data from 1 day, 3 days, 7 days, 15 days, 20 days, or 30 days, corresponding indicators are calculated based on the historical data. The percentage of historical missing fields can be obtained from the audit logs of the historical data; the historical reception delay can be calculated based on the time difference between the data generation time and the reception time; and the historical data error rate is calculated based on the percentage of data that exceeds the standard range (e.g., the standard range of numerical values ​​or the standard range of data types).

[0056] In the above embodiments of the present invention, when performing standardized preprocessing on massive amounts of acquired business data, the quality of data from the same data source is first evaluated based on the historical data quality from different data sources. For data sources with high and relatively stable quality, the quality of the raw data stream from that data source is trusted, thus reducing the degree of standardization processing (only performing necessary standardization). For raw data streams with unstable or low quality, a more advanced standardization method (e.g., data cleaning) is used to perform initial standardization, thereby improving data quality. By using a proposed quality weight calculation method to evaluate the current data quality, the posterior quality of the raw data stream (e.g., based on logs or idle-time sampling) can be used as a foundation to accurately determine the degree of standardization of real-time data from different data sources. This reduces the pressure on the system to preprocess massive raw data streams from different data sources while ensuring data quality, thereby improving system performance.

[0057] Since it is impossible to judge the data quality or error data in real time for the raw data stream obtained in real time (it requires a lot of computing resources), but considering that data transmission factors have a key impact on the quality of the data stream when receiving the raw data stream transmitted in the business scenario (transmission delay is usually accompanied by packet loss, error data, etc.), the quality weight calculation method proposed in the above implementation method specifically uses the historical data transmission quality and delay of the data source corresponding to the raw data stream as a standard to evaluate the quality of the raw data stream transmitted by the data source and reflect the data transmission quality of the data source.

[0058] Preferably, the data management module includes a database unit and a relationship unit;

[0059] The database unit is used to build a shared database based on standardized business data, and to classify, store and manage the standardized business data and corresponding metadata.

[0060] The association unit is used to extract association features from business data in the shared database, and to identify the associations of related business data based on the extracted features.

[0061] After storing standardized business data in a shared database, further association processing is performed on the stored business data. This allows related business data to be stored together, making it easier to retrieve data based on the characteristics and associations of the business data when calling data, thus improving the targeting and effectiveness of data retrieval.

[0062] In the association unit, the association features extracted from the business data include metadata features such as data source, region, security level, business data type, and business data type. Association data tags are established based on one or more metadata features, and business data that simultaneously meet one or more metadata features are uniformly labeled with the same association data tag.

[0063] Furthermore, based on the data association based on metadata, it is possible to add advanced features such as semantic features of business data (e.g., semantic vectors) as a foundation to further associate data that are the same or similar (e.g., based on semantic cosine similarity).

[0064] Preferably, the resource allocation module includes a request acquisition unit, an allocation response unit, and a dynamic allocation unit; wherein,

[0065] The request acquisition unit is used to acquire data call requests initiated by the requester based on the business scenario. The data call request includes requester information, business scenario description, and the range of requested data.

[0066] The allocation response unit is used to analyze the requester information, business scenario description and requested data range carried in the data call request to obtain the data call priority intensity;

[0067] The dynamic allocation unit is used to allocate data processing nodes of the corresponding level according to the obtained priority intensity to complete the corresponding data retrieval tasks and further data processing tasks. The data processing nodes retrieve the corresponding business data memory image from the shared database and return it to the requester.

[0068] In one scenario, data requesters include fault management centers, power grid control consoles, dispatch analysis servers, general residential users, and payment service centers. The specific business data required by each requester varies depending on the business scenario. For example, a fault management center might require data on the operation of the power grid within its region, while a payment service center typically requires data on the monthly cumulative electricity consumption of nodes within its region.

[0069] The business scenario description is the description of the business data request initiated for a specific business scenario. It includes descriptions of the business scenario content, such as "fault location", "load forecast", "monthly report summary", "electricity bill inquiry", etc.

[0070] The requested data scope refers to the range of business data required, including characteristics that limit the scope of business data, such as time characteristics, geographical characteristics, object characteristics, and security level characteristics; business scenario information includes a description of the application scenario or function of the required business data.

[0071] When a data requester initiates a data call request based on a business scenario, the allocation and response unit first analyzes the corresponding data call priority based on the received request. According to different data call priorities, the dynamic allocation unit assigns appropriate data processing nodes to complete the data distribution task corresponding to the data call request. This makes the allocation of data processing nodes more reasonable and adaptable to the needs of data call requests under different priorities and business scenarios. This improves the overall performance of the shared data system and optimizes the system's response to data call tasks.

[0072] Preferably, the allocation response unit analyzes the requester information, business scenario description, and requested data range carried in the data call request, specifically including:

[0073] Extract the data transmission path delay between the current server and the requester based on the requester's information;

[0074] Extract the corresponding semantic features of the business scenario based on the business scenario description, and calculate the similarity between the semantic scene business features and the standard features of each standard business scenario to obtain the standard business type corresponding to the data extraction request.

[0075] Based on the obtained standard business type and requested data range, the required business data range is determined, and the corresponding data call chain information is obtained based on the required business data range, and the complexity of the data call chain is calculated.

[0076] Get the data retrieval priority intensity of the current data retrieval request:

[0077] In the formula, This indicates the priority level of the current data request k. This indicates the business level corresponding to the standard business type of the data retrieval request; This indicates a delay in the data transmission path of the requester. Represents the delay sensitivity coefficient. This indicates the complexity of the data call chain.

[0078] Specifically, the data transmission path latency of the requester can be obtained from the real-time latency information of the requester accessing the API gateway. Based on the content of the data call request initiated by the requester, semantic features are extracted from the business scenario description. Similarity analysis is then performed between these semantic features and preset standard business scenarios (such as aggregation, location, query, emergency repair, prediction, and analysis). Based on the similarity analysis results (usually the highest similarity), the standard business type and corresponding business level corresponding to the data extraction request are obtained. A higher business level indicates a higher priority for the corresponding data call request. The complexity of the data call chain represents the dispersion of the data required by the data call request, calculated by the server's response speed to historically similar data call requests.

[0079] By proposing a priority intensity calculation method, a unified evaluation of the semantics, timeliness, and complexity of business data call requests can be conducted to comprehensively analyze the priority intensity of these requests. Higher priority intensity indicates that the data call request requires more server resources for priority processing to meet the needs of shared data calls in critical business scenarios; therefore, higher-performance data nodes are used to handle the corresponding data call requests. Conversely, lower priority intensity indicates a reduced real-time requirement for the current data call task; therefore, to ensure a more balanced allocation of performance across the shared data server, general or edge data nodes are used to process the corresponding data call requests. This method of prioritizing data call requests allows for a comprehensive evaluation of the priority of data call tasks based on actual conditions, providing a basis for further allocating appropriate data nodes to handle the corresponding data call tasks.

[0080] In one scenario, traditional priority allocation methods that solely consider business scenarios lack consideration for actual data transmission and data call complexity. This single-dimensional approach can easily lead to resource lockouts on high-priority data nodes even when a single high-priority task experiences data transmission or call anomalies, impacting the performance of subsequent data call tasks. In contrast, the priority strength evaluation method provided by the aforementioned implementation, which comprehensively evaluates priority based on semantics, timeliness, and complexity, further analyzes the requester's data transmission and task complexity beyond the business scenario. This allows for the downgrading of "abnormal" high-priority tasks, resulting in better resource allocation on high-priority data nodes and improved overall performance in the shared data allocation process.

[0081] Preferably, the dynamic allocation unit specifically includes:

[0082] Based on the analysis of the data call priority intensity of the data call request, when the data call priority intensity is within the set first priority range, a first-level data processing node is allocated to respond to the data call request, so that the first-level data processing node can directly access the shared database according to the data call request, directly obtain the mirror of the required business data from the shared database and transmit it to the requester.

[0083] When the data call priority is within the set second priority range, a secondary data processing node is allocated to respond to the data call request. This allows the secondary data processing node to utilize the load balancer to allocate idle secondary data processing nodes to respond to the data call request and complete the forwarding of the required data.

[0084] When the data call priority is within the set third priority range, a third-level data processing node is assigned to complete the response to the data call request. This allows the third data processing node to asynchronously obtain the required data through edge processing and then transmit the obtained data to the requester via mirror image.

[0085] Based on the priority of the data call request, the dynamic allocation unit assigns three different data nodes to complete the corresponding data call tasks. By setting three data processing nodes, data call tasks with different characteristics can be completed respectively, thereby improving the overall performance of the server in responding to shared data calls.

[0086] In one scenario, after range limiting and normalization, the data call priority intensity is... The value range of is [0,1], where when When, a first-level data processing node is allocated to complete the corresponding data scheduling task k. When, a secondary data processing node is allocated to complete the corresponding data scheduling task k. When the time comes, three-level data processing nodes are allocated to complete the corresponding data scheduling task k.

[0087] In one scenario, the first-level data processing node is a dedicated data node, the second-level data node is a general data node, and the third-level data node is an edge data node.

[0088] Among them, establishing a physical channel (such as through an RDMA network connection) between the dedicated data node and the shared database enables the dedicated data node to directly access the data mirror within the shared database and assist in completing the corresponding data scheduling tasks through the "exclusive" computing node, thereby obtaining the shared data resources required by the requester in the highest priority and lowest latency manner.

[0089] The general data nodes first establish a connection with the load balancing module. The load balancing module controls the performance of the general data nodes in obtaining shared data from the shared database. Based on the current access and data acquisition load of the shared database, the load balancing module coordinates the access priority and performance (data transmission rate and data transmission resources) of each general data node to maximize the satisfaction of the shared data scheduling requests of each general data node.

[0090] Edge data nodes retrieve relevant business data from the shared database using low-priority methods (such as asynchronous scheduling). Based on the load of the shared server or the set idle time period, they retrieve data mirrors from the shared database and store them in the local edge node. They complete the corresponding data scheduling tasks based on the asynchronous data. In other words, edge data nodes can handle real-time, low-priority data scheduling tasks, and additional edge node settings can share the computing power and data transmission pressure of the shared database.

[0091] In one scenario, for a data retrieval task requesting measurement data within the current fault section from a fault location system, the data retrieval and transmission are accomplished in the following way:

[0092] The system obtains data request requests from the fault location system and analyzes the corresponding data request priority level. The system allocates dedicated data nodes to complete the data retrieval task. These dedicated data nodes retrieve the required data set from the shared database based on the data retrieval request and establish a direct interface between the fault location system and the shared database (IP: 192.168.10.3:9001). The fault location system then directly retrieves the required data (millisecond-level updated power grid data within the region) from the shared database based on the location of the data set, enabling the fault location system to complete the fault location task. The dedicated data node is released after the task is completed.

[0093] In another scenario, for historical transmission line data requested by the forecasting system, the data retrieval and transmission are accomplished in the following way: the system obtains the data retrieval request issued by the forecasting system, and analyzes the corresponding data retrieval priority based on the data retrieval request. The system allocates general data nodes to complete the data retrieval task. The general data nodes are uniformly controlled by the load balancer. Based on the obtained values ​​and business scenario tags, the load balancer calculates the current load status of each general data node (node ​​A load 0.42, node B load 0.58) and allocates the corresponding general data node (node ​​A (IP: 10.0.0.12)) to establish a connection with the prediction system according to the principle of balance. Node A then accesses the corresponding data pool from the shared database according to the data request and pulls the required data. Node A then returns the data to the prediction system.

[0094] In another scenario, for the monthly electricity consumption data of regional users requested by the electricity billing system, the data retrieval and transmission are accomplished in the following way: the system obtains the data retrieval request issued by the electricity billing system, and analyzes the corresponding data retrieval priority based on the data retrieval request. The system allocates edge data nodes to complete the data retrieval task. The edge nodes retrieve copies of the required data (including user ID, electricity data within a specified time period, etc.) from the shared database during off-peak hours (such as a preset time period or based on conditions) according to the current load of the shared database. The edge data nodes then perform statistics on the data copies, obtain the electricity statistics results for the user, and return the statistics results directly to the electricity billing system.

[0095] It should be noted that the functional units / modules in the various embodiments of the present invention can be integrated into one processing unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated into one unit / module. The integrated unit / module described above can be implemented in hardware or in the form of software functional units / modules.

[0096] From the above description of the embodiments, those skilled in the art will clearly understand that the embodiments described herein can be implemented in hardware, software, firmware, middleware, code, or any suitable combination thereof. For hardware implementation, the processor can be implemented in one or more of the following units: Application-Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field-Programmable Gate Array (FPGA), processor, controller, microcontroller, microprocessor, other electronic units designed to implement the functions described herein, or combinations thereof. For software implementation, some or all of the processes of the embodiments can be implemented by a computer program instructing the associated hardware. During implementation, the program can be stored in a computer-readable medium or transmitted as one or more instructions or code on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transmission of a computer program from one place to another. Storage media can be any available medium accessible to a computer. Computer-readable media can include, but is not limited to, RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code having the form of instructions or data structures and accessible to a computer.

[0097] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit the scope of protection of the present invention. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should be able to analyze that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the essence and scope of the technical solutions of the present invention.

Claims

1. A digital operation management and control platform based on shared service operation management, characterized in that, It includes a data acquisition module, a data management module, and a resource allocation module; among which, The data acquisition module is used to access heterogeneous data sources from different business scenarios, collect business data in real time under different business scenarios, and standardize the acquired business data to obtain standardized business data. The data management module is used to build a shared database based on standardized business data and to manage the association and identification of business data in the shared database. The resource allocation module is used to obtain data call requests initiated by the requester based on the business scenario, respond to the data call requests according to the dynamic resource allocation rules, obtain the priority level of the data call requests, allocate the corresponding processing nodes according to the priority level to retrieve the corresponding data and complete the data processing, and return the data processing results to the requester. The data acquisition module includes a data access unit, a metadata extraction unit, and a standardization processing unit; among which, The data access unit is used to connect to the data interfaces of various business scenarios and obtain the raw data streams of each business scenario; The metadata extraction unit is used to extract metadata from the acquired raw data stream and form a feature vector corresponding to the raw data stream. The extracted metadata includes data type, update frequency, single data volume and transmission delay. The standardization processing unit is used to standardize the original data stream based on the obtained feature vectors, by invoking the corresponding standardization processing standard to obtain standardized business data, specifically including: The quality weight of the current raw data stream is determined based on the historical data quality of the data source to which the raw data stream belongs. The quality weight acquisition function used is as follows: In the formula, This represents the quality weight of the i-th original data stream. This represents the percentage of historical missing fields in the i-th original data stream. This represents the historical reception delay of the i-th raw data stream, obtained from the difference between the timestamp when the data was generated and the timestamp when the data was received; This indicates the maximum allowed reception delay. This represents the historical data error rate of the i-th raw data stream, obtained by statistically analyzing the proportion of data in the historical raw data streams that exceeds the corresponding standard value range. This indicates the maximum allowable data error rate. , , These represent the set weighting factors; Based on the quality weight of the original data stream, if the quality weight exceeds the preset weight standard, then only the current original data stream is format converted to the preset standard format to obtain standardized business data; otherwise, if the quality weight does not exceed the preset weight standard, the original data stream is further cleaned by removing data differences and outliers, and then the cleaned data is format converted to obtain standardized business data.

2. The digital operation and management platform based on shared service operation and management according to claim 1, characterized in that, It also includes a permissions management module; among which, The permission management module is used to verify the identity information of the requester who initiates the data retrieval request, and to obtain the corresponding data retrieval permission of the requester based on the verified identity information.

3. The digital operation and management platform based on shared service operation and management according to claim 1, characterized in that, The data management module includes a database unit and a relationship unit; The database unit is used to build a shared database based on standardized business data, and to classify, store and manage the standardized business data and corresponding metadata. The association unit is used to extract association features from business data in the shared database, and to identify the associations of related business data based on the extracted features.

4. A digital operation and control platform based on shared service operation and management according to claim 3, characterized in that, The resource allocation module includes a request acquisition unit, an allocation response unit, and a dynamic allocation unit; among which, The request acquisition unit is used to acquire data call requests initiated by the requester based on the business scenario. The data call request includes requester information, business scenario description, and requested data range. The allocation response unit is used to analyze the requester information, business scenario description and requested data range carried in the data call request to obtain the data call priority intensity; The dynamic allocation unit is used to allocate data processing nodes of the corresponding level according to the obtained priority intensity to complete the corresponding data retrieval tasks and further data processing tasks. The data processing nodes retrieve the corresponding business data memory image from the shared database and return it to the requester.

5. A digital operation and control platform based on shared service operation and management according to claim 4, characterized in that, The allocation response unit analyzes the requester information, business scenario description, and requested data range carried in the data call request, specifically including: Extract the data transmission path delay between the current server and the requester based on the requester's information; Extract the corresponding semantic features of the business scenario based on the business scenario description, and calculate the similarity between the semantic scene business features and the standard features of each standard business scenario to obtain the standard business type corresponding to the data extraction request. Based on the obtained standard business type and requested data range, the required business data range is determined, and the corresponding data call chain information is obtained based on the required business data range, and the complexity of the data call chain is calculated. Get the data retrieval priority intensity of the current data retrieval request: In the formula, This indicates the priority level of the current data request k. This indicates the business level corresponding to the standard business type of the data retrieval request; This indicates a delay in the data transmission path of the requester. Represents the delay sensitivity coefficient. This indicates the complexity of the data call chain.

6. A digital operation and control platform based on shared service operation and management according to claim 5, characterized in that, The dynamic allocation unit specifically includes: Based on the analysis of the data call priority intensity of the data call request, when the data call priority intensity is within the set first priority range, a first-level data processing node is allocated to respond to the data call request, so that the first-level data processing node can directly access the shared database according to the data call request, directly obtain the mirror of the required business data from the shared database and transmit it to the requester. When the data call priority is within the set second priority range, a secondary data processing node is allocated to respond to the data call request. This allows the secondary data processing node to utilize the load balancer to allocate idle secondary data processing nodes to respond to the data call request and complete the forwarding of the required data. When the data call priority is within the set third priority range, a third-level data processing node is assigned to complete the response to the data call request. This allows the third data processing node to asynchronously obtain the required data through edge processing and then transmit the obtained data to the requester via mirror image.

Images