A data center design method for power marketing service system

Abstract

The present invention discloses a data middle platform design method for an electric power marketing service system, comprising: step 1, data aggregation: including data indicator combing, business master data combing, enterprise-level database table establishment and specification update, and building a data foundation for the electric power marketing service system; step 2, data fusion: organizing data in different subject domains of the business to form a common data layer model, and carrying out code development and deployment and maintenance; step 3, data extraction to provide a data foundation for upper-layer services; step 4, analysis result sedimentation; step 5, data service refinement and full-process data operation. This design method complies with the corresponding methodology, standard specifications and requirements, and the front and back links are mutually verified and iterated. After data aggregation, fusion and extraction, data support is provided for data services. Through data services and data operation management, data value mining is realized, and reverse empowerment of business is achieved.

Description

Technical Field

[0001] The present invention relates to the field of power marketing technology, and in particular to a method for designing a data center platform for a power marketing service system. Background Art

[0002] The layers of barriers between the various IT systems in the existing power industry hinder the effective flow of information within and outside companies. Much high-value data is confined to the narrow confines of their own systems, unable to realize its value within the larger ecosystem. Within enterprises, whether for specific topics, reports, or data collection, data production currently relies on a siloed or project-based approach. This prevents the accumulation and continuous development of data knowledge, hindering the development of models into reusable components and the ability to support rapid response and innovation in data analysis.

[0003] The data resources accumulated over years of operation in power marketing information systems are widely dispersed across heterogeneous systems. Data models and standards for related businesses are not unified, resulting in poor data consistency and liquidity. This prevents effective sharing and prevents maximizing data value. The core of the enterprise middle platform, consisting of the business middle platform and the data middle platform, supports the digitization of enterprise businesses, the commercialization of enterprise data, builds the enterprise's digital operations capabilities, and supports flexible business iteration. The enterprise middle platform not only refers to the information system support of the middle platform, but also includes the management system and standards that match it.

[0004] The data middle platform refers to the use of data technology to collect, calculate, store, and process massive amounts of data, while unifying data standards and calibers. It also includes the model services and algorithm services required in the process of building the data middle platform, as well as the organization, processes, standards, specifications, and management systems required for building the data middle platform. After the data middle platform unifies the data, it will form standard data, which will then be stored to form a big data asset layer. Through data mining and analysis tools, data service capabilities can be realized, thereby providing efficient services to customers or the ecosystem. At the same time, these services are strongly related to the company's business, are unique to the company, and can be reused. They are the accumulation of the company's business and data. They not only reduce the cost of duplication and siloed collaboration, but also provide differentiated competitive advantages, enhance the company's rapid innovation and help the company build a business ecosystem.

[0005] The core of the data middle platform is empowerment. By integrating enterprise data capabilities and supporting new IT technologies, it effectively enhances the ability of enterprises to operate, extend, and create businesses. With data flow as the connection and market-driven response as the driving force, it provides rapid data feedback across the entire region, reducing trial and error costs and accelerating response, helping enterprises build a business ecosystem. The data middle platform is the driving force for digital transformation of market-oriented competitive enterprises. It continuously evolves to produce new innovative services and products for enterprises. Combined with the collaborative linkage of the business middle platform, it effectively promotes the two-dimensional growth of enterprise business capabilities and operational efficiency, building a new support model of "setting the stage and performing the show", promoting the expansion of various businesses, and efficiently supporting smart operations. Summary of the Invention

[0006] In response to the problems existing in the prior art, the purpose of the present invention is to provide a data middle platform design method for an electric power marketing service system. The OneData, OneID, and OneService system methodology can be adopted as the construction methodology of the data middle platform, and a data middle platform service system including a basic data center, a global data center, an extraction data center, a data service center, etc. is built. The overall construction is carried out in accordance with the construction idea of ​​"two-line advancement, iterative evolution, and operation-driven". First, data access preparation and data standardized storage are carried out from the bottom up to realize the integration and precipitation of global marketing data; second, data analysis model construction and data service refinement are carried out from the top down to gradually realize service-oriented support for the business and refine marketing data sharing service capabilities.

[0007] In order to achieve the above-mentioned purpose, the present invention adopts the following technical solutions to achieve it.

[0008] A method for designing a data center platform for a power marketing service system includes the following steps:

[0009] Step 1: Data Aggregation: This includes data indicator analysis, business master data analysis, and the establishment and standardization of enterprise-level database tables. Using online data collection and access tools, we connect various businesses with external data, synchronize existing and incremental data, and gradually improve data timeliness from T+1 to near-real time based on business drivers, building the data foundation for the power marketing service system.

[0010] Step 2, Data Integration: Through the data unification system methodology, based on the standardized design of the middle platform, organize the data of different business subject domains to form a common data layer model, and carry out code development and deployment operations;

[0011] Step 3, Data Extraction: Guided by business needs analysis, establish a multi-dimensional business-oriented system and a data application layer for business scenario applications, complete data extraction, and provide a data foundation for upper-layer services;

[0012] Step 4: Analyze and summarize the results: Conduct a summary based on business needs, business application query requirements, and data maintenance requirements;

[0013] Step 5: Data service extraction and full-process data operation: Business personnel design data quality inspection rules and data operation mechanisms. The data team uses the data asset catalog and data quality rules after sorting out to implement information technology and conduct regular scans. As the business develops and time passes, the inspection rules are continuously iterated.

[0014] Compared with the prior art, the present invention has the following beneficial effects:

[0015] The data center design method for the power marketing service system provided in this paper establishes a data center service system that includes a basic data center, a global data center, an extraction data center, and a data service center. The design process adheres to relevant methodologies, standards, and requirements, with mutual verification and iteration between the previous and subsequent links. Through data aggregation, fusion, and extraction, data support is provided for data services. Through data services and data operations management, data value mining is achieved, which in turn empowers the business. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 A flowchart of an embodiment of a method for designing a data center platform for a power marketing service system provided by the present invention;

[0018] Figure 2 This is a schematic diagram of the data development implementation process provided by the present invention. DETAILED DESCRIPTION

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0020] refer to Figure 1This invention provides a method for designing a business platform for a power marketing service system. The design process consists of five steps: data aggregation, data fusion, data extraction, analysis result sedimentation, data service refinement, and full-process data operations. The process adheres to relevant methodologies, standards, and requirements, and each step is mutually verified and iterated. Data aggregation, fusion, and extraction provide data support for data services. Through data services and data operations management, data value mining is achieved, which in turn empowers the business.

[0021] The details are as follows:

[0022] Step 1: Data Aggregation: This includes data indicator sorting, business master data sorting, enterprise-level database table establishment, and standardized updates. Through online data collection and access tools, the system connects various businesses with external data, synchronizes existing and incremental data, and gradually increases data timeliness from T+1 to quasi-real time based on business drivers, building the data foundation for the power marketing service system.

[0023] Step 2, Data Integration: Through the data unification system methodology, based on the standardized design of the middle platform, organize the data of different business subject domains to form a common data layer model, and carry out code development and deployment operations;

[0024] Step 3, Data Extraction: Guided by business needs analysis, establish a multi-dimensional business-oriented system and a data application layer for business scenario applications, complete data extraction, and provide a data foundation for upper-layer services;

[0025] Step 4: Analyze and summarize the results: Conduct a summary based on business needs, business application query requirements, and data maintenance requirements;

[0026] Step 5: Data service extraction and full-process data operation: Business personnel design data quality inspection rules and data operation mechanisms. The data team uses the data asset catalog and data quality rules after sorting out to implement information technology and conduct regular scans. As the business develops and time passes, the inspection rules are continuously iterated.

[0027] Furthermore, in step 1, data aggregation is a dynamic process. To cope with different data update frequencies, it is necessary to be compatible with six data input methods: batch aggregation, streaming aggregation, local incremental aggregation, streaming to batch aggregation, streaming to local incremental aggregation, and streaming to zipper table aggregation, and select the appropriate aggregation method based on actual needs.

[0028] Batch aggregation is the most commonly used offline data aggregation method. This method reads full data snapshots from the source at a certain frequency and writes them in batches to the target. This can be done by overwriting or adding date partitions.

[0029] Streaming aggregation is the most common method for real-time data aggregation. This method is triggered by data changes (additions or modifications) at the data source, transmitting the changed data to the target. On the target, the changed data is written in an append-only manner or stored in partitions based on the change date.

[0030] Local incremental aggregation is a common method for incremental data aggregation. This method captures data changes (new additions and modifications) at the data source at a certain frequency. On the target side, new data is written to a new date partition; modified data is modified directly in the partition where the original data resides. For targets that don't support modification operations, a "refresh" method can be used to write the modified data over the unmodified data.

[0031] Stream-to-batch aggregation combines streaming and batch aggregation and can be built on top of streaming aggregation. On the data source side, data is read in a streaming manner, driven by data changes, and the changed data is transmitted to the target side. When the changed data accumulates to a certain scale or at a certain frequency, the target side merges historical data with the change log and writes it to a new partition in batches.

[0032] Streaming-to-local incremental convergence combines streaming and local incremental convergence, building on top of streaming convergence. On the data source side, data is read in a streaming manner, driven by data changes, and the changed data is transmitted to the target side. When the changed data accumulates to a certain scale or at a certain frequency, it is written to the target side. New data is written to a new date partition; modified data is modified directly in the partition where the original data resides. For targets that do not support modification operations, a "refresh" method can be used to write the modified data, overwriting the unmodified data.

[0033] Streaming zipper table aggregation is an extension of streaming aggregation and can be built on top of it. On the data source side, data is read in a streaming manner, driven by data changes, and the changed data is transmitted to the target side. When the changed data accumulates to a certain volume or at a certain frequency, it is written to the target side. The written data only contains the changed data and maintains the mapping relationship with the original record.

[0034] For further reference, Figure 2 When building a public data layer model, we must first conduct sufficient data research; secondly, divide the data domain according to dimensional modeling theory, build a bus matrix, abstract the business processes and dimensions, and abstract and organize the analysis and mining business needs; finally, design the physical model, carry out code development and deployment and operation and maintenance.

[0035] Specifically, it includes the following sub-steps:

[0036] Sub-step 2.1, data research: Research the business in the business system and conduct demand analysis.

[0037] Data research includes business research and demand analysis. Before and after the project officially launches, business research requires relevant business personnel to provide detailed business information to help understand and document the needs of analysts and operations staff from each team. There are two approaches to demand analysis: one is to understand the needs through communication with analysts and operations staff; the other is to research and analyze existing reports in the reporting system. After conducting demand research and analysis, the future format and style of the data are clarified. Often, specific data requirements drive the data warehouse team to understand the business system; there is no strict order between the two. The ultimate goal is to enable the data warehouse team to better understand the business data, data, format and style.

[0038] Sub-step 2.2, data domain division: Based on the results of the data survey, abstract the business processes or dimensions and divide the data domain according to the dimensional modeling theory.

[0039] A data domain is an abstract collection of business processes or dimensions for business analysis. Business processes can be summarized as inseparable behavioral events, such as user logins, favorites, and subscriptions. To ensure the vitality of the entire system, the data domain needs to be abstracted and refined, undergoing long-term maintenance and regular updates, but not easily changed. The basic requirements for dividing data domains are that they cover all current business needs and can be seamlessly incorporated into existing data domains or expanded into new ones when new businesses are launched. Data domain division can be conducted after business research, requiring analysis of the business activities within each business module.

[0040] Sub-step 2.3, constructing a bus matrix: This includes clarifying the data domains described by the business process and clarifying the relationship between the business process and the dimensions.

[0041] After conducting sufficient business research and demand research, it is necessary to build a bus matrix. During the construction process, two things need to be done: first, clarify which business processes are under each data domain; second, which dimensions the business processes are related to.

[0042] Sub-step 2.4, clarify statistical indicators: including clarifying atomic indicators and clarifying derived indicators.

[0043] Clarifying statistical indicators includes: (1) Indicators are divided into atomic indicators and derived indicators. Atomic indicators = business process + measurement; derived indicators = time period + modifier + atomic indicator. The creation of atomic indicators must be confirmed after the business process is defined, while the business process is already defined when the measurement is defined. (2) The creation of derived indicators generally needs to be carried out after understanding the specific reporting requirements and does not need to be created in the early stage.

[0044] Sub-step 2.5, specification definition: Based on the research and organization in the bus matrix, design consistency dimensions; based on the business processes in the bus matrix, abstract the metrics included in the business processes.

[0045] Sub-step 2.6, design the detailed model: first design the consistency dimension table, then design the consistency detail fact table.

[0046] Specifically, the following steps are followed: (1) Dimension table design. In the specification definition, the dimensions and their attributes have been defined. The next step is to design the "dimension table." (2) Detailed fact table design. In the consistency measurement, the procurement business process and its measurement are defined. The detailed fact table is essentially a model design for the business process. When designing a detailed fact table, we follow the four steps of fact table design: select the business process ---> determine the granularity ---> select the dimension ---> determine the fact (measurement). Granularity is more of a semantic description of the granularity of business activities when the dimension is not expanded. When the enterprise data team builds a detailed fact table, it needs to choose an existing table based on which to develop detailed layer data. This table can be called the "base table", so the base table must be analyzed and annotated for the business process in advance.

[0047] Sub-step 2.7, design summary model: first design the common summary model, then design the application summary model.

[0048] Summary tables are mainly divided into two categories: DWS and ADS. The process of building table models for these two types of summary tables is basically the same. The steps and processes for creating them using model tools are almost the same, except for the types of indicators stored. (Among them, the scope of indicator processing at the DWS summary layer includes: transactional indicators defined in the specification definition and inventory indicators (optional), this layer mainly corresponds to public data centers. The scope of indicator processing at the ADS summary layer includes: composite indicators defined in the specification definition, plus user-defined non-standard indicators), this layer mainly corresponds to extraction data centers.

[0049] Sub-step 2.8: Carry out code development and deployment operations.

[0050] Furthermore, in step 3, the extraction system of the data middle platform can be divided into three methods as a whole: first, data extraction driven by business processes with reference to mature methodologies of Internet companies; second, data extraction driven by operation or management indicators; third, data extraction guided by business analysis needs, and then deposited in the extraction center in the form of labels, models, indicators, etc., to provide support for upper-level service.

[0051] Among them, the business process-driven data extraction, based on the input of the full business model, sorts out the relationships between entities in the public data center, completes the entity relationship logic table, and provides a clear data context for subsequent extraction analysis. The specific steps are as follows: (1) With the cooperation of business personnel, based on the business model and data model, sort out the business entity relationship view with each major analysis object as the core, such as the customer ID entity relationship view, the device ID entity relationship view, etc. The diagram contains business object entities, business processes and result entities. (2) Based on the business entity relationship view, combined with the public data center data model, sort out the primary key and associated fields of the data object, determine the relevant data objects in the business process and the dependency relationships between objects. (3) Using the data objects and relationships in step 2 as input, determine the relationship view with the analysis object as the core, complete the object entity relationship logic table, and precipitate it into a data association analysis model with the user ID as the perspective in the extraction layer.

[0052] Among them, data extraction driven by operational or management indicators is divided into knowledge extraction and data indicator-driven extraction: (1) Knowledge extraction uses entity concepts as nodes and relationships as edges, providing a way to describe customers from a relationship perspective. Its overall extraction and fusion process includes the overall process of business concept combing, knowledge extraction, knowledge fusion, and knowledge calculation.

[0053] Among them, data extraction guided by business analysis needs analyzes business needs, sorts out the traceability information of data extraction indicators, and then carries out key processes such as data extraction task development and data extraction task execution based on the traceability information of indicators to complete data extraction and provide a data foundation for upper-level services.

[0054] (1) Data extraction indicator tracing: According to business needs, determine the indicators that need to be counted and extract data around the indicators. First, it is necessary to trace the indicators and sort out the basic information of the indicators, including the indicator name, category, indicator definition, business scope, analysis granularity, corresponding supporting data entities, corresponding supporting data attributes of analysis dimensions, indicator calculation data formulas and other key information.

[0055] The key information is listed below for explanation:

[0056]

[0057] (2) Data extraction task development: First, through the indicator definition, business scope and data model, determine the data entities, fields and indicator calculation formulas that support the indicator analysis; second, determine the analysis dimension based on the data attributes that support the analysis dimension; finally, guide the development of data extraction tasks based on the calculation formula, analysis dimension and other indicator requirement information (such as measurement unit, indicator accuracy, etc.).

[0058] (3) Data extraction task execution: The statistical cycle in the traceability information is used to determine the analysis granularity of the indicators, such as year / month / week / day / hour. The tasks are then scheduled in sequence as execution cycles to complete data extraction and to form a data extraction center.

[0059] Furthermore, in step 4, the analysis results are summarized and consolidated based on business needs, business application query requirements, and data maintenance requirements. Physically stored business application results are then provided to relevant business applications in a unified service format. Analysis results include, but are not limited to, customer tags, analysis models and results, customer relationship maps, and self-service analysis pages. The data center establishes a service capability system based on data, tags, models, strategies, and analytical capabilities.

[0060] Taking data services as an example, with the continuous evolution of data middle-office concepts and methods, the rapid construction of lightweight microservice APIs and the adoption of a serverless architecture for automatic elastic scaling have become a new generation of data service solutions. Thanks to the microservices architecture, API developers only need to focus on the API logic itself, without having to worry about the API service infrastructure. They can generate a data API using a visual API generation wizard or by customizing API query SQL. For APIs with particularly complex logic, they can also implement them using functions written in languages ​​such as Python, creating Function as a Service (FAS). Therefore, this new generation of data services can significantly improve API development efficiency, reduce service operation and maintenance costs, and facilitate unified management and planning, avoiding inefficient "siloed" customization and ensuring flexible and timely satisfaction of business needs. The data middle-office utilizes a unified data service bus to carry out business domain modeling, service architecture modeling, service design, implementation, governance, and evolution.

[0061] Furthermore, in step 5, data service extraction and full-process data operations support provide comprehensive functionality for data integration, processing, management, monitoring, and output services. This includes a visual workflow designer, enabling multi-person collaboration, role-based task development, online scheduling, operations and maintenance, and data permissions management. Complex operational processes can be completed without requiring data or tasks to be deployed. Simply focusing on data calculation logic and defining statistical indicators in a functional manner allows for model development and automated code generation for computational execution, including metadata management, data quality management, and data security management.

[0062] Specifically,

[0063] (1) Metadata management: Metadata management is the core control method of data management. As a bridge between business, technology and management, it provides auxiliary support for data security, data architecture and standards, and data lifecycle management, and promotes the goal of data value creation.

[0064] Management objectives:

[0065] Form a unified enterprise-level indicator system: Starting from the establishment of an enterprise-level indicator system, gradually incorporate business classification, business rules, etc. into business metadata management, provide a unified explanation of business processing, and improve data credibility.

[0066] Trace the source of data and perform data impact analysis: sort out the data source, data definition, data storage location, storage type and data relationship within the data system to form a unified metadata management to achieve data traceability

[0067] Ensure the orderly progress of the data processing process: demonstrate the correlation between data and processing, comprehensively, truthfully, intuitively and timely reflect the system business logic and actual technical implementation, make the entire process of system technology implementation transparent and visual, and ensure the orderly progress of the data processing process.

[0068] Assisting in data integration: Using metadata systems can help systems gain a deeper understanding of data, find effective means and methods for data integration, verify the corresponding rationality, and ensure the orderly progress of data integration.

[0069] Supporting changes in requirements: Using metadata management can make it easier for business personnel to understand the current status of the business or system, and increase the rationality of the requirements put forward.

[0070] Management objects:

[0071] To effectively support enterprise production and operations, it's necessary to include metadata objects generated throughout the lifecycle of each enterprise project within the scope of enterprise metadata management. Metadata can be divided into business metadata, technical metadata, and management metadata, depending on the subject it describes.

[0072] Business metadata is a description of concepts and relationships related to the business field, including business management specifications (business definitions, business management rules, business processes), business requirements, and indicators.

[0073] Technical metadata is a description of relevant concepts and relationships in the technical field, including metadata types such as system planning, system requirements, architecture metadata, model metadata, design metadata, program metadata, interface metadata, data encapsulation metadata, application metadata, and environment metadata.

[0074] Management metadata is a description of concepts and relationships related to the management field, including metadata types such as role metadata, permission metadata, administrator metadata, and management specification process metadata.

[0075] Management Organization:

[0076] Metadata management is an important part of enterprise data management, and the metadata management responsibility system follows the enterprise-level data management organizational system architecture.

[0077] Management Process:

[0078] The metadata management process primarily involves using phased plans to guide the collection and development of daily metadata issues, mobilizing business and support departments to collaborate on metadata management, and then evaluating the relevant results, forming a cyclical and continuously progressive process. There are seven metadata management processes: ① Metadata Acquisition Process: This is the process by which relevant departments initiate metadata additions and changes throughout the system lifecycle, based on the content requirements of metadata management objects; ② Daily Metadata Management Process: This is the process by which relevant departments initiate metadata changes throughout the system lifecycle, based on their business needs; ③ Metadata Quality Management Process: This describes the data quality review and assessment process for metadata management objects to ensure metadata quality reliability; ④ Metadata Assessment Process: This assessment process assesses the metadata management system's and organizational support capabilities, comprehensively assessing metadata from the source, process, and organization perspectives to ensure metadata quality reliability, validity, and authority. ⑤ Metadata Delisting Process: This is the metadata visualization process that synchronizes metadata with the actual system, ensuring the validity of the mapping relationship between metadata and the actual system. ⑥ Metadata Application Authorization Process: This is the metadata application authorization process that enables controllable metadata application management and ensures the security of metadata information. ⑦ Metadata issue post-evaluation process: The metadata issue post-evaluation process describes the evaluation process for related issues within the metadata work cycle. This process aims to complete the evaluation of metadata quality issues found in the work, and complete the metadata work rectification after proposing rectification suggestions for the corresponding issues.

[0079] Assessment method:

[0080] Metadata quality assessment comprehensively considers process and result assessments, and conducts a comprehensive assessment of metadata quality results and metadata quality management processes. It is divided into two major indicators: metadata quality health level and metadata quality management level.

[0081] (2) Data quality management:

[0082] Management objectives:

[0083] Establish a cross-disciplinary, full-process data quality control system: establish daily data quality management processes; establish a data quality management organizational responsibility system; formulate intra-disciplinary and cross-disciplinary data audit rules; formulate quality monitoring methods for the entire process of data generation, acquisition, processing, release and use; and establish a data quality inspection and assessment system.

[0084] Ensure data accuracy, standardization, completeness and consistency: Through cross-disciplinary data audit management and the application of a unified enterprise-level indicator system, we will gradually achieve a unified enterprise-level data caliber to ensure that data services are accurate, standardized and consistent.

[0085] Manage content:

[0086] The basic objects of data quality management include interfaces, jobs, entities, indicators, data applications, and environmental information.

[0087] Focusing on the objects of data quality management, the data quality management content framework is divided into three levels: business, technology and management.

[0088] The business class completes the definition of data quality management objects and audit rules, and is the business basis for data quality management, including quality requirements, audit rules, etc.

[0089] The technical category involves the implementation of monitoring, early warning, problem solving, and other tasks for data quality management objects.

[0090] Management is the support and guarantee for data quality management work, including management organization, management system, assessment methods, etc.

[0091] Management Organization:

[0092] Improving data quality cannot be accomplished by relying solely on a data management department or a professional production department, but must rely on an enterprise-level data management organizational responsibility system. Data quality management is divided into eight roles: business department, data service, data management, and data quality management.

[0093] Management Process:

[0094] Data quality management is to collect and evaluate existing business data through a series of interlocking control processes and measures, ultimately improving the quality of enterprise data. Specifically, 11 management processes reflect the work execution process.

[0095] Quality data collection process: Each data providing department submits quality data files in accordance with quality data specifications, which is the data source for data quality management.

[0096] Monitoring process: Monitor all types of monitoring data in accordance with audit rules, and distribute monitoring reports to relevant departments to ensure that quality data is monitored effectively in real time.

[0097] Early warning process: Implement early warning for various types of monitoring abnormal data according to early warning rules, and distribute the early warning data to various departments.

[0098] Complaint Process: The effectiveness and controllability of the complaint process fundamentally ensures the resolution of complaint issues. This process standardizes the complaint flow and ensures that tasks are handled effectively. This process controls the timing and quality of each step in the complaint processing process, facilitating tracking and analysis of the complaint processing process and timely identification of issues. This further enhances problem-solving and analysis capabilities.

[0099] Problem solving process: The problem solving process can classify problems according to specific circumstances and provide a variety of data quality problem solving methods; the data quality management department coordinates other departments to solve data quality problems.

[0100] Data quality reporting process: The data quality department regularly initiates the collection of requirements, and each relevant department submits relevant materials (abnormal monitoring reports, problem solving reports, etc.). After analysis and summary by the data quality department, a periodic data quality report is issued.

[0101] Data quality management assessment process: Assess the support capabilities of the data quality management system and the organizational support capabilities, and conduct a comprehensive assessment from the data source, process, and organization to ensure the reliability, validity, and authority of data quality.

[0102] Data quality rule management process: The data quality management role receives management requirements and business requirements and takes the lead in organizing the formation of different types of data quality management processes. The purpose is to formulate different types of rules to ensure the order and efficiency of rule management work.

[0103] Quality requirement change management process: Quality requirement change management is the business basis for data quality management. Quality requirement management includes the management of addition and change of quality requirements. Quality requirement management can effectively control the quality requirement version and ensure the orderly progress of data quality management work.

[0104] Data quality management process: The data quality management role receives data quality requirements (major problem reporting, key business data quality requirements, etc.) from business departments, data services, system maintenance, and provincial branch information technology departments to coordinate personnel from relevant departments to discuss and resolve data quality issues to ensure the timeliness, completeness, and accuracy of data.

[0105] Data quality post-assessment process: This process aims to complete the assessment of data quality issues found during work, and complete the rectification of data quality work after proposing rectification suggestions for the corresponding issues.

[0106] Assessment method:

[0107] Data quality assessment comprehensively considers process and result assessments, and conducts a comprehensive assessment of data quality results and data quality management processes. It is divided into two categories of indicators: data quality health level and data quality management level.

[0108] (3) Data security management:

[0109] Management objectives:

[0110] The goal of data security management is to establish a security control system for the entire data management process from the three aspects of personnel, environment and technology for various types of data to ensure the security and reliability of data assets.

[0111] Management objects:

[0112] Data security management objects include three categories: personnel, data, and environment.

[0113] Personnel objects are divided into company users and third-party personnel

[0114] Data class objects are divided into core data and non-core data according to their importance.

[0115] Environmental objects mainly refer to the software and hardware environment that data storage, transmission, and processing rely on

[0116] Management Organization:

[0117] Data security management is an important part of enterprise data management. The roles of data security management organizations mainly include business departments, data management, data services, data security management, system construction, system operation and maintenance, and infrastructure management.

[0118] Management Process:

[0119] Data security management involves understanding and measuring the current security status of data, implementing a series of control measures, and ultimately improving the security level of enterprise data, thereby continuously maintaining data security in a healthy state. The specific implementation of this work is primarily reflected in the management process.

[0120] Assessment method:

[0121] The assessment method for data security management follows the principles of publicity, objectivity, openness and regularity, and the assessment indicators are divided into result and process categories.

[0122] Based on the above embodiments, the present invention has developed a data middle platform design method for the power marketing system, adopting the OneData, OneID, and OneService system methodology as the data middle platform construction methodology, and built a data middle platform service system including a basic data center, a global data center, an extraction data center, a data service center, etc.

[0123] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A data center design method for a power marketing service system, characterized in that: The following steps are involved: Step 1: Data Aggregation: This includes data indicator analysis, business master data analysis, and the establishment and standardization of enterprise-level database tables. Using online data collection and access tools, we connect various businesses with external data, synchronize existing and incremental data, and gradually improve data timeliness from T+1 to near-real time based on business drivers, building the data foundation for the power marketing service system. Step 2, Data Integration: Through the data unification system methodology, based on the standardized design of the middle platform, organize the data of different business subject domains to form a common data layer model, and carry out code development and deployment operations; Step 3, Data Extraction: Guided by business needs analysis, establish a multi-dimensional business-oriented system and a data application layer for business scenario applications, complete data extraction, and provide a data foundation for upper-layer services; Step 4: Analyze and summarize the results: Conduct summaries based on business needs, business application query requirements, or data maintenance requirements. Step 5: Data service refinement and full-process data operations: Business personnel design data quality inspection rules and data operation mechanisms. The data team uses the data asset catalog and data quality rules to implement informationization and conduct regular scans. As the business develops and time passes, the inspection rules are continuously iterated. In step 1, the data aggregation method includes batch aggregation, streaming aggregation, local incremental aggregation, streaming to batch aggregation, streaming to local incremental aggregation and streaming to zipper table aggregation; Step 2 contains the following sub-steps: Sub-step 2.1, Data Research: Research the business in the business system and conduct demand analysis; Sub-step 2.2, Data Domain Division: Based on the results of the data survey, abstract the business processes or dimensions and divide the data domain according to dimensional modeling theory; Sub-step 2.3, constructing a bus matrix: This includes clarifying the data domains described by the business process and clarifying the relationship between the business process and the dimensions; Sub-step 2.4, clarify statistical indicators: including clarifying atomic indicators and clarifying derived indicators; Sub-step 2.5, Specification Definition: Based on the research and organization in the bus matrix, design consistency dimensions; based on the business processes in the bus matrix, abstract the metrics included in the business processes; Sub-step 2.6, design the detail model: first design the consistency dimension table, then design the consistency detail fact table; Sub-step 2.7, design the aggregation model: first design the common aggregation model, then design the application aggregation model; Sub-step 2.8: Carry out code development and deployment operations; The data extraction in step 3 adopts the following three methods: data extraction driven by business processes, data extraction driven by operational or management indicators, and data extraction guided by business analysis needs; Step 3 specifically includes the following sub-steps: Sub-step 3.1, Data Extraction and Indicator Tracing: Based on business needs, determine the indicators to be counted, trace the indicators' sources, and organize the basic information of the indicators; Sub-step 3.2, Data Extraction Task Development: First, determine the data entities, fields, and calculation formulas supporting the indicator analysis based on the indicator definition, business scope, and data model. Second, determine the analysis dimensions based on the corresponding data attributes. Finally, guide the development of the data extraction task based on the calculation formula, analysis dimensions, and other indicator requirements. Sub-step 3.3, data extraction task execution: Determine the analysis granularity of the indicator through the statistical cycle in the traceability information, and use it as the execution cycle to schedule tasks and complete data extraction.

Images