Enterprise ESG data processing method, device, equipment and medium

Abstract

The invention provides an enterprise ESG data processing method and device, equipment and a medium. The method comprises the following steps: acquiring enterprise ESG basic data from a set data source interface; processing the acquired ESG basic data based on an AI large model; storing the governed data by adopting a distributed hierarchical storage architecture; based on user permission display data, through the full-process design of automatic collection, AI intelligent treatment, distributed hierarchical storage and permission display, the technical problems that in traditional enterprise ESG data processing, collection efficiency is low, data quality is poor, calibers are not uniform, storage is not flexible, data security is difficult to guarantee, and an enabling decision cannot be made are solved.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a method, apparatus, equipment and medium for processing enterprise ESG data. Background Technology

[0002] There are three major pain points in current corporate ESG data governance: First, data sources are scattered, covering multiple heterogeneous sources such as corporate annual reports, regulatory announcements, third-party rating reports, sensor data, social media sentiment, and corporate rewards and punishments. Traditional collection methods are difficult to achieve comprehensive coverage and format uniformity. Second, data quality is uneven, with problems such as missing, redundant, contradictory, and semantically ambiguous data. Manual cleaning is inefficient and easily affected by subjective factors. Third, data presentation is monotonous, mostly in the form of static reports, which cannot meet the personalized analysis needs of different users (management, regulatory agencies, investors) and makes it difficult to uncover the underlying logic and potential risks of the data.

[0003] In existing technologies, ESG data governance largely relies on rule engines and traditional machine learning algorithms, which suffer from drawbacks such as poor cross-source data adaptability, weak semantic understanding capabilities, and insufficient visualization and interactivity. Large AI models, with their powerful natural language processing, multimodal understanding, and logical reasoning capabilities, offer a technological possibility for solving these problems. Summary of the Invention

[0004] The technical problem to be solved by this invention is to provide an enterprise ESG data processing method, device, equipment and medium. Through the full-process design of automated collection, AI intelligent governance, distributed hierarchical storage and permission-based display, it solves the technical problems of low collection efficiency, poor data quality, inconsistent standards, inflexible storage, difficulty in ensuring data security and inability to empower decision-making in traditional enterprise ESG data processing.

[0005] In a first aspect, the present invention provides a method for processing enterprise ESG data, comprising the following steps: S1. Collect basic ESG data of the enterprise from the set data source interface; S2. Governance of collected ESG basic data based on AI large-scale models, including: Build basic resources and establish a three-level feature library of "industry-enterprise-indicator". The industry-level features include industry code, indicator code, data distribution pattern, and anomaly judgment threshold. The enterprise-level features include enterprise ID, historical data sequence, trend features, and personalized threshold. The indicator-level features include indicator code, calculation logic, list of related indicators, and related constraint rules. Perform hierarchical anomaly detection, using a large AI model to perform industry-level feature detection, enterprise-level feature detection, and indicator-level feature detection, and generate anomaly data reports that include anomaly level, anomaly type, and judgment criteria; Perform data completion and correction for abnormal and missing data, through internal time-series reasoning, external industry benchmarking, and policy constraint correction, to generate completed data and confidence scores; To achieve data standardization, non-standard expressions are mapped to standard indicators through text vector conversion and semantic matching, with a similarity threshold set at 0.85. Construct a four-dimensional knowledge graph, establish a four-dimensional relational structure of "ESG data - business scenarios - policy standards - industry benchmarking", and store it in the Neo4j graph database; S3. Use a distributed, tiered storage architecture to store the managed data; S4. Display data based on user permissions.

[0006] Secondly, the present invention provides an enterprise ESG data processing apparatus, comprising: The data acquisition module collects basic ESG data from the designated data source interface. The data governance module, based on a large AI model, governs the collected ESG basic data, including: Build basic resources and establish a three-level feature library of "industry-enterprise-indicator". The industry-level features include industry code, indicator code, data distribution pattern, and anomaly judgment threshold. The enterprise-level features include enterprise ID, historical data sequence, trend features, and personalized threshold. The indicator-level features include indicator code, calculation logic, list of related indicators, and related constraint rules. Perform hierarchical anomaly detection, using a large AI model to perform industry-level feature detection, enterprise-level feature detection, and indicator-level feature detection, and generate anomaly data reports that include anomaly level, anomaly type, and judgment criteria; Perform data completion and correction for abnormal and missing data, through internal time-series reasoning, external industry benchmarking, and policy constraint correction, to generate completed data and confidence scores; To achieve data standardization, non-standard expressions are mapped to standard indicators through text vector conversion and semantic matching, with a similarity threshold set at 0.85. Construct a four-dimensional knowledge graph, establish a four-dimensional relational structure of "ESG data - business scenarios - policy standards - industry benchmarking", and store it in the Neo4j graph database; The data storage module uses a distributed, hierarchical storage architecture to store the processed data. The data display module displays data based on user permissions.

[0007] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect.

[0008] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect.

[0009] One or more technical solutions provided by this invention have at least the following technical effects or advantages: 1. This invention significantly improves the efficiency and stability of ESG data collection, ensuring data integrity from the source. Existing technologies mostly employ manual collection or simple batch retrieval methods, which suffer from low collection efficiency, susceptibility to interface rate limiting, and the need to repeat collection after interruption. This application adopts a "pagination / batch retrieval + Redis caching breakpoint resumption" mechanism. During pagination collection, requests are retrieved in a loop according to the configuration and the current page number is cached. During batch collection, requests are split into batches of 100, and a request interval of 500ms / 1s is used to avoid interface rate limiting. After collection is interrupted, collection can be resumed directly from the cached page number, improving efficiency by more than 80% compared to traditional collection methods. At the same time, an exponential backoff retry strategy is adopted to handle recoverable anomalies such as network fluctuations and 5xx errors. Combined with a timed refresh mechanism for authentication credentials (valid for 2 hours), the interface call is ensured to remain valid, and the collection success rate reaches over 99%. In addition, through the dual verification design of "field-level verification + order-of-magnitude verification", field-level verification ensures the completeness and format compliance of required fields, while order-of-magnitude verification controls the consistency between the collected data and the total number of records returned by the interface through a 5% difference rate threshold. When the difference rate exceeds the threshold, an alarm is automatically triggered and data is re-collected, effectively preventing missing or erroneous data from flowing into subsequent stages and improving the basic quality of data from the source. 2. This invention achieves intelligent and precise ESG data governance, significantly improving data quality and standardization. Existing ESG data governance technologies largely rely on manual rules, which suffer from problems such as low accuracy in anomaly detection, lack of basis for data completion, and difficulty in unifying standards. This invention relies on a three-tiered feature library of "industry-enterprise-indicator" and a large AI model to achieve hierarchical anomaly detection: industry-level anomalies are determined by the 5% / 95th percentile; enterprise-level anomalies are determined by time-series trend deviation combined with verification of major event correlations; and indicator-level anomalies are determined by the logical consistency of association rules. The anomaly identification accuracy is improved to over 95%, effectively avoiding false positives and false negatives. Through a triple completion logic of "internal time-series reasoning + external industry benchmarking + policy constraints," time-series reasoning is based on historical data prediction, industry benchmarking matches 10-20 comparable enterprises for weighted calculation, and policy constraints meet compliance requirements. At the same time, a 95% confidence interval is quantified and manual verification is triggered for low-confidence results, ensuring both data integrity and the rationality of data completion. With the help of semantic matching (similarity threshold ≥0.85), non-standard indicators are accurately mapped to standard indicators, and numerical units are unified (e.g., carbon emissions are unified to tons of carbon dioxide equivalent). This completely solves the industry pain point of inconsistent ESG indicator standards across data sources and documents, with a standardization rate of over 95%. 3. This invention constructs a highly efficient and flexible distributed hierarchical storage architecture, achieving an optimal balance between performance and cost. Existing technologies often use a single storage engine to store ESG data, resulting in slow access speeds for high-frequency data and high costs for storing massive amounts of cold data. This application divides the storage system into three layers—hot, warm, and cold—based on data access frequency and characteristics: The hot data layer (high-frequency data within the last year) uses a combination of MySQL and Redis, with Redis caching hot data (expiring in 1 hour) to ensure a high-frequency query response time of ≤1 second; the warm data layer (historical data from 1 to 3 years) uses a MySQL partitioned table, partitioning by time to improve query efficiency; the cold data layer (result data and unstructured documents older than 3 years) uses MinIO object storage, archived by "year-month-data type," with a 5-year retention period for cold data. Compared to traditional single-storage methods, storage costs are reduced by more than 60%. Simultaneously, the DataX data synchronization tool ensures data consistency across multiple storage engines (MySQL, Neo4j, MinIO, and Redis), and mechanisms such as MySQL transactions and MD5 checksums ensure no data loss or tampering during writes, solving the problems of scattered data storage and difficulty in guaranteeing consistency in traditional ESG data storage. 4. This invention achieves fine-grained data security management and adapts to the business needs of multiple roles. Existing ESG data display technologies often lack a robust access control mechanism, which can easily lead to the leakage of sensitive data. This application adopts a JWT+RBAC access control mechanism, dividing roles into five categories: system administrator, operations and maintenance personnel, business analysts, enterprise users, and auditors. Through a full-process access control of "login token generation - request carrying token - access verification - data filtering," it achieves "data isolation and access control to individuals": enterprise users can only view their own enterprise's data, business analysts can view authorized industry / enterprise data, and system administrators have the highest privileges, effectively preventing the leakage of sensitive data and meeting ESG data compliance management requirements. Simultaneously, the data display process supports full-scenario functions such as data query, trend analysis, industry comparison, graph visualization, early warning management, data export, and manual processing, realizing a business closed loop of "data-analysis-decision-handling" and adapting to the core needs of different roles. 5. This invention upgrades data processing to data empowerment, providing precise support for ESG management decisions. Existing ESG data processing technologies often remain at the data cleaning level, failing to achieve deep integration of data with business, policies, and industries. This application constructs a four-dimensional knowledge graph of "ESG data – business scenarios – policy standards – industry benchmarking," establishing relationships between data nodes and business scenarios, policy provisions, and industry benchmarks. Combined with Neo4j's multi-hop inference capabilities, it can automatically generate compliance risk warnings and root cause analyses of abnormal indicators, providing precise data support for enterprise ESG compliance rectification, industry benchmarking, and strategic optimization, significantly improving the scientific rigor and efficiency of ESG management. The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and in order to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0010] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0011] Figure 1 This is a flowchart of the method in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the device in Embodiment 2 of the present invention. Detailed Implementation

[0012] The overall concept of the technical solution in this application is as follows: The system is divided into four main modules: data acquisition, data processing, data storage, and visualization.

[0013] Data Acquisition: Various basic data are obtained by calling the API interface provided by Fujian Big Data Group.

[0014] Data processing: Intelligent data cleaning and integration driven by AI-powered large-scale models.

[0015] Data storage: Stores a three-level feature library, a four-dimensional knowledge graph, and all process and result data.

[0016] Visualization: The processed data is presented to users in a visual interface for easy viewing.

[0017] The specific implementation steps are as follows: S1: Through the data acquisition module, it calls the data source interface provided by Fujian Big Data Group to obtain structured and unstructured enterprise ESG basic data. This includes: S11: Construct request parameters according to the interface documentation requirements. Construct the data in JSON format, ensuring that field names and data types match the API requirements. Also include authentication information, data format, and request source identifier.

[0018] S12: Pagination / Batch Data Collection The pagination method will request data in a loop using the page (page number) and pageSize (number of items per page) parameters until all data is retrieved; the batch method will split the ID to be queried according to the maximum batch size limit of the interface, construct request parameters in batches, and retrieve data in batches.

[0019] S13: Data Reception and Storage Parse the response data (JSON / XML format), extract core fields, and retain the complete original response data for easy troubleshooting. Simultaneously, store the parsed field data in a temporary cache to prevent data loss due to subsequent processing failures.

[0020] S14: Data Integrity Verification Check if required fields are missing and if field formats meet requirements. Data that does not meet the rules is marked as "pending processing," and the reason for the validation failure is recorded. At the same time, compare the total number of records collected this time with the "total number of records" returned by the interface. If the difference exceeds the threshold, trigger an alarm and re-collect data.

[0021] S15: Handling API Call Exceptions For recoverable anomalies, an exponential backoff retry strategy is implemented to ensure complete data collection. If a retry fails, an anomaly log is recorded and an alarm is triggered.

[0022] S2: A data processing module driven by AI large-scale models, which manages the basic data using AI large-scale models such as DeepSeek and Tongyi Qianwen. This includes: S21: Based on a pre-built three-level feature library of "industry-enterprise-indicator", a large model is used to perform hierarchical anomaly detection and graded processing on the input data. Industry-level features: By learning the distribution patterns of ESG data in multiple key industries through large-scale models, industry-specific anomaly detection thresholds are established (large-scale models are learned through manually labeled data, and different anomaly detection thresholds can be defined according to different industries); for example, a reasonable range of 30%-40% for "R&D investment as a percentage of revenue" is set for the fintech industry, and anomaly detection is triggered if a company's data exceeds this range.

[0023] Furthermore, the industry dynamic baseline update mechanism: the system automatically calls the large model periodically (e.g., quarterly) to recalculate the distribution patterns and thresholds based on newly added publicly available industry data, thereby realizing the dynamic evolution of the industry baseline and ensuring the timeliness of the benchmark.

[0024] Furthermore, the threshold setting method: the anomaly detection threshold is not a fixed value, but a dynamic probability distribution learned by a large model. The system uses the probability of a data point being placed at the tail of this distribution (such as below the 5th percentile or above the 95th percentile) as the trigger condition, which is more adaptable to the long-tail distribution in the industry than a fixed interval.

[0025] Enterprise-level features: Utilize large models to perform time-series analysis on the target company's historical ESG data, extract its own data change trends, and form personalized anomaly detection benchmarks (which are expressed in the form of historical data change trends, such as: if the change trend is abnormally obvious in a certain period of time, it is the threshold of the previous change trend); for example, analyze the downward trend of the company's "total greenhouse gas emissions" over the years, and if the data shows an abnormal surge in a certain year, it is marked as an anomaly.

[0026] Furthermore, trend decomposition and event correlation: When performing time series analysis, large models need to decompose the series into trend terms, seasonal terms, and residual terms. For significant anomalies in the residual terms, the system will cross-validate them with publicly disclosed major events of the company during the same period (such as mergers and acquisitions, expansions, and accidents). If there are correlated events, the anomaly level will be downgraded; otherwise, the level will be upgraded.

[0027] Indicator-level features: By analyzing the connotation and calculation logic of each ESG indicator through a large model, we identify the correlation and contradictions between indicators (by semantically analyzing the correlation of indicators through the large model to determine whether there are logical conflicts; the purpose is to obtain high-quality, real ESG data to provide a basis for subsequent business processing. Finally, it is provided to users in the form of documents, tables, interfaces, etc., so that they can use it as a basis for handling subsequent business); for example, if it is detected that "employee information security training coverage" is 100% but "information security leakage events" is greater than 0, it is judged as a logical conflict anomaly.

[0028] Furthermore, the system constructs and infers the association constraint library: It pre-defines an indicator association rule library, including strong logical associations (such as mathematical derivation relationships), weak statistical associations (such as historical positive / negative correlations), and common-sense associations. During detection, the large model not only determines whether strong rules are violated, but also provides probabilistic anomaly alerts based on weak associations and common sense (e.g., a significant drop in "employee satisfaction" but no significant increase in "employee turnover rate" suggests potential data lag or survey bias).

[0029] S22: For abnormal or missing data, a large model completion and correction strategy is adopted that integrates internal time-series reasoning, external industry benchmarking, and policy constraints (if the large model can be automatically completed and corrected through correlation calculation, it will be automatically completed and corrected; if it cannot be completed through correlation calculation, manual completion and correction are required). Internal inference: Based on the temporal correlation of the company's own historical ESG data, the missing data is supplemented by the time-series prediction algorithm of the large model. If there is a large gap in the historical prediction, an alarm will be issued and manual intervention will be required to verify it. For example, the missing value of the fourth quarter of the year will be supplemented based on the "total investment in public welfare" of the first three quarters.

[0030] Furthermore, uncertainty quantification and confidence labeling: When providing completed values, large models must also output the confidence interval or uncertainty score for those values. Completed results with low confidence (such as large prediction variance due to insufficient historical data) must be clearly labeled during storage and display for user reference.

[0031] External benchmarking: The large model retrieves ESG data from comparable companies of the same industry and size, and performs weighted calculations based on the characteristics of the target company to generate suggested values ​​for completion; for example, for companies that are missing the "percentage of female management", the average value of listed companies in the same industry (fintech) is used to complete the data.

[0032] Furthermore, intelligent selection of comparable enterprises: The large model is not only based on industry and scale, but also on multiple dimensions such as financial indicators (such as revenue and profit margin), region, and technical attributes (such as number of patents). Through clustering or similarity calculation, it can dynamically identify the most comparable enterprise group, making the benchmarking more accurate.

[0033] Policy constraints: Incorporate the latest ESG policy requirements as lower or upper limits for data completion; for example, when completing the "labor contract signing rate", the result must be 100% to meet the requirements of the "Labor Law of the People's Republic of China".

[0034] Furthermore, the policy knowledge base is linked to the clauses: the system maintains a structured policy knowledge base, where each policy clause is parsed into computable constraints (such as "≥100%" or "≤emission standard X"). During completion, the large model needs to retrieve and apply all relevant clauses, and record the specific policy clause number used in the completion log.

[0035] S23: Utilize a large model to perform semantic parsing and standard mapping (based on standards such as GRI specifications) on the same ESG indicator data from different sources, thereby achieving data standardization. The large model automatically identifies the definition text in the data source documents and extracts key limiting conditions; for example, it analyzes whether "carbon emissions" in enterprise reports includes "Scope 3 (indirect emissions from the supply chain)".

[0036] The extracted non-standard definitions are mapped to the standardized definitions in the ESG semantic map; for example, "carbon emissions from direct and indirect electricity use by the company" is mapped to the standard definition "greenhouse gas emissions in range two".

[0037] Numerical transformation calculations are performed using a large model based on mapping relationships and industry parameters.

[0038] Furthermore, the caliber resolution and mapping process is as follows: (1) Document Slicing and Key Information Extraction: First, a text segmentation model is used to slice long documents (such as annual reports) by chapters and paragraphs. Then, for each segment containing numerical values, the large model locates and extracts five core elements: indicator name, numerical value, unit, calculation method description, and statistical period.

[0039] (2) Semantic disambiguation and standard mapping: The extracted "indicator name" and "calculation scope description" are input into the large model and matched with the standard ESG indicator semantic map. Each standard indicator node in the map is associated with all possible aliases, common non-standard expressions, and precise calculation scope definitions. The large model completes the mapping from non-standard expressions to standard indicators.

[0040] S24: Based on the high-quality data after governance, construct a four-dimensional knowledge graph of "ESG data - business scenarios - policy standards - industry benchmarking" to achieve deep data correlation and integration. Data layer: Structured indicator data after correlation governance, such as a company's "R&D investment: RMB 2.458 billion" and "Total greenhouse gas emissions: 8,544.84 tons of carbon dioxide equivalent" in 2024.

[0041] Business scenario layer: Establish the correlation between enterprise business processes and ESG indicators, such as associating the "software development and testing" business scenario with indicators such as "number of quality incidents" and "percentage of product technical employees".

[0042] Policy and Standards Layer: Link relevant policy provisions, such as associating the "Guaranteeing Data Security" chapter with specific provisions of the "Cybersecurity Law of the People's Republic of China" and the "Data Security Law of the People's Republic of China".

[0043] Industry benchmarking layer: Access industry benchmark data, such as comparing a company's "37.36% R&D investment ratio" with the "industry average of the top 100 fintech companies".

[0044] Based on this map, the system can respond to user queries, such as answering "What industry best practices and policy incentives can be referenced to reduce carbon emissions in product operation?" Furthermore, the dynamic reasoning and early warning functions of the graph include: Proactive compliance risk warning: The "policy and standards layer" nodes of the knowledge graph contain specific compliance thresholds. The system periodically traverses the graph, and when the value of an indicator node in the "data layer" exceeds its associated policy threshold, it automatically generates a compliance risk warning event and pushes it to relevant users.

[0045] Root cause analysis and suggestion generation: When a user queries an abnormal indicator (such as "Why is carbon emission intensity rising?"), the system can use graphs to perform multi-hop reasoning. For example, it can link carbon emission nodes to business scenarios such as a surge in "production volume", and then to supplier changes such as "increased transportation distance of raw materials", thereby generating insight reports that include root causes and improvement suggestions (such as "optimize supplier location"), rather than simply presenting numerical comparisons.

[0046] S3: A tiered storage system is built using a distributed storage architecture through the data storage module, including... S31: Data Classification and Storage Mapping The system receives three types of data output from the data processing module: feature data (basic features, derived features, and higher-order features), knowledge graph data (entities, relationships, attributes, and time information), and process and result data.

[0047] Complete data labeling according to preset classification rules. For feature data: add feature type tags ("Basic Feature - Environmental Category - Emission Concentration"), feature extraction timestamps, and data confidence scores. For example, add the tag "Basic Feature - Environmental Category - COD Emission Concentration", timestamp "2025-011-30 09:00", and confidence score "98%". For knowledge graph data: add classification tags according to four dimensions (entity / relationship / attribute / time) and associate them with unique entity IDs. For example, add tags "Entity - Enterprise", "Relationship - Compliance", "Attribute - Industry", and "Time - 2025-11", and associate them with entity ID "ENT_0001" (corresponding to Enterprise A). For process data: label the data source interface, processing stage (collection / cleaning / integration), and processing log ID. For example, the annotation interface is "FJ_BIGDATA_ESG_E_001", the processing stage is "cleaning", and the log ID is "LOG_CLEAN_0023"; the result data includes the annotation data standardization version number and the output time. For example, the annotation version number is "V1.0" and the output time is "2025-11-30 10:00". Establish a mapping relationship between data and storage engines: Three-level feature library data (feature vectors): stored in a relational database, with data tables divided according to feature hierarchy and relationships established using feature ID as the primary key; Four-dimensional knowledge graph data: stored in a graph database, with entity data stored as graph nodes (including attribute key-value pairs) and relational data stored as node-related edges (including time attributes); Process data (structured logs, intermediate results): stored in a relational database, with log tables divided according to processing stages; Result data (standardized datasets, unstructured output files): stored in an object storage system, with a directory structure established according to "year-month-data type," and metadata information synchronized to the relational database.

[0048] S32: Data Writing and Consistency Guarantee When writing feature data, multi-dimensional indexes are built simultaneously (e.g., by feature type, data time range, and ESG dimension) to improve subsequent query efficiency. When knowledge graph data is written, the graph database's relationship verification is triggered, and missing entity relationships are automatically filled in (e.g., the relationship between enterprise entities and corresponding policy entities) to ensure the integrity of the graph; When process and result data are written, a data checksum (such as an MD5 value) is generated synchronously and stored in the metadata for subsequent data integrity verification.

[0049] S4: Present the processed data to relevant users to facilitate business analysis and decision-making. Based on different user permission levels, allow users to view a range of data, ensuring data isolation and security. Example 1

[0050] like Figure 1 As shown, this embodiment provides a method for processing enterprise ESG data, including the following steps: S1. Collect basic ESG data of the enterprise from the set data source interface; S2. Governance of collected ESG basic data based on AI large-scale models, including: Build basic resources and establish a three-level feature library of "industry-enterprise-indicator". The industry-level features include industry code, indicator code, data distribution pattern, and anomaly judgment threshold. The enterprise-level features include enterprise ID, historical data sequence, trend features, and personalized threshold. The indicator-level features include indicator code, calculation logic, list of related indicators, and related constraint rules. Perform hierarchical anomaly detection, using a large AI model to perform industry-level feature detection, enterprise-level feature detection, and indicator-level feature detection, and generate anomaly data reports that include anomaly level, anomaly type, and judgment criteria; Perform data completion and correction for abnormal and missing data, through internal time-series reasoning, external industry benchmarking, and policy constraint correction, to generate completed data and confidence scores; To achieve data standardization, non-standard expressions are mapped to standard indicators through text vector conversion and semantic matching, with a similarity threshold set at 0.85. Construct a four-dimensional knowledge graph, establish a four-dimensional relational structure of "ESG data - business scenarios - policy standards - industry benchmarking", and store it in the Neo4j graph database; S3. Use a distributed, tiered storage architecture to store the managed data; S4. Display data based on user permissions.

[0051] In this embodiment, preferably, S1 specifically comprises: Perform interface authentication initialization, read authentication information from the configuration file, complete identity verification through the authentication interface, obtain access credentials and store them in the Redis cache, set the expiration time and start a scheduled refresh task; Construct request parameters, define a request parameter structure containing basic parameters and filter parameters, validate parameter data types, value ranges and the completeness of required fields using validation functions, and serialize the validated parameters into JSON strings; Perform pagination or batch collection. For pagination collection, calculate the total number of pages based on the total number of items (totalCount) and the number of items per page (pageSize), and send requests in a loop until collection is complete. For batch collection, split the enterprise ID list into multiple sub-lists according to the maximum batch limit of the interface, and send requests in batches. The system receives and parses data, determines the data format based on the Content-Type response header, and extracts core fields such as enterprise ID, indicator name, indicator value, statistical time, and data source to form a standardized basic data structure. Perform data integrity checks, including field-level and order-of-magnitude checks, generate a check report, and process the data that passes the checks.

[0052] In this embodiment, preferably, the pagination collection specifically includes: Initiate an initial request with page=1 and pageSize=100 parameters to obtain the first page of data and the total number of records (totalCount). Calculate the total number of pages using the formula totalPage = ceil(totalCount / pageSize); Starting from page=2, requests are initiated in a loop, with each request interval set to 500ms, until page>totalPage; After each data collection is completed, the current page number is stored in the Redis cache. Upon restart, data collection continues from the cached page number. The data integrity verification includes: Field-level validation: Validate whether the core fields of enterprise ID, indicator name, indicator value, and statistical time are missing; validate whether the statistical time format is the set format; validate whether the indicator value is a numeric type; and validate whether the enterprise ID is a string with a set fixed length. Order of magnitude verification: Calculate the difference rate = |number of collected records - total number of records| / total number of records. If the difference rate > 5%, trigger an alarm and automatically start re-collection. If the difference rate ≤ 5%, record the difference information.

[0053] In this embodiment, preferably, the hierarchical anomaly detection in S2 specifically includes: Industry-level feature detection: The DeepSeek-67B model is used to calculate the quantile of the data to be detected in the industry data distribution. If it is lower than the 5th quantile or higher than the 95th quantile, it is marked as "industry-level anomaly". Enterprise-level feature detection: The historical data sequence is decomposed into trend, seasonal and residual terms using the LSTM algorithm, and the deviation between the current data and the trend term is compared to determine anomalies; Indicator-level feature detection: Verify the logical consistency between indicators through large AI models, and identify logical conflicts and weak correlation anomalies.

[0054] In this embodiment, preferably, the data completion and correction in S2 includes: Internal time-series inference completion: Based on at least 3 consecutive historical data, the missing value is predicted using an AI large model, and a 95% confidence interval is calculated. If the span of the confidence interval exceeds 50% of the fluctuation range of the historical data, it is marked as low confidence. External industry benchmarking and completion: Using the DeepSeek-67B model, 10-20 selected companies are clustered based on multi-dimensional features such as industry, size, revenue, profit margin, region, and number of patents, and the weighted average completion suggestion value is calculated. Policy constraint modification: Relevant policy provisions are retrieved from the policy knowledge base, and policy constraints are used as hard rules for supplementation and modification.

[0055] In this embodiment, preferably, the data standardization in S2 includes: The document is sliced ​​by chapter and paragraph using a large AI model, with each slice limited to 500 characters. Extract five core elements from the slice containing numerical values: indicator name, value, unit, calculation method description, and statistical period. Hugging Face Transformers converts indicator names and calculation method descriptions into text vectors. The similarity between the text vector and the standard indicator vector in the ESG semantic graph is calculated. When the similarity is ≥0.85, the mapping is completed automatically. When the similarity is <0.85, it is marked as "ambiguous" and manual confirmation is triggered. The construction of the four-dimensional knowledge graph in S2 includes: Data layer: Store enterprise ID, indicator code, value, and statistical time as data nodes in Neo4j; Business Scenario Layer: Establish business scenario nodes for software development and testing, manufacturing, and supply chain management, and associate ESG indicator nodes through "business scenario-indicator" relationship edges; Policy Standards Layer: Policy clauses are used as policy nodes, and data layer indicator nodes are linked through "policy-set indicators" and "policy-set constraints" relationship edges; Industry benchmarking layer: Establish industry benchmark nodes and associate data layer indicator nodes through the "enterprise indicator - industry benchmark" relationship edge.

[0056] In this embodiment, preferably, S3 specifically comprises: A hierarchical storage system is established. The hot data layer uses a combination of MySQL and Redis to store feature data for the past year, the warm data layer uses MySQL partitioned tables to store process data for 1-3 years, and the cold data layer uses MinIO objects to store result data for more than 3 years. Perform data classification and storage mapping, store feature data in MySQL feature data table, knowledge graph data in Neo4j, process data in MySQL process data table, and result data in MinIO object storage; To ensure data write consistency, MySQL uses a transaction mechanism, Neo4j performs correlation verification, and MinIO generates MD5 checksums. Specifically, S5 is: Establish a permission system and define five roles: system administrator, operations and maintenance personnel, business analyst, enterprise user, and auditor. Implement permission control through JWT tokens and role-based access control. It provides data query, visualization analysis, early warning management, and data export functions. The front end uses the Vue 3.0 framework, the visualization uses the ECharts 5.4 component, and the back end uses the FastAPI framework.

[0057] Based on the same inventive concept, this application also provides an apparatus corresponding to the method in Embodiment 1, as detailed in Embodiment 2. Example 2

[0058] like Figure 2 As shown, this embodiment provides an enterprise ESG data processing device, including: The data acquisition module collects basic ESG data from the designated data source interface. The data governance module, based on a large AI model, governs the collected ESG basic data, including: Build basic resources and establish a three-level feature library of "industry-enterprise-indicator". The industry-level features include industry code, indicator code, data distribution pattern, and anomaly judgment threshold. The enterprise-level features include enterprise ID, historical data sequence, trend features, and personalized threshold. The indicator-level features include indicator code, calculation logic, list of related indicators, and related constraint rules. Perform hierarchical anomaly detection, using a large AI model to perform industry-level feature detection, enterprise-level feature detection, and indicator-level feature detection, and generate anomaly data reports that include anomaly level, anomaly type, and judgment criteria; Perform data completion and correction for abnormal and missing data, through internal time-series reasoning, external industry benchmarking, and policy constraint correction, to generate completed data and confidence scores; To achieve data standardization, non-standard expressions are mapped to standard indicators through text vector conversion and semantic matching, with a similarity threshold set at 0.85. Construct a four-dimensional knowledge graph, establish a four-dimensional relational structure of "ESG data - business scenarios - policy standards - industry benchmarking", and store it in the Neo4j graph database; The data storage module uses a distributed, hierarchical storage architecture to store the processed data. The data display module displays data based on user permissions.

[0059] In this embodiment, preferably, the data acquisition module specifically comprises: Perform interface authentication initialization, read authentication information from the configuration file, complete identity verification through the authentication interface, obtain access credentials and store them in the Redis cache, set the expiration time and start a scheduled refresh task; Construct request parameters, define a request parameter structure containing basic parameters and filter parameters, validate parameter data types, value ranges and the completeness of required fields using validation functions, and serialize the validated parameters into JSON strings; Perform pagination or batch collection. For pagination collection, calculate the total number of pages based on the total number of items (totalCount) and the number of items per page (pageSize), and send requests in a loop until collection is complete. For batch collection, split the enterprise ID list into multiple sub-lists according to the maximum batch limit of the interface, and send requests in batches. The system receives and parses data, determines the data format based on the Content-Type response header, and extracts core fields such as enterprise ID, indicator name, indicator value, statistical time, and data source to form a standardized basic data structure. Perform data integrity checks, including field-level and order-of-magnitude checks, generate a check report, and process the data that passes the checks.

[0060] In this embodiment, preferably, the pagination collection specifically includes: Initiate an initial request with page=1 and pageSize=100 parameters to obtain the first page of data and the total number of records (totalCount). Calculate the total number of pages using the formula totalPage = ceil(totalCount / pageSize); Starting from page=2, requests are initiated in a loop, with each request interval set to 500ms, until page>totalPage; After each data collection is completed, the current page number is stored in the Redis cache. Upon restart, data collection continues from the cached page number. The data integrity verification includes: Field-level validation: Validate whether the core fields of enterprise ID, indicator name, indicator value, and statistical time are missing; validate whether the statistical time format is the set format; validate whether the indicator value is a numeric type; and validate whether the enterprise ID is a string with a set fixed length. Order of magnitude verification: Calculate the difference rate = |number of collected records - total number of records| / total number of records. If the difference rate > 5%, trigger an alarm and automatically start re-collection. If the difference rate ≤ 5%, record the difference information.

[0061] In this embodiment, preferably, the hierarchical anomaly detection in the data governance module specifically includes: Industry-level feature detection: The DeepSeek-67B model is used to calculate the quantile of the data to be detected in the industry data distribution. If it is lower than the 5th quantile or higher than the 95th quantile, it is marked as "industry-level anomaly". Enterprise-level feature detection: The historical data sequence is decomposed into trend, seasonal and residual terms using the LSTM algorithm, and the deviation between the current data and the trend term is compared to determine anomalies; Indicator-level feature detection: Verify the logical consistency between indicators through large AI models, and identify logical conflicts and weak correlation anomalies.

[0062] In this embodiment, preferably, the data completion and correction in the data governance module includes: Internal time-series inference completion: Based on at least 3 consecutive historical data, the missing value is predicted using an AI large model, and a 95% confidence interval is calculated. If the span of the confidence interval exceeds 50% of the fluctuation range of the historical data, it is marked as low confidence. External industry benchmarking and completion: Using the DeepSeek-67B model, 10-20 selected companies are clustered based on multi-dimensional features such as industry, size, revenue, profit margin, region, and number of patents, and the weighted average completion suggestion value is calculated. Policy constraint modification: Relevant policy provisions are retrieved from the policy knowledge base, and policy constraints are used as hard rules for supplementation and modification.

[0063] In this embodiment, preferably, the data standardization in the data governance module includes: The document is sliced ​​by chapter and paragraph using a large AI model, with each slice limited to 500 characters. Extract five core elements from the slice containing numerical values: indicator name, value, unit, calculation method description, and statistical period. Hugging Face Transformers converts indicator names and calculation method descriptions into text vectors. The similarity between the text vector and the standard indicator vector in the ESG semantic graph is calculated. When the similarity is ≥0.85, the mapping is completed automatically. When the similarity is <0.85, it is marked as "ambiguous" and manual confirmation is triggered. The construction of the four-dimensional knowledge graph in the data governance module includes: Data layer: Store enterprise ID, indicator code, value, and statistical time as data nodes in Neo4j; Business Scenario Layer: Establish business scenario nodes for software development and testing, manufacturing, and supply chain management, and associate ESG indicator nodes through "business scenario-indicator" relationship edges; Policy Standards Layer: Policy clauses are used as policy nodes, and data layer indicator nodes are linked through "policy-set indicators" and "policy-set constraints" relationship edges; Industry benchmarking layer: Establish industry benchmark nodes and associate data layer indicator nodes through the "enterprise indicator - industry benchmark" relationship edge.

[0064] In this embodiment, preferably, the data storage module specifically comprises: A hierarchical storage system is established. The hot data layer uses a combination of MySQL and Redis to store feature data for the past year, the warm data layer uses MySQL partitioned tables to store process data for 1-3 years, and the cold data layer uses MinIO objects to store result data for more than 3 years. Perform data classification and storage mapping, store feature data in MySQL feature data table, knowledge graph data in Neo4j, process data in MySQL process data table, and result data in MinIO object storage; To ensure data write consistency, MySQL uses a transaction mechanism, Neo4j performs correlation verification, and MinIO generates MD5 checksums. The data display module specifically includes: Establish a permission system and define five roles: system administrator, operations and maintenance personnel, business analyst, enterprise user, and auditor. Implement permission control through JWT tokens and role-based access control. It provides data query, visualization analysis, early warning management, and data export functions. The front end uses the Vue 3.0 framework, the visualization uses the ECharts 5.4 component, and the back end uses the FastAPI framework.

[0065] Since the apparatus described in Embodiment 2 of the present invention is an apparatus used to implement the method of Embodiment 1 of the present invention, those skilled in the art can understand the specific structure and variations of the apparatus based on the method described in Embodiment 1 of the present invention, and therefore will not be described again here. All apparatuses used in the method of Embodiment 1 of the present invention fall within the scope of protection of the present invention.

[0066] Based on the same inventive concept, this application provides an electronic device embodiment corresponding to Embodiment 1, as detailed in Embodiment 3. Example 3

[0067] This embodiment provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can implement any of the implementation methods in Embodiment 1.

[0068] Since the electronic device described in this embodiment is the device used to implement the method in Embodiment 1 of this application, those skilled in the art can understand the specific implementation method and various variations of the electronic device in this embodiment based on the method described in Embodiment 1 of this application. Therefore, how the electronic device implements the method in the embodiment of this application will not be described in detail here. Any device used by those skilled in the art to implement the method in the embodiment of this application falls within the scope of protection of this application.

[0069] Based on the same inventive concept, this application provides a storage medium corresponding to Embodiment 1, as detailed in Embodiment 4. Example 4

[0070] This embodiment provides a computer-readable storage medium storing a computer program thereon. When the computer program is executed by a processor, it can implement any of the implementation methods in Embodiment 1.

[0071] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0072] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0073] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0074] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0075] While specific embodiments of the present invention have been described above, those skilled in the art should understand that the specific embodiments described are merely illustrative and not intended to limit the scope of the present invention. Equivalent modifications and variations made by those skilled in the art in accordance with the spirit of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for processing enterprise ESG data, characterized in that, Includes the following steps: S1. Collect basic ESG data of the enterprise from the set data source interface; S2. Governance of collected ESG basic data based on AI large-scale models, including: Build basic resources and establish a three-level feature library of "industry-enterprise-indicator". The industry-level features include industry code, indicator code, data distribution pattern, and anomaly judgment threshold. The enterprise-level features include enterprise ID, historical data sequence, trend features, and personalized threshold. The indicator-level features include indicator code, calculation logic, list of related indicators, and related constraint rules. Perform hierarchical anomaly detection, using a large AI model to perform industry-level feature detection, enterprise-level feature detection, and indicator-level feature detection, and generate anomaly data reports that include anomaly level, anomaly type, and judgment criteria; Perform data completion and correction for abnormal and missing data, through internal time-series reasoning, external industry benchmarking, and policy constraint correction, to generate completed data and confidence scores; To achieve data standardization, non-standard expressions are mapped to standard indicators through text vector conversion and semantic matching, with a similarity threshold set at 0.

85. Construct a four-dimensional knowledge graph, establish a four-dimensional relational structure of "ESG data - business scenarios - policy standards - industry benchmarking", and store it in the Neo4j graph database; S3. Use a distributed, tiered storage architecture to store the managed data; S4. Display data based on user permissions.

2. The enterprise ESG data processing method according to claim 1, characterized in that, Specifically, S1 is: Perform interface authentication initialization, read authentication information from the configuration file, complete identity verification through the authentication interface, obtain access credentials and store them in the Redis cache, set the expiration time and start a scheduled refresh task; Construct request parameters, define a request parameter structure containing basic parameters and filter parameters, validate parameter data types, value ranges and the completeness of required fields using validation functions, and serialize the validated parameters into JSON strings; Perform pagination or batch collection. For pagination collection, calculate the total number of pages based on the total number of items (totalCount) and the number of items per page (pageSize), and send requests in a loop until collection is complete. For batch collection, split the enterprise ID list into multiple sub-lists according to the maximum batch limit of the interface, and send requests in batches. The system receives and parses data, determines the data format based on the Content-Type response header, and extracts core fields such as enterprise ID, indicator name, indicator value, statistical time, and data source to form a standardized basic data structure. Perform data integrity checks, including field-level and order-of-magnitude checks, generate a check report, and process the data that passes the checks.

3. The enterprise ESG data processing method according to claim 2, characterized in that, The pagination data collection specifically includes: Initiate an initial request with page=1 and pageSize=100 parameters to obtain the first page of data and the total number of records (totalCount). Calculate the total number of pages using the formula totalPage = ceil(totalCount / pageSize); Starting from page=2, requests are initiated in a loop, with each request interval set to 500ms, until page>totalPage; After each data collection is completed, the current page number is stored in the Redis cache. Upon restart, data collection continues from the cached page number. The data integrity verification includes: Field-level validation: Validate whether the core fields of enterprise ID, indicator name, indicator value, and statistical time are missing; validate whether the statistical time format is the set format; validate whether the indicator value is a numeric type; and validate whether the enterprise ID is a string with a set fixed length. Order of magnitude verification: Calculate the difference rate = |number of collected records - total number of records| / total number of records. If the difference rate > 5%, trigger an alarm and automatically start re-collection. If the difference rate ≤ 5%, record the difference information.

4. The enterprise ESG data processing method according to claim 1, characterized in that, The hierarchical anomaly detection in S2 specifically includes: Industry-level feature detection: The DeepSeek-67B model is used to calculate the quantile of the data to be detected in the industry data distribution. If it is lower than the 5th quantile or higher than the 95th quantile, it is marked as "industry-level anomaly". Enterprise-level feature detection: The historical data sequence is decomposed into trend, seasonal and residual terms using the LSTM algorithm, and the deviation between the current data and the trend term is compared to determine anomalies; Indicator-level feature detection: Verify the logical consistency between indicators through large AI models, and identify logical conflicts and weak correlation anomalies.

5. The enterprise ESG data processing method according to claim 1, characterized in that, The data completion and correction in S2 includes: Internal time-series inference completion: Based on at least 3 consecutive historical data, the missing value is predicted using an AI large model, and a 95% confidence interval is calculated. If the span of the confidence interval exceeds 50% of the fluctuation range of the historical data, it is marked as low confidence. External industry benchmarking and completion: Using the DeepSeek-67B model, 10-20 selected companies are clustered based on multi-dimensional features such as industry, size, revenue, profit margin, region, and number of patents, and the weighted average completion suggestion value is calculated. Policy constraint modification: Relevant policy provisions are retrieved from the policy knowledge base, and policy constraints are used as hard rules for supplementation and modification.

6. The enterprise ESG data processing method according to claim 1, characterized in that, The unified data caliber in S2 includes: The document is sliced ​​by chapter and paragraph using a large AI model, with each slice limited to 500 characters. Extract five core elements from the slice containing numerical values: indicator name, value, unit, calculation method description, and statistical period. Hugging Face Transformers converts indicator names and calculation method descriptions into text vectors. The similarity between the text vector and the standard indicator vector in the ESG semantic graph is calculated. When the similarity is ≥0.85, the mapping is completed automatically. When the similarity is <0.85, it is marked as "ambiguous" and manual confirmation is triggered. The construction of the four-dimensional knowledge graph in S2 includes: Data layer: Store enterprise ID, indicator code, value, and statistical time as data nodes in Neo4j; Business Scenario Layer: Establish business scenario nodes for software development and testing, manufacturing, and supply chain management, and associate ESG indicator nodes through "business scenario-indicator" relationship edges; Policy Standards Layer: Policy clauses are used as policy nodes, and data layer indicator nodes are linked through "policy-set indicators" and "policy-set constraints" relationship edges; Industry benchmarking layer: Establish industry benchmark nodes and associate data layer indicator nodes through the "enterprise indicator - industry benchmark" relationship edge.

7. The enterprise ESG data processing method according to claim 1, characterized in that, Specifically, S3 is: A hierarchical storage system is established. The hot data layer uses a combination of MySQL and Redis to store feature data for the past year, the warm data layer uses MySQL partitioned tables to store process data for 1-3 years, and the cold data layer uses MinIO objects to store result data for more than 3 years. Perform data classification and storage mapping, store feature data in MySQL feature data table, knowledge graph data in Neo4j, process data in MySQL process data table, and result data in MinIO object storage; To ensure data write consistency, MySQL uses a transaction mechanism, Neo4j performs correlation verification, and MinIO generates MD5 checksums. Specifically, S5 is: Establish a permission system and define five roles: system administrator, operations and maintenance personnel, business analyst, enterprise user, and auditor. Implement permission control through JWT tokens and role-based access control. It provides data query, visualization analysis, early warning management, and data export functions. The front end uses the Vue 3.0 framework, the visualization uses the ECharts 5.4 component, and the back end uses the FastAPI framework.

8. An enterprise ESG data processing device, characterized in that: include: The data acquisition module collects basic ESG data from the designated data source interface. The data governance module, based on a large AI model, governs the collected ESG basic data, including: Build basic resources and establish a three-level feature library of "industry-enterprise-indicator". The industry-level features include industry code, indicator code, data distribution pattern, and anomaly judgment threshold. The enterprise-level features include enterprise ID, historical data sequence, trend features, and personalized threshold. The indicator-level features include indicator code, calculation logic, list of related indicators, and related constraint rules. Perform hierarchical anomaly detection, using a large AI model to perform industry-level feature detection, enterprise-level feature detection, and indicator-level feature detection, and generate anomaly data reports that include anomaly level, anomaly type, and judgment criteria; Perform data completion and correction for abnormal and missing data, through internal time-series reasoning, external industry benchmarking, and policy constraint correction, to generate completed data and confidence scores; To achieve data standardization, non-standard expressions are mapped to standard indicators through text vector conversion and semantic matching, with a similarity threshold set at 0.

85. Construct a four-dimensional knowledge graph, establish a four-dimensional relational structure of "ESG data - business scenarios - policy standards - industry benchmarking", and store it in the Neo4j graph database; The data storage module uses a distributed, hierarchical storage architecture to store the processed data. The data display module displays data based on user permissions.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 7.

Images