A cloud-based financial data processing method and system
By collecting and standardizing financial data through distributed acquisition, combining scenario prioritization and network status monitoring, dynamically allocating resources, and optimizing server failures, the system has solved the financial data processing problems caused by cloud platform hardware and network failures, achieving efficient and stable financial data management and analysis.
Patent Information
- Application Number
- CN202510550741.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-29
- Publication Date
- 2025-11-14
- Estimated Expiration
- 2045-04-29
AI Technical Summary
Hardware failures, network failures, and performance limitations of communication equipment on cloud platforms lead to slow and inefficient processing of financial data. In particular, during periods of high concurrency, some users cannot obtain sufficient computing resources and network bandwidth, affecting the real-time processing and analysis of financial data.
By collecting multi-source financial data in a distributed manner and performing standardized processing, financial scenario identifiers are generated. Based on scenario priority and network status monitoring, resources are dynamically allocated, server failures are identified and optimized, a financial database is built, and reports are generated.
It improves the efficiency and stability of financial data processing, ensures that critical tasks are prioritized, optimizes resource utilization, reduces the impact of failures, and enables centralized management and reliable transmission of data.
Smart Images

Figure CN120338972B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of financial data processing technology, and in particular to a financial data processing method and system based on a cloud platform. Background Technology
[0002] Cloud platforms utilize distributed computing technology to process financial data in a decentralized manner, improving data processing speed and efficiency. Based on changes in business needs, cloud platforms can rapidly expand computing resources to ensure data processing capabilities always meet demand. However, hardware failures in cloud platforms, such as hard drive damage or memory failures, can lead to financial data loss or system crashes. Furthermore, aging or improperly maintained communication equipment can cause network failures, impacting the stability of the cloud platform. The performance and stability of communication equipment directly affect the availability of the cloud platform. Insufficient network bandwidth, high network latency, or network outages can prevent users from accessing financial data on the cloud platform in a timely manner, affecting the normal operation of financial work. For example, during critical financial settlement periods, network connectivity issues can prevent the entry of financial data, impacting normal business operations. Limited transmission speed and processing capacity of communication equipment can cause delays in the transmission of financial data between the cloud platform and user terminals. For financial data requiring real-time processing and analysis, an imperfect hardware resource allocation mechanism on the cloud platform can result in some users not receiving sufficient computing resources and network bandwidth during high-concurrency access, thus affecting the speed and efficiency of financial data processing. Summary of the Invention
[0003] Therefore, it is necessary to provide a cloud-based financial data processing method and system to solve at least one of the aforementioned technical problems.
[0004] To achieve the above objectives, a cloud-based financial data processing method is provided, comprising the following steps:
[0005] Step S1: Distributed collection of multi-source raw financial data; data standardization of multi-source raw financial data to obtain standard multi-source financial data; matching the standard multi-source financial data with financial scenarios and generating financial scenario identifiers;
[0006] Step S2: Determine the priority ranking of financial scenarios based on the financial scenario identifier to obtain the scenario priority value; monitor the communication network status of the financial cloud platform based on the scenario priority value to obtain scenario network status data; use the scenario network status data to detect cloud platform resource demand information, and determine financial task resource allocation data based on the cloud platform resource demand information.
[0007] Step S3: Based on the financial task resource allocation data, identify faults on the server side of the financial cloud platform and generate server fault data; based on the server fault data, optimize the server side of the financial cloud platform to obtain an optimized financial cloud platform.
[0008] Step S4: Upload standard multi-source financial data to the optimized financial cloud platform and build a financial database; present financial data reports based on the financial database.
[0009] This invention utilizes distributed acquisition technology to obtain financial data from multiple different data sources, ensuring the breadth and diversity of data sources and providing a comprehensive data foundation for subsequent processing. It standardizes financial data from different sources and with varying formats, making the data format uniform and standardized, facilitating subsequent matching and processing, improving data availability and consistency, and ensuring accurate analysis and application of financial data. By matching the standardized data with specific financial scenarios and generating corresponding financial scenario identifiers, it achieves precise association between data and financial business scenarios, providing a clear basis for subsequent scenario prioritization and resource allocation, and improving the targeting and efficiency of financial data processing. By prioritizing different financial scenarios based on financial scenario identifiers, a scenario priority value is obtained. This allows for the rational arrangement of processing order according to the importance and urgency of financial business, ensuring that critical financial tasks are processed first, thus improving the efficiency of financial data processing and the rationality of resource utilization. Monitoring the communication network status of the financial cloud platform based on the scenario priority value acquires scenario network status data, enabling real-time understanding of the impact of the network environment on the processing of different financial scenarios. This provides accurate network status information for subsequent resource allocation, ensuring the stability and reliability of the financial data processing process. By analyzing the scenario network status data, the resource requirements of the cloud platform when processing different financial scenarios are detected, and based on this, financial task resource allocation data is determined. This achieves dynamic allocation and optimized configuration of resources, improving the utilization rate of cloud platform resources and ensuring that financial data processing tasks can be executed efficiently and stably. By identifying server-side faults based on financial task resource allocation data and generating server fault data, this invention can promptly detect server failures during financial task processing, providing accurate data support for subsequent fault handling and optimization, and reducing the impact of server failures on financial data processing. Based on the server fault data, fault risk optimization is performed on the server side, resulting in an optimized financial cloud platform. This effectively reduces the probability of server failures, improves the stability and reliability of the financial cloud platform, and ensures the continuity and security of financial data processing. Standardized multi-source financial data is uploaded to the optimized financial cloud platform, and a financial database is built, enabling centralized management and storage of data. This facilitates subsequent data querying, analysis, and report generation, improving the efficiency and convenience of financial data management. Financial data reports are generated based on the constructed financial database, providing a clear and accurate display of financial data analysis results and business status. Therefore, this invention, through data processing technology, network status detection technology, and network fault optimization technology, detects server fault risks and optimizes the server side for fault risks, improving the stability of the cloud platform; it also dynamically adjusts the financial task resource allocation data of the cloud platform to ensure the reliability of financial data.
[0010] Preferably, step S1 includes the following steps:
[0011] Step S11: At the preset multiple data collection nodes, collect financial data from different sources according to the time interval of each layer; wherein, the first-layer collection node collects financial bill data at a 1-minute interval; the second-layer collection node collects financial object data at a 5-minute interval; and the third-layer collection node collects business type data at a 15-minute interval.
[0012] Step S12: Merge financial billing data, financial object data, and business type data to generate multi-source raw financial data;
[0013] Step S13: Perform format unification processing on the date field in the multi-source raw financial data to obtain standardized date data; perform encoding unification processing on the text field in the multi-source raw financial data to obtain standardized text data; perform format unification processing on the timestamp field in the multi-source raw financial data to obtain standardized timestamp data; perform encoding mapping processing on the classification field in the multi-source raw financial data to map the classification labels from different sources to a unified numerical code to obtain standardized classification data.
[0014] Step S14: Integrate standardized date data, standardized text data, standardized timestamp data, and standardized classification data to obtain standardized multi-source financial data;
[0015] Step S15: Match standard multi-source financial data with financial scenarios and generate financial scenario identifiers.
[0016] This invention collects financial data from different sources at preset multiple layers of data collection nodes according to the time interval of each layer. The first layer of collection nodes collects financial bill data at a 1-minute interval, the second layer of collection nodes collects financial object data at a 5-minute interval, and the third layer of collection nodes collects business type data at a 15-minute interval. This layered data collection method accurately matches the update frequency of different financial data, ensuring timely collection of high-frequency data and reasonable acquisition of low-frequency data, thereby improving the timeliness and completeness of data collection. It merges financial billing data, financial object data, and business type data, achieving data integration from different financial business processes. It unifies the format of date fields, unifies the encoding of text fields, unifies the format of timestamp fields, and maps the encoding of classification fields, mapping classification labels from different sources to unified numerical codes, ensuring data format consistency and eliminating format differences caused by different data sources. It integrates standardized date data, standardized text data, standardized timestamp data, and standardized classification data, forming a structured and standardized financial dataset, facilitating subsequent financial scenario matching and analysis, and improving the data's universality and operability. Finally, it matches standard multi-source financial data with financial scenarios and generates financial scenario identifiers, achieving precise association between data and specific financial business scenarios.
[0017] Preferably, step S15 includes the following steps:
[0018] Step S151: Perform interval partitioning on the numerical fields in the standard multi-source data to obtain numerical interval identifiers;
[0019] Step S152: Perform time period segmentation processing on the time field in the standard multi-source data to obtain the time period identifier;
[0020] Step S153: Extract keywords from the text fields in the standard multi-source data, count the frequency of keyword occurrence, and classify the text fields according to the frequency of occurrence to obtain category labels;
[0021] Step S154: Map the categorical fields in the standard multi-source data to obtain categorical field data;
[0022] Step S155: Hash the classification field data based on the numerical range identifier, time period identifier, and category identifier to generate a scene identifier.
[0023] The numerical range identifier generation of this invention provides a clear classification basis for the numerical dimensions of financial data, facilitating rapid filtering and location of data within a specific range, and improving data retrieval efficiency. The time period identifier division provides a unified periodic framework for the time dimension of financial data, facilitating the analysis of data trends across different time periods and enhancing the accuracy of time-series analysis. Keyword extraction and category identifier division quantify and structure key information in text data, facilitating classification and analysis based on text content and enhancing the characteristics of text data in financial analysis. The mapping processing of categorical field data standardizes categorical fields in multi-source data, ensuring data consistency and providing a reliable foundation for subsequent data integration and analysis. The hash processing based on multi-dimensional identifiers generates scenario identifiers, which can efficiently integrate the multi-dimensional features of financial data, enabling rapid data location and scenario-based differentiation, meeting the data processing needs of complex financial scenarios.
[0024] Preferably, the step S2 of determining the priority ranking of financial scenarios based on the financial scenario identifier includes:
[0025] The scene identifier is decoded to extract multiple feature fields; the feature fields include a time period field, a numerical range field, a category field, and a scene type field; the decoding process includes decomposing the scene identifier into multiple parts, each part corresponding to a feature field;
[0026] The feature fields are weighted to obtain feature field weight values; the weight of the time period field is 0.3, the weight of the numerical range field is 0.4, the weight of the category field is 0.2, and the weight of the scene type field is 0.1.
[0027] The priority score is obtained by weighting the feature field weights. The weighting formula is: Priority Score = Time Period Field Value × Time Period Weight + Numerical Range Field Value × Numerical Range Weight + Category Field Value × Category Weight + Scene Type Field Value × Scene Type Weight.
[0028] The data items are sorted according to their priority scores and arranged in descending order to obtain the scene priority value.
[0029] This invention decomposes scenario identifiers into time period fields, numerical range fields, category fields, and scenario type fields, enabling precise extraction of multi-dimensional feature information from financial data. This provides clear feature basis for subsequent refined analysis, ensuring the integrity and traceability of data features. By explicitly setting the weight of the time period field to 0.3, the numerical range field to 0.4, the category field to 0.2, and the scenario type field to 0.1, the importance of different feature fields is quantitatively differentiated, ensuring that the contribution of each feature field in subsequent calculations meets the needs of actual application scenarios and providing an objective weight basis for priority calculation. Weighted calculations based on the set weight values yield accurate priority scores, comprehensively considering the multi-dimensional features of financial data, ensuring comprehensive and accurate priority calculation results, providing a scientific quantitative basis for data sorting, and avoiding the limitations of single-dimensional analysis. Data items are sorted from high to low according to the priority scores to obtain scenario priority values, achieving dynamic priority sorting of financial data. This facilitates rapid identification and processing of important financial scenarios, improves the efficiency and targeting of data processing, and ensures that key financial data is processed first.
[0030] Preferably, the monitoring of the communication network status of the financial cloud platform based on scenario priority values in step S2 includes:
[0031] The monitoring range of the communication network status for each scenario is determined based on the scenario priority value, where the higher the priority value, the larger the monitoring range.
[0032] On multiple nodes of the cloud platform, test signals are periodically sent according to the communication network status monitoring range, and the timestamps of sending and receiving are recorded. The test signals are set to fixed-size data packets, each containing a unique timestamp, to detect the network's signal round-trip time, signal quantity, and actual amount of data transmitted.
[0033] Collect network status data, including network latency, packet loss rate, and bandwidth utilization. Network latency is assessed by measuring the round-trip time of test signals; packet loss rate is assessed by counting the number of test signals that did not receive a response; and bandwidth utilization is assessed by monitoring the actual amount of data transmitted.
[0034] The network status data is uploaded to the central monitoring system of the cloud platform, and then aggregated and stored to form scenario network status data.
[0035] This invention dynamically adjusts the monitoring range of the communication network status based on the priority value of the scenario, ensuring that high-priority scenarios receive broader monitoring coverage. This allows for precise allocation of monitoring resources, optimizing the efficiency and targeting of network status monitoring, and providing more reliable network protection for the transmission of important financial data. By periodically sending test signals of fixed size with unique timestamps at multiple nodes and recording the sending and receiving timestamps, it can accurately measure signal round-trip time, count the number of signals, and the actual amount of data transmitted, providing basic data for the quantitative assessment of network status and ensuring the accuracy and real-time performance of network performance monitoring. By measuring the round-trip time of test signals to assess network latency, counting the number of test signals that did not receive a response to assess packet loss rate, and monitoring the actual amount of data transmitted to assess bandwidth utilization, it achieves comprehensive quantitative monitoring of network status. This provides detailed and accurate data support for subsequent network optimization and troubleshooting, ensuring that network performance meets the requirements for financial data transmission. The collected network status data is uploaded to the central monitoring system of the cloud platform and aggregated and stored to form complete scenario network status data, facilitating centralized management and analysis. This provides a unified data basis for the cloud platform's network resource allocation, fault early warning, and optimization strategy formulation, improving the efficiency and reliability of the cloud platform's network management.
[0036] Preferably, step S2, which involves detecting cloud platform resource demand information from scene network status data and determining financial task resource allocation data based on cloud platform resource demand information, includes:
[0037] Network latency, packet loss rate, and bandwidth utilization are extracted from the scene's network status data to obtain network parameter indicators;
[0038] The storage resource requirements of the cloud platform are detected based on network parameter indicators; the CPU utilization, memory utilization, and disk I / O utilization of financial tasks are determined based on the storage resource requirements in order to generate storage resource allocation data.
[0039] The computing resource requirements of the cloud platform are detected based on network parameter indicators; the number of tasks and resource usage of financial tasks are determined based on the computing resource requirements, so as to generate computing resource allocation data;
[0040] Detect the network resource requirements of the cloud platform based on network parameter indicators; determine the network bandwidth of financial tasks based on network resource requirements, and generate network resource allocation data;
[0041] The data on storage resource allocation, computing resource allocation, and network resource allocation are merged to obtain the data on financial task resource allocation.
[0042] This invention extracts key network parameters such as network latency, packet loss rate, and bandwidth utilization to provide precise quantitative data for subsequent resource demand detection, ensuring that resource allocation decisions are based on actual network performance. By determining the CPU utilization, memory utilization, and disk I / O utilization of financial tasks based on these network parameters, it can accurately assess storage resource requirements, generate targeted storage resource allocation data, optimize storage resource allocation, and improve storage resource utilization efficiency. Furthermore, by determining the number of running tasks and resource usage of financial tasks using network parameters, it generates computing resource allocation data, enabling on-demand allocation of computing resources and ensuring efficient operation of financial tasks. Based on these network parameters, it determines the network bandwidth requirements of financial tasks, generating network resource allocation data, enabling dynamic adjustment of network resources and ensuring the stability and efficiency of financial data transmission. Finally, by integrating storage, computing, and network resource allocation data, it forms comprehensive financial task resource allocation data, achieving unified management and optimized resource configuration, and enhancing the cloud platform's overall support capabilities for financial data processing.
[0043] Preferably, the fault identification of the server side of the financial cloud platform based on financial task resource allocation data in step S3 includes:
[0044] Based on storage resource allocation data, the storage hardware structure on the server side of the financial cloud platform is detected: the disk type of the storage hardware structure is identified and the disk read and write speed is recorded; read and write latency is detected on the disk read and write speed to obtain disk read and write fault data;
[0045] Based on the computing resource allocation data, the computing hardware structure in the server side of the financial cloud platform is detected: the number of CPU cores in the computing hardware structure is identified and the CPU cache usage is recorded; cache ratio fault detection is performed on the CPU cache usage to obtain CPU cache fault data;
[0046] Based on network resource allocation data, detect the network hardware structure on the server side of the financial cloud platform: identify the network interface types of the network hardware structure and record the switching volume of the network response interfaces; perform switching volume fault detection on the switching volume of the network response interfaces to obtain switching volume fault data;
[0047] Integrate disk read / write failure data, CPU cache failure data, and exchange volume failure data to generate server-side failure data.
[0048] This invention, by identifying the disk type and recording disk read / write speeds in the storage hardware structure, can accurately grasp the actual performance of the storage hardware. Read / write latency detection of disk read / write speeds yields disk read / write fault data, enabling timely detection of potential storage hardware failures and ensuring the reliability of financial data storage. Identifying the number of CPU cores and recording CPU cache usage in the computing hardware structure clarifies the usage status of computing resources. Cache ratio fault detection of CPU cache usage yields CPU cache fault data, effectively preventing computing resource failures and ensuring efficient operation of financial tasks. Identifying the network interface type and recording network interface exchange volume in the network hardware structure provides a comprehensive understanding of the network hardware's operational status. Exchange volume fault detection of network interface exchange volume yields exchange volume fault data, enabling timely detection of network transmission problems and ensuring the stability of financial data transmission. By integrating multi-dimensional fault data, comprehensive server-side fault data is formed, facilitating centralized management and analysis, providing unified data support for cloud platform fault diagnosis and resource optimization, and improving the operational efficiency and reliability of the cloud platform.
[0049] Preferably, the fault risk optimization of the financial cloud platform server based on server-side fault data in step S3 includes:
[0050] Based on disk read / write failure data, the server side of the financial cloud platform is used to locate disk blocks with read / write failures, and the financial data of the disk blocks with read / write failures is migrated to generate disk failure optimization measures.
[0051] The cache kernel control group is determined based on CPU cache fault data. The cache kernel control group restricts the use of cache for non-critical tasks and sets critical tasks to real-time scheduling in order to generate cache fault optimization measures.
[0052] Based on the data on switching failures, the server side of the financial cloud platform is tested for switching paths, and the switching path volume is dynamically adjusted to generate optimization measures for switching failures.
[0053] Based on disk failure optimization measures, cache failure optimization measures, and exchange volume failure optimization measures, the server side of the financial cloud platform was optimized and adjusted to obtain an optimized financial cloud platform.
[0054] This invention effectively avoids data loss and read / write errors by precisely locating faulty disk blocks and migrating related financial data, ensuring the integrity and reliability of financial data. By limiting the caching of non-critical tasks and optimizing the scheduling of critical tasks, it improves CPU resource utilization efficiency, ensuring the efficient operation of financial tasks. By detecting the switching path and dynamically adjusting the switching volume, it optimizes network resource allocation, reduces network latency and packet loss rate, and ensures the stability of financial data transmission. By comprehensively adjusting the server side using multiple optimization measures, it can comprehensively improve the performance and stability of the financial cloud platform, ensuring the reliability of financial data processing.
[0055] Preferably, step S4 includes the following steps:
[0056] Step S41: Divide the standard multi-source financial data into multiple financial data blocks, each block being no larger than 1MB; upload the financial data blocks one by one to the financial cloud platform via a network interface to obtain financial data upload information;
[0057] Step S42: Define the database table structure based on the uploaded financial data, including table name, field name and field type, and create the database table, initialize the table structure and indexes; import the financial data blocks one by one into the database table to build the financial database;
[0058] Step S43: Execute an SQL query on the financial database and extract data that meets the query criteria;
[0059] Step S44: Set the title, header, and footer structure of the report for data that meets the query conditions, obtain the report data, and export the report data as a financial data report.
[0060] This invention divides standard multi-source financial data into data blocks of no more than 1MB and uploads them one by one to a financial cloud platform. This ensures efficient and stable data transmission, avoiding transmission failures or network congestion caused by excessively large data blocks, while also facilitating monitoring and management of the upload process. Based on the financial data upload information, it defines and creates a database table structure, initializes the table structure and indexes, providing a standardized and efficient storage environment for financial data, ensuring fast data retrieval and query performance, and laying the foundation for subsequent data processing and analysis. By executing SQL queries on the financial database and extracting data that meets the query conditions, it efficiently filters out target financial data, meeting diverse data analysis needs and improving the flexibility and accuracy of data processing. Finally, by setting the title, header, and footer structure of reports based on the data that meets the query conditions and exporting them as financial data reports, it presents the data in a standardized format, facilitating quick understanding and use by users, improving the readability and usability of financial data, and meeting the output requirements of financial reports.
[0061] This specification provides a cloud-based financial data processing system for executing the aforementioned cloud-based financial data processing method. The cloud-based financial data processing system includes:
[0062] The financial data acquisition module is used for distributed acquisition of multi-source raw financial data; it standardizes the multi-source raw financial data to obtain standard multi-source financial data; it matches the standard multi-source financial data with financial scenarios and generates financial scenario identifiers.
[0063] The communication network status detection module is used to determine the priority ranking of financial scenarios based on financial scenario identifiers and obtain scenario priority values; monitor the communication network status of the financial cloud platform based on scenario priority values to obtain scenario network status data; detect cloud platform resource demand information using scenario network status data, and determine financial task resource allocation data based on cloud platform resource demand information.
[0064] The server fault optimization module is used to identify faults on the server side of the financial cloud platform based on financial task resource allocation data and generate server fault data; based on the server fault data, the module optimizes the server side of the financial cloud platform to obtain an optimized financial cloud platform.
[0065] The financial database management module is used to upload standard multi-source financial data to the optimized financial cloud platform and build a financial database; it then presents financial data reports based on the financial database.
[0066] This invention utilizes a financial data acquisition module to achieve distributed acquisition and standardized processing of multi-source raw financial data, generating financial scenario identifiers to provide a high-quality foundation and clear classification basis for data processing. A communication network status detection module determines scenario priority values based on the financial scenario identifiers, monitors the communication network status, and generates scenario network status data. This allows for accurate assessment of cloud platform resource requirements and determination of financial task resource allocation data, ensuring efficient and stable data processing. A server fault optimization module identifies and optimizes server faults based on resource allocation data, improving system performance and reliability and reducing fault risks. A financial database management module uploads standardized data to the optimized cloud platform and constructs a financial database, ultimately generating financial data reports for centralized data management and intuitive presentation, meeting financial reporting requirements. Overall, this system achieves end-to-end optimization of financial data processing, improving data processing efficiency, stability, and reliability. Attached Figure Description
[0067] Figure 1 This is a flowchart illustrating the steps of a cloud-based financial data processing method.
[0068] Figure 2 for Figure 1 A detailed flowchart illustrating the implementation steps of step S1.
[0069] Figure 3 for Figure 1 A detailed flowchart illustrating the implementation steps of step S4.
[0070] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0071] The technical method of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.
[0072] Furthermore, the accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor methods and / or microcontroller methods.
[0073] It should be understood that although the terms "first," "second," etc., may be used herein to describe various units, these units should not be limited by these terms. These terms are used merely to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.
[0074] To achieve the above objectives, please refer to Figures 1 to 3 A cloud-based financial data processing method, the method comprising the following steps:
[0075] Step S1: Distributed collection of multi-source raw financial data; data standardization of multi-source raw financial data to obtain standard multi-source financial data; matching the standard multi-source financial data with financial scenarios and generating financial scenario identifiers;
[0076] Step S2: Determine the priority ranking of financial scenarios based on the financial scenario identifier to obtain the scenario priority value; monitor the communication network status of the financial cloud platform based on the scenario priority value to obtain scenario network status data; use the scenario network status data to detect cloud platform resource demand information, and determine financial task resource allocation data based on the cloud platform resource demand information.
[0077] Step S3: Based on the financial task resource allocation data, identify faults on the server side of the financial cloud platform and generate server fault data; based on the server fault data, optimize the server side of the financial cloud platform to obtain an optimized financial cloud platform.
[0078] Step S4: Upload standard multi-source financial data to the optimized financial cloud platform and build a financial database; present financial data reports based on the financial database.
[0079] In this embodiment of the invention, reference is made to Figure 1 The diagram shown illustrates the steps of a cloud-based financial data processing method according to the present invention. In this example, the cloud-based financial data processing method includes the following steps:
[0080] Step S1: Distributed collection of multi-source raw financial data; data standardization of multi-source raw financial data to obtain standard multi-source financial data; matching the standard multi-source financial data with financial scenarios and generating financial scenario identifiers;
[0081] In this embodiment of the invention, a distributed data acquisition system architecture is adopted, including multiple acquisition nodes (Agents) and a central management node (Master). The acquisition nodes are responsible for acquiring data from their respective data sources, while the central management node is responsible for task scheduling, status monitoring, and data aggregation. Multiple acquisition nodes are configured to connect to different data sources based on the source of the financial data. These data sources include internal enterprise financial systems, ERP systems, bank reconciliation systems, and external financial data interfaces. Each acquisition node selects an appropriate connection protocol and authentication method based on the type of data source. The central management node assigns acquisition tasks to each acquisition node according to the distribution of data sources and the needs of the acquisition tasks. The acquisition nodes acquire data from the designated data sources according to the assigned tasks. The acquired raw financial data is transmitted over a network to a central processing unit or a distributed storage system (such as HDFS) for centralized storage. During transmission, data compression and encryption technologies are used to improve transmission efficiency and data security. The acquired multi-source financial data undergoes format conversion to ensure that all data uses a unified format. For example, the date format is unified to "YYYY-MM-DD", and the amount format is unified to two decimal places. Data cleaning tools are used to preprocess the raw data, including removing duplicate data, filling in missing values, and correcting outliers. For example, missing monetary data can be filled by interpolation or using default values. A unified coding rule is established for various items and accounts in the financial data. For example, different expense items are coded as "EXPENSE_001" and "EXPENSE_002" to facilitate subsequent data processing and analysis. A pre-defined financial scenario rule base is established, defining matching conditions for different financial scenarios. For example, the matching conditions for the sales scenario include "revenue type" as "sales revenue" and "transaction status" as "completed". The standardized financial data is matched according to the pre-defined rules. For data matching the sales scenario, a financial scenario identifier "SALES_SCENE_001" is generated; for data matching the procurement scenario, an identifier "PURCHASE_SCENE_002" is generated. The matched financial data and their corresponding scenario identifiers are stored in the database, and an index is created for subsequent querying and analysis.
[0082] Step S2: Determine the priority ranking of financial scenarios based on the financial scenario identifier to obtain the scenario priority value; monitor the communication network status of the financial cloud platform based on the scenario priority value to obtain scenario network status data; use the scenario network status data to detect cloud platform resource demand information, and determine financial task resource allocation data based on the cloud platform resource demand information.
[0083] In this embodiment of the invention, a weighted scoring model is used to prioritize financial scenarios. First, multiple evaluation dimensions are defined, including business value, urgency, technical complexity, and resource requirements. Each dimension is assigned a corresponding weight; for example, business value has a weight of 0.4, urgency has a weight of 0.3, technical complexity has a weight of 0.2, and resource requirements have a weight of 0.1. For each financial scenario, relevant data is extracted from the database based on its identifier (e.g., "SALES_SCENE_001"), and financial experts score each dimension according to preset standards. For example, a sales scenario might receive a high score (90 points) on the business value dimension, a medium score (60 points) on the urgency dimension, a low score (30 points) on the technical complexity dimension, and a medium score (50 points) on the resource requirements dimension. The priority value for this scenario is calculated using a weighted average: Priority Value = (90 × 0.4) + (60 × 0.3) + (30 × 0.2) + (50 × 0.1) = 62. In this way, a priority value is calculated for each financial scenario, and all scenarios are sorted from highest to lowest priority. Network performance monitoring tools (such as Zabbix or Prometheus) are used to monitor the communication network status of the financial cloud platform in real time. Based on the priority value of the financial scenarios, higher priority scenarios are assigned a higher monitoring frequency. For example, scenarios with a priority value higher than 80 are monitored every minute, scenarios with a priority value between 60 and 80 are monitored every 5 minutes, and scenarios with a priority value lower than 60 are monitored every 15 minutes. Monitoring indicators include network bandwidth utilization, latency, and packet loss rate. The monitored network status data is stored in the cloud platform's monitoring database, with data format including scenario identifier, monitoring timestamp, bandwidth utilization, latency, and packet loss rate fields. The scenario network status data is analyzed to determine the cloud platform's resource requirements. A resource demand alert is triggered when network bandwidth utilization exceeds 80% or latency exceeds 200 milliseconds. Resource allocation requirements are calculated based on the alert level and the priority value of the financial scenario. For example, for high-priority scenarios (priority value above 80), additional bandwidth resources are automatically allocated from the resource pool when network bandwidth is insufficient; for medium-priority scenarios (priority value between 60 and 80), moderate allocation is performed when resources allow; for low-priority scenarios (priority value below 60), allocation is only performed when resources are sufficient. Resource allocation data includes scenario identifiers, resource types (such as CPU, memory, and bandwidth), allocation quantity, and allocation time, and this data is stored in the cloud platform's resource management database.
[0084] Step S3: Based on the financial task resource allocation data, identify faults on the server side of the financial cloud platform and generate server fault data; based on the server fault data, optimize the server side of the financial cloud platform to obtain an optimized financial cloud platform.
[0085] In this embodiment of the invention, based on financial task resource allocation data, real-time monitoring tools (such as Prometheus) are used to identify faults on the server side of the financial cloud platform. Monitoring indicators include CPU utilization, memory usage, disk I / O read / write speed, network bandwidth utilization, latency, and application response time and error rate. When monitoring indicators exceed preset thresholds, such as CPU utilization exceeding 85%, memory usage exceeding 90%, or network latency exceeding 200 milliseconds, the system triggers a fault alarm and records relevant data to generate server fault data. The automated root cause analysis function of the AIOps platform is used to perform in-depth mining and correlation analysis of the fault data. For example, by analyzing error codes in system logs, bottlenecks in performance indicators (such as disk I / O blocking), and abnormal behaviors in application logs, the root cause of the fault is determined. Based on the root cause analysis results and combined with a preset risk assessment model, the impact of the fault on the financial cloud platform is assessed, and optimization strategies are formulated. For example, if the failure is caused by disk I / O blocking, optimization strategies include increasing disk capacity, optimizing database indexes, or adjusting disk scheduling policies. These optimization strategies are executed using automation tools on the AIOps platform (such as Ansible and Terraform). For instance, this might involve automatically adjusting server resource allocation, restarting relevant services, or switching to a standby instance. Simultaneously, detailed information about the optimization operations is recorded in the optimization log, including the operation time, the optimization measures performed, and their results. After the optimization operations are completed, the server's operating status is continuously monitored to evaluate the optimization effect. If the performance indicators return to normal and operate stably after optimization, the optimization strategy is marked as successful; if the expected results are not achieved, the optimization strategy is readjusted based on new failure data.
[0086] Step S4: Upload standard multi-source financial data to the optimized financial cloud platform and build a financial database; present financial data reports based on the financial database.
[0087] In this embodiment of the invention, firstly, a distributed data transmission system is used to transmit standard multi-source financial data from local storage or a data lake to an optimized financial cloud platform. This transmission system employs a high-availability architecture, configuring multiple data transmission nodes to ensure the stability and reliability of data transmission. During transmission, the data is divided into multiple small blocks, each of which is verified after transmission to ensure data integrity. After transmission, the data is temporarily stored in the distributed storage system of the cloud platform for subsequent processing and storage. Next, a relational database management system is deployed on the cloud platform to build the financial database. A financial database instance is created using database management tools, and a reasonable table structure is designed based on the characteristics of the financial data. The table structure design follows normalization principles to reduce data redundancy and improve data consistency. For example, an account information table is designed to store basic account information, and a transaction details table is designed to record specific information for each transaction. During the table structure design process, primary keys and foreign key constraints are set for each table to ensure data relevance and integrity. Subsequently, an ETL tool is used to import the standard multi-source financial data temporarily stored in the distributed storage system into the financial database. The ETL tool performs data cleaning, transformation, and verification during the data import process. Data cleaning operations include removing duplicate data, filling in missing values, and correcting formatting errors. Data transformation operations adjust the data format uniformly according to the table structure requirements of the financial database. Data validation operations ensure that the imported data conforms to preset business rules and data integrity constraints. Automated processing by ETL tools ensures efficient and accurate data import into the financial database. After the financial database is built, a professional report generation tool connects to it to generate and display reports. The report generation tool extracts the required data from the financial database using preset report templates and query conditions. The report templates are designed according to the needs of financial analysis, including various commonly used financial statement formats such as balance sheets, income statements, and cash flow statements. The report templates define the data display methods, formatting rules, and dynamic parameter settings to generate corresponding report content based on different query conditions. When extracting data, the report generation tool executes corresponding query operations from the financial database based on user-defined time ranges, business types, and other dynamic parameters to obtain the data required for the reports. Query operations are performed using optimized SQL query statements to ensure efficient and accurate data extraction. The extracted data is formatted by the report generation tool and displayed according to the preset report template style. The report display supports multiple formats, including HTML web pages, PDF documents, and Excel spreadsheets, to meet the needs of different users. Finally, the report generation tool provides automated report distribution capabilities. Users can configure scheduled tasks to set the report generation and distribution times.For example, at the beginning of each month, the system automatically generates the previous month's financial statements and sends them to relevant personnel in the finance department via email or the company's internal messaging system. The report distribution function supports multiple distribution channels to ensure that reports are delivered to target users in a timely and accurate manner.
[0088] As an example of the present invention, reference is made to... Figure 2 As shown, in this example, step S1 includes:
[0089] Step S11: At the preset multiple data collection nodes, collect financial data from different sources according to the time interval of each layer; wherein, the first-layer collection node collects financial bill data at a 1-minute interval; the second-layer collection node collects financial object data at a 5-minute interval; and the third-layer collection node collects business type data at a 15-minute interval.
[0090] Step S12: Merge financial billing data, financial object data, and business type data to generate multi-source raw financial data;
[0091] Step S13: Perform format unification processing on the date field in the multi-source raw financial data to obtain standardized date data; perform encoding unification processing on the text field in the multi-source raw financial data to obtain standardized text data; perform format unification processing on the timestamp field in the multi-source raw financial data to obtain standardized timestamp data; perform encoding mapping processing on the classification field in the multi-source raw financial data to map the classification labels from different sources to a unified numerical code to obtain standardized classification data.
[0092] Step S14: Integrate standardized date data, standardized text data, standardized timestamp data, and standardized classification data to obtain standardized multi-source financial data;
[0093] Step S15: Match standard multi-source financial data with financial scenarios and generate financial scenario identifiers.
[0094] In this embodiment of the invention, a distributed data acquisition system is deployed, with multiple layers of data acquisition nodes. Each node collects financial data from different sources. The first-layer acquisition node is configured to collect financial bill data every minute, connecting to the billing module of the financial system and obtaining real-time bill information through an API interface. The second-layer acquisition node collects financial object data at 5-minute intervals, connecting to the ERP system and receiving object data through a message queue. The third-layer acquisition node collects business type data at 15-minute intervals. The collected financial bill data, financial object data, and business type data are transmitted to a data processing center. The data processing center uses data fusion technology to merge different types of data according to preset field mapping rules. During the merging process, data is associated using unique identifiers (such as transaction IDs) to achieve data consistency and integrity. The merged multi-source original financial data is stored in a distributed storage system. The multi-source original financial data is standardized. For date fields, a date formatting tool is used to uniformly convert all dates to "YYYY-MM-DD" format. For text fields, a unified encoding scheme using Unicode is adopted. For timestamp fields, they are uniformly converted to UTC time format, accurate to milliseconds. For categorization fields, a categorization label mapping table is established, mapping categorization labels from different sources to a unified numerical code, such as mapping "income" to "1" and "expenses" to "2". Standardized date data, text data, timestamp data, and categorization data are integrated. During integration, data validation tools are used to check the integrity and consistency of the data, ensuring each data entry contains the necessary fields. The integrated standard multi-source financial data is stored in a data warehouse using a star schema, organizing the data in the form of fact tables and dimension tables for easy subsequent querying and analysis. The standard multi-source financial data is matched with preset financial scenarios. A scenario matching engine assigns data to corresponding financial scenarios based on the data's business attributes and financial characteristics. For example, data related to sales is matched to the "sales scenario," and data related to procurement is matched to the "procurement scenario." After matching, a unique financial scenario identifier is generated for each data entry, using UUID format. The generated financial scenario identifiers are stored in the data warehouse along with the data for subsequent scenario-based analysis and processing.
[0095] Preferably, step S15 includes the following steps:
[0096] Step S151: Perform interval partitioning on the numerical fields in the standard multi-source data to obtain numerical interval identifiers;
[0097] Step S152: Perform time period segmentation processing on the time field in the standard multi-source data to obtain the time period identifier;
[0098] Step S153: Extract keywords from the text fields in the standard multi-source data, count the frequency of keyword occurrence, and classify the text fields according to the frequency of occurrence to obtain category labels;
[0099] Step S154: Map the categorical fields in the standard multi-source data to obtain categorical field data;
[0100] Step S155: Hash the classification field data based on the numerical range identifier, time period identifier, and category identifier to generate a scene identifier.
[0101] In this embodiment of the invention, a data processing framework (such as Apache Spark) is used to partition numerical fields in standard multi-source financial data into intervals. Based on the statistical distribution characteristics of the numerical fields, an appropriate interval partitioning method is selected. For example, for a certain numerical field, a quantile-based partitioning strategy is adopted to divide it into several intervals. The interval boundaries are determined by calculating the quantiles of the field (such as the 25th, 50th, and 75th quantiles). Using Spark's distributed computing capabilities, large-scale datasets are processed efficiently, mapping the value of each numerical field to the corresponding interval and assigning a unique numerical interval identifier to each interval. For example, intervals with values less than the 25th quantile are labeled "Interval 1," intervals between the 25th and 50th quantiles are labeled "Interval 2," and so on. The time field in the standard multi-source financial data is then partitioned into time periods. First, the time field is uniformly converted to a standard time format (such as ISO 8601 format). Based on business requirements and the time granularity of the data, the time period partitioning method is determined. For example, for transaction time fields, partitioning is performed according to daily, weekly, and monthly periods. Using time processing libraries (such as Python's `datetime` module or Java's `java.time` package), extract information such as year, month, day, and week from time fields to generate time period identifiers. Extract keywords from text fields in standard multi-source financial data. Use natural language processing tools (such as NLTK or spaCy) to perform word segmentation, part-of-speech tagging, and keyword extraction on text fields. Calculate the frequency of each keyword in the text field through statistical analysis. Divide the text fields into different categories based on the frequency distribution of keywords. For example, use the TF-IDF algorithm to calculate keyword importance and divide the text fields into "high-frequency keyword categories" and "low-frequency keyword categories" based on their values. Assign a corresponding category identifier to each text field through keyword extraction and frequency statistics. Map the categorical fields in standard multi-source financial data. Establish mapping rules for categorical fields, uniformly mapping categorical labels from different sources to predefined numerical codes. For example, map "sales" to "1", "purchasing" to "2", and "reimbursement" to "3". Convert the original labels of categorical fields into standardized numerical codes using the mapping rules to obtain categorical field data. This process can be implemented through configuration files or database tables, ensuring that category labels from different sources can be accurately mapped to a unified coding system. The category field data is hashed based on the numerical range identifier, time period identifier, and category identifier. The numerical range identifier, time period identifier, and category identifier are then combined with the category field data to form a unique string. A hash algorithm (such as SHA-256) is used to perform a hash calculation on this string to generate a unique scene identifier.For example, for a data entry with a numerical range identifier of "Range 2", a time period identifier of "Year X Month", a category identifier of "High-Frequency Keyword Category", and a category field data of "Sales", the combined string would be "Range 2_Year X Month_High-Frequency Keyword Category_Sales". A scenario identifier for this data entry is calculated using a hash algorithm. The generated scenario identifier is stored in the data warehouse, associated with the original data, facilitating subsequent scenario-based analysis and processing.
[0102] Preferably, the step S2 of determining the priority ranking of financial scenarios based on the financial scenario identifier includes:
[0103] The scene identifier is decoded to extract multiple feature fields; the feature fields include a time period field, a numerical range field, a category field, and a scene type field; the decoding process includes decomposing the scene identifier into multiple parts, each part corresponding to a feature field;
[0104] The feature fields are weighted to obtain feature field weight values; the weight of the time period field is 0.3, the weight of the numerical range field is 0.4, the weight of the category field is 0.2, and the weight of the scene type field is 0.1.
[0105] The priority score is obtained by weighting the feature field weights. The weighting formula is: Priority Score = Time Period Field Value × Time Period Weight + Numerical Range Field Value × Numerical Range Weight + Category Field Value × Category Weight + Scene Type Field Value × Scene Type Weight.
[0106] The data items are sorted according to their priority scores and arranged in descending order to obtain the scene priority value.
[0107] In this embodiment of the invention, data processing tools (such as Python's `re` module or regular expression tools) are used to decode the scene identifier. The scene identifier is a unique string generated by a hash algorithm. The decoding process requires reverse parsing of this string to extract multiple feature fields. The specific operation steps are as follows: The scene identifier is decomposed into multiple parts, each corresponding to a feature field. For example, if the scene identifier is "Period A_Interval B_Category C_Type D", the time period field "Period A", the numerical interval field "Interval B", the category field "Category C", and the scene type field "Type D" are extracted through regular expression matching and segmentation. According to a preset weight allocation rule, a weight value is assigned to each feature field. The specific weight allocation is as follows: Time period field weight is 0.3; numerical interval field weight is 0.4; category field weight is 0.2; scene type field weight is 0.1. The weighted calculation formula is used to calculate the feature field weight values to obtain the priority score. The formula is as follows: Priority score = Time period field value × Time period weight + Numerical interval field value × Numerical interval weight + Category field value × Category weight + Scene type field value × Scene type weight. The specific steps are as follows: Convert the feature field values into numerical form (e.g., through encoding or mapping) for weighted calculation. For example, the time period field "Period A" can be mapped to the value 1, the numerical range field "Range B" can be mapped to the value 2, the category field "Category C" can be mapped to the value 1, and the scene type field "Type D" can be mapped to the value 1. Calculate the priority score for each data item according to the weighting formula. Associate the calculated priority scores with the data items and sort the data items in descending order of priority scores. The specific steps are as follows: Create a data structure (such as a list or array) to store the data items and their corresponding priority scores. Sort the data structure using a sorting algorithm (such as quicksort or mergesort) to ensure that the data items are arranged in descending order of priority scores. Store the sorted data items and their priority scores in a database or data warehouse for subsequent analysis and processing.
[0108] Preferably, the monitoring of the communication network status of the financial cloud platform based on scenario priority values in step S2 includes:
[0109] The monitoring range of the communication network status for each scenario is determined based on the scenario priority value, where the higher the priority value, the larger the monitoring range.
[0110] On multiple nodes of the cloud platform, test signals are periodically sent according to the communication network status monitoring range, and the timestamps of sending and receiving are recorded. The test signals are set to fixed-size data packets, each containing a unique timestamp, to detect the network's signal round-trip time, signal quantity, and actual amount of data transmitted.
[0111] Collect network status data, including network latency, packet loss rate, and bandwidth utilization. Network latency is assessed by measuring the round-trip time of test signals; packet loss rate is assessed by counting the number of test signals that did not receive a response; and bandwidth utilization is assessed by monitoring the actual amount of data transmitted.
[0112] The network status data is uploaded to the central monitoring system of the cloud platform, and then aggregated and stored to form scenario network status data.
[0113] In this embodiment of the invention, a cloud platform's resource management system dynamically allocates the monitoring range of the communication network status based on scenario priority values. The system pre-defines mapping rules between priority values and monitoring ranges, dividing the priority values into multiple intervals, each corresponding to a different monitoring range. For example, scenarios with priority values between 80 and 100 are allocated the largest monitoring range, covering all core nodes and key edge nodes of the cloud platform; scenarios with priority values between 60 and 79 are allocated a medium monitoring range, covering some core nodes and key edge nodes; and scenarios with priority values below 60 are allocated the smallest monitoring range, covering only key edge nodes. Based on these rules, the resource management system generates specific monitoring task configuration files for each scenario, specifying parameters such as the list of monitoring nodes and the monitoring frequency. Network status monitoring tools (such as Ping tools based on the ICMP protocol or custom UDP / TCP testing tools) are deployed on multiple nodes of the cloud platform. These tools can generate fixed-size data packets, the size of which is set according to actual needs, for example, 128 bytes. Each data packet contains a unique timestamp in standard UTC time format, accurate to milliseconds. Based on the monitoring range determined by the scenario priority value, the monitoring tool periodically sends test signals at a preset frequency. For example, high-priority scenarios send signals every 30 seconds, medium-priority scenarios every 2 minutes, and low-priority scenarios every 5 minutes. The monitoring tool records the sending timestamp when sending a test signal and the receiving timestamp when receiving a response signal, while also counting the number of signals sent and the number of signals that did not receive a response. Network status data collection includes three key indicators: network latency, packet loss rate, and bandwidth utilization. Network latency is assessed by calculating the round-trip time of the test signal, i.e., the receiving timestamp minus the sending timestamp. Packet loss rate is assessed by the ratio of the number of test signals that did not receive a response to the total number of test signals sent, calculated as: Packet Loss Rate = (Number of Test Signals That Did Not Receive a Response / Total Number of Test Signals Sent) × 100%. Bandwidth utilization is assessed by monitoring the actual amount of data transmitted. The monitoring tool records the amount of data successfully transmitted within a certain time interval and compares it with the maximum available bandwidth within that time interval, calculated as: Bandwidth Utilization = (Actual Data Transmission Amount / Maximum Available Bandwidth) × 100%. The monitoring tool stores the collected network latency, packet loss rate, and bandwidth utilization data in local log files. These log files are in a structured format, such as CSV, to facilitate subsequent data uploading and parsing. Data transmission tools (such as HTTP-based RESTful API interfaces or message queue systems like Kafka) are used to upload the collected network status data from each monitoring node to the central monitoring system on the cloud platform. The uploaded data includes fields such as scenario identifier, timestamp, network latency, packet loss rate, and bandwidth utilization.After receiving the uploaded data, the central monitoring system parses and verifies the data to ensure its integrity and accuracy. The parsed data is stored in a distributed database (such as Cassandra or HBase), with database table structures designed to include fields such as scenario identifier, timestamp, network latency, packet loss rate, and bandwidth utilization, facilitating subsequent data querying and analysis. The central monitoring system periodically aggregates the stored network status data, generating time series data for scenario network status, providing data support for network management and optimization on the cloud platform.
[0114] Preferably, step S2, which involves detecting cloud platform resource demand information from scene network status data and determining financial task resource allocation data based on cloud platform resource demand information, includes:
[0115] Network latency, packet loss rate, and bandwidth utilization are extracted from the scene's network status data to obtain network parameter indicators;
[0116] The storage resource requirements of the cloud platform are detected based on network parameter indicators; the CPU utilization, memory utilization, and disk I / O utilization of financial tasks are determined based on the storage resource requirements in order to generate storage resource allocation data.
[0117] The computing resource requirements of the cloud platform are detected based on network parameter indicators; the number of tasks and resource usage of financial tasks are determined based on the computing resource requirements, so as to generate computing resource allocation data;
[0118] Detect the network resource requirements of the cloud platform based on network parameter indicators; determine the network bandwidth of financial tasks based on network resource requirements, and generate network resource allocation data;
[0119] The data on storage resource allocation, computing resource allocation, and network resource allocation are merged to obtain the data on financial task resource allocation.
[0120] In this embodiment of the invention, data processing tools (such as Apache Spark or Pandas) are used to extract key indicators such as network latency, packet loss rate, and bandwidth utilization from scene network status data. This data is stored in a distributed database on a cloud platform, and data for corresponding fields is extracted from the database using preset query statements. For example, the SQL statement `SELECT network_latency, packet_loss_rate, bandwidth_utilization FROM network_status_data WHERE scene_id = 'specific scene identifier'` is used to obtain network parameter indicators for a specific scene. By analyzing these network parameter indicators, machine learning algorithms (such as linear regression or decision trees) are used to predict storage resource requirements. For example, high network latency or increased packet loss rate indicates insufficient storage resources, requiring increased disk I / O utilization. Storage resource requirements are assessed based on threshold values for network parameter indicators (such as network latency exceeding 100 milliseconds or packet loss rate exceeding 5%). Specific operations include: using storage resource monitoring tools (such as Prometheus) to monitor CPU utilization, memory utilization, and disk I / O utilization in real time; and dynamically adjusting storage resource allocation based on changes in network parameter indicators. For example, when network latency exceeds a threshold, disk I / O utilization is increased to alleviate storage bottlenecks. By analyzing network parameter metrics, machine learning algorithms are used to predict computing resource demands. For example, when bandwidth utilization approaches its limit, it indicates increased computing resource demand. Specific operations include: using computing resource monitoring tools (such as Ganglia) to monitor the number of tasks and resource usage in real time; dynamically adjusting computing resource allocation based on changes in network parameter metrics. For example, when bandwidth utilization exceeds 80%, computing resources are increased to support more tasks. By analyzing network parameter metrics, machine learning algorithms are used to predict network resource demands. For example, when network latency or packet loss rate increases, it indicates insufficient network bandwidth. Specific operations include: using network resource monitoring tools (such as Zabbix) to monitor network bandwidth in real time; dynamically adjusting network resource allocation based on changes in network parameter metrics. For example, when network latency exceeds a threshold, network bandwidth is increased to alleviate network bottlenecks. Data integration tools (such as Apache NiFi) are used to merge storage resource allocation data, computing resource allocation data, and network resource allocation data. The specific steps include: extracting storage resource allocation data, computing resource allocation data, and network resource allocation data from a distributed database; using data integration tools to merge this data into a unified data structure, such as creating a table structure containing scenario identifiers, storage resource allocation information, computing resource allocation information, and network resource allocation information; and storing the merged data in the cloud platform's central resource management system for subsequent resource allocation and management.
[0121] Preferably, the fault identification of the server side of the financial cloud platform based on financial task resource allocation data in step S3 includes:
[0122] Based on storage resource allocation data, the storage hardware structure on the server side of the financial cloud platform is detected: the disk type of the storage hardware structure is identified and the disk read and write speed is recorded; read and write latency is detected on the disk read and write speed to obtain disk read and write fault data;
[0123] Based on the computing resource allocation data, the computing hardware structure in the server side of the financial cloud platform is detected: the number of CPU cores in the computing hardware structure is identified and the CPU cache usage is recorded; cache ratio fault detection is performed on the CPU cache usage to obtain CPU cache fault data;
[0124] Based on network resource allocation data, detect the network hardware structure on the server side of the financial cloud platform: identify the network interface types of the network hardware structure and record the switching volume of the network response interfaces; perform switching volume fault detection on the switching volume of the network response interfaces to obtain switching volume fault data;
[0125] Integrate disk read / write failure data, CPU cache failure data, and exchange volume failure data to generate server-side failure data.
[0126] In this embodiment of the invention, a monitoring tool is used to detect the number of CPU cores in the server-side computing hardware, and the CPU cache usage is recorded using the monitoring metric "CPU cache usage". The monitoring tool displays the CPU cache usage as an average value in MB. When the CPU cache usage exceeds a preset threshold (e.g., exceeding 80% of the total cache), it is recorded as CPU cache fault data. The fault data includes the timestamp of the fault occurrence, the number of CPU cores, the current cache usage, and the duration of the fault. The monitoring tool is also used to detect the network interface type of the server-side network hardware (e.g., 1Gbps, 10Gbps, etc.), and the network interface switching volume is recorded using the monitoring metrics "NIC inbound packet rate" and "NIC outbound packet rate". The monitoring tool displays the network interface switching volume as an average value in pps (packets per second). When the network interface switching volume exceeds a preset threshold (e.g., the inbound or outbound packet rate exceeds 90% of the interface capacity), it is recorded as switching volume fault data. The fault data includes the timestamp of the fault occurrence, the network interface type, the current switching volume, and the duration of the fault. Data processing tools (such as Apache NiFi) are used to integrate disk read / write failure data, CPU cache failure data, and exchange volume failure data. The integrated data is stored in the central fault management system of the cloud platform. The data structure includes fields such as: failure type (disk read / write failure, CPU cache failure, exchange volume failure), failure timestamp, failure duration, and failure description. The integrated server-side failure data is then stored in a distributed database (such as Cassandra) for subsequent fault analysis and troubleshooting. The query function provided by the fault management system allows for real-time viewing and analysis of failure data, enabling rapid location and resolution of server-side hardware problems.
[0127] Of particular importance is the identification of the disk type in the storage hardware structure and the recording of disk read / write speeds;
[0128] Detect the physical interface type of each disk in the storage hardware structure and record its corresponding interface version number;
[0129] In the storage hardware structure, for each disk, read and write speed tests are conducted in the inner, middle and outer areas of the disk, and the differences in read and write speeds in different areas of the disk are recorded by the hardware controller.
[0130] Read the disk's SMART data, including the number of read / write errors, the number of remapped sectors, the number of sectors to be mapped, and the disk temperature; and record the disk's cumulative read / write count and power-on time.
[0131] During the testing process, based on the difference in read and write speeds, the caching mechanism in the storage hardware structure was used to test the disk's read and write speeds with and without caching enabled, and the results were recorded as disk read and write speeds.
[0132] In this embodiment of the invention, a hardware management tool that supports the detection of multiple storage interfaces is used, such as the system's built-in hardware detection tool or a third-party tool (such as CrystalDiskInfo). This tool scans all disk devices in the storage hardware structure, identifying and recording the physical interface type of each disk. Common disk interface types include SATA, SAS, and M.2. For SATA interfaces, its version number is further detected, such as SATA 1.0, SATA 2.0, SATA 3.0, etc.; for SAS interfaces, its version number is recorded, such as SAS 1.0, SAS 2.0, SAS 3.0, etc.; for M.2 interfaces, its supported protocol version is detected, such as PCIe x2, PCIe x4, etc. The detected interface type and its corresponding version number of each disk are stored in the cloud platform's database. A professional disk performance testing tool, such as CrystalDiskMark, is then used. First, the testing tool is connected to the storage hardware structure, ensuring it can access all disks. Then, for each disk, read and write speed tests are performed on the inner, middle, and outer disk areas. The specific operation is as follows: Set the test areas of the testing tool to the inner circle (near the center of the disk), middle circle (the middle part of the disk), and outer circle (near the edge of the disk) of the disk. Perform read and write operations in each area. The data block size can be selected according to the disk type and test requirements. For example, for mechanical hard drives, larger data blocks can be selected for sequential read and write tests, and for solid-state drives, smaller data blocks can be selected for random read and write tests. Record the read and write speed of each area through the hardware controller, including sequential read and write speeds and random read and write speeds. Store the test results in the cloud platform's database to analyze the differences in read and write speeds in different areas of the disk. Use a tool that supports SMART data reading, such as CrystalDiskMark. After connecting the tool to the storage hardware structure, read the SMART data for each disk. SMART (Self-Monitoring, Analysis, and Reporting Technology) is a disk self-monitoring, analysis, and reporting technology that can provide disk health status information. The specific data read includes the number of read and write errors, the number of remapped sectors, the number of sectors to be mapped, and the disk temperature. In addition, the system reads the cumulative number of read / write operations and power-on time of the disk. This data is typically recorded by the disk controller and provided to the SMART data reading tool. The read SMART data is then stored in the cloud platform's database for monitoring and analysis of the disk's health status. A caching mechanism is configured through the storage hardware architecture's management interface or command-line tools. First, the disk caching mechanism is enabled, and then a disk performance testing tool (such as CrystalDiskMark) is used to test the disk's read / write speeds.During testing, appropriate test parameters, such as data block size and test file size, are selected to ensure that the test results accurately reflect the disk's read and write performance with caching enabled. Next, the disk's caching mechanism is disabled, and the same testing tools and parameters are used again to test the disk's read and write speeds. By comparing the read and write speeds with and without caching, the impact of the caching mechanism on disk performance is analyzed. The test results are recorded in the cloud platform's database for subsequent disk performance evaluation and optimization.
[0133] Of particular importance is the identification of network interface types in the network hardware structure and the recording of network response interface exchange volumes;
[0134] The physical interface types of network devices are checked one by one, including RJ45 Ethernet interfaces, fiber optic interfaces, wireless Wi-Fi interfaces, etc., and the physical connection status of each interface is recorded. At the same time, the network protocol types supported by each network interface are checked, and the bandwidth specifications and supported transmission modes of the interface are recorded.
[0135] On each network interface, a packet monitoring point is set up to record the number of packets, packet size, and packet transmission direction passing through the interface in real time, and to mark the protocol type of the packets.
[0136] The total data transmission volume of each network interface within a preset time interval is calculated, including the amount of uploaded data and the amount of downloaded data. The network latency and packet loss rate within this time interval are also recorded, as well as the peak and trough values of traffic for each network interface at different time periods, in order to obtain the network interface exchange volume.
[0137] In this embodiment of the invention, the `lshw` command is used to detect the physical interface type of the network device. Entering `lshw-classnetwork` in the command line will list detailed information about all network interfaces, including interface type (e.g., RJ45 Ethernet interface, fiber optic interface, wireless Wi-Fi interface, etc.). Simultaneously, the `ethtool` command is used to detect the physical connection status. For example, executing `sudo ethtool eth0` will show the output as `Linkdetected: yes / no`, where `yes` indicates a normal network connection and `no` indicates no connection. Alternatively, the physical connection status can be detected by reading the ` / sys / class / net / INTERFACE / carrier` file; a value of 1 indicates a normal connection, and 0 indicates no connection. The `ethtool` command is used to obtain the protocol types and bandwidth specifications supported by the network interface. For example, executing `sudo ethtool eth0` will display the supported protocol types (e.g., IPv4, IPv6, etc.), interface speed (e.g., 1000Mb / s, 10000Mb / s, etc.), and transmission mode (e.g., full-duplex, half-duplex). For wireless Wi-Fi interfaces, you can use the `iwconfig` command to view information such as the wireless protocol type and signal strength. Use tools like `tcpdump` or `Wireshark` to set up packet monitoring points on the network interface. For example, with `tcpdump`, executing `sudo tcpdump -ieth0` can capture packets passing through the `eth0` interface in real time. The tool records the size and transmission direction (inbound or outbound) of each packet and parses the protocol type (such as TCP, UDP, ICMP, etc.). Save the captured packet information to a log file for later analysis. Use tools like `iftop` or `nethogs` to count the total data transmission volume of the network interface within a preset time interval. For example, executing `sudo iftop -ieth0` will display the upload and download data volume of the `eth0` interface in real time. Simultaneously, test network latency using the `ping` command; executing `ping -c 108.8.8.8` calculates the average round-trip time (RTT) to obtain network latency. Use the `iperf` tool to test network bandwidth and packet loss rate; executing `iperf -c`...<server_ip> This allows us to obtain the packet loss rate from the test results. Furthermore, by analyzing the packet information in the log files, we can record the peak and trough traffic values for each network interface at different time periods to obtain the traffic exchange volume of the network interface.
[0138] Preferably, the fault risk optimization of the financial cloud platform server based on server-side fault data in step S3 includes:
[0139] Based on disk read / write failure data, the server side of the financial cloud platform is used to locate disk blocks with read / write failures, and the financial data of the disk blocks with read / write failures is migrated to generate disk failure optimization measures.
[0140] The cache kernel control group is determined based on CPU cache fault data. The cache kernel control group restricts the use of cache for non-critical tasks and sets critical tasks to real-time scheduling in order to generate cache fault optimization measures.
[0141] Based on the data on switching failures, the server side of the financial cloud platform is tested for switching paths, and the switching path volume is dynamically adjusted to generate optimization measures for switching failures.
[0142] Based on disk failure optimization measures, cache failure optimization measures, and exchange volume failure optimization measures, the server side of the financial cloud platform was optimized and adjusted to obtain an optimized financial cloud platform.
[0143] In this embodiment of the invention, a disk testing tool is used to check the health status of the server-side disks, focusing on the error count field in the disk's SMART information. For detected read / write faulty disk blocks, their location information is recorded; for financial data on read / write faulty disk blocks, a data migration tool is used to migrate it to a healthy disk. After migration, the disk mount point is updated to ensure the system can access the new data storage location normally. A cache kernel control group is created using Linux's cgroups feature, and non-critical tasks are assigned to this control group. CPU cache usage limits are set for the control group to restrict excessive CPU cache usage by non-critical tasks. Critical tasks are set to real-time scheduling to ensure they receive priority execution when CPU resources are scarce. A network monitoring tool is used to monitor the server-side network interfaces in real time to detect any abnormal network interface traffic. A network management tool is used to dynamically adjust network interface traffic, setting maximum traffic limits to avoid exchange failures caused by excessive traffic. Disk failure optimization measures, cache failure optimization measures, and exchange failure optimization measures are integrated into the cloud platform's operation and maintenance management system. Regularly check the server's disk, CPU cache, and network interface status using automated scripts or operation and maintenance management tools. Once a fault is detected, automatically execute corresponding optimization measures. Through these fault optimization adjustments, ensure the financial cloud platform's server-side functions correctly in terms of disk, CPU cache, and network interfaces, improving system stability and reliability.
[0144] As an example of the present invention, reference is made to... Figure 3 As shown, step S4 in this example includes:
[0145] Step S41: Divide the standard multi-source financial data into multiple financial data blocks, each block being no larger than 1MB; upload the financial data blocks one by one to the financial cloud platform via a network interface to obtain financial data upload information;
[0146] Step S42: Define the database table structure based on the uploaded financial data, including table name, field name and field type, and create the database table, initialize the table structure and indexes; import the financial data blocks one by one into the database table to build the financial database;
[0147] Step S43: Execute an SQL query on the financial database and extract data that meets the query criteria;
[0148] Step S44: Set the title, header, and footer structure of the report for data that meets the query conditions, obtain the report data, and export the report data as a financial data report.
[0149] In this embodiment of the invention, standard multi-source financial data is divided into multiple financial data blocks using data processing tools (such as Alibaba Cloud's OSS tools), with each data block not exceeding 1MB in size. Specifically, the number of data blocks after division is calculated based on the total size of the data files, ensuring that the size of each data block does not exceed 1MB. The financial data blocks are then uploaded one by one to the financial cloud platform via a network interface (such as HTTP protocol). Each data block is uploaded to a designated OSS storage bucket using Alibaba Cloud OSS's SDK or API. During the upload process, the upload timestamp, upload status (success or failure), and upload node information for each data block are recorded, forming financial data upload information. Based on the financial data upload information, a cloud database platform (such as Huawei Cloud RDS) is used to define the database table structure. The table structure includes a table name (such as financial_data), field names (such as data_block_id, upload_timestamp, upload_status, etc.), and field types (such as data_block_id as INT, upload_timestamp as TIMESTAMP, and upload_status as VARCHAR). SQL statements are used to create the database table and initialize the table structure and indexes. For example, use the SQL statement `CREATE TABLE financial_data(data_block_id INT PRIMARY KEY, upload_timestamp TIMESTAMP, upload_status VARCHAR(10))` to create a table and create an index for the `data_block_id` field. Import the financial data blocks one by one into the database table. Use a data import tool (such as the data import function of Huawei Cloud RDS) to import the data blocks into the `financial_data` table to ensure the integrity and consistency of the data. Execute an SQL query on the financial database to extract the data that meets the query conditions. For example, use the SQL statement `SELECT * FROM financial_data WHERE upload_status = 'success'` to extract the financial data blocks that have been successfully uploaded. Store the query results in a temporary table or memory for subsequent processing. The extracted data includes information such as the data block ID, upload timestamp, and upload status. Set the report title, header, and footer structure for the data that meets the query conditions. For example, the report title is "Financial Data Upload Report", the header includes fields such as "data block ID", "upload timestamp", and "upload status", and the footer includes statistical information (such as the total number of successful uploads). Use a report generation tool (such as FineBI) to export report data as financial data reports. Reports can be exported in various formats, such as PDF, Excel, or HTML, for users to view and use.
[0150] This specification provides a cloud-based financial data processing system for executing the aforementioned cloud-based financial data processing method. The cloud-based financial data processing system includes:
[0151] The financial data acquisition module is used for distributed acquisition of multi-source raw financial data; it standardizes the multi-source raw financial data to obtain standard multi-source financial data; it matches the standard multi-source financial data with financial scenarios and generates financial scenario identifiers.
[0152] The communication network status detection module is used to determine the priority ranking of financial scenarios based on financial scenario identifiers and obtain scenario priority values; monitor the communication network status of the financial cloud platform based on scenario priority values to obtain scenario network status data; detect cloud platform resource demand information using scenario network status data, and determine financial task resource allocation data based on cloud platform resource demand information.
[0153] The server fault optimization module is used to identify faults on the server side of the financial cloud platform based on financial task resource allocation data and generate server fault data; based on the server fault data, the module optimizes the server side of the financial cloud platform to obtain an optimized financial cloud platform.
[0154] The financial database management module is used to upload standard multi-source financial data to the optimized financial cloud platform and build a financial database; it then presents financial data reports based on the financial database.
[0155] Therefore, the embodiments should be considered as exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of the equivalents of the application are intended to be included within the invention.
[0156] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features of the invention herein.
Claims
1. A financial data processing method based on a cloud platform, characterized in that, Includes the following steps: Step S1: Distributed collection of multi-source raw financial data; data standardization of multi-source raw financial data to obtain standard multi-source financial data; matching the standard multi-source financial data with financial scenarios and generating financial scenario identifiers; Step S2: Determine the priority ranking of financial scenarios based on the financial scenario identifier to obtain the scenario priority value; monitor the communication network status of the financial cloud platform based on the scenario priority value to obtain scenario network status data; detect cloud platform resource demand information based on the scenario network status data, and determine financial task resource allocation data based on the cloud platform resource demand information. Step S3: Based on the financial task resource allocation data, identify faults on the server side of the financial cloud platform and generate server fault data; Based on server-side fault data, the server side of the financial cloud platform is optimized to obtain an optimized financial cloud platform. The process of identifying server-side faults in the financial cloud platform based on financial task resource allocation data includes: Based on storage resource allocation data, the storage hardware structure on the server side of the financial cloud platform is detected: the disk type of the storage hardware structure is identified and the disk read and write speed is recorded; read and write latency is detected on the disk read and write speed to obtain disk read and write fault data; Based on the computing resource allocation data, the computing hardware structure in the server side of the financial cloud platform is detected: the number of CPU cores in the computing hardware structure is identified and the CPU cache usage is recorded; cache ratio fault detection is performed on the CPU cache usage to obtain CPU cache fault data; Based on network resource allocation data, detect the network hardware structure on the server side of the financial cloud platform: identify the network interface types of the network hardware structure and record the network interface switching volume; perform switching volume fault detection on the network interface switching volume to obtain switching volume fault data. Integrate disk read / write failure data, CPU cache failure data, and swap volume failure data to generate server-side failure data; Step S4: Upload standard multi-source financial data to the optimized financial cloud platform and build a financial database; present financial data reports based on the financial database.
2. The financial data processing method based on a cloud platform according to claim 1, characterized in that, Step S1 includes the following steps: Step S11: At the preset multiple data collection nodes, collect financial data from different sources according to the time interval of each layer; wherein, the first-layer collection node collects financial bill data at a 1-minute interval; the second-layer collection node collects financial object data at a 5-minute interval; and the third-layer collection node collects business type data at a 15-minute interval. Step S12: Merge financial billing data, financial object data, and business type data to generate multi-source raw financial data; Step S13: Perform format unification processing on the date field in the multi-source raw financial data to obtain standardized date data; perform encoding unification processing on the text field in the multi-source raw financial data to obtain standardized text data; perform format unification processing on the timestamp field in the multi-source raw financial data to obtain standardized timestamp data; perform encoding mapping processing on the classification field in the multi-source raw financial data to map the classification labels from different sources to a unified numerical code to obtain standardized classification data; Step S14: Integrate standardized date data, standardized text data, standardized timestamp data, and standardized classification data to obtain standardized multi-source financial data; Step S15: Match standard multi-source financial data with financial scenarios and generate financial scenario identifiers.
3. The financial data processing method based on a cloud platform according to claim 2, characterized in that, Step S15 includes the following steps: Step S151: Perform interval partitioning on the numerical fields in the standard multi-source data to obtain numerical interval identifiers; Step S152: Perform time period segmentation processing on the time field in the standard multi-source data to obtain the time period identifier; Step S153: Extract keywords from the text fields in the standard multi-source data, count the frequency of keyword occurrence, and classify the text fields according to the frequency of occurrence to obtain category labels; Step S154: Map the categorical fields in the standard multi-source data to obtain categorical field data; Step S155: Hash the classification field data based on the numerical range identifier, time period identifier, and category identifier to generate a scene identifier.
4. The financial data processing method based on a cloud platform according to claim 1, characterized in that, Step S2, which involves determining the priority ranking of financial scenarios based on the financial scenario identifier, includes: The scene identifier is decoded to extract multiple feature fields; the feature fields include a time period field, a numerical range field, a category field, and a scene type field; the decoding process includes decomposing the scene identifier into multiple parts, each part corresponding to a feature field; The feature fields are weighted to obtain the feature field weight values; the weight of the time period field is set to 0.3, the weight of the numerical range field is set to 0.4, the weight of the category field is set to 0.2, and the weight of the scene type field is set to 0.
1. The priority score is obtained by weighting the feature field weights. The weighting formula is: Priority Score = Time Period Field Value × Time Period Weight + Numerical Range Field Value × Numerical Range Weight + Category Field Value × Category Weight + Scene Type Field Value × Scene Type Weight. The data items are sorted according to their priority scores and arranged in descending order to obtain the scene priority value.
5. The financial data processing method based on a cloud platform according to claim 1, characterized in that, Step S2, which involves monitoring the communication network status of the financial cloud platform based on scenario priority values, includes: The monitoring range of the communication network status for each scenario is determined based on the scenario priority value, where the higher the priority value, the larger the monitoring range. On multiple nodes of the cloud platform, test signals are periodically sent according to the communication network status monitoring range, and the timestamps of sending and receiving are recorded. The test signals are set to fixed-size data packets, each containing a unique timestamp, to detect the network's signal round-trip time, signal quantity, and actual amount of data transmitted. Collect network status data, including network latency, packet loss rate, and bandwidth utilization. Network latency is assessed by measuring the round-trip time of test signals; packet loss rate is assessed by counting the number of test signals that did not receive a response; and bandwidth utilization is assessed by monitoring the actual amount of data transmitted. Network status data is uploaded to the central monitoring system of the cloud platform, and then aggregated and stored to form scenario network status data.
6. The financial data processing method based on a cloud platform according to claim 1, characterized in that, Step S2, which involves detecting cloud platform resource demand information based on scenario network status data and determining financial task resource allocation data based on cloud platform resource demand information, includes: Network latency, packet loss rate, and bandwidth utilization are extracted from the scene's network status data to obtain network parameter indicators; The storage resource requirements of the cloud platform are detected based on network parameter indicators; the CPU utilization, memory utilization, and disk I / O utilization of financial tasks are determined based on the storage resource requirements in order to generate storage resource allocation data. The computing resource requirements of the cloud platform are detected based on network parameter indicators; the number of tasks and resource usage of financial tasks are determined based on the computing resource requirements, so as to generate computing resource allocation data; Detect the network resource requirements of the cloud platform based on network parameter indicators; determine the network bandwidth of financial tasks based on network resource requirements, and generate network resource allocation data; The data on storage resource allocation, computing resource allocation, and network resource allocation are merged to obtain the data on financial task resource allocation.
7. The financial data processing method based on a cloud platform according to claim 1, characterized in that, Step S3, which involves optimizing the server-side fault risk of the financial cloud platform based on server-side fault data, includes: Based on disk read / write failure data, the server side of the financial cloud platform is used to locate disk blocks with read / write failures, and the financial data of the disk blocks with read / write failures is migrated to generate disk failure optimization measures. The cache kernel control group is determined based on CPU cache fault data. The cache kernel control group restricts the use of cache for non-critical tasks and sets critical tasks to real-time scheduling in order to generate cache fault optimization measures. Based on the data on switching failures, the server side of the financial cloud platform is tested for switching paths, and the switching path volume is dynamically adjusted to generate optimization measures for switching failures. Based on disk failure optimization measures, cache failure optimization measures, and exchange volume failure optimization measures, the server side of the financial cloud platform was optimized and adjusted to obtain an optimized financial cloud platform.
8. The financial data processing method based on a cloud platform according to claim 1, characterized in that, Step S4 includes the following steps: Step S41: Divide the standard multi-source financial data into multiple financial data blocks, each block being no larger than 1MB; upload the financial data blocks one by one to the financial cloud platform via a network interface to obtain financial data upload information; Step S42: Define the database table structure based on the uploaded financial data, including table name, field name and field type, and create the database table, initialize the table structure and indexes; import the financial data blocks one by one into the database table to build the financial database; Step S43: Execute an SQL query on the financial database and extract data that meets the query criteria; Step S44: Set the title, header, and footer structure of the report for data that meets the query conditions, obtain the report data, and export the report data as a financial data report.
9. A cloud-based financial data processing system, characterized in that, For performing the cloud-based financial data processing method as described in claim 1, the cloud-based financial data processing system comprises: The financial data acquisition module is used for distributed acquisition of multi-source raw financial data; it standardizes the multi-source raw financial data to obtain standard multi-source financial data; it matches the standard multi-source financial data with financial scenarios and generates financial scenario identifiers. The communication network status detection module is used to determine the priority ranking of financial scenarios based on financial scenario identifiers and obtain scenario priority values; monitor the communication network status of the financial cloud platform based on scenario priority values and obtain scenario network status data; detect cloud platform resource demand information based on scenario network status data, and determine financial task resource allocation data based on cloud platform resource demand information. The server fault optimization module is used to identify faults on the server side of the financial cloud platform based on financial task resource allocation data and generate server fault data; based on the server fault data, the module optimizes the server side of the financial cloud platform to obtain an optimized financial cloud platform. The financial database management module is used to upload standard multi-source financial data to the optimized financial cloud platform and build a financial database; it then presents financial data reports based on the financial database.
Citation Information
Patent Citations
Intelligent financial cloud auditing method and system based on big data
CN118917956A
An automatic inspection management system based on cloud platform
CN119743507A