A multi-department real-time profit aggregation calculation method and system
Patent Information
- Application Number
- CN202611175239.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-04
- Publication Date
- 2026-10-13
AI Technical Summary
[0009]本发明的目的在于:克服传统离线批量核算滞后、数据割裂、核算不准、变更无法即时重算、对账溯源困难的问题,提供五层架构的多部门实时利润聚合计算方法及配套系统,实现业务数据毫秒级采集、流式逐条实时核算、多维度分级聚合、利润数据实时展示与全流程日志溯源
[0024]1.本发明摒弃传统定时全量拉取模式,通过五层架构的多源数据接入层实现CDC增量数据毫秒级捕获,实时监听商机、收款、报销、提成、公共分摊等全品类业务数据的增删改事件,无需周期性批量查询数据库,大幅降低业务库查询压力。彻底解决原有利润数据数小时至隔日的滞后问题,实现部门利润数据实时更新。
Abstract
Description
Technical Field
[0001] This invention relates to the field of enterprise business and financial data processing technology, and more specifically to a method and system for real-time profit aggregation calculation across multiple departments. Background Technology
[0002] The current profit accounting of enterprises generally relies on ERP, and the financial ledgers adopt a timed full data extraction and T+1 batch offline calculation mode, which has the following defects:
[0003] The data collection method is periodic full data retrieval, which puts a lot of pressure on the database and cannot simultaneously capture the addition, modification and deletion of business opportunities, payments, reimbursements, salaries, commissions and public sharing ledgers. Profit data has a lag of several hours to the next day.
[0004] Multiple types of business and financial data are stored separately and lack a unified preprocessing and verification mechanism. Dirty data, missing fields, and abnormal allocation coefficients will directly cause profit calculation to be distorted, and abnormal data cannot be quickly located and archived.
[0005] Profit accounting is divided into multiple processes, including revenue collection, deduction of various costs, and cross-departmental expense allocation. The existing batch calculation relies on a fixed time window for unified calculation, and the profit value of the corresponding department cannot be updated in real time when a single refund, expense adjustment, or commission modification occurs.
[0006] Most of them only output the final profit result value, lacking the ability to drill down to multiple levels from department to salesperson to project. At the same time, they do not retain records of profit changes brought about by each business document, resulting in a huge workload for tracing and reconciliation when discrepancies between accounts and actual figures occur.
[0007] The general streaming big data architecture is only used for general indicator statistics and does not match financial accounting rules such as tiered commission, refund reduction, and public cost allocation. Rule changes often require modification of the underlying code, resulting in high maintenance costs.
[0008] While existing basic components such as streaming data processing, CDC data capture, and in-memory databases are well-known and commonly used technologies, these technologies have not integrated these basic components with the full-chain accounting logic of financial profit to form a layered closed-loop system. This fails to address the actual business pain points of real-time and accurate profit accounting, dynamic updates, and audit traceability for multiple departments within an enterprise. Summary of the Invention
[0009] The purpose of this invention is to overcome the problems of traditional offline batch accounting, such as lag, data fragmentation, inaccurate accounting, inability to recalculate changes in real time, and difficulty in reconciliation and traceability. It provides a five-layer architecture for multi-department real-time profit aggregation and calculation method and supporting system, which realizes millisecond-level collection of business data, real-time streaming and item-by-item accounting, multi-dimensional hierarchical aggregation, real-time display of profit data, and full-process log traceability.
[0010] To achieve the above objectives, the present invention specifically adopts the following technical solution:
[0011] A method for real-time profit aggregation calculation across multiple departments includes five sequentially executed processing steps:
[0012] S1. Multi-source data access steps: Connect to seven business data sources: business opportunity database, business status database, expense reimbursement database, payment collection database, commission calculation database, salary cost database, and cross-department allocation ledger. Use a data synchronization mechanism to monitor and capture the addition, modification, and deletion of incremental data in each data table in real time. Push all incremental data to the message queue to complete the distribution. Use incremental collection mode to replace full data retrieval to achieve millisecond-level collection of business data.
[0013] S2. Data Preprocessing Steps: Obtain incremental data streams transmitted via message queues, unify heterogeneous data formats, filter invalid and dirty data, and automatically complete key fields such as department codes, business types, commission rates, and cost allocation rules; configure finance-specific verification rules to verify the legality of payment amounts, commission rates, and cost allocation coefficients; mark the abnormal data that fails verification and store it together with the original data in the error log database, and send the standardized data that passes verification to the streaming computing engine;
[0014] S3, Real-time Streaming Computing Steps: Deploy resident streaming computing tasks in the computing engine to perform real-time calculations on individual business data entries without waiting for batch data aggregation. Specifically, this includes three computational logics: ① Revenue Calculation: Match business opportunity information with payment records and aggregate business revenue based on the department to which the data belongs; ② Cost Deduction Calculation: Simultaneously calculate employee commissions, departmental fixed salaries, reimbursed expenses, and shared operating costs, and deduct them from revenue accordingly; ③ Rule Matching Calculation: Call preset tiered commission rules, cross-departmental allocation rules, and refund profit reduction rules to complete all cost deduction calculations in real time.
[0015] S4. Multi-department profit aggregation steps: Secondary streaming aggregation calculation is carried out with department code as the first-level aggregation dimension, and real-time statistics of real-time revenue, real-time total cost, real-time gross profit and real-time net profit of a single department are calculated; it supports three-level drill-down aggregation of department-salesperson-account project; whenever business data is added, modified or deleted, the profit value of the corresponding department is immediately recalculated.
[0016] S5. Storage and Visualization Output Steps: Write the real-time aggregated profit indicators into the in-memory database to support real-time visualization on the front-end page; perform snapshot persistence storage of profit data at each time point; the system opens up data call interfaces to the outside world, and the front-end dynamically refreshes the real-time profits of each department through the interfaces; retain the profit change log corresponding to each business data change, and complete the full-link traceability and reconciliation of profit data based on the transaction identifier.
[0017] A multi-department real-time profit aggregation and calculation system is used to execute the multi-department real-time profit aggregation and calculation method. The system has a five-layer hierarchical architecture from top to bottom, namely, multi-source data access layer, data preprocessing layer, streaming real-time calculation layer, multi-dimensional profit aggregation layer, and data storage and visualization output layer.
[0018] The multi-source data access layer includes a multi-source data source connection module, an incremental data capture module, and a message queue distribution module. The multi-source data source connection module connects to the business opportunity database, business status database, expense reimbursement database, payment collection database, commission calculation database, salary cost database, and cross-departmental allocation ledger. The incremental data capture module collects the incremental changes in business data in real time and pushes the incremental data down in milliseconds via the message queue distribution module.
[0019] The data preprocessing layer includes a format standardization module, a dirty data filtering module, a key field completion module, a financial data verification module, and an abnormal data diversion and archiving module; the compliant data after preprocessing flows to the downstream streaming real-time computing layer, and the abnormal data is uniformly archived to the error log library;
[0020] The streaming real-time computing layer is the core computing unit of the system. It deploys resident streaming computing tasks and integrates a revenue matching and collection unit, a comprehensive cost deduction calculation unit, and a financial rule matching unit to complete the real-time accounting and processing of revenue and various costs for each stream of streaming data.
[0021] The multi-dimensional profit aggregation layer is configured with department-dimensional aggregation units, multi-level drill-down aggregation units, and data change triggering recalculation units; it aggregates and calculates various profit indicators with the department code as the primary key, and automatically triggers the immediate recalculation of the corresponding department's profit value when a business data change event occurs.
[0022] The data storage and visualization output layer includes a real-time in-memory database, a historical snapshot persistent library, an external API interface service module, and a front-end visualization display module. The in-memory database is responsible for supporting the real-time profit data display on the front end, the persistent library stores historical profit snapshots and full profit change logs, and the API interface enables external system calls, data display, and business reconciliation and traceability.
[0023] The beneficial effects of this invention are as follows:
[0024] 1. This invention abandons the traditional timed full data retrieval mode and achieves millisecond-level capture of incremental data in CDC through a five-layer multi-source data access layer. It monitors the addition, deletion, and modification events of all categories of business data, such as business opportunities, payments, expense reimbursements, commissions, and public allocations, in real time, eliminating the need for periodic batch database queries and significantly reducing the query pressure on the business database. It completely solves the problem of the original profit data being several hours to the next day behind, enabling real-time updates of departmental profit data.
[0025] 2. This invention adds an independent data preprocessing layer to unify the format of multi-source heterogeneous business and financial data, filter dirty data, complete key business fields, and configure financial-specific legality verification rules. Abnormal data is automatically routed, archived, and logged, avoiding profit calculation distortion caused by missing fields or abnormal allocation coefficients from the source. This enables rapid location, traceability, and archiving of abnormal data, ensuring the accuracy of accounting data.
[0026] 3. This invention employs a streaming, real-time, item-by-item calculation combined with an event-driven local recalculation mechanism, replacing the traditional fixed-time-window batch offline calculation. For single business changes such as refunds, fee adjustments, and commission modifications, the corresponding department's profit recalculation can be triggered immediately without waiting for batch aggregation, achieving dynamic real-time updates of profit values and solving the pain point of untimely profit updates for single business changes.
[0027] 4. This invention features a multi-dimensional profit aggregation layer, supporting drill-down aggregation analysis of data at three levels: department, salesperson, and project. Simultaneously, it retains profit change logs and time snapshots for each business transaction throughout the entire process, enabling full-chain traceability of profit changes. This completely solves the problem of traditional models only outputting the final result and the enormous workload of tracing and reconciliation when discrepancies exist, thus meeting the needs of financial auditing and accurate reconciliation.
[0028] 5. This invention deeply integrates streaming big data components with dedicated financial accounting rules, incorporating proprietary financial logic such as tiered commission rates, refund reductions, and cross-departmental cost allocation, thus decoupling business rules from the underlying code. Rule iteration and parameter adjustments can be achieved through configuration without modifying the underlying calculation code, significantly reducing system development and maintenance costs and adapting to the dynamic iteration needs of enterprise financial accounting rules.
[0029] 6. This invention adopts a layered, decoupled, closed-loop architecture, with clear functional boundaries at each level, stable unidirectional data flow, low module coupling, low failure risk, and each level can be independently iterated and upgraded, further improving system stability and scalability. Detailed Implementation
[0030] The present invention will be further described in detail below with reference to specific embodiments. These embodiments are implemented based on the technical solution of the present invention, providing detailed implementation methods and specific operating procedures; however, the scope of protection of the present invention is not limited to the following embodiments.
[0031] This invention constructs a five-layer hierarchical closed-loop accounting architecture, relying on CDC incremental capture technology, Flink streaming computing framework, and Redis in-memory database to build a complete multi-department real-time profit aggregation and calculation system. It is adapted to the real-time accounting needs of medium and large enterprises with multiple business departments and multiple accounting scenarios for integrated business and finance. It can accurately realize real-time profit statistics, dynamic recalculation, and full-link traceability and reconciliation at the department, salesperson, and project levels, effectively solving the industry pain points of traditional offline T+1 accounting mode such as data lag, accounting distortion, difficulty in traceability, and high operation and maintenance costs.
[0032] 1. Overall System Architecture Implementation
[0033] The system of this invention adopts a top-down five-layer decoupled closed-loop architecture, namely a multi-source data access layer, a data preprocessing layer, a streaming real-time computing layer, a multi-dimensional profit aggregation layer, and a data storage and visualization output layer. Each layer has clear functional boundaries, data flows in a unidirectional closed loop, and modules are independent and iteratively upgradeable. The hardware and software selections for each layer are adapted to the enterprise production environment. The specific implementation configuration is as follows:
[0034] Multi-source data access layer: Based on CDC change data capture technology, it monitors the binlog logs of the enterprise's MySQL business database and non-intrusively connects to seven core business data sources, namely, business opportunity database, business status database, expense reimbursement database, payment collection database, commission calculation database, salary and cost database, and cross-departmental allocation ledger; it uses RabbitMQ message queue as the data distribution carrier to achieve millisecond-level caching and distribution of incremental data, replacing the traditional full-data timed pull mode.
[0035] Data preprocessing layer: Build an independent data preprocessing service, integrating five major functional modules: format conversion, dirty data filtering, field completion, financial rule verification, and anomaly archiving. It has a built-in financial-specific verification rule library to achieve standardized and unified processing of multi-source heterogeneous business and financial data.
[0036] Real-time streaming computing layer: Deploys resident streaming computing tasks based on the Flink distributed streaming computing framework, sets up a real-time computing thread pool, continuously receives pre-processed standardized data streams, and realizes real-time computation of single business data without the need for batch data collection.
[0037] Multi-dimensional profit aggregation layer: Configure multi-level aggregation calculation rules and event triggering mechanisms, with department code as the core aggregation dimension, supporting three-level data drill-down of department-salesperson-account project, and binding business data add, delete and modify event-driven local recalculation logic.
[0038] Data storage and visualization output layer: Redis in-memory database is used to store real-time profit indicators, ensuring millisecond-level data refresh at the front end; MySQL time-series database is used to store historical profit snapshots, error logs, and profit change logs; standardized API interface services and front-end visualization console are built to support external data calls, real-time display, and end-to-end reconciliation and traceability.
[0039] 2. Specific Implementation Steps for Multi-Department Real-Time Profit Aggregation Calculation Method
[0040] S1. Implementation of Multi-Source Data Access
[0041] After system startup, the multi-source data access layer continuously monitors the binlog logs of the seven major business databases. Through CDC non-intrusive incremental capture technology, it identifies new, modified, and deleted business events in each data table in real time, accurately capturing all incremental business data such as opportunity entry, payment receipts, expense submissions, payroll accruals, commission calculations, and cross-departmental expense allocation. Compared to traditional hourly or daily full data retrieval methods, this step only captures changed data, with single data collection latency controlled within 50ms, significantly reducing the query pressure on the business databases.
[0042] All captured incremental data is uniformly pushed to the RabbitMQ message queue and distributed according to business type. Incremental data from different data sources is pushed to the downstream preprocessing stage in an orderly manner, realizing unified access and orderly scheduling of multi-source heterogeneous business data, and ensuring the real-time performance and stability of data transmission.
[0043] S2, Data Preprocessing Implementation
[0044] After the incremental data stream distributed by the message queue enters the data preprocessing layer, it undergoes full-process standardization processing. First, the format standardization module unifies the data field formats, data types, and encoding standards of different business databases, eliminating heterogeneous differences between multi-source data. Then, the dirty data filtering module automatically filters out various types of dirty data, such as empty data, duplicate data, and invalid character data.
[0045] For data with missing fields, the system automatically triggers a key field completion mechanism. Relying on the enterprise's organizational structure ledger and business rule templates, it automatically completes core key fields such as department codes, business types, commission rates, and cost allocation rules, avoiding accounting failures caused by missing fields.
[0046] After data standardization is completed, a dedicated financial legality check is performed through the financial data verification module. The verification dimensions include the legality of positive and negative payment amounts, the compliance of commission ratio ranges, the reasonableness of cross-departmental cost allocation coefficients, and the validity of business data ownership relationships. Standardized data that passes the verification is directly sent to the Flink streaming computing engine; abnormal data that fails the verification is automatically marked with the abnormality type, business transaction ID, and abnormality generation timestamp, and is then diverted and stored in the error log database along with the original data.
[0047] Managers can view all abnormal data through the abnormal data management interface in the system backend. After correcting the data, they can inject it back into the preprocessing process with one click and re-participate in profit accounting, thus achieving closed-loop processing of abnormal data.
[0048] S3, Streaming Real-Time Computing Implementation
[0049] The streaming real-time computing layer deploys 24 / 7 resident Flink streaming computing tasks, continuously receiving preprocessed standardized data streams. It adopts a one-by-one real-time operation mode, eliminating the need to wait for batch data aggregation within a fixed time window. The latency for completing the calculation of a single business data item is controlled within 200ms. Specifically, it includes three core computing logics:
[0050] ① Revenue Collection and Calculation: The system automatically matches business opportunity information with corresponding payment records, and accurately completes the collection of business revenue based on the department code, salesperson ID, and project number to which the business belongs, distinguishing between valid revenue, advance payments, and revenue pending verification, thus eliminating errors and omissions in revenue statistics.
[0051] ② Comprehensive cost deduction calculation: Synchronously link the salary cost database, expense reimbursement database, commission accrual database, and cross-departmental allocation ledger to calculate four types of core costs in sequence, including employee personal business commissions, departmental fixed salary costs, employee daily expense reimbursements, and corporate public operating costs, and deduct them from the current period's business revenue in sequence according to financial accounting rules.
[0052] ③ Financial Rule Matching Calculation: The system has built-in configurable tiered commission rules, cross-departmental expense allocation rules, and refund profit reduction rules, automatically matching the accounting parameters for corresponding business scenarios. For refund business scenarios, the system can reverse the recorded revenue of the corresponding department, salesperson, and project, simultaneously roll back the accrued commissions and allocated costs, and reduce the current period's net profit in real time, accurately adapting to special business scenarios such as refunds, expense adjustments, and commission changes.
[0053] S4. Implementation of Multi-Department Profit Aggregation
[0054] After completing the real-time calculation of a single business data item, the system enters the multi-dimensional profit aggregation stage. Using the department code as the first-level aggregation dimension, it conducts secondary streaming aggregation calculations to summarize and statistically analyze the four core profit indicators of each department in real time: total revenue, total cost expenditure, gross profit, and net profit.
[0055] The system also supports three-level drill-down aggregation, allowing users to drill down from the department level to the individual profit data of the corresponding salesperson, and then further down to the profit data of specific account sets and projects. This enables refined and hierarchical breakdown and analysis of profit data. This step employs an event-driven recalculation mechanism. Whenever new, modified, or deleted business data is added, the system automatically locates the corresponding department, salesperson, and project, triggering only a partial recalculation of the profit data at that level. This eliminates the need for a full data recalculation, significantly improving accounting efficiency and ensuring real-time dynamic updates of profit data.
[0056] S5, Storage and Visualization Output Implementation
[0057] The system aggregates profit metrics at all levels in real time and writes them to a Redis in-memory database. Leveraging the high-speed read capability of in-memory data, the front-end visualization page refreshes and displays real-time profit data for each department within seconds. The system also performs snapshots of all profit data at fixed 5-minute intervals for persistent storage, preserving profit accounting ledgers at different time points.
[0058] Meanwhile, the system assigns a unique identifier (ID) to each business transaction, binding and storing each transaction ID with its corresponding profit change record. This establishes a precise mapping between documents and profit changes, and fully preserves the profit change log for each business data change. Enterprise managers can access data from third-party systems and dynamically refresh the front-end page through the system's open API interface. Furthermore, relying on the unique transaction identifier, they can complete full-chain traceability of profit data, reconcile accounts with physical assets, and conduct financial audits.
[0059] 3. Specific Implementation of System Functional Modules
[0060] 3.1 Implementation of Multi-Source Data Access Layer Module
[0061] The multi-source data source integration module completes port integration and data adaptation for seven major business databases, ensuring compatibility with different versions of database data formats; the incremental data capture module uses CDC technology to achieve non-intrusive data monitoring without consuming business system resources; the message queue distribution module implements incremental data caching, partitioning, and orderly distribution, ensuring that data is not lost, duplicated, or out of order in high-concurrency business scenarios.
[0062] 3.2 Implementation of the Data Preprocessing Layer Module
[0063] The modules work together to achieve comprehensive data governance. The format standardization module unifies the format of common fields such as date, amount, and code; the dirty data filtering module intercepts invalid data based on multiple rules; the key field completion module automatically completes missing information based on related ledgers; the financial data verification module adapts to the enterprise's specific financial accounting standards; and the abnormal data diversion and archiving module enables independent storage, marking, and reprocessing of abnormal data, ensuring the accuracy of accounting data from the source.
[0064] 3.3 Implementation of the Streaming Real-Time Computing Layer Module
[0065] With the Flink framework as its core, persistent streaming tasks run continuously, and the three major computing units work together. The revenue matching and collection unit accurately locks the ownership of business data and completes accurate revenue statistics; the comprehensive cost deduction computing unit calculates various costs in layers to achieve full cost deduction; and the financial rule matching unit dynamically adapts to various financial accounting rules to support the accounting needs of complex business scenarios and ensure that the accounting logic conforms to the company's financial system.
[0066] 3.4 Implementation of the Multi-Dimensional Profit Aggregation Layer Module
[0067] The department-level aggregation unit enables real-time summary and statistics of overall profit indicators for each department; the multi-level drill-down aggregation unit supports the layer-by-layer breakdown and separate query and statistics of data at the three levels: department, salesperson, and project; the data change trigger recalculation unit monitors business data change events in real time, accurately triggers partial recalculation, and avoids resource waste and data delays caused by full recalculation.
[0068] 3.5 Implementation of Data Storage and Visualization Output Layer Module
[0069] The Redis in-memory database handles real-time profit data read and write, ensuring smooth and efficient front-end display; the combination of time-series and relational databases stores historical snapshots, exception logs, and profit change records, adapting to the needs of time-series data storage and structured data query; the API interface service module supports data integration with multiple terminals and systems; the front-end visualization module provides functions such as profit data dashboards, detailed queries, data export, and traceability reconciliation, which are convenient to operate and intuitive to display.
[0070] 4. Implementation of Visual Rule Configuration Function
[0071] This invention features a system with an independent, visual rule configuration panel. Staff can configure and modify various financial accounting parameters and rules online without modifying the underlying calculation program code. Specific configurable content includes tiered commission gradient ranges, commission rates for each range, cross-departmental cost sharing coefficients, refund profit reduction ratios, abnormal data verification thresholds, and data snapshot generation time granularity. Rule parameter modifications take effect immediately without requiring a system service restart, achieving complete decoupling between business rules and underlying code. This significantly reduces system iteration and maintenance costs and adapts to the dynamic adjustment needs of enterprise financial accounting rules.
[0072] 5. Specific Application Implementation Cases
[0073] Taking the multi-department profit accounting scenario of a medium-sized technology service enterprise as an example, the enterprise has multiple business departments such as Sales Department 1, Sales Department 2, Project Department, and Operations Department. There are complex accounting scenarios such as cross-departmental cooperation projects, tiered calculation of employee commissions, allocation of public operating costs, and reduction of profits by customer refunds.
[0074] After adopting the solution of this invention, the system monitors incremental data from the enterprise's seven major business databases in real time. After business operations such as business opportunity entry, customer payment collection, employee expense reimbursement, salary accrual, and expense allocation are completed, data collection is completed within 50ms, and single data calculation and profit update are completed within 200ms. When a customer refund occurs, the system immediately triggers reverse accounting logic, automatically reducing the income and profit of the corresponding department, salesperson, and project without manual intervention.
[0075] Managers can view real-time revenue, costs, gross profit, and net profit for each department through a front-end dashboard. They can also drill down to view detailed profit data for individual salespersons or projects with a single click. For any discrepancies in accounting data, the system automatically archives and retains the data, supporting rapid location, correction, and data re-injection. All profit changes are logged completely, enabling end-to-end traceability and reconciliation via business transaction IDs. This completely solves the problems of data lag, inaccurate accounting, difficulty in traceability, and cumbersome maintenance associated with traditional T+1 offline accounting.
[0076] 6. Implementation Results Description
[0077] By implementing the technical solution of this invention, compared with the traditional accounting model, the following technical effects can be achieved: First, it enables millisecond-level real-time collection of business and financial data and real-time profit calculation, completely eliminating data lag; second, through full-dimensional data preprocessing and financial verification, the profit calculation error rate is reduced to below 0.5%; third, the event-driven local recalculation mechanism significantly improves accounting efficiency, and the system operates stably without delay in high-concurrency scenarios; fourth, it enables three-level data drill-down and full-link traceability, meeting the needs of enterprises for refined accounting and audit compliance; fifth, the rule visualization configuration replaces code modification, improving system operation and maintenance iteration efficiency by more than 80%, adapting to the long-term business development and rule iteration needs of enterprises.
Claims
1. A method for real-time profit aggregation calculation across multiple departments, characterized in that, This includes five processing steps executed sequentially: S1. Multi-source data access steps: Connect to seven business data sources: business opportunity database, business status database, expense reimbursement database, payment collection database, commission calculation database, salary cost database, and cross-department allocation ledger. Use a data synchronization mechanism to monitor and capture the addition, modification, and deletion of incremental data in each data table in real time. Push all incremental data to the message queue to complete the distribution. Use incremental collection mode to replace full data retrieval to achieve millisecond-level collection of business data. S2. Data Preprocessing Steps: Obtain incremental data streams transmitted via message queues, unify heterogeneous data formats, filter invalid and dirty data, and automatically complete key fields such as department codes, business types, commission rates, and cost allocation rules; configure finance-specific verification rules to verify the legality of payment amounts, commission rates, and cost allocation coefficients; mark the abnormal data that fails verification and store it together with the original data in the error log database, and send the standardized data that passes verification to the streaming computing engine; S3, Real-time Streaming Computing Steps: Deploy resident streaming computing tasks in the computing engine to perform real-time calculations on individual business data entries without waiting for batch data aggregation. Specifically, this includes three computational logics: ① Revenue Calculation: Match business opportunity information with payment records and aggregate business revenue based on the department to which the data belongs; ② Cost Deduction Calculation: Simultaneously calculate employee commissions, departmental fixed salaries, reimbursed expenses, and shared operating costs, and deduct them from revenue accordingly; ③ Rule Matching Calculation: Call preset tiered commission rules, cross-departmental allocation rules, and refund profit reduction rules to complete all cost deduction calculations in real time. S4. Multi-department profit aggregation steps: Secondary streaming aggregation calculation is carried out with department code as the first-level aggregation dimension, and real-time statistics of real-time revenue, real-time total cost, real-time gross profit and real-time net profit of a single department are calculated; it supports three-level drill-down aggregation of department-salesperson-account project; whenever business data is added, modified or deleted, the profit value of the corresponding department is immediately recalculated. S5. Storage and Visualization Output Steps: Write the real-time aggregated profit indicators into the in-memory database to support real-time visualization on the front-end page; perform snapshot persistence storage of profit data at each time point; the system opens up data call interfaces to the outside world, and the front-end dynamically refreshes the real-time profits of each department through the interfaces; retain the profit change log corresponding to each business data change, and complete the full-link traceability and reconciliation of profit data based on the transaction identifier.
2. The method for real-time profit aggregation calculation across multiple departments according to claim 1, characterized in that: Step S1 employs CDC change data capture technology to monitor the database binlog logs, enabling non-intrusive incremental extraction of business data addition, deletion, and modification events.
3. The method for real-time profit aggregation calculation across multiple departments according to claim 1, characterized in that: In step S2, the error log library stores the original message, error type, and generation timestamp of the abnormal data. After the administrator corrects the abnormal data, it can be fed back into the preprocessing process to participate in the accounting again.
4. The method for real-time profit aggregation calculation across multiple departments according to claim 1, characterized in that: Step S3: Upon receiving refund business data, the corresponding department's already recorded revenue is deducted in reverse, the accrued commissions and allocated costs are rolled back simultaneously, and the department's current net profit is updated in real time.
5. The method for real-time profit aggregation calculation across multiple departments according to claim 1, characterized in that: Step S5 generates and persists a profit snapshot at a fixed time granularity. Each business transaction ID is bound to a profit change record for storage, establishing a one-to-one mapping relationship between documents and profit changes.
6. A multi-department real-time profit aggregation calculation system, used to execute the multi-department real-time profit aggregation calculation method according to any one of claims 1 to 5, characterized in that, The system is structured in five layers from top to bottom: multi-source data access layer, data preprocessing layer, streaming real-time computing layer, multi-dimensional profit aggregation layer, and data storage and visualization output layer. The multi-source data access layer includes a multi-source data source connection module, an incremental data capture module, and a message queue distribution module. The multi-source data source connection module connects to the business opportunity database, business status database, expense reimbursement database, payment collection database, commission calculation database, salary cost database, and cross-departmental allocation ledger. The incremental data capture module collects the incremental changes in business data in real time and pushes the incremental data down in milliseconds via the message queue distribution module. The data preprocessing layer includes a format standardization module, a dirty data filtering module, a key field completion module, a financial data verification module, and an abnormal data diversion and archiving module. The pre-processed compliant data flows to the downstream streaming real-time computing layer, while abnormal data is uniformly archived to the error log library; The streaming real-time computing layer is the core computing unit of the system. It deploys resident streaming computing tasks and integrates a revenue matching and collection unit, a comprehensive cost deduction calculation unit, and a financial rule matching unit to complete the real-time accounting and processing of revenue and various costs for each stream of streaming data. The multi-dimensional profit aggregation layer is configured with department-dimensional aggregation units, multi-level drill-down aggregation units, and data change triggering recalculation units; it aggregates and calculates various profit indicators with the department code as the primary key, and automatically triggers the immediate recalculation of the corresponding department's profit value when a business data change event occurs. The data storage and visualization output layer includes a real-time in-memory database, a historical snapshot persistence library, an external API interface service module, and a front-end visualization display module; The in-memory database is responsible for supporting the real-time profit data display on the front end, while the persistent database stores historical profit snapshots and full profit change logs. It also relies on API interfaces to enable external system calls, data display, and business reconciliation and traceability.
7. A multi-department real-time profit aggregation calculation system according to claim 6, characterized in that: The streaming real-time computing layer uses the Flink framework to build resident streaming computing tasks.
8. A multi-department real-time profit aggregation calculation system according to claim 6, characterized in that: Redis is used as the real-time in-memory database, while historical profit snapshots, error logs, and profit change logs are persistently stored using a combination of relational databases and time-series databases.
9. A multi-department real-time profit aggregation calculation system according to claim 6, characterized in that: The system features a visual rules configuration panel, allowing staff to configure and modify tiered commission parameters, cross-departmental cost allocation coefficients, and refund reduction rules online without modifying the underlying calculation program code.