Big data based audit analysis method

Abstract

The application provides an audit analysis method based on big data, comprising: collecting data of a marketing domain from the audit data penetration coverage of the network, province, city, and county, and obtaining the audit suspicious point data of each unit; for the audit data after preliminary processing, using in-memory computing technology to analyze the data in real time, extracting key indicators including business type, transaction amount, and transaction frequency, and obtaining a multi-dimensional index data set; according to the multi-dimensional index data set, using a preset anomaly detection algorithm to analyze the data, if the transaction amount or transaction frequency of a certain data point exceeds the preset threshold, the data point is marked as a potential suspicious point, and a suspicious point candidate set is obtained; for the violation behavior detection result, using a preset visual display template to display the audit result on a map, a pie chart and other different dimensions, and generating a visual audit report.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of information technology, and particularly relates to an audit analysis method based on big data. BACKGROUND

[0002] In the audit suspicious data analysis scene, there is a technical challenge of how to effectively integrate and analyze massive audit data. Audit data usually comes from different business systems and databases, and the data formats and structures are different, and the data volume is huge. How to quickly and accurately extract, clean, transform and load these heterogeneous data, and carry out correlation analysis and mining, is a complex technical problem. In addition, the judgment of audit suspicion needs to consider multiple dimensions of indicators, how to design a reasonable algorithm and model to automatically identify suspicious abnormal points and violations is also a problem to be solved. At the same time, the visualization of audit results also faces challenges, how to clearly and intuitively present the audit situation on different dimensions such as maps and pie charts needs carefully designed visualization solutions and interaction technology. In the context of massive data, how to ensure the performance and efficiency of data processing and analysis also puts higher requirements on the technical architecture and algorithm optimization. These technical problems are interrelated, and need in-depth research and innovation in data processing, algorithm design, visualization display and other aspects, in order to finally realize efficient, accurate and comprehensive audit suspicious data analysis. SUMMARY

[0003] The present application provides an audit analysis method based on big data, mainly including:

[0004] From the audit data penetration coverage of the network, province, city, district and county, the data of the marketing domain is collected to obtain the audit suspicious point data of each unit; according to the preset distributed computing framework, the audit suspicious point data is divided into multiple data blocks and distributed to different computing nodes for parallel processing to obtain the preliminary processed audit data, and the preliminary processed audit data includes the transaction records, account information and business operation logs after standardization and preprocessing; for the preliminary processed audit data, the memory computing technology is used to analyze the data in real time, extract key indicators including business type, transaction amount and transaction frequency, and obtain a multi-dimensional index data set; according to the multi-dimensional index data set, a preset anomaly detection algorithm is used to analyze the data in the multi-dimensional index data set, if the transaction amount of a certain data point exceeds a preset threshold A or the transaction frequency exceeds a preset threshold B, it is marked as a potential suspicious point to obtain a suspicious point candidate set; a transaction data set is obtained, which includes transaction amount, transaction frequency and transaction time, by setting the normal range of each indicator, if multiple indicators exceed the normal range at the same time, it is marked as a suspicious abnormal point, the suspicious abnormal point is combined with the suspicious point candidate set to generate a violation behavior detection result, and the data processing time of the violation behavior detection is recorded; for the violation behavior detection result, a preset visual display template is used to display the audit result on the map and pie chart to generate a visual audit report; according to the visual audit report, the dynamic interaction between the user and the data is realized through the interaction technology, if the user selects a certain data point, the detailed information of the data point is displayed, including the violation behavior type, the involved amount and the occurrence time; if the data processing time of the violation behavior detection exceeds a preset threshold C, the algorithm parameters are automatically adjusted, including the number of trees or the abnormal score threshold D of the anomaly detection algorithm, to improve the processing efficiency, and the performance optimization is throughout the entire data processing process to ensure the real-time response capability of the system; the visual audit report is fed back to each business system according to the preset interface specification to realize the closed-loop management of the data, and the closed-loop management includes pushing the violation behavior detection result to the related business system to trigger the early warning or blocking mechanism and recording the subsequent processing situation to form a complete audit tracking chain.

[0005] The technical scheme provided by the embodiment of the application can include the following beneficial effects:

[0006] The application discloses an audit analysis method based on big data. The method collects marketing domain data from multiple levels, uses a distributed computing framework to perform parallel processing and real-time analysis on suspicious data. A multi-dimensional index comprehensive model is used for irregular behavior detection, and a pre-set anomaly detection algorithm is used to identify potential suspicious points. The application also integrates visualization display and interactive technology to generate intuitive audit reports and support dynamic interaction between users and data. During data processing, the application can automatically adjust algorithm parameters according to performance requirements to ensure real-time response capabilities. Finally, by feeding back the analysis results to the business system, the application realizes closed-loop management of audit data. This method not only improves audit efficiency and accuracy, but also enhances the visualization and application value of audit results, providing strong support for enterprise compliance management. BRIEF DESCRIPTION OF DRAWINGS

[0007] Figure 1 A flowchart of the audit analysis method based on big data of the application. DETAILED DESCRIPTION

[0008] To further understand the content of the application, the application is described in detail in conjunction with the drawings and examples. The application is further described in conjunction with the drawings and examples. It can be understood that the specific examples described herein are only used to explain the related application, and not to limit the application. In addition, it should be noted that only the parts related to the application are shown in the drawings for ease of description.

[0009] As Figure 1 , the audit analysis method based on big data of the embodiment can specifically include:

[0010] S101, collect data of the marketing domain from the audit data penetration coverage of the network, province, city, and county, and obtain suspicious data of each unit.

[0011] An initial audit data set is obtained from a network-level audit system. Data cleaning is performed on the initial audit data set to remove duplicate records, handle missing values, and unify data formats, forming a cleaned data set. For the cleaned data set, a province screening rule is pre-established, which includes an identification field of each province. According to the province screening rule, audit data corresponding to each province is screened from the cleaned data set, and the audit data of each province is associated with the network-level audit data to form a province-level associated data set. The association operation uses a database table join operation, and the join field is a unit code. For the province-level associated data set, a city-level screening rule is pre-established, which includes an identification field of each city. According to the city-level screening rule, audit data corresponding to each city is screened from the province-level associated data set. It is judged whether the unit code field in the city-level audit data is consistent with the unit code field in the province-level associated data set. If they are consistent, the city-level audit data and the province-level associated data set are combined to form a city-level associated data set. For the city-level associated data set, a county-level screening rule is pre-established, which includes an identification field of each county. According to the county-level screening rule, audit data corresponding to each county is screened from the city-level associated data set. Data verification is performed on the county-level audit data to check whether the data is within the preset business rules. If it meets the preset business rules, a county-level data set is formed. For the county-level data set, a marketing domain field rule is pre-established, which includes a field name related to marketing business. According to the marketing domain field rule, all field data related to the marketing domain is screened from the county-level data set to form a marketing domain data set. For the marketing domain data set, the interquartile range of each unit is calculated. If the value of a certain index of a certain unit in the marketing domain data set exceeds the threshold value of the upper limit or lower limit of the interquartile range, the value of the index of the unit is marked as suspicious data to form a unit suspicious data set. For the unit suspicious data set, grouping is performed according to the unit code, and the suspicious data in each group is counted and summarized to generate an audit suspicious data report of each unit, from which the audit suspicious data of each unit is obtained.

[0012] The initial audit dataset obtained from the network-level audit system is the starting point of the entire audit process. This data may include various network devices, system logs, and user activity records. For example, it may include router traffic data, firewall logs, application access records, etc. These raw data often have different formats and may have duplicates, missing values, or errors. Data cleaning is a crucial step to ensure data quality. In this stage, we may encounter the following situations: duplicate login records, missing IP addresses, inconsistent timestamp formats, etc. By removing duplicates, filling in missing values, and unifying formats, we obtain a more standardized and reliable dataset. The establishment of province screening rules helps to geographically classify the data. Provinces may have unique identifiers such as administrative division codes. Such classification makes subsequent analysis more targeted and enables the discovery of regional problems or trends. Using database table join operations, we can associate province-level data with network-level data, with the unit code as the connection field, to enrich the content and context of the data. For example, abnormal traffic patterns in a certain region may be significantly different from overall trends, which may suggest unique network security issues in that region.

[0013] The data screening at the city and county levels further refines the data granularity. This multi-level data structure enables auditors to comprehensively grasp the situation from macro to micro. For example, an abnormal increase in network traffic in a sub-region may be related to the construction of a new data center in the area. Data verification ensures the accuracy and compliance of the data. Pre-established marketing domain field rules may include traffic thresholds, access frequency limits, etc. The formation of the marketing domain dataset focuses on specific business areas. This may include customer acquisition cost, customer retention rate, marketing campaign effectiveness, and other indicators. By analyzing these data, we can evaluate the effectiveness of marketing strategies and ROI (Return on Investment). Calculating the interquartile range of each unit's indicators helps identify outliers. For example, if the customer acquisition cost of a certain unit is much higher than the upper quartile range of the industry, it may indicate low marketing efficiency or abnormal expenditures, and data beyond the quartile range is marked as suspicious data, forming a unit suspicious data set. Grouping the unit suspicious data set by unit code, counting and summarizing the suspicious data within each group, and generating an audit suspicious data report for each unit. Finally, the generated audit suspicious data report provides management with a clear overview of risks. This report may show that some units have multiple abnormal indicators, such as abnormally high network traffic, frequent security alerts, and excessive marketing expenditures, which may indicate comprehensive management problems or potential violations.

[0014] S102, according to the preset distributed computing framework, the audit suspicious data case is divided into multiple data blocks, and is distributed to different computing nodes for parallel processing to obtain the preliminary processed audit data, and the preliminary processed audit data includes standardized and preprocessed transaction records, account information and business operation logs.

[0015] A distributed computing framework is adopted to split the audit suspicious data case into multiple data blocks, each data block containing transaction records, account information and operation logs within a specific range. Through a preset distribution mechanism, the data blocks are distributed to different computing nodes to ensure balanced data volume for each node. Each computing node performs standardization and preprocessing operations in parallel, the standardization process including data format unification, field completion and outlier processing, and the preprocessing process including data cleaning, deduplication and feature extraction to obtain processed data blocks. The processed data blocks are aggregated to a central node for integration to generate a preliminary audit data set containing standardized transaction records, account information and business operation logs. Quality checks are performed on the preliminary audit data set, and if data loss or abnormalities are found, the corresponding computing nodes are traced back for correction. Machine learning algorithms are used for further analysis of the preliminary audit data, including clustering algorithms to identify abnormal transaction patterns, classification algorithms to assess account risks and association rule mining to analyze potential risks in operation logs, to obtain the preliminary processed audit data.

[0016] For example, assuming that the preset distributed computing framework is Hadoop, the suspicious data case is the transaction records of a bank in a year, the total data volume is 10 TB, and the fields include transaction serial number, transaction time, transaction amount, transaction account number, and transaction type. First, the 10 TB of transaction records are divided into 365 data blocks according to the transaction date, each data block is about 28 GB, and the HDFS distributed file system of Hadoop is used to store these data blocks. Then, the MapReduce programming model is used for parallel processing. In the Map stage, 365 data blocks are allocated to 100 computing nodes in the cluster, and each node is responsible for processing about 3 data blocks. In each Map task, first, data cleaning is performed, for example, records with transaction amount less than 0 or more than 100 million RMB in the transaction amount field are marked as suspicious transactions, and records with empty transaction type field are filtered out. Then, data standardization is performed, the transaction time field is uniformly converted to Unix timestamp format, and the transaction account number field is uniformly converted to fixed-length string format, and the insufficient bits are padded with zeros. For example, the account number "12345" is converted to "000000000012345". Then, data preprocessing is performed, for example, grouping according to transaction amount, calculating the number of transactions and total amount of each group (e.g., 0-1000 yuan, 1001-10000 yuan, 10001-100000 yuan, etc.), which can use, for example, a fast grouping aggregation algorithm to perform local aggregation in the Map stage, reducing the data transmission volume in the Reduce stage. In the Reduce stage, the output results of 100 Map tasks are combined. For example, all suspicious transaction records marked in the Map tasks are aggregated to generate a suspicious transaction list; the statistical results of transaction amount grouping in all Map tasks are combined to obtain the transaction amount distribution of the whole year. Finally, the preliminary processed audit data is obtained, including suspicious transaction records after cleaning, standardization and preprocessing, Unix timestamp and standardized account number information of all transaction records, and transaction amount grouping statistics of the whole year, etc. After obtaining the preliminary processed audit data, quality inspection is performed, and if data is missing or abnormal, the corresponding computing node is modified. For example, the suspicious transaction records, Unix timestamp and standardized account number information of all transaction records, and transaction amount grouping statistics of the whole year, etc. are checked for missing or obvious abnormalities, and if so, the corresponding node of the data is modified. These data can be further used for subsequent audit analysis, for example, analyzing the pattern of suspicious transactions through clustering algorithm, discovering abnormal transaction relationships through association rule mining, etc.

[0017] S103, for the preliminary processed audit data, real-time analysis of the data is performed using in-memory computing technology to extract key indicators, including business type, transaction amount, transaction frequency, to obtain a multi-dimensional indicator data set.

[0018] The preliminary processed audit data is loaded into a memory computing framework for data preprocessing, including data cleaning and data formatting, to ensure data quality and obtain preprocessed audit data. Real-time analysis is performed on the preprocessed audit data through memory computing technology, and parallel processing capability is utilized to quickly extract key indicator data including business type, transaction amount, and transaction frequency. The extracted key indicator data is subjected to multi-dimensional aggregation operations to generate a preliminary multi-dimensional indicator dataset. Based on the preliminary multi-dimensional indicator dataset, a machine learning algorithm is used for pattern recognition to construct a transaction behavior model for predicting future transaction trends. The transaction behavior model is continuously optimized in combination with historical data and real-time data to obtain a multi-dimensional indicator dataset.

[0019] For example, assume that the pre-processed audit data has been loaded into an in-memory database, such as Apache Spark or SAP HANA. Using Apache Spark as an example, it can load the pre-processed audit data into distributed memory as a Resilient Distributed Dataset (RDD). This approach can significantly improve data processing speed, especially for data that needs to be accessed repeatedly. During the data preprocessing phase, Spark's transformation operations can be used for data cleaning and formatting. For example, for outliers in transaction records, such as negative amounts or excessively large amounts, filters can be used to remove or correct them. For the problem of inconsistent date and time formats, custom functions can be used to convert them uniformly to a standard format. These operations can ensure the consistency and reliability of the data, laying the foundation for subsequent analysis. Real-time analysis is a major advantage of in-memory computing. Through technologies such as Spark Streaming, pre-processed audit data can be processed in parallel to extract key indicators in real time. For example, a sliding window can be set to calculate the total transaction amount, average amount, and transaction frequency within the past hour every 5 minutes. This method can quickly detect abnormal transactions and improve audit efficiency. Multi-dimensional aggregation operations are an important part of data analysis. Using Spark SQL, complex aggregation queries can be performed on the extracted indicator data to obtain a preliminary multi-dimensional indicator dataset. For example, transactions can be grouped by business type, time period, and amount interval to calculate the number of transactions and total amount in each group. This allows us to observe transaction patterns from different angles and identify potential risk points. Machine learning algorithms play a key role in building transaction behavior models. Clustering algorithms such as K-means in the Spark MLlib library can be used to classify transaction data and identify different transaction patterns. For example, customers can be divided into high-risk, medium-risk, and low-risk categories based on transaction frequency, amount, and time, which can help to target risk control. Predicting future transaction trends is an important aspect of model application. Through time series analysis algorithms such as the ARIMA model, historical transaction data can be used to predict transaction volume and amount in the future. This is important for fund management and risk assessment. Continuous optimization of the model is crucial to its effectiveness. Online learning algorithms such as random forests can be used to continuously absorb new transaction data and update model parameters. At the same time, through regular cross-validation, the accuracy of the model can be evaluated, and feature selection and algorithm parameters can be adjusted in a timely manner to adapt to changing transaction environments. This in-memory computing-based audit data analysis method can significantly improve processing speed and analysis depth. Through real-time monitoring, multi-dimensional analysis, and intelligent prediction, potential risks can be detected earlier, improving the efficiency and accuracy of audit work. At the same time, continuously optimized models can adapt to changing financial environments, providing reliable data support for audit decision-making.

[0020] S104, according to the multi-dimensional index data set, using a preset anomaly detection algorithm to analyze the data in the multi-dimensional index data set, if a data point transaction amount exceeds a preset threshold A, or the transaction frequency exceeds a preset threshold B, it is marked as a potential suspect, and a suspect candidate set is obtained.

[0021] Obtain a multi-dimensional index data set containing key indicators such as business type, transaction amount, and transaction frequency. Adopt a preset anomaly detection algorithm to analyze each item in the multi-dimensional index data set. If a data point transaction amount exceeds a preset threshold A, mark the data point as a money abnormal suspect. If a data point transaction frequency exceeds a preset threshold B, mark the data point as a frequency abnormal suspect. Summarize the potential suspects marked as money abnormal suspects and frequency abnormal suspects to generate a suspect candidate set.

[0022] For example, assume we have obtained the multi-dimensional indicator dataset described in the previous step, which includes information such as business type, transaction amount, transaction frequency, and number of unique users. Now, we use the Isolation Forest algorithm for anomaly detection. For instance, for business type "A001," its historical average transaction amount is 120,000 yuan, with a standard deviation of 15,000 yuan. We set threshold A to three times the standard deviation, meaning transactions exceeding 120,000 + 3 * 15,000 = 165,000 yuan or falling below 120,000 - 3 * 15,000 = 75,000 yuan will be flagged as potentially suspicious. Regarding transaction frequency, assuming user "user123" has an average of 5 transactions in the past hour, with a standard deviation of 1, we set threshold B to three times the standard deviation. Transaction frequencies exceeding 5 + 3 * 1 = 8 transactions or falling below 5 - 3 * 1 = 2 transactions are considered abnormal. At the same time, combined with the rate of change of unique users (UV), for example, if the growth rate of UV (150) in the current time window compared with the previous time window (135) exceeds the preset threshold (e.g., 10%), i.e. (150-135) / 135>10%, even if the transaction frequency of a single user does not exceed the threshold, there may be a risk of batch registration of fake accounts. Therefore, all transactions within these time windows are included in the potential suspicious points. All potentially suspicious transactions (including anomalies detected based on amount, frequency, and UV change rate) are consolidated into a JSON-formatted candidate set of suspicious transactions, for example: `{"suspicious_transactions": [{"user_id":"user123", "business_type": "A001", "amount": 180000, "timestamp": "2024-07-24 10:35:00", "reason": "Transaction amount exceeds threshold"}, {"user_id": "user456", "business_type": "B002", "count": 10, "timestamp": "2024-07-24 10:40:00", "reason": "Transaction frequency exceeds threshold"}, {"timestamp": "2024-07-24 10:30:00", "reason": "Abnormal UV growth"}]}`.

[0023] S105, acquire a transaction data set containing transaction amount, transaction frequency, transaction time, set a normal range for each index, if multiple indexes exceed the normal range at the same time, mark as a suspicious abnormal point, combine the suspicious abnormal point with the suspicious point candidate set to generate a violation behavior detection result, and record the data processing time of the violation behavior detection.

[0024] A transaction data set containing basic information such as transaction amount, transaction frequency, and transaction time is acquired. The missing values of the data in the transaction data set are filled using an interpolation method, and the abnormal values are removed using a 3 times standard deviation method to ensure data quality. The normal range threshold of the transaction data in each transaction data set is set in the amount, frequency, and time dimensions, and if multiple dimension index values of a transaction record exceed the normal range threshold at the same time, it is marked as a suspicious abnormal point. The marked suspicious abnormal points are further screened, combined with the suspicious point candidate set, and a random forest algorithm is used for violation behavior prediction. If the prediction result is a violation, it is confirmed as a violation behavior. A violation behavior detection result containing detailed information of the violation transaction record and violation judgment basis is generated, the data processing time of the violation behavior detection is recorded, and stored in a database.

[0025] For example, the acquisition of transaction data sets is the basis of risk monitoring. This data set usually contains key information such as transaction amount, transaction frequency and transaction time. For example, a typical transaction record may include fields such as user ID, transaction amount, transaction timestamp and transaction type. Data quality is crucial for accurate risk assessment. Interpolation method is used to fill in missing values, such as using linear interpolation to fill in missing transaction amounts in time series data. The three-sigma rule is used to eliminate outliers, which helps to eliminate extreme data points that may distort the analysis results. For example, if the average transaction amount of a user is 1000 yuan and the standard deviation is 200 yuan, transactions exceeding 1600 yuan or less than 400 yuan may be marked as abnormal and need further review. Determining the normal range threshold is a key step in detecting suspicious transactions. This usually involves analyzing historical transaction data and calculating the mean and standard deviation of each dimension (amount, frequency, time). For example, if the normal amount range of a certain type of transaction is 500-2000 yuan, the normal frequency is 1-5 times per day, and the normal transaction time is from 9 am to 10 pm, a transaction that occurs at 2 am with an amount of 5000 yuan may be marked as suspicious. Further screening of suspicious outliers involves more complex analysis. Here, factors such as suspect candidate set, transaction behavior characteristics and historical transaction patterns are introduced. For example, if a user suddenly starts making a large number of small transactions, which is inconsistent with his historical transaction patterns, even if each transaction is within the normal range, it may be marked as suspicious. The use of random forest algorithm combines suspicious outliers and suspect candidate sets, which reflects the application of machine learning in risk detection. This algorithm predicts whether a transaction is in violation by building multiple decision trees. Each tree may focus on different features, such as the degree of abnormality of transaction time, the degree of deviation of amount, the change of user's recent behavior, etc. Finally, the algorithm will integrate the results of all decision trees to give a violation probability. The confirmation of violation behavior is the key output of the whole process. This not only includes the transaction records that are judged to be in violation, but also the basis for making such a judgment. For example, a transaction may be judged to be in violation because of its unusually high amount, unusual time of occurrence and the user's recent multiple similar suspicious behaviors. Finally, the data processing time of each violation detection is recorded, and the processing time result is stored in the database. Recording data processing time not only records, but also provides a data basis for future analysis and model optimization, which helps to evaluate and optimize the efficiency of the entire detection process. This complete process reflects a comprehensive risk management method from data collection, cleaning, analysis to decision-making, effectively combining statistical analysis and machine learning techniques to provide strong protection for financial security.

[0026] S106, for the violation behavior detection result, a preset visual display template is used to display the audit result on the map and pie chart, and a visual audit report is generated.

[0027] The violation detection result is obtained, and data preprocessing is performed, including data cleaning and data standardization. Through data cleaning, invalid and redundant data is removed to ensure data quality; through data standardization, data format is unified to facilitate subsequent processing. The preprocessed data is classified and aggregated according to the preset visualization template requirements. For the map display dimension, the data is classified according to the geographic location information to generate a geographic distribution dataset; for the pie chart display dimension, the data is classified according to the violation type to generate a type distribution dataset. The geographic distribution dataset is converted into a map visualization image using a map generation algorithm, including map base map loading, data point labeling, and heat map rendering. The type distribution dataset is converted into a pie chart visualization image using a pie chart generation algorithm, including pie chart drawing, data proportion calculation, and color filling. The generated map and pie chart visualization images are integrated according to the preset report format to generate a visual audit report.

[0028] For example, assume we have an audit dataset containing the violation behavior of employees in different regions, including fields such as "region identifier", "violation type", "violation amount", "handling status", etc. First, the system reads the dataset and uses predefined rules to preliminarily filter the violation behavior. After filtering the violation records, the system begins to generate a visualization report. In terms of map visualization, the system calculates the total violation amount of each region, such as region A with a total violation amount of 125,800 yuan, region B with a total violation amount of 87,650 yuan, and region C with a total violation amount of 36,300 yuan, etc. Then, using geographic information data, the violation amount is mapped to the corresponding map, with color depth representing the violation amount, using, for example, Jet color band for color mapping, with region A displaying as dark red for the highest violation amount and region B displaying as light blue for the lower violation amount. In terms of pie chart visualization, the system first groups the violation records according to the "violation type" field, calculates the proportion of each violation type, such as "violation type one" accounting for 45%, "violation type two" accounting for 30%, "violation type three" accounting for 15%, and "other" accounting for 10%. Then, a pie chart is generated, with each sector representing a violation type and the sector size representing the proportion of that type of violation. After the pie chart is generated, the pie chart data needs to be associated with the map data, and when the mouse hovers over a certain region of the map, the pie chart can be dynamically updated to display the violation type distribution of that region, such as hovering the mouse over region A to pop up a new pie chart showing the violation type proportion of region A as "violation type one" accounting for 60%, "violation type two" accounting for 25%, "violation type three" accounting for 10%, and "other" accounting for 5%. Through this correlation analysis, it can be found that the violation type one problem in region A is more prominent than the average level. Finally, the system integrates the generated map, pie chart, and related statistical tables into an HTML file to form a visual audit report.

[0029] S107, according to the visual audit report, the dynamic interaction between the user and the data is realized through the interactive technology, if the user selects a data point, the detailed information of the data point is displayed, including the type of violation behavior, the involved amount and the occurrence time.

[0030] The visual audit report data is obtained, the integrated interactive technology is used, the chart has the interactive function, the user can interact with the chart through clicking and hovering operations, if the user selects a data point, the detailed information display logic is triggered. Determine the data point selected by the user, extract the detailed information of the data point from the visual audit report data, including the type of violation behavior, the involved amount and the occurrence time. Dynamically generate a detailed information display interface and present the extracted detailed information of the data point on the interface to facilitate the user to quickly obtain information.

[0031] For example, the acquisition and preprocessing of audit report data are the basis of visualization analysis. Taking an annual audit of a certain enterprise as an example, different types of violation behavior data may be collected. During the preprocessing process, the data format needs to be unified, the abnormal values need to be cleaned, and the integrity of the fields needs to be ensured. For example, the amount is unified to ten thousand yuan unit, and the time is formatted to "year-month-day". The choice of visualization technology directly affects the data display effect. For the distribution of violation behavior types, pie chart or column chart can intuitively show the proportion of each type. For the trend of the amount involved over time, line chart is more suitable. If you want to display three dimensions of violation type, amount and time at the same time, scatter plot is a good choice, where the horizontal axis represents time and the vertical axis represents amount, and different colored points represent different violation types. The integration of interactive functions greatly improves user experience. Taking scatter plot as an example, you can achieve the function of displaying brief information when the mouse hovers over it, and displaying detailed information window when you click it. This design allows users to quickly browse the overall situation while deeply understanding specific cases of interest. After the user selects a data point, the system needs to quickly extract relevant information from the preprocessed data. Suppose the user clicks on a data point representing violation type one, the system will immediately display detailed information such as "violation type: violation type one; amount involved: 500 million yuan; occurrence time: July 15, 2023; department involved: business department A; handling status: case has been investigated". The design of the detailed information display interface needs to consider the hierarchy and readability of the information. You can use a card layout to display different categories of information in blocks. Important information such as violation type and amount can be emphasized with larger font size or eye-catching color, while secondary information can be presented in smaller font size. Analysis of user interaction behavior is crucial for system optimization. By recording the violation types or amount intervals that users frequently focus on, you can adjust the priority of data display. For example, if you find that users frequently view large violation cases, you can highlight this information in the initial interface to improve user efficiency. This visualization analysis system based on audit reports not only helps auditors find problems more efficiently, but also provides intuitive decision support for management. Through dynamic and interactive data display, complex audit information becomes easy to understand and analyze, thereby promoting the improvement of internal control system and the enhancement of risk management capability.

[0032] S108, if the data processing time for detecting violation behavior exceeds the preset threshold C, automatically adjust the algorithm parameters, including the number of trees of the anomaly detection algorithm or the anomaly score threshold D, to improve processing efficiency. Performance optimization runs through the entire data processing process to ensure the real-time response capability of the system.

[0033] The data processing time for performing the violation behavior detection is obtained and compared with a preset threshold C. If the data processing time for performing the violation behavior detection exceeds the preset threshold C, a parameter adjustment mechanism is triggered. The abnormal detection algorithm parameters that need to be adjusted are determined, including but not limited to the number of trees of the isolation forest algorithm and the abnormal score threshold D. According to a preset adjustment rule, a new parameter value is calculated to ensure that the new parameter can significantly improve the processing efficiency. The new parameter value is applied to the abnormal detection algorithm to update the algorithm configuration. The running situation of the updated algorithm is monitored in real time, a new data processing time for performing the violation behavior detection is obtained, and the new data processing time is compared with the preset threshold C again. If the new data processing time for performing the violation behavior detection still exceeds the preset threshold C, the above adjustment process is repeated until the real-time response requirement is met, and the real-time response capability of the system is ensured.

[0034] For example, assume that the system preset data processing time threshold C is 500 milliseconds, the current processing of a batch of 10,000 network traffic data anomaly detection tasks, initially, the number of Isolation Forest algorithm trees is set to 100, and the anomaly score threshold D is set to 60. The system obtains the data processing time for performing the violation behavior detection through the performance monitoring module, and finds that the current batch data processing time is 650 milliseconds, which exceeds the preset threshold C of 500 milliseconds. The automatic parameter adjustment mechanism is triggered. First, analyze the relationship between historical data processing time and algorithm parameters, and construct a simple linear regression model: processing time = a * tree number + b * anomaly score threshold + c. Use the past 10 batches of data (for example, tree numbers are 80, 90, 110, 120, etc., and anomaly score thresholds D are 55, 60, 65, etc., corresponding to processing times of 450 ms, 520 ms, 700 ms, 780 ms, etc.) to train the model, and obtain the model parameters a = 5, b = 100, and c = 50. Then, based on the model, the parameters are optimized. To reduce the processing time, the number of trees or the anomaly score threshold D can be reduced (which may cause the false positive rate to rise, which needs to be balanced). To simplify the calculation and prioritize efficiency, the system decides to adjust the number of trees first. Set the target processing time to 480 milliseconds (slightly lower than the threshold to leave a margin), and substitute it into the model formula: 480 = 5 * 74 + 100 * 60 + 50. The new tree number is calculated to be about 74. The system automatically adjusts the number of Isolation Forest algorithm trees to 74. After adjustment, the next batch of 10,000 network traffic data tasks is processed, and the processing time is reduced to 490 milliseconds, meeting the preset threshold C requirement. At the same time, the system continuously monitors the false positive rate, and if the false positive rate significantly rises (for example, exceeds the preset upper limit of 5%), the anomaly score threshold D will be adjusted, for example, the anomaly score threshold is gradually increased from 60 to 62, 65, etc., and the impact on processing time and false positive rate is observed, and a balance point is selected. All of these data and model parameters are recorded in the system log and can be exported as a CSV format file for subsequent offline analysis and model improvement.

[0035] S109, adopt the preset interface specification to feed back the visual audit report to each business system, realize the closed-loop management of data, the closed-loop management includes pushing the violation behavior detection result to the related business system, triggering the early warning or blocking mechanism, and recording the subsequent processing situation, forming a complete audit tracking chain.

[0036] After obtaining the visualized audit report, data cleaning is performed to remove duplicate data and invalid fields, and missing values are imputed to obtain cleaned data. The cleaned data is formatted, the field types and coding rules are unified to ensure data quality and consistency, and formatted data is obtained. The formatted data is input into an audit analysis module, and SQL queries and Python scripts are used for classification statistics to identify abnormal transaction records and push them to related business systems. After receiving the abnormal transaction records, the related business systems judge the abnormal type of each record one by one. If the abnormal type is high risk, warning information is sent to the relevant responsible person through email and SMS. If the abnormal type is serious violation, the relevant user account is immediately locked and its transaction authority is suspended, and detailed logs of the locking operation are recorded. According to the detailed logs of the locking operation, the business system generates a violation investigation task and assigns it to a designated risk control personnel. The risk control personnel checks the violation behavior according to the task requirements and enters the checking result into the business system. After receiving the checking result, the business system updates the state of the abnormal transaction record to form a complete audit tracking chain, which includes information such as abnormal type, processing operation, and checking result.

[0037] For example, the data processing flow of the audit report starts with data cleaning after obtaining the visualized report. This step aims to improve data quality and ensure the accuracy of subsequent analysis. For instance, when processing financial transaction records of a company, repeated numbers or invalid transactions with zero amount may be found. By removing these data and imputing the missing transaction dates (e.g., using the average of adjacent dates), the reliability of the data can be significantly improved. Data formatting is a crucial step to ensure data consistency. Taking cross-department transaction data as an example, different departments may use different date formats or currency units. By converting dates to the "year-month-day" format and unifying currency units to RMB, subsequent data analysis can be greatly simplified. In the audit analysis stage, the combination of SQL queries and Python scripts can efficiently identify abnormal transactions. For example, SQL queries are used to filter out transactions that exceed the department budget, and Python scripts are used to conduct in-depth analysis of these transactions, such as calculating their deviation from the historical average, to identify potential abnormal behavior. The push of abnormal transaction records reflects the collaboration between systems. When the audit system detects a large amount of money purchase transaction, it will immediately push the record to the procurement management system. This real-time push mechanism can greatly improve the efficiency of risk management. The processing flow of the business system after receiving the abnormal record demonstrates the importance of risk classification management. For example, for a transaction that exceeds the authorized limit but is not large in magnitude, the system may classify it as "high risk" and notify the relevant supervisor through email and SMS for review. For a suspected abnormal large transfer, the system will determine it as "serious violation" and immediately freeze the account and notify the risk control department. The investigation work of the risk control personnel is an important part of the audit tracking chain. Taking an account abnormal login event as an example, the risk control personnel may check the login log, contact the relevant personnel to verify the situation, check whether there is unauthorized operation, etc. These investigation results will be recorded in the system, forming a complete event processing track. The design of the entire process embodies the concept of "prevention before danger". Through automated data processing and analysis, combined with multi-level risk response mechanism, the system can discover and handle abnormal situations in time before the problem expands. This not only improves the efficiency of audit, but also greatly enhances the risk management ability of the enterprise. At the same time, the complete audit tracking chain provides valuable data support for subsequent process optimization and risk assessment, which helps the enterprise continuously improve its internal control system.

[0038] In addition, it should be noted that the various specific technical features described in the above specific embodiments can be combined in any appropriate manner without contradiction. In order to avoid unnecessary repetition, the present application will not further describe various possible combinations. Furthermore, various different embodiments of the present application can be combined in any manner as long as they do not contradict the idea of the present application, and they should also be considered as disclosed by the present application.

Claims

1. A big data-based audit analysis method, characterized in that, The method includes: From the audit data penetration coverage at the network, provincial, municipal, and county levels, marketing domain data is collected to obtain audit suspicion data for each unit. Based on a pre-set distributed computing framework, the audit suspicion data is divided into multiple data blocks and distributed to different computing nodes for parallel processing, resulting in pre-processed audit data. This pre-processed audit data includes standardized and pre-processed transaction records, account information, and business operation logs. For this pre-processed audit data, in-memory computing technology is used for real-time data analysis to extract key indicators, including business type, transaction amount, and transaction frequency, resulting in a multi-dimensional indicator dataset. Based on this multi-dimensional indicator dataset, a pre-set anomaly detection algorithm is used to analyze the data. If the transaction amount of a data point exceeds a pre-set threshold A, or the transaction frequency exceeds a pre-set threshold B, it is marked as a potential suspicion, resulting in a candidate set of suspicions. Finally, a transaction dataset is obtained, containing transaction amount, transaction frequency, and transaction time. By setting normal ranges for each indicator, if multiple indicators simultaneously exceed the normal range... If a data point is detected, it is marked as a suspicious anomaly. This suspicious anomaly is then combined with a candidate set of suspicious points to generate a violation detection result, and the data processing time for the violation detection is recorded. For the violation detection result, a preset visualization template is used to display the audit results on a map and pie chart, generating a visual audit report. Based on the visual audit report, interactive technology enables dynamic interaction between the user and the data. If the user selects a data point, detailed information about that data point is displayed, including the type of violation, the amount involved, and the time of occurrence. If the data processing time for violation detection exceeds a preset threshold C, the algorithm parameters are automatically adjusted, including the number of trees in the anomaly detection algorithm or the anomaly score threshold D, to improve processing efficiency. Performance optimization is implemented throughout the entire data processing flow to ensure the system's real-time responsiveness. A preset interface specification is used to feed the visual audit report back to various business systems, achieving closed-loop data management. Closed-loop management includes pushing violation detection results to relevant business systems, triggering early warning or blocking mechanisms, and recording subsequent processing, forming a complete audit trail chain. The audit data after initial processing is analyzed in real time using in-memory computing technology to extract key indicators, including business type, transaction amount, and transaction frequency, resulting in a multi-dimensional indicator dataset, including: The pre-processed audit data is loaded using an in-memory computing framework. Data preprocessing, including data cleaning and formatting, is performed to ensure data quality and obtain pre-processed audit data. The preprocessed audit data is analyzed in real time using in-memory computing technology, and key indicator data, including business type, transaction amount and transaction frequency, are quickly extracted using parallel processing capabilities. The extracted key indicator data is aggregated in multiple dimensions to generate a preliminary multi-dimensional indicator dataset. Based on the aforementioned preliminary multi-dimensional indicator dataset, machine learning algorithms are used for pattern recognition to construct a trading behavior model for predicting future trading trends. By combining historical and real-time data, the trading behavior model is continuously optimized to obtain a multi-dimensional indicator dataset; The process involves analyzing the data in a multi-dimensional indicator dataset using a pre-defined anomaly detection algorithm. If the transaction amount of a data point exceeds a pre-defined threshold A, or the transaction frequency exceeds a pre-defined threshold B, it is marked as a potential suspicious point, resulting in a candidate set of suspicious points, including: Obtain a multi-dimensional indicator dataset, including key indicators such as business type, transaction amount, and transaction frequency; A preset anomaly detection algorithm is used to analyze the multi-dimensional indicator dataset item by item; If the transaction amount of a certain data point exceeds the preset threshold A, then the data point is marked as a suspicious amount. If the transaction frequency of a certain data point exceeds the preset threshold B, then the data point will be marked as a frequency anomaly. The potential suspicious points marked as abnormal amounts and abnormal frequencies are summarized to generate a candidate set of suspicious points; If the data processing time for violation detection exceeds a preset threshold C, the algorithm parameters are automatically adjusted, including the number of trees in the anomaly detection algorithm or the anomaly score threshold D, to improve processing efficiency. Performance optimization is implemented throughout the entire data processing flow to ensure the system's real-time responsiveness, including: Obtain the data processing time for detecting violations and compare it with a preset threshold C; If the data processing time for detecting violations exceeds the preset threshold C, a parameter adjustment mechanism is triggered. Determine the anomaly detection algorithm parameters that need to be adjusted, including the number of trees and the anomaly score threshold D for the Isolation Forest algorithm; Based on the preset adjustment rules, calculate new parameter values ​​to ensure that the new parameters can significantly improve processing efficiency; Apply the new parameter values ​​to the anomaly detection algorithm and update the algorithm configuration; Monitor the operation of the updated algorithm in real time, obtain the new data processing time for detecting violations, and compare it again with the preset threshold C; If the processing time for new violation detection data still exceeds the preset threshold C, the process of automatically adjusting the algorithm parameters will be repeated until the real-time response requirements are met, ensuring the system's real-time response capability.

2. The method according to claim 1, characterized in that, The audit data penetration coverage from the network, province, city, and county levels includes collecting data from the marketing domain to obtain audit suspicion data for each unit, including: Obtain the initial audit dataset from the network-level audit system; The initial audit dataset is cleaned by removing duplicate records, handling missing values, and standardizing the data format to form a cleaned dataset. For the cleaned dataset, province filtering rules are pre-established, and the province filtering rules include an identifier field for each province; According to the province selection rules, the audit data corresponding to each province is selected from the cleaned dataset, and the audit data of each province is associated with the network-level audit data to form a provincial associated dataset. The association operation uses a database table join operation, and the join field is the unit code. For the aforementioned provincial-level associated dataset, pre-establish city-level filtering rules, which include an identifier field for each city; Based on the city-level screening rules, the audit data corresponding to each city is selected from the provincial-level associated dataset; Determine whether the unit code field in the municipal audit data is consistent with the unit code field in the provincial associated dataset. If they are consistent, merge the municipal audit data with the provincial associated dataset to form a municipal associated dataset. For the aforementioned city-level associated dataset, district / county-level filtering rules are pre-established, and the district / county-level filtering rules include an identifier field for each district / county; Based on the district / county-level screening rules, the audit data corresponding to each district / county is selected from the city-level associated dataset; Data validation is performed on district and county-level audit data to check whether the data is within the scope of preset business rules. If it meets the preset business rules, a district and county-level dataset is formed. For the aforementioned district / county-level dataset, marketing domain field rules are pre-established, which include field names related to marketing business. Based on the aforementioned marketing domain field rules, all marketing domain-related field data are filtered out from the district / county-level dataset to form a marketing domain dataset; For the aforementioned marketing domain dataset, calculate the interquartile range of each indicator for each unit; If a certain indicator value of a unit in the marketing domain dataset exceeds the upper or lower threshold of the interquartile range, then that indicator value of the unit is marked as suspicious data, forming a unit suspicious dataset. The dataset of suspected audit points is grouped according to the unit code. The suspected audit points in each group are counted and summarized to generate an audit suspected audit point data report for each unit. The audit suspected audit point data of each unit is obtained from the report.

3. The method according to claim 1, characterized in that, According to a pre-defined distributed computing framework, the audit data is divided into multiple data blocks and distributed to different computing nodes for parallel processing to obtain pre-processed audit data. This pre-processed audit data includes standardized and pre-processed transaction records, account information, and business operation logs, including: A distributed computing framework is used to segment the audit data to generate multiple data blocks. Each data block contains a specific range of transaction records, account information, and operation logs. The data blocks are distributed to different computing nodes through a preset distribution mechanism to ensure that the amount of data processed by each node is balanced. Each computing node performs standardization and preprocessing operations in parallel. The standardization process includes data format unification, field completion, and outlier handling. The preprocessing process includes data cleaning, deduplication, and feature extraction to obtain processed data blocks. The processed data blocks are aggregated to the central node, integrated, and a preliminary audit dataset containing standardized transaction records, account information, and business operation logs is generated. The preliminary audit dataset is subjected to quality checks. If data is missing or abnormal, the process is traced back to the corresponding computing node for correction. The preliminary audit dataset is further analyzed using machine learning algorithms, including clustering algorithms to identify abnormal transaction patterns, classification algorithms to assess account risks, and association rule mining to analyze potential risks in operation logs, resulting in pre-processed audit data.

4. The method according to claim 1, characterized in that, The process involves acquiring a transaction dataset, which includes transaction amount, transaction frequency, and transaction time. By setting a normal range for each indicator, if multiple indicators simultaneously exceed the normal range, they are marked as suspicious anomalies. These suspicious anomalies are then combined with a candidate set of suspicious anomalies to generate violation detection results. The data processing time for violation detection is also recorded, including: Obtain a transaction dataset containing basic information such as transaction amount, transaction frequency, and transaction time; Interpolation was used to fill in missing values ​​in the transaction dataset, and outliers were removed using the 3x standard deviation method to ensure data quality. Set normal range thresholds for transaction data in each transaction dataset in three dimensions: amount, frequency, and time. If the values ​​of multiple dimensions of a transaction record exceed the normal range thresholds at the same time, it is marked as a suspicious anomaly. Further screening of marked suspicious anomalies, combined with the candidate set of suspicious points, is used to predict violations using the random forest algorithm; If the prediction result is found to be in violation, then it is confirmed as a violation. Generate violation detection results containing detailed information on violation transaction records and the basis for violation determination, and record the data processing time for violation detection in the database.

5. The method according to claim 1, characterized in that, The results of violation detection are displayed using a preset visualization template on maps and pie charts, generating a visual audit report, including: Obtain the results of violation detection and perform data preprocessing, including data cleaning and data standardization; Data cleaning removes invalid and redundant data, ensuring data quality. Data standardization unifies data formats, facilitating subsequent processing; The preprocessed data is then categorized and aggregated according to the preset visualization template requirements; For map display dimensions, the data is categorized according to geographical location information to generate a geographic distribution dataset; For the pie chart display dimension, the data is categorized according to the type of violation to generate a type distribution dataset; A map generation algorithm is used to convert a geographic distribution dataset into a map visualization image. The specific steps include loading the base map, labeling data points, and rendering a heat map. A pie chart generation algorithm is used to convert a type distribution dataset into a pie chart visualization image. The specific steps include pie chart drawing, data proportion calculation, and color filling. The generated map and pie chart visualizations are integrated according to a preset report format to generate a visual audit report.

6. The method according to claim 1, characterized in that, Based on the visualized audit report, interactive technology enables dynamic interaction between users and data. If a user selects a data point, detailed information about that data point is displayed, including the type of violation, the amount involved, and the time of occurrence. To obtain data from the visual audit report, integrated interactive technology is used to enable interactive charts. Users can interact with the charts by clicking or hovering. If a user selects a data point, the logic for displaying detailed information is triggered. Determine the data point selected by the user, and extract detailed information about the data point from the visualized audit report data, including the type of violation, the amount involved, and the time of occurrence; A dynamic, detailed information display interface is generated, presenting the extracted data points in detail on the screen, making it easy for users to quickly obtain information.

7. The method according to claim 1, characterized in that, The system employs a pre-defined interface specification to feed the visual audit report back to various business systems, achieving closed-loop data management. This closed-loop management includes pushing violation detection results to relevant business systems, triggering early warning or blocking mechanisms, and recording subsequent processing, forming a complete audit trail chain, including: After obtaining the visual audit report, data cleaning is performed to remove duplicate data and invalid fields, and missing values ​​are imputed to obtain the cleaned data. The cleaned data is formatted to unify field types and encoding rules, ensuring data quality and consistency, resulting in formatted data. The formatted data is input into the audit analysis module, where SQL queries and Python scripts are used for classification and statistics to identify abnormal transaction records, which are then pushed to relevant business systems. After receiving abnormal transaction records, the relevant business system determines the type of abnormality for each record. If the type of abnormality is high-risk, it sends a warning message to the relevant responsible persons via email and SMS. If the anomaly type is a serious violation, the relevant user account will be immediately locked and its trading privileges suspended, and a detailed log of the locking operation will be recorded. Based on the detailed logs of the locking operation, the business system generates a violation investigation task and assigns it to the designated risk control personnel; The risk control personnel will verify the violations according to the task requirements and enter the verification results into the business system; After receiving the verification results, the business system updates the status of the abnormal transaction records to form a complete audit trail chain, which includes the abnormality type, processing operation, and verification result information.

Images