Processing method and device for report data volume abnormity, electronic equipment and storage medium

Abstract

The invention discloses a processing method and device for report data volume abnormity, electronic equipment and a storage medium, and relates to the technical field of artificial intelligence or other related technical fields, and the method comprises the steps: collecting report data volume parameters in a target time period, and carrying out the preprocessing of the report data volume parameters; constructing report time sequence data based on the preprocessed report data volume parameters, inputting the report time sequence data into a data volume prediction model, performing trend prediction on the report data volume through the data volume prediction model, and outputting a trend prediction sequence of the report data volume in a future preset time period; and identifying report data volume abnormity based on the trend prediction sequence, generating an optimization strategy based on the report data volume abnormity, and executing the optimization strategy to perform performance optimization on the report database. According to the method and the device, the technical problem of relatively high hysteresis of an exception identification mode based on a database running log in related technologies is solved.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology or other related technical fields. Specifically, it relates to a method and apparatus for processing abnormal report data volume, an electronic device, and a storage medium. Background Technology

[0002] In modern financial systems, the generation and analysis of report data are key components of daily operations and decision support. With the continuous expansion of business and the exponential growth of data volume, databases often exhibit significant performance bottlenecks when faced with massive amounts of report data. This not only affects report download speeds but also reduces user experience and increases costs and the complexity of resource planning.

[0003] In related technologies, analyzing abnormal report data volumes based on database operation logs to determine whether there are performance bottlenecks in the database and taking corresponding solutions to address the current performance bottleneck problem in a timely manner has a strong lag and cannot respond promptly to database performance bottlenecks caused by abnormal report data volumes, thus lacking real-time capability.

[0004] There is currently no effective solution to the above problems. Summary of the Invention

[0005] This invention provides a method, apparatus, electronic device, and storage medium for handling abnormal report data volume, in order to at least solve the technical problem that the abnormal identification method based on database operation logs in related technologies has a strong lag.

[0006] According to one aspect of the present invention, a method for handling abnormal report data volume is provided, comprising: collecting report data volume parameters within a target time period and preprocessing the report data volume parameters; constructing report time-series data based on the preprocessed report data volume parameters, inputting the report time-series data into a data volume prediction model, performing trend prediction on the report data volume through the data volume prediction model, and outputting a trend prediction sequence of the report data volume within a future preset time period, wherein the data volume prediction model is an autoregressive integrated moving average model optimized using a random search variable selection method; identifying report data volume anomalies based on the trend prediction sequence, generating an optimization strategy based on the report data volume anomalies, and executing the optimization strategy to optimize the performance of the report database.

[0007] Furthermore, the data volume prediction model is pre-built. The steps for building the data volume prediction model include: Step 1, constructing training samples based on historical report data volume parameters, and dividing the training samples into a training set and a test set; Step 2, determining the candidate range of model parameters, selecting model parameters from the candidate range using random sampling, and constructing an initial data volume prediction model by adding an influence factor; Step 3, training the initial data volume prediction model using the training set, calculating the posterior probability of each model parameter being selected using a random variable selection mechanism, and selecting new model parameters from the candidate range using random sampling; Step 4, repeating Step 3 to iteratively train the initial data volume prediction model until the number of iterations exceeds a preset iteration threshold, and selecting the model parameter with the highest posterior probability as the final model parameter of the data volume prediction model, thus obtaining the trained data volume prediction model; Step 5, testing the trained data volume prediction model using the test set, and obtaining the final data volume prediction model if the data volume prediction model passes the test.

[0008] Further, the step of inputting the time-series data of the report into the data volume prediction model and using the data volume prediction model to predict the trend of the report data volume includes: inputting the time-series data of the report into the data volume prediction model; performing linear calculation through the autoregressive prediction layer of the data volume prediction model and outputting the autoregressive prediction result; performing residual calculation using the random error term through the moving average prediction layer of the data volume prediction model and outputting the residual calculation result; and fusing the autoregressive prediction result and the residual calculation result through the data volume prediction model to obtain the trend prediction sequence of the report data volume.

[0009] Furthermore, the step of identifying abnormal report data volume based on the trend prediction sequence includes: comparing the predicted report data volume value at each prediction time in the trend prediction sequence with a preset data volume threshold to obtain a comparison result; and determining that the report data volume is abnormal when the comparison result indicates that the predicted report data volume value is greater than the data volume threshold.

[0010] Furthermore, the optimization strategy includes at least a report download optimization strategy. The step of executing the optimization strategy to optimize the performance of the report database includes: partitioning the report database based on a preset partitioning rule to obtain N virtual partitions, where N is a positive integer; configuring a parallel report data download strategy for the N virtual partitions, wherein the parallel report data download strategy indicates that when downloading report data, the report data in the report database is downloaded in parallel according to the virtual partitions.

[0011] Furthermore, the optimization strategy includes at least a query statement optimization strategy. The steps of performing the optimization strategy to optimize the performance of the report database include: analyzing the query logs to identify slow query statements whose execution time exceeds a preset time threshold; optimizing the slow query statements, wherein optimizing the slow query statements includes at least one of the following: using joins instead of subqueries, adding query indexes, and constructing range filtering statements.

[0012] Furthermore, the optimization strategy includes at least a hot report data caching strategy. The step of performing the optimization strategy to optimize the performance of the report database includes: identifying hot report data based on historical report data download records, wherein the hot report data refers to report data with a download frequency greater than a preset download frequency threshold; and caching the hot report data.

[0013] According to another aspect of the present invention, a processing device for abnormal report data volume is also provided, comprising: a data acquisition unit, configured to acquire report data volume parameters within a target time period and preprocess the report data volume parameters; a prediction unit, configured to construct report time-series data based on the preprocessed report data volume parameters, input the report time-series data into a data volume prediction model, perform trend prediction on the report data volume through the data volume prediction model, and output a trend prediction sequence of the report data volume within a future preset time period, wherein the data volume prediction model is an autoregressive integrated moving average model optimized using a random search variable selection method; and an optimization unit, configured to identify abnormal report data volume based on the trend prediction sequence, generate an optimization strategy based on the abnormal report data volume, and execute the optimization strategy to optimize the performance of the report database.

[0014] Furthermore, the device for handling abnormal report data volume further includes: a first construction module, used in step one to construct training samples based on historical report data volume parameters, and divide the training samples into a training set and a test set; a first determination module, used in step two to determine the candidate range of model parameters, select model parameters from the candidate range using random sampling, and construct an initial data volume prediction model by adding an influence factor; a first calculation module, used in step three to train the initial data volume prediction model using the training set, calculate the posterior probability of each model parameter being selected using a random variable selection mechanism, and select new model parameters from the candidate range using random sampling; a first iteration module, used in step four to repeatedly execute step three to iteratively train the initial data volume prediction model until the number of iterations exceeds a preset iteration threshold, and select the model parameter with the highest posterior probability as the final model parameter of the data volume prediction model, thereby obtaining the trained data volume prediction model; and a first testing module, used in step five to test the trained data volume prediction model using the test set, and obtain the final data volume prediction model if the data volume prediction model passes the test.

[0015] Furthermore, the prediction unit includes: a second calculation module, used to input the report time series data into the data volume prediction model, perform linear calculation through the autoregressive prediction layer of the data volume prediction model, and output the autoregressive prediction result; a third calculation module, used to perform residual calculation using the random error term through the moving average prediction layer of the data volume prediction model, and output the residual calculation result; and a first fusion module, used to perform fusion calculation on the autoregressive prediction result and the residual calculation result through the data volume prediction model to obtain the trend prediction sequence of the report data volume.

[0016] Furthermore, the optimization unit includes: a first comparison module, used to compare the predicted value of the report data volume at each prediction time in the trend prediction sequence with a preset data volume threshold to obtain a comparison result; and a first determination module, used to determine that the report data volume is abnormal when the comparison result indicates that the predicted value of the report data volume is greater than the data volume threshold.

[0017] Furthermore, the optimization strategy includes at least a report download optimization strategy, and the optimization unit further includes: a first partitioning module, used to partition the report database based on a preset partitioning rule to obtain N virtual partitions, where N is a positive integer; and a first configuration module, used to configure a parallel report data download strategy for the N virtual partitions, wherein the parallel report data download strategy indicates that when downloading report data, the report data in the report database is downloaded in parallel according to the virtual partitions.

[0018] Furthermore, the optimization strategy includes at least a query statement optimization strategy, and the optimization unit further includes: a first analysis module for analyzing query logs and identifying slow query statements whose execution time exceeds a preset time threshold; and a first optimization module for optimizing the slow query statements, wherein optimizing the slow query statements includes at least one of the following: using joins to replace subqueries, adding query indexes, and constructing range filtering statements.

[0019] Furthermore, the optimization strategy includes at least a hot report data caching strategy, and the optimization unit further includes: a second identification module, used to identify hot report data based on historical report data download records, wherein the hot report data represents report data with a download frequency greater than a preset download frequency threshold; and a first caching module, used to cache the hot report data.

[0020] According to another aspect of the present invention, a computer-readable storage medium is also provided, the computer-readable storage medium including a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform any of the above-described methods for handling abnormal report data volume.

[0021] According to another aspect of the present invention, an electronic device is also provided, including one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement any of the above-described methods for handling abnormal report data volume.

[0022] In this application, the following steps are performed: collecting report data volume parameters within a target time period and preprocessing the report data volume parameters; constructing report time-series data based on the preprocessed report data volume parameters; inputting the report time-series data into a data volume prediction model; predicting the trend of report data volume using the data volume prediction model; and outputting a trend prediction sequence of report data volume within a future preset time period. The data volume prediction model is an autoregressive integrated moving average model optimized using a random search variable selection method. Finally, anomalies in report data volume are identified based on the trend prediction sequence, optimization strategies are generated based on the anomalies, and the optimization strategies are executed to optimize the performance of the report database.

[0023] In this application, the report data volume parameters within a target time period are obtained, and the parameters are analyzed using a pre-constructed autoregressive integrated moving average model optimized by a random search variable selection method. This accurately predicts the changing trend of the report data volume over a future period, thereby accurately predicting report data volume anomalies, identifying potential performance bottlenecks in the report database in advance, and taking relevant strategies to optimize the database. This allows for timely response to potential performance bottlenecks, improves the response speed to data volume anomalies, reduces the lag in anomaly handling, and solves the technical problem of strong lag in anomaly identification methods based on database operation logs in related technologies. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0025] Figure 1 A hardware structure block diagram of a computer terminal (or mobile device) for implementing a method for handling abnormal report data volume is shown.

[0026] Figure 2 This is a flowchart of an optional method for handling abnormal report data volume according to an embodiment of the present invention;

[0027] Figure 3 This is a schematic diagram of an optional report data volume abnormality processing device according to an embodiment of the present invention;

[0028] Figure 4 This is a hardware structure block diagram of an electronic device (or mobile device) according to an optional method for handling abnormal report data volume according to an embodiment of the present invention. Detailed Implementation

[0029] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0030] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0031] It should be noted that the method and apparatus for handling abnormal report data volume in this application can be used in the field of artificial intelligence technology to optimize databases with abnormal data volume based on artificial intelligence, or in any field other than the field of artificial intelligence technology to optimize databases with abnormal data volume based on artificial intelligence. This application does not limit the application field of the method and apparatus for handling abnormal report data volume.

[0032] It should be noted that the information collected in this application (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for display, data used for analysis, etc.) are information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of this data all comply with relevant laws, regulations, and standards, necessary confidentiality measures have been taken, and they do not violate public order and good morals. Corresponding access points are provided for users to choose to authorize or refuse. For example, interfaces are set up between this system and relevant users or organizations, providing users with corresponding access points to choose to agree to or refuse automated decision-making results; if the user chooses to refuse, the process proceeds to the expert decision-making stage.

[0033] The following embodiments of the present invention can be applied to various systems / applications / devices for handling abnormal report data volumes. This invention addresses database performance issues by using seemingly unrelated models to predict data volume growth trends and identify bottlenecks. Accurate bottleneck identification points guide performance tuning. This allows for precise prediction of report database storage growth, enabling proactive database optimization, addressing potential performance bottlenecks, reducing lag in anomaly handling, and providing timely responses to abnormal data volume issues.

[0034] The present invention will now be described in detail with reference to various embodiments.

[0035] Example 1

[0036] According to an embodiment of the present invention, an embodiment of a method for handling abnormal report data volume is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0037] The method embodiment provided in Embodiment 1 of this application can be executed on a mobile terminal, computer terminal, or similar computing device. Figure 1 A hardware block diagram of a computer terminal (or mobile device) for implementing a method to handle abnormal report data volumes is shown. Figure 1 As shown, the computer terminal 10 (or mobile device) may include one or more processors 102 (shown as 102a, 102b, ..., 102n in the figure) 102 (processor 102 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 104 for storing data, and a transmission device 106 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of a BUS bus), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 1 The structure shown is for illustrative purposes only and does not limit the structure of the aforementioned electronic device. For example, computer terminal 10 may also include... Figure 1 The more or fewer components shown, or having the same Figure 1 The different configurations shown.

[0038] It should be noted that the aforementioned one or more processors 102 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the computer terminal 10 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0039] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the report data volume abnormality handling method in this embodiment. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, thereby realizing the above-mentioned report data volume abnormality handling method. The memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory remotely located relative to the processor 102, and these remote memories can be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0040] The transmission device 106 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the computer terminal 10. In one example, the transmission device 106 includes a Network Interface Controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the transmission device 106 may be a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0041] The display may be, for example, a touchscreen liquid crystal display (LCD) that allows the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0042] Under the aforementioned operating environment, this application provides the following: Figure 2 The method shown is for handling abnormal report data volume. The main body implementing this method is the report data volume abnormality handling system.

[0043] Figure 2 This is a flowchart of an optional method for handling abnormal report data volume according to an embodiment of the present invention, such as... Figure 2 As shown, the method includes the following steps:

[0044] Step S201: Collect the report data volume parameters within the target time period and preprocess the report data volume parameters.

[0045] In step S201 above, report data volume parameters for the target time period are collected. The purpose of collecting this data is to analyze it to capture patterns in data volume changes, including trends, seasonality, and periodicity, thereby enabling accurate data volume prediction. Report data volume parameters include the report data generation timestamp, report data volume value, and report data identifier. Subsequently, the acquired report data volume parameters are preprocessed, including data cleaning and standardization, to provide a reliable data foundation for subsequent data volume prediction.

[0046] Step S202: Construct report time series data based on the preprocessed report data volume parameters, input the report time series data into the data volume prediction model, perform trend prediction on the report data volume through the data volume prediction model, and output the trend prediction sequence of the report data volume within a future preset time period.

[0047] In step S202 above, the data volume prediction model is an autoregressive integrated moving average model optimized using a random search variable selection method.

[0048] In this embodiment of the invention, for the preprocessed report data volume parameters, report time-series data is constructed according to the report data generation timestamps, providing a structured data format for the construction of time series analysis and prediction models. The constructed report time-series data is then input into an optimized autoregressive integrated moving average model, i.e., a data volume prediction model. This model can capture the autocorrelation and trend of the data. Simultaneously, the model is optimized and trained using a random search variable selection method, enabling it to more accurately identify which historical data points are most critical for predicting future report data volume trends. The optimized data volume prediction model is used to analyze the preprocessed report data volume parameters, predicting the future trend of report data volume, thereby obtaining a trend prediction sequence for a preset future time period. This prediction sequence not only reflects the potential growth of data volume but also considers the volatility and non-stationarity of the data itself, providing a more accurate and comprehensive prediction of future report data volume trends.

[0049] Furthermore, the data volume prediction model is pre-built. The steps for building the data volume prediction model include: Step 1, constructing training samples based on historical report data volume parameters and dividing the training samples into training and testing sets; Step 2, determining the candidate range of model parameters, selecting model parameters from the candidate range using random sampling, and constructing an initial data volume prediction model by adding an influence factor; Step 3, training the initial data volume prediction model using the training set, calculating the posterior probability of each model parameter being selected using a random variable selection mechanism, and selecting new model parameters from the candidate range using random sampling; Step 4, repeating Step 3 to iteratively train the initial data volume prediction model until the number of iterations exceeds a preset iteration threshold, and selecting the model parameter with the highest posterior probability as the final model parameter of the data volume prediction model, thus obtaining the trained data volume prediction model; Step 5, testing the trained data volume prediction model using the testing set, and obtaining the final data volume prediction model if the data volume prediction model passes the test.

[0050] Specifically, the training process of the data volume prediction model includes: collecting historical data volume parameters within a historical time period, configuring data labels through the data volume sequences of future time periods corresponding to the historical time periods, thereby constructing training samples, dividing the training samples into training sets and test sets, with the training set used for model learning and parameter estimation, and the test set used to evaluate the model's performance on unseen data.

[0051] Subsequently, based on the principle of the autoregressive integrated moving average model, candidate ranges for the model parameters of the autoregressive (AR) and moving average (MA) components were preset. In addition to historical data, virtual performance crisis variables (i.e., influencing factors) were also considered to reflect the impact of specific events (such as data migration and holidays) on the data volume. Random sampling techniques were used to select specific model parameters from the candidate range to construct an initial data volume prediction model based on the selected model parameters.

[0052] The initial data set is used to iteratively train the prediction model. In each iteration, a random variable selection mechanism is used to calculate the posterior probability of each model parameter being selected. This step helps identify which model parameters are most critical to the prediction model, thereby eliminating insignificant parameters and optimizing the model structure. Based on the calculated posterior probabilities, new model parameters are selected again using random sampling for the next iteration calculation. This process is repeated until the number of iterations exceeds a preset threshold.

[0053] During iterative training, the model parameters with the highest probability after each iteration are retained. By comparing the model performance under different parameter combinations, the best-performing set is selected as the final model parameters for the data volume prediction model. Once the iteration threshold is reached, the model parameter combination with the highest posterior probability is identified, and the final data volume prediction model is built based on this combination.

[0054] Using the reserved test set data, the constructed data volume prediction model is tested to evaluate its prediction accuracy and stability, and the final data volume prediction model is obtained.

[0055] More specifically, by building predictive models based on seemingly unrelated data volumes, we can predict data volume bottlenecks.

[0056] First, a Bayesian inference model based on an autoregressive model is established to fit the historical data of the database table.

[0057] That is .

[0058] y represents the data volume growth rate of a certain report within a preset range and a certain period. These are the autoregressive coefficients. For the random error term, random sampling is used to obtain the solution to the equation. The random variable search method quickly and accurately determines the model parameters and eliminates insignificant independent variables.

[0059] The Bayesian inference model of the autoregressive model can then be rewritten as:

[0060] ;

[0061] For lag operators, , , , It is the autoregressive coefficient.

[0062] Add a constant term c:

[0063] ;

[0064] c>0 indicates that the sequence after differencing has an upward trend;

[0065] c<0 indicates that the sequence after differencing has a downward trend;

[0066] c represents the time trend item.

[0067] Finally, add a performance-critical dummy variable. (Simulating data migration events, with certain holidays represented as 1), the final autoregressive model is obtained:

[0068] ;

[0069] 'c' represents the time trend term. For example, if consumers in a certain city enjoy shopping online every day, and as online shopping becomes increasingly convenient, more and more consumers are joining the ranks of online shoppers, the table's growth exhibits exponential growth. (This refers to a dummy variable related to performance issues.) For example, during promotional events such as Singles' Day (November 11th) each year, there are a large number of orders, which leads to an increase in the amount of data in the reports.

[0070] Furthermore, the steps of inputting the report time-series data into the data volume prediction model and using the data volume prediction model to predict the trend of the report data volume include: inputting the report time-series data into the data volume prediction model, performing linear calculations through the autoregressive prediction layer of the data volume prediction model, and outputting the autoregressive prediction result; performing residual calculations using the random error term through the moving average prediction layer of the data volume prediction model, and outputting the residual calculation result; and fusing the autoregressive prediction result and the residual calculation result through the data volume prediction model to obtain the trend prediction sequence of the report data volume.

[0071] Specifically, when predicting the amount of report data, preprocessed report time-series data is used as input to the model. The data is standardized and differencing to achieve a stationary state, enabling the model to accurately identify patterns in the data sequence. The autoregressive prediction layer of the data volume prediction model predicts future data volume by linearly combining past data points. Specifically, based on the autoregressive order (AR order) determined by the model—for example, in the ((1,4,8),1,0) model—it uses data from 1, 4, and 8 time points prior to the current moment to perform linear calculations to predict the report data volume at the next time point. Through this linear calculation, the model outputs the autoregressive prediction result, i.e., the predicted report data volume value, which reflects the inherent trend and dependence in the data sequence. Subsequently, in the moving average prediction layer, the predicted value is adjusted by calculating a random error term (i.e., residual) on the report data volume parameters to reflect the random fluctuations in the data. Finally, the autoregressive prediction results and residual calculation results are fused together to obtain the final report data volume trend prediction sequence. This sequence not only includes the linear trend information of the data, but also takes into account the impact of random fluctuations, providing a more comprehensive and accurate prediction result.

[0072] Step S203: Identify abnormal report data volume based on trend prediction sequence, generate optimization strategy based on abnormal report data volume, and execute the optimization strategy to optimize the performance of report database.

[0073] In step S203 above, data points that significantly deviate from the preset data volume threshold are identified by comparing the trend prediction sequence with the threshold. These deviations may indicate a surge in report data volume that could lead to a performance bottleneck in the report database, and are therefore considered abnormal. After identifying the abnormal report data volume, optimization strategies are constructed to preemptively optimize the database to address potential performance bottlenecks. Optimization strategies may include optimizing query statements, adjusting the report file download method, and modifying database configurations. After the strategy is implemented, database performance is continuously monitored to ensure that the optimization strategy effectively improves the performance of the report database, reduces response time, and increases data processing speed.

[0074] Furthermore, the steps for identifying abnormal report data volume based on trend prediction sequences include: comparing the predicted report data volume values ​​at each prediction time in the trend prediction sequence with a preset data volume threshold to obtain a comparison result; and determining that there is an abnormality in the report data volume if the comparison result indicates that the predicted report data volume value is greater than the data volume threshold.

[0075] Specifically, the preset report data volume threshold is an upper limit set based on the current database processing capacity and business needs. It reflects the level of data volume the system can efficiently process. The purpose of setting this threshold is to monitor whether the growth of report data volume exceeds the normal processing range of the database, thereby promptly identifying potential performance issues. The predicted report data volume value at each prediction time in the report data volume trend prediction sequence obtained through the data volume prediction model is compared one by one with the aforementioned preset data volume threshold. This comparison process aims to identify which predicted values ​​exceed the upper limit of normal processing capacity, thus indicating potential data volume anomalies. By dynamically identifying anomalies in report data volume, potential database performance bottlenecks can be warned in advance.

[0076] Furthermore, the optimization strategy includes at least the following: a report download optimization strategy. The steps for optimizing the performance of the report database by executing the optimization strategy include: partitioning the report database based on preset partitioning rules to obtain N virtual partitions, where N is a positive integer; configuring a parallel report data download strategy for the N virtual partitions, where the parallel report data download strategy means that when downloading report data, the report data in the report database is downloaded in parallel according to the virtual partitions.

[0077] Specifically, the partitioning and parallel downloading strategies for the report database are key steps in optimizing its performance. This aims to accelerate the report data download process and improve the overall responsiveness of the database without compromising data integrity and consistency. The report database is logically divided using preset partitioning rules, creating N virtual partitions. Here, N is a positive integer representing the number of partitions, which can be flexibly set according to the database size and the characteristics of the report data. Each virtual partition covers a subset of the report database, aiming to balance the data volume across partitions, thereby enabling more efficient parallel downloading operations.

[0078] Based on the creation of N virtual partitions, a parallel download strategy for report data is implemented. This means that when downloading report data, data from different virtual partitions can be processed simultaneously, significantly improving data transfer and processing speed. In practice, data from each virtual partition is assigned to an independent download task, which can run in parallel on different processors or threads. In this way, even when the amount of data in a single partition is very large, the overall download time can be significantly reduced.

[0079] By using preset partitioning rules, report data from the database can be evenly distributed across multiple virtual partitions, avoiding excessive pressure on a single node caused by hot data, achieving load balancing, and improving overall processing efficiency and response speed. The parallel download strategy allows report data to be downloaded and processed simultaneously across multiple partitions, greatly reducing report generation waiting time and improving the user experience of obtaining reports.

[0080] Furthermore, the optimization strategy includes at least: query statement optimization strategy. The steps for optimizing the performance of the report database by implementing the optimization strategy include: analyzing the query log to identify slow query statements whose execution time exceeds a preset time threshold; optimizing the slow query statements, wherein optimizing the slow query statements includes at least one of the following: using joins instead of subqueries, adding query indexes, and constructing range filtering statements.

[0081] Specifically, optimizing the reporting database can also be achieved through query optimization. By analyzing query logs, slow queries that take longer than a preset threshold can be identified. This threshold can be set based on system performance and user needs, for example, 3 seconds. For slow queries, statements involving subqueries can be converted to statements using join operations, as joins are generally more efficient than subqueries, especially when processing large amounts of data. Indexes can be added to frequently used query fields to speed up data retrieval. Indexes can significantly improve query performance, especially for statements involving range queries. Optimize queries by using more precise range filtering conditions to limit the range of data scans, reducing unnecessary data processing and thus improving query speed. Once slow queries are identified and corresponding optimization strategies are developed, these strategies are then implemented to tangibly improve the performance of the reporting database. This includes actually modifying the identified slow queries and deploying these optimized queries in the reporting database. Optimized queries execute faster, significantly reducing report generation time and improving user experience.

[0082] Furthermore, the optimization strategy includes at least the following: a hot report data caching strategy. The steps for implementing the optimization strategy to optimize the performance of the report database include: identifying hot report data based on historical report data download records, where hot report data refers to report data whose download frequency is greater than a preset download frequency threshold; and caching the hot report data.

[0083] Specifically, report database optimization can also be achieved by caching frequently accessed report data. First, extract all data download information from the report database's historical records, including the report type, download time, and download frequency for each download. Define a preset download frequency threshold to determine which report data is considered frequently accessed. Typically, this threshold is set based on historical data and business needs; for example, if a report is downloaded more than 100 times in a week, it might be considered frequently accessed. By analyzing the download records, identify report data with download frequencies exceeding the threshold and mark them as frequently accessed, laying the foundation for subsequent caching strategies. After identifying frequently accessed report data, pre-download and store it in a cache for fast response on subsequent requests. The caching strategy should include the data update frequency and cache expiration time to ensure the cached data is up-to-date while avoiding excessive resource consumption.

[0084] Since frequently accessed report data is cached, subsequent requests for this data do not require database queries, significantly reducing query time and improving the user experience. Furthermore, caching frequently accessed report data reduces direct database queries, lowering the database load.

[0085] More specifically, the report data processing flow of the report database is as follows:

[0086] (1) The source application generates a file and sends it to the data lake;

[0087] (2) Documents are processed in the lake and then processed into new documents;

[0088] (3) Data lake transmits files to reporting applications;

[0089] (4) The report application receives the file, reads the file, and loads it into a new table;

[0090] (5) Reports displayed for each region;

[0091] For situations where abnormal report data volume leads to potential database performance bottlenecks, optimization strategies can be configured through the steps above to improve database performance.

[0092] For step (5), which displays reports for each region, the database performance bottleneck can be reduced by optimizing query statements. Specifically, this includes: avoiding the use of statements such as "!=", "NOT IN", and "NOTEXISTS" in the WHERE clause that may cause a full table scan; moving the conversion of various parameters to step (4) above; and finally, adding a sorting batch processing option during batch processing, sorting the imported tables according to specified conditions before inserting them into the database table.

[0093] In step “(4) Report application receives file, reads file and loads into new table”, the financial institution’s reporting system involves data from multiple regions. The same business data in the same table has different volumes of business in different regions. Assuming the system is divided into 10 partitions to process data from multiple regions in parallel, the system will try to distribute the report data from hot regions with large amounts of data and regions with small amounts of data evenly among the 10 partitions, so as to minimize the longest running time of the 10 partitions and the total parallel running time.

[0094] Collect historical download information for each table record by region, mainly the total number of tables downloaded within the download period of these tables record by region, and then calculate the table download time based on this history.

[0095] Table download time = Unit table download time Total number of tables downloaded in this region / Table partitions in this region;

[0096] When estimating data download time, if the table download time exceeds the set time threshold, various optimization methods need to be adopted to shorten the download time.

[0097] Virtual partitioning: In addition to partitioning according to each region, virtual regions can also be created based on other conditions of the table, such as the network point number or card type. This allows the database table data to be divided into more virtual regions for parallel downloading. For example, if there were originally only 10 regions, virtual region numbers can be used to create 100 partitions for simultaneous downloading.

[0098] Add batch job nodes: Download jobs are distributed across the server cluster for execution. By leveraging the characteristics of multi-processor multi-core processors and using cluster parallelism and thread parallelism technologies, high-concurrency processing can be achieved. For downloads with large workloads, elastic computing methods such as adding batch job nodes can be used to amortize the workload and ensure the processing timeliness of the entire batch job.

[0099] Hotspot report caching: When downloading reports from hotspot regions, temporary jobs are triggered on unpredictable dates to avoid days with high business volume, such as reconciliation dates, the end of the month, the end of the quarter, and the end of the year. The reports are downloaded and cached in advance using a scheduled method.

[0100] Migrating large amounts of data from reports can also cause performance bottlenecks. During report usage, changes to the report format may lead to modifications to the corresponding table structure. In such cases, it's necessary to migrate data from the old table to the new table. For tables with tens of millions of records, the amount of data to migrate is significant. Combining multiple migration methods can help reduce performance bottlenecks.

[0101] Through the above steps, report data volume parameters within the target time period are collected and preprocessed. Then, report time-series data is constructed based on the preprocessed report data volume parameters and input into the data volume prediction model. The data volume prediction model predicts the trend of report data volume and outputs the trend prediction sequence of report data volume within the future preset time period. The data volume prediction model is an autoregressive integrated moving average model optimized using a random search variable selection method. Finally, report data volume anomalies are identified based on the trend prediction sequence, optimization strategies are generated based on the report data volume anomalies, and the optimization strategies are executed to optimize the performance of the report database.

[0102] In this embodiment, the report data volume parameters within the target time period are obtained, and the parameters are analyzed using a pre-constructed autoregressive integrated moving average model optimized by a random search variable selection method. This accurately predicts the changing trend of the report data volume in the future, thereby accurately predicting report data volume anomalies, identifying potential performance bottlenecks in the report database in advance, and taking relevant strategies to optimize the database. This allows for timely response to potential performance bottlenecks, improves the response speed to data volume anomalies, reduces the lag in anomaly handling, and solves the technical problem of strong lag in anomaly identification methods based on database operation logs in related technologies.

[0103] The following is a detailed description with reference to another embodiment.

[0104] Example 2

[0105] The report data volume abnormality processing device provided in this embodiment includes multiple implementation units, each of which corresponds to the implementation steps in the above embodiment one. Its specific implementation method and beneficial effects can be referred to the foregoing method embodiment, and will not be repeated here.

[0106] Figure 3 This is a schematic diagram of an optional report data volume anomaly processing device according to an embodiment of the present invention, such as... Figure 3 As shown, the device for handling abnormal report data volume may include: a data acquisition unit 31, a prediction unit 32, and an optimization unit 33, wherein,

[0107] The data acquisition unit 31 is used to acquire report data volume parameters within the target time period and to preprocess the report data volume parameters.

[0108] Prediction unit 32 is used to construct report time series data based on preprocessed report data volume parameters, input the report time series data into the data volume prediction model, perform trend prediction on the report data volume through the data volume prediction model, and output the trend prediction sequence of the report data volume within a future preset time period. The data volume prediction model is an autoregressive integrated moving average model optimized by a random search variable selection method.

[0109] The optimization unit 33 is used to identify abnormal report data volume based on the trend prediction sequence, generate optimization strategies based on the abnormal report data volume, and execute the optimization strategies to optimize the performance of the report database.

[0110] The aforementioned report data volume anomaly processing device collects report data volume parameters within a target time period through acquisition unit 31 and preprocesses these parameters. Prediction unit 32 constructs report time-series data based on the preprocessed data volume parameters and inputs this data into a data volume prediction model. The model then predicts the trend of the report data volume and outputs a trend prediction sequence for the report data volume within a preset future time period. The data volume prediction model is an autoregressive integrated moving average model optimized using a random search variable selection method. Optimization unit 33 identifies report data volume anomalies based on the trend prediction sequence, generates optimization strategies based on these anomalies, and executes these strategies to optimize the performance of the report database.

[0111] In this embodiment, the report data volume parameters within the target time period are obtained, and the parameters are analyzed using a pre-constructed autoregressive integrated moving average model optimized by a random search variable selection method. This accurately predicts the changing trend of the report data volume in the future, thereby accurately predicting report data volume anomalies, identifying potential performance bottlenecks in the report database in advance, and taking relevant strategies to optimize the database. This allows for timely response to potential performance bottlenecks, improves the response speed to data volume anomalies, reduces the lag in anomaly handling, and solves the technical problem of strong lag in anomaly identification methods based on database operation logs in related technologies.

[0112] Furthermore, the device for handling abnormal report data volume also includes: a first construction module, used in step one, to construct training samples based on historical report data volume parameters and divide the training samples into a training set and a test set; a first determination module, used in step two, to determine the candidate range of model parameters, select model parameters from the candidate range using random sampling, and construct an initial data volume prediction model by adding an influence factor; a first calculation module, used in step three, to train the initial data volume prediction model using the training set, calculate the posterior probability of each model parameter being selected using a random variable selection mechanism, and select new model parameters from the candidate range using random sampling; a first iteration module, used in step four, to repeatedly execute step three, iteratively train the initial data volume prediction model until the number of iterations exceeds a preset iteration threshold, and select the model parameter with the highest posterior probability as the final model parameter of the data volume prediction model, thus obtaining the trained data volume prediction model; and a first testing module, used in step five, to test the trained data volume prediction model using a test set, and obtain the final data volume prediction model if the data volume prediction model passes the test.

[0113] Furthermore, the prediction unit includes: a second calculation module, used to input the report time series data into the data volume prediction model, perform linear calculations through the autoregressive prediction layer of the data volume prediction model, and output the autoregressive prediction result; a third calculation module, used to perform residual calculations using the random error term through the moving average prediction layer of the data volume prediction model, and output the residual calculation result; and a first fusion module, used to fuse the autoregressive prediction result and the residual calculation result through the data volume prediction model to obtain the trend prediction sequence of the report data volume.

[0114] Furthermore, the optimization unit includes: a first comparison module, used to compare the predicted value of the report data volume at each prediction time in the trend prediction sequence with a preset data volume threshold to obtain a comparison result; and a first determination module, used to determine that there is an anomaly in the report data volume when the comparison result indicates that the predicted value of the report data volume is greater than the data volume threshold.

[0115] Furthermore, the optimization strategy includes at least: a report download optimization strategy, and the optimization unit also includes: a first partitioning module, used to partition the report database based on preset partitioning rules to obtain N virtual partitions, where N is a positive integer; and a first configuration module, used to configure a parallel report data download strategy for the N virtual partitions, wherein the parallel report data download strategy means that when downloading report data, the report data in the report database is downloaded in parallel according to the virtual partitions.

[0116] Furthermore, the optimization strategy includes at least: a query statement optimization strategy, and the optimization unit also includes: a first analysis module, used to analyze the query log and identify slow query statements whose execution time exceeds a preset time threshold; and a first optimization module, used to optimize the slow query statements, wherein optimizing the slow query statements includes at least one of the following: using a join to replace a subquery, adding a query index, or constructing a range filtering statement.

[0117] Furthermore, the optimization strategy includes at least the following: a hot report data caching strategy. The optimization unit also includes: a second identification module, used to identify hot report data based on historical report data download records, wherein hot report data refers to report data whose download frequency is greater than a preset download frequency threshold; and a first caching module, used to cache the hot report data.

[0118] It should be noted that the acquisition unit 31, prediction unit 32, and optimization unit 33 mentioned above correspond to steps S201 to S203 in Embodiment 1. The instances and application scenarios implemented by the above units and corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules or units can be hardware or software components stored in memory (e.g., memory 104) and processed by one or more processors (e.g., processors 102a, 102b, ..., 102n). The above modules or units can also be part of a device and run in the computer terminal 10 provided in Embodiment 1.

[0119] The invention will now be described in conjunction with another alternative embodiment.

[0120] Example 3

[0121] The present invention can also provide an electronic device. Figure 4 This is a hardware structure block diagram of an electronic device (or mobile device) according to an optional method for handling abnormal report data volume according to an embodiment of the present invention, such as... Figure 4 As shown, the electronic device may include: one or more ( Figure 4 (Only one is shown) Processor 402, memory 404, memory controller, and peripheral interface, wherein the peripheral interface is connected to the radio frequency module, audio module and display.

[0122] The memory can be used to store software programs and modules, such as the program instructions / modules corresponding to the methods and apparatus in the embodiments of this application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, thereby implementing the above-described methods. The memory may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0123] The processor can access information and applications stored in the memory via a transmission device to perform the following steps: collect report data volume parameters within a target time period and preprocess the report data volume parameters; construct report time-series data based on the preprocessed report data volume parameters, input the report time-series data into a data volume prediction model, predict the trend of report data volume using the data volume prediction model, and output a trend prediction sequence of report data volume within a future preset time period. The data volume prediction model is an autoregressive integrated moving average model optimized using a random search variable selection method; identify report data volume anomalies based on the trend prediction sequence, generate optimization strategies based on the report data volume anomalies, and execute the optimization strategies to optimize the performance of the report database.

[0124] The processor can access information and applications stored in memory via a transmission device to execute the following steps: Step 1, construct training samples based on historical report data volume parameters and divide the training samples into training and testing sets; Step 2, determine the candidate range of model parameters, select model parameters from the candidate range using random sampling, and construct an initial data volume prediction model by adding an influence factor; Step 3, train the initial data volume prediction model using the training set, calculate the posterior probability of each model parameter being selected using a random variable selection mechanism, and select new model parameters from the candidate range using random sampling; Step 4, repeat Step 3 to iteratively train the initial data volume prediction model until the number of iterations exceeds a preset iteration threshold, and select the model parameter with the highest posterior probability as the final model parameter of the data volume prediction model, thus obtaining the trained data volume prediction model; Step 5, test the trained data volume prediction model using the testing set, and obtain the final data volume prediction model if the data volume prediction model passes the test.

[0125] The processor can access the information and application programs stored in the memory via the transmission device to perform the following steps: inputting the report time series data into the data volume prediction model; performing linear calculations through the autoregressive prediction layer of the data volume prediction model and outputting the autoregressive prediction results; performing residual calculations using the random error term through the moving average prediction layer of the data volume prediction model and outputting the residual calculation results; and fusing the autoregressive prediction results and the residual calculation results through the data volume prediction model to obtain the trend prediction sequence of the report data volume.

[0126] The processor can call the information and application stored in the memory through the transmission device to perform the following steps: compare the predicted value of the report data volume at each prediction time in the trend prediction sequence with the preset data volume threshold to obtain the comparison result; if the comparison result indicates that the predicted value of the report data volume is greater than the data volume threshold, determine that there is an anomaly in the report data volume.

[0127] The processor can access information and applications stored in memory via a transmission device to perform the following steps: The optimization strategy includes at least a report download optimization strategy. The steps for optimizing the performance of the report database by executing the optimization strategy include: partitioning the report database based on preset partitioning rules to obtain N virtual partitions, where N is a positive integer; configuring a parallel report data download strategy for the N virtual partitions, where the parallel report data download strategy means that when downloading report data, the report data in the report database is downloaded in parallel according to the virtual partitions.

[0128] The processor can access information and applications stored in memory via a transmission device to perform the following steps: The optimization strategy includes at least a query optimization strategy. The steps for optimizing the performance of the report database by executing the optimization strategy include: analyzing query logs to identify slow query statements whose execution time exceeds a preset time threshold; optimizing the slow query statements, wherein optimizing the slow query statements includes at least one of the following: using joins instead of subqueries, adding query indexes, or constructing range filtering statements.

[0129] The processor can access information and applications stored in the memory via a transmission device to perform the following steps: The optimization strategy includes at least a hot report data caching strategy. The steps of performing the optimization strategy to optimize the performance of the report database include: identifying hot report data based on historical report data download records, wherein hot report data refers to report data whose download frequency is greater than a preset download frequency threshold; and caching the hot report data.

[0130] This invention provides a solution for handling abnormal report data volume. It acquires report data volume parameters for a target time period and analyzes these parameters using a pre-constructed autoregressive integrated moving average model optimized with a random search variable selection method. This accurately predicts the trend of report data volume changes over a future period, thereby precisely predicting abnormal report data volume. It also identifies potential performance bottlenecks in the report database in advance and implements relevant strategies to optimize the database, responding promptly to potential performance bottlenecks, improving the response speed to data volume anomalies, and reducing the lag in anomaly handling. This solves the technical problem of significant lag in anomaly identification methods based on database operation logs in related technologies.

[0131] Those skilled in the art will understand that Figure 4 The structure shown is for illustrative purposes only. Electronic devices can also be smartphones, tablets, handheld computers, mobile internet devices (MIDs), PADs, and other terminal devices. Figure 4 This does not limit the structure of the aforementioned electronic device. For example, electronic devices may also include components that are more... Figure 4 The more or fewer components shown (such as network interfaces, display devices, etc.), or having the same Figure 4 The different configurations shown.

[0132] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the terminal device. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0133] The invention will now be described in conjunction with another alternative embodiment.

[0134] Example 4

[0135] This invention also provides a computer-readable storage medium. Optionally, in this invention, the computer-readable storage medium can be used to store the program code executed by the method for handling abnormal report data volume provided in Embodiment 1.

[0136] Optionally, in this embodiment of the invention, the storage medium may be located in any computer terminal in a group of computer terminals in a computer network, or in any mobile terminal in a group of mobile terminals.

[0137] This invention also provides a computer program product, which, when executed on a data processing device, is suitable for performing the following steps of a method for handling abnormal report data volume: collecting report data volume parameters within a target time period and preprocessing the report data volume parameters; constructing report time-series data based on the preprocessed report data volume parameters, inputting the report time-series data into a data volume prediction model, performing trend prediction on the report data volume through the data volume prediction model, and outputting a trend prediction sequence of the report data volume within a future preset time period, wherein the data volume prediction model is an autoregressive integrated moving average model optimized using a random search variable selection method; identifying report data volume anomalies based on the trend prediction sequence, generating optimization strategies based on the report data volume anomalies, and executing the optimization strategies to optimize the performance of the report database.

[0138] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0139] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0140] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0141] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0142] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0143] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0144] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for handling abnormal report data volume, characterized in that, include: Collect report data volume parameters within the target time period and preprocess the report data volume parameters; Based on the preprocessed report data volume parameters, report time series data is constructed, and the report time series data is input into the data volume prediction model. The data volume prediction model is used to predict the trend of the report data volume and output the trend prediction sequence of the report data volume within a future preset time period. The data volume prediction model is an autoregressive integrated moving average model optimized by a random search variable selection method. Based on the trend prediction sequence, anomalies in the report data volume are identified. An optimization strategy is generated based on the anomalies in the report data volume, and the optimization strategy is executed to optimize the performance of the report database.

2. The method according to claim 1, characterized in that, The data volume prediction model is pre-built, and the steps for building the data volume prediction model include: Step 1: Construct training samples based on the historical report data volume parameters, and divide the training samples into training set and test set; Step 2: Determine the candidate range of model parameters, select model parameters from the candidate range by random sampling, and construct an initial data volume prediction model by adding an influence factor; Step 3: Train the prediction model with the initial data volume using the training set, calculate the posterior probability of each model parameter being selected using a random variable selection mechanism, and select new model parameters from the candidate range using random sampling. Step four: Repeat step three to iteratively train the initial data volume prediction model until the number of iterations exceeds the preset iteration threshold, and select the model parameter with the highest posterior probability as the final model parameter of the data volume prediction model to obtain the trained data volume prediction model. Step 5: Test the trained data volume prediction model using the test set. If the data volume prediction model passes the test, obtain the final data volume prediction model.

3. The method according to claim 1, characterized in that, The steps of inputting the time-series data of the report into the data volume prediction model and using the data volume prediction model to predict the trend of the report data volume include: The time-series data of the report is input into the data volume prediction model, and linear calculation is performed through the autoregressive prediction layer of the data volume prediction model to output the autoregressive prediction result. The residual calculation is performed by using the random error term in the moving average prediction layer of the data volume prediction model, and the residual calculation result is output. The autoregressive prediction results and the residual calculation results are fused together by the data volume prediction model to obtain the trend prediction sequence of the report data volume.

4. The method according to claim 1, characterized in that, The steps for identifying abnormal report data volume based on the trend prediction sequence include: The predicted data volume of the report at each prediction time in the trend prediction sequence is compared with the preset data volume threshold to obtain the comparison result; If the comparison result indicates that the predicted value of the report data volume is greater than the data volume threshold, it is determined that the report data volume is abnormal.

5. The method according to claim 1, characterized in that, The optimization strategy includes at least: a report download optimization strategy, and the steps of executing the optimization strategy to optimize the performance of the report database include: The report database is partitioned based on preset partitioning rules to obtain N virtual partitions, where N is a positive integer; For N virtual partitions, a parallel download strategy for report data is configured, wherein the parallel download strategy for report data means that when downloading report data, the report data in the report database is downloaded in parallel according to the virtual partitions.

6. The method according to claim 1, characterized in that, The optimization strategy includes at least: a query statement optimization strategy, and the steps of executing the optimization strategy to optimize the performance of the report database include: Analyze the query logs to identify slow queries whose execution time exceeds a preset time threshold; Optimizing the slow query statement includes at least one of the following: using a join instead of a subquery, adding a query index, or constructing a range filter statement.

7. The method according to claim 1, characterized in that, The optimization strategy includes at least: a hot report data caching strategy, and the steps of executing the optimization strategy to optimize the performance of the report database include: Hotspot report data is identified based on historical report data download records, wherein the hotspot report data refers to report data whose download frequency is greater than a preset download frequency threshold; The hotspot report data is cached.

8. A device for processing abnormal report data volume, characterized in that, include: The data acquisition unit is used to acquire report data volume parameters within a target time period and to preprocess the report data volume parameters. The prediction unit is used to construct report time series data based on the preprocessed report data volume parameters, input the report time series data into the data volume prediction model, perform trend prediction on the report data volume through the data volume prediction model, and output the trend prediction sequence of the report data volume within a future preset time period. The data volume prediction model is an autoregressive integrated moving average model optimized by a random search variable selection method. An optimization unit is used to identify abnormal report data volume based on the trend prediction sequence, generate an optimization strategy based on the abnormal report data volume, and execute the optimization strategy to optimize the performance of the report database.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program, wherein, when the computer program is executed, it controls the device where the computer-readable storage medium is located to perform the report data volume abnormality handling method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, It includes one or more processors and a memory, the memory being used to store one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors cause the one or more processors to implement the report data volume abnormality handling method according to any one of claims 1 to 7.

Images