Business data processing method, apparatus, device, medium, and product

Abstract

This paper provides a business data processing method, apparatus, device, medium, and product applied in the field of data analysis. The method includes: obtaining a first data object in a first business application in response to a data sampling trigger command; obtaining a first sampling rate of the first data object based on a first mapping relationship between the data object and the sampling rate; the first mapping relationship is determined by simulation based on a downsampling simulation model corresponding to each data object, the downsampling simulation model being constructed based on datasets collected at least two different sampling rates for each data object; performing randomized downsampling according to the first sampling rate to obtain a first data set of the first data object; and performing performance management or resource allocation for a first business scenario in the first business application based on the first data set. This paper solves the problem of high data sampling costs in performance management, reducing data sampling costs and improving the efficiency of business data processing.

Description

Technical Field

[0001] This article relates to the field of data analysis and mining technology, and in particular to a business data processing method, device, equipment, medium and product. Background Technology

[0002] In the performance management of business applications, performance metrics data during application operation are typically obtained through event tracking. Here, "business application" refers to the software system that performs a specific business function. While historical predictions of the effects of random sampling exist, only methods for predicting the mean exist; effective predictions using quantiles are not yet possible. In scenarios involving random downsampling, performance metrics expressed using quantiles can become severely distorted. Summary of the Invention

[0003] This paper provides a business data processing method, apparatus, equipment, medium, and product that can significantly reduce data processing costs while ensuring the statistical stability of core business indicator data, and ensuring that the effect of business data processing based on downsampled data is consistent with that based on full data.

[0004] Firstly, a business data processing method is provided, the method comprising: In response to a data sampling trigger command, obtain the first data object in the first business application; Based on the first mapping relationship between data objects and sampling rates, the first sampling rate of the first data object is obtained; wherein, the first mapping relationship is determined by simulation based on the downsampling simulation model corresponding to each data object, each data object is a data object in the first business application, and the downsampling simulation model is constructed based on at least two datasets of each data object, and the at least two datasets are data collected through different sampling rates; Random downsampling is performed based on the first sampling rate to obtain the first data set of the first data object; Based on the first data set, performance management or resource allocation is performed on the first business scenario in the first business application, wherein the first business scenario is the business scenario associated with the first data object.

[0005] Secondly, a business data processing apparatus is also provided, the apparatus comprising: The sampling object acquisition module is used to obtain the first data object in the first business application in response to the data sampling trigger command; The sampling rate acquisition module is used to obtain the first sampling rate of the first data object based on the first mapping relationship between the data object and the sampling rate; wherein, the first mapping relationship is determined by simulation based on the downsampling simulation model corresponding to each data object, each data object is a data object in the first business application, and the downsampling simulation model is constructed based on at least two datasets of each data object, and the at least two datasets are data collected through different sampling rates; The data sampling module is used to perform random downsampling based on a first sampling rate to obtain a first data set of the first data object; The data analysis and application module is used to perform performance management or resource allocation on the first business scenario in the first business application based on the first data set, wherein the first business scenario is the business scenario associated with the first data object.

[0006] Thirdly, an electronic device is also provided, comprising: One or more processors; Storage device for storing one or more programs. When one or more programs are executed by one or more processors, the one or more processors implement the business data processing methods described in any of the contents of this specification.

[0007] Fourthly, a storage medium containing computer-executable instructions is also provided, which, when executed by a computer processor, are used to perform business data processing methods as described in any of the contents of this specification.

[0008] Fifthly, a computer program product is also provided, including a computer program that, when executed by a processor, implements a business data processing method as described in any of the contents of this specification. Attached Figure Description

[0009] The above and other features, advantages, and aspects described herein will become more apparent when taken in conjunction with the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0010] Figure 1 This is an exemplary system architecture that can implement the business data processing methods described in this article; Figure 2 This is a flowchart illustrating one of the business data processing methods presented in this article; Figure 3 This is a schematic diagram of the structure of a business data processing device provided in this article; Figure 4This is a schematic diagram of the structure of an electronic device used to implement the business data processing method described in this paper. Detailed Implementation

[0011] The implementation of the methods described herein will now be described in more detail with reference to the accompanying drawings. While some implementations of the methods described herein are shown in the drawings, it should be understood that the methods described herein can be implemented in various forms and should not be construed as limited to those described herein. Rather, these implementations are provided to facilitate a more thorough and complete understanding of the present invention. It should be understood that the accompanying drawings and implementations are for illustrative purposes only and are not intended to limit the scope of this invention.

[0012] It should be understood that the steps described in the method embodiments herein may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this document is not limited in this respect.

[0013] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one method implementation" means "at least one method implementation"; the term "another method implementation" means "at least one additional method implementation"; the term "some method implementations" means "at least some method implementations". Definitions for other terms will be given in the following description.

[0014] It should be noted that the concepts of "first" and "second" mentioned in this article are only used to distinguish different devices, modules or units, and are not used to limit the order of the functions performed by these devices, modules or units or their interdependencies.

[0015] It should be noted that the terms "one" and "more" used in this document are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0016] The names of messages or information exchanged between multiple devices in the embodiments herein are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0017] It is understandable that before using any of the solutions described in this article, users should be informed of the type, scope of use, and usage scenarios of the personal information involved in this article in an appropriate manner in accordance with relevant laws and regulations, and their authorization should be obtained.

[0018] For example, upon receiving a user's proactive request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to autonomously choose whether to provide personal information to the software or hardware such as electronic devices, applications, servers, or storage media implementing the scheme described herein, based on the prompt message.

[0019] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0020] It is understood that the above notification and user authorization process is merely illustrative and does not limit the implementation method described in this article. Other methods that comply with relevant laws and regulations may also be applied to the implementation method described in this article.

[0021] It is understood that the data involved in this article (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0022] It should also be noted that the system mentioned in this article can be... Figure 1 The system shown can implement the business data processing methods described in this paper.

[0023] Figure 1 An exemplary system architecture is shown that can be applied to the business data processing methods described in this paper.

[0024] The technical solution described in this paper can be applied to the business data processing system architecture shown in Figure 1. In practical applications, this business data processing system may include a client 101 and a server 102.

[0025] Client 101 can be a browser, application (App), HTML5 application or other web application, lightweight application (also known as a mini-program, a type of lightweight application), or cloud application, etc. Client 101 can be deployed on an electronic device and relies on the operation of that device or certain applications on the device to implement its functions. The electronic device can be, for example, a device with a display screen that supports information browsing, such as a smartphone, tablet, personal computer or other client terminal. For ease of understanding, Figure 1 The client is primarily represented in the form of a device. Various applications can also be configured on these electronic devices, such as multimedia applications, e-commerce platform applications, search applications, instant messaging applications, etc.

[0026] Server 102 can be one or more servers providing various services. That is, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server; in addition, it can also be a server of a distributed system, a server integrating blockchain technology, a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology, etc.

[0027] In the implementation described in this paper, server 102 can interact with client 101 to obtain application function running data of a specified application on the client. This data is used for analysis and mining to guide the improvement of application function performance. Server 102 can also synchronize the function configuration parameters determined based on the data analysis results to client 101 to optimize the running performance of the specified application function on client 101 and optimize business decisions related to the application function.

[0028] Understandable, Figure 1 The number of clients and servers shown is for illustrative purposes only. Any number of clients and servers can be configured to meet specific implementation requirements.

[0029] It should be noted that the implementation methods described in this article may involve the use of user data. In practical applications, within the scope permitted by applicable laws and regulations, the personal data of specific users may be used in the scheme described in this article, provided that the relevant laws and regulations of the user's region are met (such as obtaining the user's explicit consent and ensuring that the user receives appropriate notification).

[0030] Figure 2 This paper presents a flowchart illustrating a business data processing method applicable to scenarios involving business decision optimization of application functions. This method can be executed by a business data processing device, which can be implemented in software and / or hardware, optionally through an electronic device such as a mobile terminal, PC, or server.

[0031] like Figure 2 As shown, a business data processing method in one scenario can be applied to a first business application, and may specifically include: S210, In response to the data sampling trigger command, obtain the first data object in the first business application.

[0032] In this paper, the first business application can be any specific application that requires performance monitoring and cost control, such as a document application, instant messaging application, or collaborative office platform. The first data object can be performance metrics estimated using quantiles. Examples include page load time, frame rate, and server resource (CPU, memory, connection count, etc.) utilization. Quantiles can more comprehensively and stably reflect the central tendency, dispersion, and tail characteristics of data distribution. Performance metrics estimated using quantiles can be obtained by sorting all data from smallest to largest and proportionally dividing the values ​​at each position, reflecting extreme values, long tails, and scenarios of lag and degradation. The median represents the middle level, with half being high and half low; the 95th and 99th percentiles represent the performance metrics for the worst 5% and worst 1% of users, respectively. The primary data object can be used for analyzing online lag rates, degradation on low-end devices, performance safety nets, or compliance experience metrics during application runtime. For example, the 95th percentile of page load time data recorded by a core tracking point can more comprehensively reflect the tail characteristics of the data distribution.

[0033] Data sampling trigger commands can be start signals issued during application performance management when random downsampling operations need to be performed, such as when a preset time period, data volume threshold, or user-initiated trigger.

[0034] After receiving the sampling trigger command, the system extracts the specified performance indicator data from the performance monitoring database.

[0035] For example, in one scenario, the system can use a timed trigger to check the core tracking point reporting volume every five minutes. If it exceeds a cost threshold, sampling is automatically triggered to determine the corresponding performance metric value, i.e., the first data object. In another scenario, the first data object can be obtained through an event-driven approach. For instance, when a sudden increase in tracking point reporting volume is detected in a certain business scenario, a sampling command is immediately triggered to capture the current data window from the real-time data stream and determine the first data object. In yet another scenario, manual data collection can be triggered by operations personnel issuing sampling commands through a configuration platform, and the system responds to determine the first data object that needs to be sampled.

[0036] S220. Based on the first mapping relationship between the data object and the sampling rate, obtain the first sampling rate of the first data object.

[0037] The first mapping relationship is determined by simulation based on the downsampling simulation model corresponding to each data object. Each data object is a data object in the first business application. The downsampling simulation model is constructed based on at least two datasets of each data object. The at least two datasets are data collected through different sampling rates.

[0038] In one scenario, the first mapping relationship is determined by simulation based on the downsampling simulation model corresponding to each data object. The first mapping relationship satisfies the business stability conditions of the business scenario to which each data object belongs. The downsampling simulation model is constructed based on at least two datasets of the corresponding data object, and the at least two datasets are data collected through different sampling rates.

[0039] The downsampling simulation model can be a mathematical or machine learning model constructed using datasets collected at least two different sampling rates for a specific data object (such as a performance metric). This model determines the sampling rate range that satisfies business stability conditions by comparing the deviations of the quantiles corresponding to the datasets collected at different sampling rates.

[0040] Business stability conditions can be understood as the deviation between the quantiles of the downsampled data and the quantiles of the original data being controlled within an acceptable range for the business. For example, the relative error of the 95th percentile of page loading time does not exceed 5%, and the deviation does not change drastically with slight fluctuations in the sampling rate.

[0041] The system can query a pre-built mapping table based on the first data object to match the sampling rate that meets the business stability requirements and has the optimal cost as the first sampling rate. For example, offline simulation mapping can be built using the core data points of a certain application as an example. The system collects the full amount of data from the past seven days as a baseline, or data collected at a high sampling rate above the first sampling rate standard as a baseline. The system then performs repeated downsampling at different sampling rates such as 10%, 20%, and 50%, calculates the deviation of the quantiles at each sampling rate, and filters out the sampling rates with a deviation of less than 5%, forming a mapping table between quantile intervals and sampling rates.

[0042] In one scenario, after obtaining new quantile data objects each time, the system can retrain the downsampling simulation model using high sampling rate data (e.g., 80 percent or more) and low sampling rate data (e.g., 10 percent) from the past few days, dynamically updating the sampling rate threshold in the mapping relationship.

[0043] In one scenario, the process of obtaining the first mapping relationship includes the following steps: Step A: For each data object, combine the second data set with each sampling rate in the first candidate sampling rate set to obtain the model input data.

[0044] The second dataset is a set of control data collected for each data object at a second sampling rate. Each sampling rate in the first candidate sampling rate set is lower than the second sampling rate. This can be understood as a set of control data obtained for a specific data object at a higher second sampling rate (e.g., 80% or higher, or full sampling). This set is considered a benchmark that approximates the "true distribution." The first candidate sampling rate set may contain multiple sampling rates lower than the second sampling rate, such as 5%, 10%, 20%, 60%, etc. These sampling rates are candidate values ​​for the downsampling ratio to be evaluated.

[0045] The model input data can be a data pair consisting of a second data set and a certain sampling rate from the first candidate sampling rate set, used to input the downsampling simulation model to simulate the dataset that can be collected at that lower sampling rate.

[0046] For example, in one scenario, the system can pair a second set of data, which serves as a baseline, with each candidate sampling rate to generate multiple sets of model input data, each set corresponding to a downsampling rate to be tested.

[0047] Step B: Input the model input data into the corresponding downsampling simulation model to obtain the third data set output by the downsampling simulation model.

[0048] The third dataset can be the dataset output by the downsampling simulation model based on a set of model input data (the second dataset and a candidate sampling rate), where the corresponding sampling rate is the sampling rate in the model input data.

[0049] Downsampling simulation models can be pre-built mathematical models for specific data objects, capable of simulating the data distribution characteristics obtained by randomly sampling data from high-sampling-rate data at a specified low sampling rate. In one scenario, a downsampling simulation model can also be a system that sends each set of model input data generated in step A into the corresponding downsampling simulation model, and the model outputs the corresponding third data set after performing simulated downsampling.

[0050] The process of obtaining the third dataset from the downsampling simulation model can be as follows: for each record in the second dataset, the downsampling simulation model independently and randomly retains or discards records with a retention probability based on the candidate sampling rate. The retained records constitute the third dataset. For example, if the second dataset has 10,000 records and the candidate sampling rate is 10%, the simulation output will be approximately 1,000 records. Alternatively, for performance data with time-series characteristics, the process of obtaining the third dataset from the downsampling simulation model can also be as follows: the downsampling simulation model first divides the second dataset into time windows, independently samples within each segment according to the candidate sampling rate, and then merges all segment results to form the third dataset, thus preserving temporal locality. Or, if the downsampling simulation model is a machine learning model, it can directly output the corresponding downsampling result based on the input second dataset and the corresponding sampling rate.

[0051] Step C: Perform data object difference and significance analysis on each third dataset and the second dataset respectively, and obtain the difference analysis results and significance analysis results.

[0052] Difference analysis can compare the numerical differences between the third and second datasets on key indicators (especially quantiles), such as calculating the absolute or relative deviation of their 95th percentiles. Significance analysis can determine whether the differences are statistically significant through statistical tests, i.e., whether the differences are unlikely to be caused by random fluctuations. Commonly used methods include hypothesis testing.

[0053] The system can compare the third dataset corresponding to each candidate sampling rate with the benchmark second dataset, calculate the difference, and test its significance. For example, in one scenario, the 95th percentile of the second dataset can be calculated, followed by the 95th percentile of the third dataset, and the difference can be subtracted to obtain the absolute deviation. If the absolute value of the deviation is less than a preset threshold (e.g., 5% of the original value), the difference is considered acceptable. In another scenario, a non-parametric statistical test method can be used to compare the overall distribution of the two datasets and output a p-value. If the p-value is greater than the significance level (e.g., 0.05), the difference is considered insignificant. Alternatively, the deviations of multiple quantiles, such as the 90th, 95th, and 99th, can be compared simultaneously and scored comprehensively. Only when the deviations of all key quantiles meet the business requirements is the difference considered acceptable.

[0054] Step D: Under the constraints of preset difference conditions and preset significance conditions, based on the difference analysis results and significance analysis results, obtain the third sampling rate in the sampling rate corresponding to each third data set, and establish the first mapping relationship between the data object and the third sampling rate.

[0055] The preset difference condition can be the maximum acceptable quantile deviation in business operations, such as a relative deviation of no more than 5% from the 95th percentile. The preset significance condition can be the statistically acceptable significance level, such as a p-value greater than 0.05, indicating that the difference is not significant.

[0056] The third sampling rate is one or more sampling rates selected from all candidate sampling rates that meet the above two conditions. The system can filter out those sampling rates that simultaneously meet the difference and significance conditions from all candidate sampling rates, and then determine the final third sampling rate based on business preferences (such as minimum cost or maximum safety margin), and store the correspondence between the data object and the third sampling rate in the first mapping relationship. For example, it can iterate through the candidate sampling rates, checking them one by one from the lowest to the highest; the first sampling rate that simultaneously meets the difference and significance conditions is selected as the third sampling rate. For example, if both the deviation and significance conditions are passed at a 10% sampling rate, then 10% is selected.

[0057] The constraints of preset difference conditions and preset significance conditions can ensure that the data collected by downsampling can meet the business stability conditions, and control the deviation between the downsampled data quantiles and the original data quantiles within the acceptable range of business. For example, the relative error of the 95th percentile of page loading time does not exceed 5%, and the deviation does not change drastically with slight fluctuations in the sampling rate.

[0058] In one scenario, step E may also be included: obtaining the data acquisition cost corresponding to each third data set based on the sampling rate corresponding to each third data set; under the constraints of preset difference conditions and preset significance conditions, obtaining a fourth sampling rate based on the difference analysis results, significance analysis results, and data acquisition cost, and establishing a first mapping relationship between the data object and the fourth sampling rate.

[0059] Data acquisition cost refers to the resource consumption or monetary cost incurred in data reporting, storage, and processing after actual downsampling at a certain sampling rate. It is typically proportional to the sampling rate. The system can calculate the corresponding cost value based on each candidate sampling rate and the estimated data volume of its corresponding third dataset.

[0060] In one scenario, the cost of full data collection can be discounted based on the sampling rate to determine the cost for that sampling rate. For example, if the unit cost of full data collection is known, a sampling rate of 10% would have a cost approximately one-tenth of the full data collection cost, allowing for the estimation of the cost for each sampling rate. Alternatively, in a real-world system, there may be fixed costs plus variable costs. For instance, each data point may have a fixed reporting fee, and storage costs may increase incrementally with the amount of data. The system can calculate the number of data points based on the sampling rate and then substitute this information into a step function to obtain the cost.

[0061] The fourth sampling rate is the final sampling rate selected from all candidate sampling rates that simultaneously meet the conditions of difference and significance, taking into account the data collection cost. Typically, the lowest cost rate is chosen while still meeting quality requirements.

[0062] In one scenario, the construction process of a downsampling simulation model includes the following steps: Step A: Input the sampling rates corresponding to the fourth and fifth data sets into the first downsampling simulation model to obtain the sixth data set.

[0063] The fourth dataset is the control dataset with the higher sampling rate among the two datasets used in constructing the downsampling simulation model, and the fifth dataset is the dataset with the lower sampling rate among the two datasets. The first downsampling simulation model can be an initial version of the downsampling simulation model, which contains some sampling algorithm (such as uniform random sampling). The sixth dataset is the output data simulated by the first downsampling simulation model based on the sampling rates corresponding to the fourth and fifth datasets, and its sampling rate should be the same as that of the fifth dataset.

[0064] In obtaining the sixth data set, the system can take the fourth data set as input, use the sampling rate of the fifth data set as the target, call the first downsampling simulation model to perform simulated downsampling, and output the sixth data set.

[0065] Step B: Perform a consistency check between the sixth data set and the fifth data set, and obtain the check result.

[0066] Consistency check: Determine whether the sixth dataset from the simulation output is statistically sufficiently similar to the fifth dataset from the actual low-sampling acquisition. This typically involves comparing key characteristics such as quantiles and distribution shapes.

[0067] The verification result can be either verification passed or verification failed (or a quantitative result with a degree of deviation).

[0068] The system can perform multi-dimensional comparisons between the sixth and fifth data sets and output consistency verification results.

[0069] For example, multiple quantiles (such as the 50th, 90th, 95th, and 99th quantiles) can be calculated for two datasets. If the relative deviation of all quantiles is less than a preset threshold (such as 10%), the validation passes. Alternatively, a statistical distance metric can be used to measure the difference in probability distributions between the two datasets; if the distance is less than a threshold, the validation passes. Alternatively, validation can be performed only on a few quantiles of primary business concern (such as the 95th quantile); if the differences in these metrics are within the business's tolerance range, the validation passes, allowing for larger deviations in other non-critical locations.

[0070] Step C: In response to the verification result being successful, the downsampling simulation model construction process ends, and the first downsampling model is taken as the model construction result.

[0071] The verification passes if the consistency check between the sixth and fifth data sets meets the preset standard, indicating that the first downsampling simulation model can accurately simulate the downsampling process from a high sampling rate to a low sampling rate.

[0072] In one scenario, step D may also be included: in response to a verification result indicating that the verification failed, the sampling method of the first downsampling simulation model is adjusted to obtain a second downsampling simulation model, until the data set output by the second downsampling simulation model passes the consistency check with the fifth data set.

[0073] The verification failed because the consistency between the sixth and fifth data sets did not meet the preset standard, indicating that the simulation accuracy of the current model was insufficient.

[0074] Adjusting the sampling method of the first downsampling simulation model can involve modifying the sampling algorithm or parameters used within the model, such as changing from simple random sampling to stratified sampling, systematic sampling, or importance sampling. The second downsampling simulation model is the new version of the model after these adjustments.

[0075] S230. Perform randomized downsampling according to the first sampling rate to obtain the first data set of the first data object.

[0076] Randomized downsampling can be used to probabilistically sample and report the original performance data stream according to a first sampling rate, with each data point being retained independently with a probability based on the sampling rate. For example, when the sampling rate is 20%, each event has a 20% probability of being collected and reported.

[0077] The first dataset is a set of performance data samples actually collected after random downsampling. This sample will be used for subsequent performance analysis and decision-making.

[0078] In the process of obtaining the first data set, the system can use the first sampling rate as the retention probability to randomly filter the original event stream corresponding to the first data object, and the retained events constitute the first data set.

[0079] For example, for each page access event of a core tracking point of an application, a random number between zero and one is generated. If the random number is less than the first sampling rate (such as 0.3), the event is retained and reported; otherwise, it is discarded.

[0080] S240. Based on the first data set, perform performance management or resource allocation on the first business scenario in the first business application, wherein the first business scenario is the business scenario associated with the first data object.

[0081] The first business scenario can be the specific business application scenario associated with the first data object, such as the scenario of monitoring the loading performance of documents in an application. Performance management can include operations such as monitoring the changing trends of performance indicators, detecting outliers, and evaluating service quality. Resource allocation can refer to adjusting the allocation of system resources based on performance data, such as adjusting the bandwidth quota of the data tracking reporting channel and the processing capacity of computing nodes.

[0082] In one scenario, performance management or resource allocation for a first business scenario based on a first data set can be achieved by obtaining the resource utilization rate of the server or terminal device corresponding to the first business scenario based on the first data set, and then allocating resources for the server or terminal device based on the resource utilization rate.

[0083] The system can calculate the actual quantile index for the first dataset, compare these indexes with the historical baseline, and trigger an alarm or automatically adjust resources if performance degradation is found.

[0084] For example, the 95th quantile of the first dataset can be calculated. If the value exceeds a preset threshold (e.g., page loading takes more than three seconds), the computing resources for that business scenario (e.g., expanding the rendering service) will be automatically increased to improve performance.

[0085] In another scenario, where a real A / B test cannot be conducted, the system can compare the indicators of the first dataset with the expected quantiles at the same sampling rate in the downsampling simulation model. If the deviation is large, it will be automatically marked as "sampling distortion" and the system will switch to a higher sampling rate to re-acquire data.

[0086] In another scenario, performance management or resource allocation for the first business scenario can also be performed in response to the first quantile index exceeding the preset performance baseline value, by issuing a performance threshold alarm based on the first quantile index, or by resampling based on the fifth sampling rate; wherein the fifth sampling rate is greater than the first sampling rate.

[0087] For example, the system pre-sets performance thresholds for the first business scenario (e.g., the 95th percentile of page loading time must not exceed three seconds). After calculating the actual quantile for the first dataset, if it exceeds the threshold, the system generates an alarm and notifies operations personnel; if it continues to exceed the threshold, it automatically records the abnormal period for subsequent root cause analysis. Alternatively, the system compares the actual data volume of the first dataset with the expected data volume based on the first sampling rate. If the actual data volume is significantly lower than expected (possibly due to sampling implementation deviations), the system automatically increases the sampling rate and resamples using the fifth sampling rate. In summary, in one scenario described in this paper, the technical solution obtains a first data object in a first business application in response to a data sampling trigger command. Based on a first mapping relationship between the data object and the sampling rate, a first sampling rate for the first data object is obtained. This first mapping relationship is determined by a downsampling simulation model corresponding to each data object. The downsampling simulation model is constructed based on at least two datasets for the corresponding data object, where the at least two datasets are data collected at different sampling rates. Randomized downsampling is performed according to the first sampling rate to obtain a first data set for the first data object. Based on the first data set, performance management or resource allocation is performed on a first business scenario in the first business application, where the first business scenario is the business scenario associated with the first data object. The solution in this paper solves the problem of high data sampling costs in performance management. It can significantly reduce data processing costs while ensuring the statistical stability of core business indicator data. It provides a scientific basis for selecting sampling rates and a verifiable downsampling process for quantile performance indicators in random downsampling scenarios, ensuring that the effect of business data processing based on sampled data is consistent with that based on full data.

[0088] In an application example of a business data processing method, a downsampling simulation model and a mapping relationship between data objects and sampling rates are first constructed. Then, random sampling of data objects is performed based on the mapping relationship.

[0089] Specifically, for effective difference analysis and offline simulation, known and different randomized sampling rate data are required, thus at least two periods of historical data are needed. The dataset collected during the high sampling rate data collection period can be represented as Whigh, with a sampling rate of 95%, and is called the "baseline dataset". The dataset collected during the low sampling rate data collection period can be represented as Wlow, with a corresponding relatively low sampling rate of 70%, and is called the "observation dataset".

[0090] Whigh can be used to simulate Wlow, and the consistency between the simulation results and actual observations can be verified. If they are consistent, the effectiveness of the offline simulation method is proven, and other unknown sampling rates can be explored using simulation.

[0091] Step 1: Observe the stability of the target object and establish a baseline. The purpose of this step is to identify a set of data source objects for analysis and corresponding data to ensure that the indicators of the target object are stable or exhibit heterogeneous patterns, so that a comparative baseline can be established subsequently.

[0092] In this process, the first step is to identify a common analytical group. In the Whigh and Wlow datasets, a common set of users, or intersection users, can be identified. The data of these intersection users in both datasets can then be used as the analysis object to eliminate interference from changes in user composition. Furthermore, the stationarity of the time series data for each core indicator in the Whigh and Wlow datasets can be tested. For example, for each core indicator (such as the 75th quantile of CPU usage), its time series data for a specific period on the intersection users can be extracted, and the Augmented Dickey-Fuller test (ADF test) can be used to determine its stationarity. Additionally, for indicators exhibiting significant natural growth or decline trends (such as seasonal variations in user activity), the influence of these trends needs to be removed. During the natural trend removal process, the natural trend rate of change (such as week-on-week growth) of the indicator can be calculated, and the observed values ​​can be adjusted based on the natural trend rate of change and the sampling time period.

[0093] Step 2: Conduct online real-world variance analysis. The purpose of this step is to quantify the impact of switching from a high-sampling-rate dataset to a low-sampling-rate dataset on core metrics in real-world scenarios. This "real-world impact" will serve as a "reference target" for subsequent offline simulations.

[0094] Specifically, the data from the intersection of Whigh and Wlow can be used as the control group data, and the data from the intersection of Wlow and Wlow can be used as the experimental group data. Then, for each core indicator, we examine whether the difference between the control group and the experimental group data is significant.

[0095] In one scenario, a normality test (e.g., the Shapiro-Wilk test) can be performed. If the data follows a normal distribution (p > 0.05), an independent samples t-test is used; if the data does not follow a normal distribution (p < 0.05), the more robust Mann-Whitney U test is employed. If the test passes, the true difference for each indicator and its corresponding significance can be obtained. Record the changes in the key indicators of interest and the significance levels as the sampling rate decreases from the high sampling rate corresponding to Whigh to the low sampling rate corresponding to Wlow. This record is crucial for verifying the reliability of the simulation method.

[0096] Step 3: Offline simulation function construction and verification. The purpose of this step is to use the data from Whigh to predict or simulate the situation of Wlow, in order to obtain an offline simulation function (corresponding to the downsampling simulation model in the above scheme). The simulation function can be a predefined simulation function, whose function is to randomly sample again from the input dataset according to the corresponding target sampling rate to generate a simulation dataset.

[0097] The simulated downsampling process involves taking the sampling rates corresponding to the control and experimental group data as inputs to the offline simulation function. The offline simulation function outputs a simulated dataset based on the input. Then, the consistency between the simulated dataset and the real results can be verified. For example, based on the simulated dataset, the core indicators for the intersecting users on this simulated dataset are calculated, yielding simulated indicator values. These simulated indicator values ​​are then compared with the corresponding indicator values ​​in the experimental group data to complete the consistency check. If there is no statistical difference between the simulated and real data, the offline simulation function is proven to be effective and reliable. If the consistency check fails, it indicates that the simulation function has a bias. The next step is to adjust the internal logic of the simulation function (such as pure random sampling or stratified sampling) to optimize it with the goal of minimizing the discrepancies.

[0098] Step 4: Optimal Randomization (Sampling Rate) Selection. The goal is to predict actual business performance under various sampling rates based on the validated and reliable offline simulation function described above. Specifically, a series of candidate sampling rates α can be selected, such as {95%, 90%, 80%, 70%, 50%}. For each candidate sampling rate, the validated simulation function can be used to simulate the data on the control group. The core indicators on the simulated dataset are calculated, and the simulated indicators are compared with the original indicators of the control group data for significance testing, yielding the differences and significance of the indicators under each candidate sampling rate.

[0099] Furthermore, cost-benefit analysis can be used to make decisions. Cost savings for each sampling rate can be quantified as a ratio to the cost of the full sampled data. The goal is to maximize cost savings while maintaining business stability. Screening criteria could include ensuring that the significance of all core metrics is greater than an acceptable threshold (e.g., 0.05 or 0.1). A constraint on variability could be that the absolute difference between metrics must be less than the upper limit of tolerable business fluctuations. The optimal choice is to select the sampling rate with the lowest sampling cost among all sampling rates that meet the above two constraints, as it corresponds to the maximum cost.

[0100] Step 5: Based on the optimal randomization level (sampling rate), downsample the corresponding data objects. This can simultaneously quantify the influence of various performance indicators and compare the influence of different randomization levels among the indicators, thereby ensuring the rationality of the randomization selection level.

[0101] For example, in page load performance degradation detection, document page load time is a core performance indicator of document applications, monitored at the 95th quantile. The sampling rate has been determined to be 60% through optimal randomization, resulting in the first data set corresponding to the first data object: document page load time.

[0102] The system can extract raw page load time data for that time period from the first dataset every five minutes, calculate the 95th quantile, and obtain a value of 3200 milliseconds. Assuming the system's preset performance threshold is 2500 milliseconds, the current value of 3200 milliseconds exceeds the threshold. The system can further compare 3200 milliseconds with the expected quantile (2400 milliseconds) at a 10% sampling rate in the downsampling simulation model, finding a deviation as high as 800 milliseconds, and the deviation indicates a deterioration in performance.

[0103] Since the deviation is significant and cannot be explained by sampling error, the system determines that the actual performance of the business is deteriorating, triggering performance management actions, such as generating a high-priority alarm, sending it to the document application performance responsibility team, and automatically marking the period as a "performance anomaly window". At the same time, the original data (corresponding records in the first data set) within the anomaly period is archived for subsequent root cause analysis.

[0104] Furthermore, the sampling rate can be dynamically adjusted to control costs and ensure data quality and resource allocation. Assume the monthly cost of collecting all data points for a certain data object is 100,000 yuan, and the business requirement is to keep the cost below 30,000 yuan per month. Based on the above, the first sampling rate is determined to be 30%, corresponding to an estimated cost of 30,000 yuan. The system runs at this first sampling rate for one week. After one week of operation, the actual data volume from the first dataset is 29% of the total data, and the actual cost is approximately 29,000 yuan, which meets the budget expectation. However, the system also finds that the 95th quantile fluctuates significantly daily during the week, and the deviation from the expected quantile in the downsampling simulation model exceeds 8% on two days (the acceptable deviation for the business is 5%). The system determines that the data stability at the current sampling rate is insufficient and triggers resource allocation actions, such as automatically calling the downsampling simulation model to try to find a higher but still cost-controllable sampling rate, provided that the difference condition (deviation less than 5%) and the significance condition are met. After recalculation, the system found that when the sampling rate was increased to 35%, the deviation decreased to 4% and the cost was 32,000 yuan (slightly exceeding the budget). At a sampling rate of 32%, the deviation was 4.5% and the cost was 30,000 yuan. The system selected 32% as the new primary sampling rate, automatically updated the sampling configuration, and executed the new sampling rate in the next cycle. This allowed for dynamic searching of the optimal sampling rate within cost constraints, controlling reporting costs while ensuring the stability of quantile indicators through moderately increasing the sampling rate, thus preventing data distortion caused by an excessively low sampling rate from affecting business decisions.

[0105] Figure 3 This paper presents a schematic diagram of a business data processing device, which is suitable for scenarios involving business decision optimization of application functions.

[0106] like Figure 3 As shown, the business data processing device includes: a sampling object acquisition module 310, a sampling rate acquisition module 320, a data sampling module 330, and a data analysis and application module 340.

[0107] The sampling object acquisition module 310 is used to obtain a first data object in the first business application in response to a data sampling trigger command; the sampling rate acquisition module 320 is used to obtain a first sampling rate of the first data object based on a first mapping relationship between the data object and the sampling rate; wherein the first mapping relationship is determined by simulation based on the downsampling simulation model corresponding to each data object, each data object is a data object in the first business application, and the downsampling simulation model is constructed based on at least two datasets of each data object, the at least two datasets being data collected through different sampling rates; the data sampling module 330 is used to perform randomized downsampling according to the first sampling rate to obtain a first data set of the first data object; the data analysis and application module 340 is used to perform performance management or resource allocation for a first business scenario in the first business application based on the first data set, wherein the first business scenario is the business scenario associated with the first data object.

[0108] In one scenario described herein, the technical solution obtains a first data object in a first business application in response to a data sampling trigger command; obtains a first sampling rate for the first data object based on a first mapping relationship between the data object and the sampling rate; wherein the first mapping relationship is determined by simulation based on a downsampling simulation model corresponding to each data object, each data object is a data object in the first business application, and the downsampling simulation model is constructed based on at least two datasets for each data object, the at least two datasets being data collected at different sampling rates; randomized downsampling is performed according to the first sampling rate to obtain a first data set for the first data object; and performance management or resource allocation is performed on a first business scenario in the first business application based on the first data set, wherein the first business scenario is the business scenario associated with the first data object. The solution in this paper solves the problem of high data sampling costs in performance management, significantly reducing data processing costs while ensuring the statistical stability of core business indicator data, and ensuring that business data processing based on downsampled data has the same effect as business data processing based on full data.

[0109] In one scenario described in this article, optionally, the process of obtaining the first mapping relationship includes: For each data object, the second data set is combined with each sampling rate in the first candidate sampling rate set to obtain the model input data; the second data set is the set of control data collected for each data object according to the second sampling rate, and each sampling rate in the first candidate sampling rate set is less than the second sampling rate; The model input data is input into the corresponding downsampling simulation model to obtain the third data set output by the downsampling simulation model; the sampling rate corresponding to the third data set is the sampling rate in the model input data. For each third dataset and the second dataset, perform data object difference and significance analysis to obtain the difference analysis results and significance analysis results; Under the constraints of preset difference conditions and preset significance conditions, based on the difference analysis results and significance analysis results, the third sampling rate is obtained from the sampling rate corresponding to each third data set, and a first mapping relationship is established between the data object and the third sampling rate.

[0110] In one scenario described in this article, the process of obtaining the first mapping relationship may optionally include: Based on the sampling rate corresponding to each third dataset, the data acquisition cost corresponding to each third dataset is obtained. Under the constraints of preset difference conditions and preset significance conditions, based on the difference analysis results, significance analysis results and data acquisition costs, a fourth sampling rate is obtained from the sampling rates corresponding to each third data set, and a first mapping relationship is established between the data object and the fourth sampling rate.

[0111] In one scenario described in this paper, the construction process of the downsampling simulation model optionally includes: The sampling rates corresponding to the fourth and fifth datasets are input into the first downsampling simulation model to obtain the sixth dataset. The fourth dataset is the control dataset with a high sampling rate in the two datasets, and the fifth dataset is the dataset with a low sampling rate in the two datasets. Perform a consistency check between the sixth and fifth data sets to obtain the check result; If the verification result is successful, the downsampling simulation model construction process ends, and the first downsampling model is taken as the model construction result.

[0112] In one scenario described in this paper, the construction process of the downsampling simulation model may optionally include: In response to a failure of the verification result, the sampling method of the first downsampling simulation model is adjusted to obtain the second downsampling simulation model, until the data set output by the second downsampling simulation model passes the consistency verification with the fifth data set.

[0113] In one scenario described herein, optionally, the data analysis and application module 340 is specifically used for: Based on the first dataset, the resource utilization rate of the server or terminal device corresponding to the first business scenario is obtained; Resource allocation for servers or terminal devices is based on resource utilization.

[0114] In one scenario described herein, optionally, the data analysis and application module 340 is specifically used for: Obtain the first quantile index in the first dataset; In response to the first quantile metric exceeding the preset performance baseline value, a performance threshold alarm is issued based on the first quantile metric, or resampling is performed based on the fifth sampling rate; wherein the fifth sampling rate is greater than the first sampling rate.

[0115] The business data processing device provided in this paper can execute the business data processing method implemented in any of the ways described in this paper, and has the corresponding functional modules and beneficial effects for executing the business data processing method.

[0116] It is worth noting that the various units and modules included in the above-mentioned device are divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be realized; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the protection scope of the implementation method in this paper.

[0117] The following is for reference. Figure 4 This document illustrates a structural diagram of an electronic device (e.g., a terminal device or server) 400 suitable for implementing the present invention. The terminal device in this document may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The electronic device shown is merely an example and should not impose any limitations on the functionality and scope of this article.

[0118] like Figure 4 As shown, electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from storage device 408 into random access memory (RAM) 403. The RAM 403 also stores various programs and data required for the operation of electronic device 400. The processing device 401, ROM 402, and RAM 403 are interconnected via bus 404. Input / output (I / O) interface 406 is also connected to bus 404.

[0119] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic device 400 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 An electronic device 400 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0120] Specifically, according to the implementation described herein, the process referred to in the flowchart above can be implemented as a computer software program. For example, the implementation herein includes a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowchart. In such an implementation, the computer program can be downloaded and installed from a network via communication device 409, or installed from storage device 408, or installed from ROM 402. When the computer program is executed by processing device 401, it performs the functions defined in the method herein.

[0121] The names of messages or information exchanged between multiple devices in this document are for illustrative purposes only and are not intended to limit the scope of these messages or information.

[0122] The electronic device provided herein and the business data processing method described above belong to the same inventive concept. Technical details not described in detail herein can be found in the above description, and have the same beneficial effects as the above description.

[0123] This article provides a computer storage medium on which a computer program is stored, which, when executed by a processor, implements the business data processing method provided by the above-described implementation.

[0124] It should be noted that the computer-readable medium described above can be a computer-readable signal medium, a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example, (but not limited to) an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this document, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0125] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0126] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0127] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: In response to a data sampling trigger command, the first data object is obtained; Based on the first mapping relationship between data objects and sampling rates, the first sampling rate of the first data object is obtained. The first mapping relationship is determined by simulation based on the downsampling simulation model corresponding to each data object. The first mapping relationship satisfies the business stability conditions of the business scenario to which each data object belongs. The downsampling simulation model is constructed based on at least two datasets of the corresponding data object. The at least two datasets are data collected through different sampling rates. Random downsampling is performed based on the first sampling rate to obtain the first data set of the first data object; Based on the first data set, performance management or resource allocation is performed on the first business scenario, where the first business scenario is the business scenario associated with the first data object.

[0128] Computer program code for performing the operations described herein may be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0129] The flowcharts and block diagrams in the accompanying figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various implementations described herein. In this respect, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the figures. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0130] The modules and units described herein can be implemented in software or hardware. The names of modules and units do not necessarily limit the specific unit; for example, a media data editing module can also be described as "a module for editing first audio data and obtaining first display data of target display text associated with the first audio data."

[0131] The functions described above in this document can be performed at least in part by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), and so on.

[0132] In the context of this document, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0133] This document also provides a computer program product, including a computer program that, when executed by a processor, implements the business data processing method provided by any implementation of this disclosure.

[0134] In the implementation of a computer program product, computer program code for performing the operations described herein can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0135] The above description is merely a preferred implementation and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this document is not limited to technical solutions formed by specific combinations of the above-described technical features, but also includes other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions in this document.

[0136] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be executed in the specific order shown or in a sequential order. Multitasking and parallel processing may be advantageous in certain contexts. Similarly, while some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of this paper. Certain features described in the context of a single implementation may also be implemented in combination within a single implementation. Conversely, various features described in the context of a single implementation may also be implemented individually or in any suitable sub-combination in multiple implementations.

[0137] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A business data processing method, applied to a first business application, comprising: In response to a data sampling trigger command, obtain the first data object in the first business application; Based on the first mapping relationship between data objects and sampling rates, the first sampling rate of the first data object is obtained; wherein, the first mapping relationship is determined by simulation based on the downsampling simulation model corresponding to each data object, the data objects are data objects in the first business application, and the downsampling simulation model is constructed based on at least two datasets of each data object, the at least two datasets being data collected through different sampling rates; Random downsampling is performed based on the first sampling rate to obtain the first data set of the first data object; Based on the first data set, performance management or resource allocation is performed on the first business scenario in the first business application, wherein the first business scenario is the business scenario associated with the first data object.

2. The method according to claim 1, wherein the process of obtaining the first mapping relationship includes: For each data object, the second data set is combined with each sampling rate in the first candidate sampling rate set to obtain the model input data; The second data set is a set of comparison data collected by each data object according to the second sampling rate, wherein each sampling rate in the first candidate sampling rate set is less than the second sampling rate; The model input data is input into the corresponding downsampling simulation model to obtain a third data set output by the downsampling simulation model; the sampling rate corresponding to the third data set is the sampling rate in the model input data. Perform data object difference and significance analysis on each of the third data sets and the second data set respectively to obtain difference analysis results and significance analysis results; Under the constraints of preset difference conditions and preset significance conditions, based on the difference analysis results and the significance analysis results, a third sampling rate is obtained in the sampling rate corresponding to each of the third data sets, and the first mapping relationship is established based on the data object and the third sampling rate.

3. The method according to claim 2, further comprising: Based on the sampling rate corresponding to each of the third data sets, the data acquisition cost corresponding to each of the third data sets is obtained; Under the constraints of preset difference conditions and preset significance conditions, based on the difference analysis results, the significance analysis results and the data acquisition cost, a fourth sampling rate is obtained from the sampling rates corresponding to each of the third data sets, and the first mapping relationship is established based on the data object and the fourth sampling rate.

4. The method according to any one of claims 1-3, wherein the process of constructing the downsampling simulation model includes: The sampling rates corresponding to the fourth and fifth data sets are input into the first downsampling simulation model to obtain the sixth data set. The fourth data set is the control data set with the high sampling rate in the two datasets, and the fifth data set is the data set with the low sampling rate in the two datasets. A consistency check is performed between the sixth data set and the fifth data set to obtain the check result; If the verification result is successful, the downsampling simulation model construction process ends, and the first downsampling model is taken as the model construction result.

5. The method according to claim 4, further comprising: In response to the verification result being a failure, the sampling method of the first downsampling simulation model is adjusted to obtain a second downsampling simulation model, until the data set output by the second downsampling simulation model passes the consistency verification with the fifth data set.

6. The method according to claim 1, wherein the step of performing performance management or resource allocation for the first business scenario based on the first data set includes: Based on the first data set, the resource utilization rate of the server or terminal device corresponding to the first business scenario is obtained; Resource allocation for the server or terminal device is performed based on the resource utilization rate.

7. The method according to claim 1, wherein the step of performing performance management or resource allocation for the first business scenario in the first business application based on the first data set includes: Obtain the first quantile index in the first dataset; In response to the first quantile index exceeding a preset performance baseline value, a performance threshold alarm is issued based on the first quantile index, or resampling is performed based on a fifth sampling rate; wherein the fifth sampling rate is greater than the first sampling rate.

8. A business data processing apparatus, configured in a first business application, comprising: The sampling object acquisition module is used to obtain the first data object in the first business application in response to the data sampling trigger command; The sampling rate acquisition module is used to obtain the first sampling rate of the first data object based on the first mapping relationship between data objects and sampling rates; wherein, the first mapping relationship is determined by simulation based on the downsampling simulation model corresponding to each data object, the data objects are data objects in the first business application, and the downsampling simulation model is constructed based on at least two datasets of each data object, the at least two datasets being data collected through different sampling rates; The data sampling module is used to perform random downsampling based on the first sampling rate to obtain a first data set of the first data object; The data analysis and application module is used to perform performance management or resource allocation on a first business scenario in the first business application based on the first data set, wherein the first business scenario is the business scenario associated with the first data object.

9. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors implement the business data processing method as described in any one of claims 1-7.

10. A storage medium comprising computer-executable instructions, which, when executed by a computer processor, are used to perform the business data processing method as described in any one of claims 1-7.

11. A computer program product comprising a computer program that, when executed by a processor, implements the business data processing method as described in any one of claims 1-7.

Images