An anti-cheating method for advertising based on big data analysis
Through real -time monitoring of advertising data and using a broad sense to add models to predict the efficiency of advertising, identify whether there is a cheating in the process of advertising, solve the problem of low detection efficiency and privacy violations in existing technology, and achieve efficient and transparent cheating. Behavior monitoring.
Patent Information
- Application Number
- CN202410854095.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-28
- Publication Date
- 2025-01-28
- Estimated Expiration
- 2044-06-28
AI Technical Summary
The prior art is inefficient and prone to infringe upon user privacy when monitoring cheating in advertising.
Through real -time monitoring of advertising data, the number of people in different access methods and calculating the proportion of the number of people can be calculated. Using a broad sense can add models to predict advertising benefits, and identify whether there is cheating in the process of advertising.
It improves the efficiency of cheating behavior monitoring, avoids infringement of user privacy, and accurately identifies and curb cheating behaviors, and protects the interests of advertisers and platform reputation.
Smart Images

Figure CN118735595B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of Internet advertising delivery, and relates to an advertising delivery anti-cheating method based on big data analysis. Background Art
[0002] On short video platforms, advertising is the main business model, but the problem of cheating is becoming increasingly serious. Common cheating behaviors include fake traffic, click fraud, fake users, device disguises, and location fraud. These behaviors cause advertisers to lose money, distort data, damage the platform's reputation, and waste resources, forcing the platform to take effective anti-cheating measures. Through big data and machine learning, the platform can analyze user behavior, identify device fingerprints, detect geographic locations, and establish anti-cheating models. These technical means can effectively identify and curb cheating behaviors, improve the authenticity and effectiveness of advertisements, protect the interests of advertisers, enhance the reputation of the platform, and optimize the user experience. The implementation of anti-cheating measures not only guarantees the advertiser's return on investment, but also enhances the advertiser's trust in the platform, which helps to attract more advertising investment and achieve a win-win situation for the platform and advertisers.
[0003] However, analyzing user behavior, identifying device fingerprints, detecting geographic locations, etc. are complex processes and easily infringe on user privacy, making it difficult to quickly and effectively monitor cheating behavior. Therefore, it is particularly important to develop a more efficient method for monitoring cheating behavior. Summary of the invention
[0004] The present invention provides an anti-cheating method for advertising based on big data analysis, aiming to use the big data analysis method to solve the problems of low efficiency in cheating behavior monitoring in the prior art and easy infringement of user privacy.
[0005] The purpose of the present invention can be achieved through the following technical solutions:
[0006] This application provides an anti-cheating method for advertising based on big data analysis, comprising the following steps:
[0007] S1. Real-time monitoring of advertising data within a specific time period, wherein the advertising data includes advertising access data and its corresponding advertising actual benefit data; the advertising access data is divided into homepage access, link access and big data push access according to access mode;
[0008] S2. Count the number of people using different access methods and calculate the proportion of people using different access methods.
[0009] S3, input the proportion of people with different access methods into a preset advertising benefit prediction model, and output the predicted advertising predicted benefit; the preset advertising benefit prediction model is configured as a generalized additive model, and the generalized additive model uses the proportion of people with different access methods as an explanatory variable and the corresponding actual advertising benefit as a response variable;
[0010] S4, by calculating the difference between the actual benefit of the advertisement and the predicted benefit of the advertisement, dividing the difference by the predicted benefit of the advertisement, and obtaining the relative benefit deviation;
[0011] S5. A relative benefit deviation threshold is set. When the relative benefit deviation exceeds the relative benefit deviation threshold, the platform determines that cheating exists in the advertisement access data of the advertisement delivery.
[0012] Furthermore, in step S2, the proportion of people using different access methods is calculated using the following formula:
[0013]
[0014] Where P o is the proportion of people using the oth access method, F o is the number of people in the oth access mode; F sum The total number of people accessing using all methods.
[0015] Specifically, in step S3, the generalized additive model includes the following construction steps:
[0016] S31. Collect data: collect data sets consisting of the proportion of people using different access methods under normal circumstances and their corresponding actual advertising benefits;
[0017] S32, data set division: divide the samples of the data set into a training set and a prediction set; the training set is used to build the model, and the prediction set is used to evaluate the prediction performance of the model;
[0018] S33, Single factor significance test: Construct a single factor model between a single explanatory variable and the response variable, and test the significance of the model;
[0019] S34, constructing a generalized additive model: adding the single factor models that pass the significance test one by one to obtain a generalized additive model;
[0020] S35. Model performance evaluation: Input the prediction set into the constructed generalized additive model, output the predicted value, calculate the coefficient of determination between the predicted value and the actual value, and evaluate the prediction performance of the model.
[0021] Furthermore, in step S32, the training set and prediction set account for 80% and 20% of the number of samples in the data set, respectively.
[0022] Furthermore, in step S33, the single factor also includes interaction terms between different explanatory variables.
[0023] Furthermore, in step S34, passing the significance test is specifically manifested as the model p value being less than or equal to 0.05.
[0024] Furthermore, in step S34, the generalized additive model is expressed as:
[0025]
[0026] In the formula, g(u y ) is an identity connection function with additive properties; u y is the response variable, i.e., the actual benefit of advertising; s(x i ,k i ) is a smooth function, and the smooth function is configured as a cubic spline function, x i is the i-th explanatory variable that passes the significant test, i.e., the proportion of people using different access methods, k i is the number of nodes of the cubic spline function, which controls the number of intervals within the numerical range of the explanatory variable. Each interval has a curve fitted by the cubic spline function. n is the number of explanatory variables that pass the significance test. T j is the jth interaction term that passes the significance test, and t is the number of interaction terms that pass the significance test.
[0027] Furthermore, in step S35, the determination coefficient is calculated by the formula:
[0028]
[0029] In the formula, R 2 is the coefficient of determination; y e is the measured value of the response variable; z e is the predicted value of the response variable; y is the average of the measured values; e represents the e-th sample of the data variable, and m is the number of samples.
[0030] Furthermore, in step S4, the relative benefit deviation is calculated by the formula:
[0031]
[0032] In the formula, E r Represents relative benefit deviation; E a Indicates the actual benefit of advertising; E b Represents the predicted advertising benefit.
[0033] Furthermore, in step S5, after the cheating behavior is determined, the cheating behavior information is sent to the advertiser, and the advertising authority of the advertiser is restricted.
[0034] Beneficial effects of the present invention:
[0035] (1) By counting the number of people using different access methods during advertising delivery and calculating the proportion of people using different access methods; collecting big data from historical periods to establish a relationship model between the proportion of people using different access methods and the actual benefits of the corresponding advertisements, the relationship model is used to predict the real-time monitored advertising access data to identify whether cheating is involved in the advertising delivery process. The present invention solves the problem of low efficiency in monitoring cheating behavior in the prior art, and the access method data acquisition channel is open and transparent, and does not require the acquisition of user device information, which is not easy to infringe on user privacy.
[0036] (2) A generalized additive model was constructed with the proportion of people using different access methods as the explanatory variable and the corresponding actual advertising benefits as the response variable. This solved the problem that the nonlinear and nonparametric relationship between the proportion of people using different access methods and the actual advertising benefits is difficult to quantify, and the comprehensive impact of the proportion of people using different access methods on the actual advertising benefits is difficult to assess. BRIEF DESCRIPTION OF THE DRAWINGS
[0037] In order to facilitate understanding by those skilled in the art, the present invention is further described below with reference to the accompanying drawings.
[0038] Figure 1 This is a structural diagram of an anti-cheating method for advertising based on big data analysis in the present invention. DETAILED DESCRIPTION
[0039] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the specific implementation mode, structure, characteristics and effects of the present invention are described in detail below in combination with the accompanying drawings and preferred embodiments.
[0040] See also Figure 1 The present application provides an anti-cheating method for advertising based on big data analysis, comprising the following steps:
[0041] S1. Real-time monitoring of advertising data within a specific time period, wherein the advertising data includes advertising access data and its corresponding advertising actual benefit data; the advertising access data is divided into homepage access, link access and big data push access according to access mode;
[0042] In this embodiment, advertisers place advertisements on short video platforms through short video bloggers or Internet celebrities, and monitor the advertisement placement data in a specific time period in real time. These data include advertisement access data and its corresponding actual advertisement benefit data, and the revenue data is characterized by the actual purchase volume of the product. Advertisement access data is divided into three categories according to the access method: home page access, link access, and big data push access. By real-time monitoring of advertisement placement data, the performance of advertisements under different access methods can be accurately captured, so as to analyze the impact of various access methods on the actual benefits of advertisements. Home page access usually represents that users actively search for relevant content, so this type of access often has a higher conversion rate. Link access usually comes from users clicking on specific links, such as advertising links in social media or emails, which can reflect the attractiveness and relevance of advertisements. Big data push access uses precise push technology to display advertisements to potential users, and evaluating its effect helps optimize push strategies and improve advertising benefits.
[0043] However, there is a certain relationship between these access methods and cheating. Cheating may have an impact on the advertising data of different access methods, thereby affecting the actual effectiveness evaluation of advertising. First, home page access is usually when users actively look for relevant content and actively enter the advertising page. Cheaters may increase home page access data through manual operations or automated tools, create false clicks and conversion rates to improve the performance of advertising effects, thereby deceiving advertisers and advertising platforms. Secondly, link access is when users click on specific links to enter the advertising page. Cheaters may increase the amount of link access data through fictitious or non-real link clicks, thereby pretending that more users are interested in the advertisement, in an attempt to mislead advertisers and platforms. Big data push access is more about the platform itself using algorithms to accurately push to users who need specific products. Bloggers or Internet celebrities cannot control the direction and quantity of big data push, and have a higher degree of authenticity.
[0044] S2. Count the number of people using different access methods and calculate the proportion of people using different access methods.
[0045] Furthermore, the calculation formula for calculating the proportion of people using different access methods is:
[0046]
[0047] Where P o is the proportion of people using the oth access method, F o is the number of people accessing in the oth way; F sum The total number of people accessing using all methods.
[0048] In this embodiment, the proportion of people with different access methods is often closely related to the actual benefits of the advertisement, that is, for the same type of product, the proportion of people with different access methods has a certain expected benefit. By obtaining the proportion of people with different access methods, the advertising benefits can be predicted. When the predicted benefits are inconsistent with the actual benefits, it means that there may be false traffic in the data of home page access and link access. To this end, this application collects big data to establish a relationship model between the proportion of people with different access methods and the actual benefits of their corresponding advertisements, which is used to predict the real-time monitored advertising access data to identify whether there is cheating in the advertising delivery process. In addition, this method does not require analysis of user behavior, identification of device fingerprints, and detection of geographic location. The source of access method data is open and transparent, and will not infringe on user privacy.
[0049] S3, input the proportion of people with different access methods into a preset advertising benefit prediction model, and output the predicted advertising predicted benefit; the preset advertising benefit prediction model is configured as a generalized additive model, and the generalized additive model uses the proportion of people with different access methods as an explanatory variable and the corresponding actual advertising benefit as a response variable;
[0050] In this embodiment, since the relationship between the proportion of people in different access modes and the actual benefits of advertisements is complex, even nonlinear and nonparametric, and there is interaction between the proportion of people in different access modes (i.e., if the proportion of people in a certain access mode is large, the proportion of people in other access modes will be small), it is difficult to quantify the comprehensive impact of the proportion of people in different access modes on the actual benefits of advertisements. For this reason, this embodiment adopts a generalized additive model to quantify the nonlinear and nonparametric relationship between the proportion of people in different access modes and the actual benefits of advertisements, as well as the comprehensive impact of the proportion of people in different access modes on the actual benefits of advertisements. The generalized additive model is a multivariate statistical model based on the framework of the generalized linear model, but unlike the generalized linear model, the generalized additive model allows the relationship between the independent variable (explanatory variable) and the dependent variable (response variable) to be nonlinear, and it uses a smoothing function to model the nonlinear relationship instead of assuming a linear relationship. The core idea of the generalized additive model is to split the independent variable into multiple components and apply a smoothing function to each component for modeling. These components can be continuous variables, categorical variables or interaction terms. By superimposing the effects of these components, the response relationship between the overall dependent variable and the independent variable is obtained. In the modeling process, commonly used smoothing functions include spline functions, B-spline functions, and local polynomials. These smoothing functions can be estimated by least squares, maximum likelihood estimation, or methods based on probability distribution.
[0051] Specifically, the generalized additive model includes the following construction steps:
[0052] S31. Collect data: collect data sets consisting of the proportion of people using different access methods under normal circumstances and their corresponding actual advertising benefits;
[0053] It should be noted that the normal situation here refers to the situation where the actual benefits of the advertisement reach the expected results. This situation means that the advertiser recognizes the traffic data and revenue results of the advertisement, and assumes that there is no cheating behavior.
[0054] S32, data set division: divide the samples of the data set into a training set and a prediction set; the training set is used to build the model, and the prediction set is used to evaluate the prediction performance of the model;
[0055] Furthermore, in step S32, the training set and prediction set account for 80% and 20% of the number of samples in the data set, respectively.
[0056] S33, Single factor significance test: Construct a single factor model between a single explanatory variable and the response variable, and test the significance of the model;
[0057] Furthermore, in step S33, the single factor also includes interaction terms between different explanatory variables.
[0058] In this embodiment, by establishing a single factor model between a single explanatory variable and a response variable, and performing a significance test on all factors, it can help screen out factors that have a significant impact on the response variable. In the model analysis process, if the coefficients of some influencing factors are not significant, that is, they have no significant effect on explaining the changes in the response variable, it can be considered to be removed from the model to improve the simplicity and interpretability of the model. This method helps to eliminate factors that contribute less to the predictive ability of the model, making the model more refined and effective. By gradually eliminating insignificant influencing factors, a more accurate and concise model can be obtained, thereby better explaining the changes in the response variable.
[0059] It should be noted that, generally speaking, the proportion of the three access modes has a significant impact on the actual effectiveness of advertising. The method of testing significance in this embodiment is suitable for variable screening when more access modes may need to be introduced to avoid introducing unnecessary variables.
[0060] In this embodiment, the interaction terms formed by different explanatory variables can be regarded as a single factor, and the interaction term refers to a new variable artificially constructed in statistical modeling to describe the mutual influence or action between different variables. In multiple regression analysis, the interaction term is a new variable obtained by multiplying two or more independent variables. By introducing the interaction term, the joint effect between different independent variables can be captured to more comprehensively explain the changes in the dependent variable. Therefore, the introduction of the interaction term can better capture the complex relationship between different factors and improve the accuracy and interpretability of the model. In addition, by introducing the interaction term, the interaction between multiple factors can be considered at the same time without adding additional factors, thereby improving the expressive power of the model. The interaction term can help us better understand the joint effect between factors and help improve the predictive power of the model.
[0061] S34, constructing a generalized additive model: adding the single factor models that pass the significance test one by one to obtain a generalized additive model;
[0062] Furthermore, in step S34, passing the significance test is specifically manifested as the model p value being less than or equal to 0.05.
[0063] It should be noted that in statistics, p-values are usually used to judge the significance of each variable in the model. The p-value indicates the probability of observing a statistic or a more extreme situation when the null hypothesis is true. Generally speaking, if the p-value of a variable is less than the set significance level (usually set to 0.05), we can reject the null hypothesis and believe that the variable has a significant effect on the result; conversely, if the p-value is greater than the significance level, we accept the null hypothesis and believe that the variable has no significant effect on the result.
[0064] Furthermore, in step S34, the generalized additive model is expressed as:
[0065]
[0066] In the formula, g(u y ) is an identity connection function with additive properties; u y is the response variable, i.e., the actual benefit of advertising; s(x i ,k i ) is a smooth function, and the smooth function is configured as a cubic spline function, x i is the i-th explanatory variable that passes the significant test, i.e., the proportion of people using different access methods, k i is the number of nodes of the cubic spline function, which controls the number of intervals within the numerical range of the explanatory variable. Each interval has a curve fitted by the cubic spline function. n is the number of explanatory variables that pass the significance test. T j is the jth interaction term that passes the significance test, and t is the number of interaction terms that pass the significance test.
[0067] S35. Model performance evaluation: Input the prediction set into the constructed generalized additive model, output the predicted value, calculate the coefficient of determination between the predicted value and the actual value, and evaluate the prediction performance of the model.
[0068] Furthermore, in step S35, the determination coefficient is calculated by the formula:
[0069]
[0070] In the formula, R 2 is the coefficient of determination; y e is the measured value of the response variable; z e is the predicted value of the response variable; is the average of the measured values; e represents the e-th sample of the data variable, and m is the number of samples.
[0071] S4, by calculating the difference between the actual benefit of the advertisement and the predicted benefit of the advertisement, dividing the difference by the predicted benefit of the advertisement, and obtaining the relative benefit deviation;
[0072] Furthermore, in step S4, the relative benefit deviation is calculated by the formula:
[0073]
[0074] In the formula, E r Represents relative benefit deviation; E a Indicates the actual benefit of advertising; E b Represents the predicted advertising benefit.
[0075] S5. A relative benefit deviation threshold is set. When the relative benefit deviation exceeds the relative benefit deviation threshold, the platform determines that cheating exists in the advertisement access data of the advertisement delivery.
[0076] In this embodiment, the predicted benefit of an advertisement can be regarded as the expected benefit of the advertisement in theory, which is derived from the prediction model established based on the data under normal circumstances. In contrast, the actual benefit of an advertisement is the effect observed after the actual advertisement is released. If there is a serious inconsistency between the actual benefit and the predicted benefit, that is, it exceeds the set relative benefit deviation threshold, then it can be inferred that there is a high possibility of cheating in the advertisement access data.
[0077] In addition, in order to avoid the risk of misjudgment, the relative benefit deviation threshold is set relatively high, and only when there is a significant difference between the actual benefit and the predicted benefit will it be identified as cheating. This design is to fully consider the unpredictable factors that may exist in the advertising process and avoid misjudging normal fluctuations as cheating. With this method, advertising platforms can promptly identify and respond to advertising access data with cheating behavior to ensure the fairness and effectiveness of advertising. At the same time, when setting the relative benefit deviation threshold, it is also necessary to evaluate and adjust it based on the actual situation to balance the potential impact of misjudgment and omission.
[0078] Furthermore, in step S5, after the cheating behavior is determined, the cheating behavior information is sent to the advertiser, and the advertising authority of the advertiser is restricted.
[0079] In this embodiment, when it is determined that there is cheating in the advertising access data, the platform will take the following measures:
[0080] 1) Notify advertisers: Detailed information about detected cheating behavior will be provided to advertisers, including specific evidence and analysis results. This will help advertisers understand the situation and avoid financial losses due to cheating.
[0081] 2) Restricting advertising rights: Implementing corresponding restrictive measures on the advertisers, including but not limited to suspending advertising, reducing advertising rights, or completely prohibiting them from advertising on the platform. This can prevent the continued generation of false traffic and maintain the fairness of the platform and the interests of advertisers.
[0082] 3) Further investigation and review: The platform will conduct a more in-depth investigation and review of the advertising parties involved in the cheating behavior to ensure that the problem is completely resolved. At the same time, the platform can strengthen the monitoring of advertising data to prevent similar situations from happening again.
[0083] Beneficial effects of the present invention:
[0084] (1) By counting the number of people using different access methods during advertising delivery and calculating the proportion of people using different access methods; collecting big data from historical periods to establish a relationship model between the proportion of people using different access methods and the actual benefits of the corresponding advertisements, the relationship model is used to predict the real-time monitored advertising access data to identify whether cheating is involved in the advertising delivery process. The present invention solves the problem of low efficiency in monitoring cheating behavior in the prior art, and the access method data acquisition channel is open and transparent, and does not require the acquisition of user device information, which is not easy to infringe on user privacy.
[0085] (2) A generalized additive model was constructed with the proportion of people using different access methods as the explanatory variable and the corresponding actual advertising benefits as the response variable. This solved the problem that the nonlinear and nonparametric relationship between the proportion of people using different access methods and the actual advertising benefits is difficult to quantify, and the comprehensive impact of the proportion of people using different access methods on the actual advertising benefits is difficult to assess.
[0086] The above description is only a preferred embodiment of the present invention and does not limit the present invention in any form. Although the present invention has been disclosed as a preferred embodiment as above, it is not used to limit the present invention. Any technical personnel in this field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present invention. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention without departing from the content of the technical solution of the present invention are still within the scope of the technical solution of the present invention.
Claims
1. An anti-cheating method for advertising based on big data analysis, characterized in that: The following steps are involved: S1. Real-time monitoring of advertising data within a specific time period, wherein the advertising data includes advertising access data and its corresponding advertising actual benefit data; the advertising access data is divided into homepage access, link access and big data push access according to access mode; S2. Count the number of people using different access methods and calculate the proportion of people using different access methods. S3, input the proportion of people with different access methods into a preset advertising benefit prediction model, and output the predicted advertising predicted benefit; the preset advertising benefit prediction model is configured as a generalized additive model, and the generalized additive model uses the proportion of people with different access methods as an explanatory variable and the corresponding actual advertising benefit as a response variable; S4, by calculating the difference between the actual benefit of the advertisement and the predicted benefit of the advertisement, dividing the difference by the predicted benefit of the advertisement, and obtaining the relative benefit deviation; S5. Setting a relative benefit deviation threshold. When the relative benefit deviation exceeds the relative benefit deviation threshold, the platform determines that cheating exists in the advertisement access data of the advertisement delivery. In step S3, the generalized additive model includes the following construction steps: S31. Collect data: collect data sets consisting of the proportion of people using different access methods under normal circumstances and their corresponding actual advertising benefits; S32, data set division: divide the samples of the data set into a training set and a prediction set; the training set is used to build the model, and the prediction set is used to evaluate the prediction performance of the model; S33, Single factor significance test: Construct a single factor model between a single explanatory variable and the response variable, and test the significance of the model; S34, constructing a generalized additive model: adding the single factor models that pass the significance test one by one to obtain a generalized additive model; S35, model performance evaluation: input the prediction set into the constructed generalized additive model, output the predicted value, calculate the coefficient of determination between the predicted value and the actual value, and evaluate the prediction performance of the model; In step S33, the single factor also includes interaction terms between different explanatory variables.
2. The method for preventing cheating in advertising based on big data analysis according to claim 1, characterized in that: In step S2, the proportion of people using different access methods is calculated using the following formula: , In the formula, P o For the o The proportion of people using different access methods, F o For the o Number of people using different access methods; F sum The total number of people accessing using all methods.
3. The method for preventing cheating in advertising based on big data analysis according to claim 1, characterized in that: In step S32, the training set and prediction set account for 80% and 20% of the sample number of the data set, respectively.
4. The method for preventing cheating in advertising based on big data analysis according to claim 1, characterized in that: In step S34, passing the significance test is specifically manifested as the model p value being less than or equal to 0.
05.
5. The method for preventing cheating in advertising based on big data analysis according to claim 1, characterized in that: In step S34, the generalized additive model is expressed as: , In the formula, g ( u y ) is an identity connection function with additive properties; u y is the response variable, i.e., actual advertising effectiveness; s ( x i , k i ) is a smooth function, and the smooth function is configured as a cubic spline function, x i For the i The explanatory variables that passed the significant test are the proportion of people using different access methods. k i is the number of nodes of the cubic spline function, which controls the number of intervals within the numerical range of the explanatory variable. Each interval has a curve fitted by the cubic spline function. n is the number of explanatory variables that passed the significance test; T j For the j There are interaction terms that pass the significance test. t is the number of interaction terms that passed the significance test.
6. The method for preventing cheating in advertising based on big data analysis according to claim 1, characterized in that: In step S35, the determination coefficient is calculated by the formula: , In the formula, R 2 is the coefficient of determination; y e is the measured value of the response variable; z e is the predicted value of the response variable; is the average of the measured values; e Indicates the data variable e samples, m is the number of samples.
7. The method for preventing cheating in advertising based on big data analysis according to claim 1, characterized in that: In step S4, the relative benefit deviation is calculated by the formula: , In the formula, E r Represents relative benefit deviation; E a Indicates the actual effectiveness of advertising; E b Represents the predicted advertising benefit.
8. The method for preventing cheating in advertising based on big data analysis according to claim 1, characterized in that: In step S5, when cheating behavior is confirmed, cheating behavior information is sent to the advertiser, and the advertising authority of the advertiser is restricted.
Citation Information
Patent Citations
Advertisement putting method, related device, equipment and storage medium
CN111667312A
Advertisement putting method and system based on real-time prediction ROI
CN113269595A