A method, device, medium and program product for determining magic number

Abstract

The purpose of the present application is to provide a method, device, medium and program product for determining a magic number, the method comprising: determining one or more first indicator features corresponding to a business goal based on product function information of a target application and historical behavioral performance information of a user; evaluating the one or more first indicator features based on statistical results corresponding to multiple dimensions, and determining at least one second indicator feature from the one or more first indicator features based on the evaluation results, wherein the multiple dimensions include a user quantity scale dimension and a historical business effect dimension; inputting at least one second indicator feature into a predetermined feature evaluation model to obtain importance information corresponding to the one or more second indicator features, and determining one or more target indicator features from the one or more second indicator features based on the importance information; determining critical value information corresponding to the one or more target indicator features, and using the critical value information as the magic number corresponding to the business goal.

Description

Technical Field

[0001] The present application relates to the field of communications, and in particular to a technology for determining a magic number. Background Art

[0002] Existing technology typically involves identifying a business's North Star Metric. Essentially, this involves finding a single metric with the greatest correlation (or even the greatest causal effect) with the business objective, breaking it down to find the critical correlation value (the "magic number"). Effective interventions are then implemented through product and operational strategies to help more users achieve this magic number, thereby improving business objectives. For example, a social app discovered that users who followed seven friends had higher retention rates. However, the main problem with current North Star Metric analysis methods is "inverted causation," particularly when analyzing returning users (this issue also exists for new users, but the impact is relatively less). For example, analysis may reveal that users who engage in a certain behavior daily have a 20% higher next-day open retention rate than those who don't. However, it's more likely that stickier users with higher open intention rates are more likely to engage in this behavior. In this case, comparing the importance of various behaviors to the objective may lead to erroneous conclusions, as the importance of a particular feature may actually be far less than expected. Furthermore, there's insufficient variability in features, meaning that some metrics may fluctuate only slightly from day to day, making it difficult to identify a "magic number." Summary of the Invention

[0003] One object of the present application is to provide a method, device, medium and program product for determining a magic number.

[0004] According to one aspect of the present application, a method for determining a magic number is provided, the method comprising:

[0005] Determine one or more first indicator features corresponding to the business goal based on the product function information of the target application and the historical behavior performance information of the user;

[0006] Evaluate the one or more first indicator features based on statistical results corresponding to multiple dimensions, and determine at least one second indicator feature from the one or more first indicator features based on the evaluation results, wherein the multiple dimensions include a user quantity and scale dimension and a historical business effect dimension;

[0007] Inputting the at least one second indicator feature into a predetermined feature evaluation model to obtain importance information corresponding to the one or more second indicator features, and determining one or more target indicator features from the one or more second indicator features based on the importance information;

[0008] Critical value information corresponding to the one or more target indicator characteristics is determined, and the critical value information is used as a magic number corresponding to the business goal.

[0009] According to one aspect of the present application, a computer device for determining a magic number is provided, the device comprising:

[0010] A module, configured to determine one or more first indicator features corresponding to a business objective based on product function information of a target application and historical behavior performance information of a user;

[0011] Module 1 and 2 are configured to evaluate the one or more first indicator features based on statistical results corresponding to multiple dimensions, and determine at least one second indicator feature from the one or more first indicator features based on the evaluation results, wherein the multiple dimensions include a user quantity and scale dimension and a historical business performance dimension;

[0012] a module 13, configured to input the at least one second indicator feature into a predetermined feature evaluation model, obtain importance information corresponding to the one or more second indicator features, and determine one or more target indicator features from the one or more second indicator features based on the importance information;

[0013] A fourth module is used to determine critical value information corresponding to the one or more target indicator characteristics, and use the critical value information as the magic number corresponding to the business goal.

[0014] According to one aspect of the present application, a computer device for determining a magic number is provided, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the operations of any of the above methods.

[0015] According to one aspect of the present application, a computer-readable storage medium is provided, on which a computer program is stored, characterized in that when the computer program is executed by a processor, the operation of any of the methods described above is implemented.

[0016] According to one aspect of the present application, a computer program product is provided, comprising a computer program, which implements the steps of any of the above methods when executed by a processor.

[0017] Compared to the prior art, the present application determines one or more first indicator features corresponding to a business objective based on product function information of a target application and historical user behavior performance information; evaluates the one or more first indicator features based on statistical results corresponding to multiple dimensions, and determines at least one second indicator feature from the one or more first indicator features based on the evaluation results, wherein the multiple dimensions include a user quantity scale dimension and a historical business performance dimension; inputs the at least one second indicator feature into a predetermined feature evaluation model to obtain importance information corresponding to the one or more second indicator features, and determines one or more target indicator features from the one or more second indicator features based on the importance information; determines critical value information corresponding to the one or more target indicator features, and uses the critical value information as the magic number corresponding to the business objective, thereby implementing a universal data science methodology for mining business North Star indicators. All key behavioral performance can be quantified and measured, and the methodology is generally applicable to all Internet and non-Internet related businesses. The methodology can combine the characteristics of macro analysis and micro analysis, typically first using macro analysis methods to determine key indicators (to reduce the impact of reverse causality), and then using micro analysis methods to find the magic number. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Other features, objects and advantages of the present application will become more apparent upon reading the detailed description of non-limiting embodiments made with reference to the following drawings:

[0019] Figure 1 A flow chart of a method for determining a magic number according to one embodiment of the present application is shown;

[0020] Figure 2 A schematic diagram showing a method for determining a magic number according to an embodiment of the present application is shown;

[0021] Figure 3 A schematic diagram showing a method for determining a magic number according to an embodiment of the present application is shown;

[0022] Figure 4 A structural diagram of a computer device for determining a magic number according to one embodiment of the present application is shown;

[0023] Figure 5 An exemplary system is shown that can be used to implement the various embodiments described in this application.

[0024] The same or similar reference numerals in the drawings represent the same or similar components. DETAILED DESCRIPTION

[0025] The present application is described in further detail below with reference to the accompanying drawings.

[0026] In a typical configuration of the present application, the terminal, the device of the service network and the trusted party all include one or more processors (eg, a central processing unit (CPU)), an input / output interface, a network interface and a memory.

[0027] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash memory. Memory is an example of a computer-readable medium.

[0028] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PCM), programmable random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device.

[0029] The devices referred to in this application include but are not limited to user devices, network devices, or devices formed by integrating user devices and network devices through a network. The user devices include but are not limited to any mobile electronic product that can interact with a user (for example, through a touchpad), such as a smartphone, a tablet computer, etc. The mobile electronic product can use any operating system, such as the Android operating system, the iOS operating system, etc. Among them, the network device includes an electronic device that can automatically perform numerical calculations and information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a programmable logic device (PLD), a field programmable gate array (FPGA), a digital signal processor (DSP), an embedded device, etc. The network device includes but is not limited to a computer, a network host, a single network server, a set of multiple network servers, or a cloud composed of multiple servers; here, the cloud is composed of a large number of computers or network servers based on cloud computing, wherein cloud computing is a type of distributed computing, a virtual supercomputer composed of a group of loosely coupled computers. The network includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network, a wireless self-organizing network (Ad Hoc network), etc. Preferably, the device may also be a program running on the user device, the network device, or a device formed by integrating the user device and the network device, the network device and the touch terminal, or the network device and the touch terminal via a network.

[0030] Of course, those skilled in the art should understand that the above-mentioned devices are only examples, and other existing or future devices that are applicable to this application should also be included in the scope of protection of this application and are included here by reference.

[0031] In the description of the present application, “plurality” means two or more, unless otherwise clearly defined.

[0032] Figure 1A flow chart of a method for determining a magic number according to an embodiment of the present application is shown, the method comprising steps S11, S12, S13, and S14. In step S11, a computer device determines one or more first indicator features corresponding to a business objective based on product function information of a target application and historical behavioral performance information of a user; in step S12, the computer device evaluates the one or more first indicator features based on statistical results corresponding to multiple dimensions, and determines at least one second indicator feature from the one or more first indicator features based on the evaluation results, wherein the multiple dimensions include a user quantity scale dimension and a historical business effect dimension; in step S13, the computer device inputs the at least one second indicator feature into a predetermined feature evaluation model, obtains importance information corresponding to the one or more second indicator features, and determines one or more target indicator features from the one or more second indicator features based on the importance information; in step S14, the computer device determines critical value information corresponding to the one or more target indicator features, and uses the critical value information as the magic number corresponding to the business objective.

[0033] In step S11, the computer device determines one or more first indicator features corresponding to the business goal based on the product function information of the target application and the user's historical behavior performance information. In some embodiments, the user first needs to specify a business goal, that is, to determine the goal that the business expects to achieve, such as "increasing the retention rate of new users the next day after opening the application." Then, the product functions of the target application and the user's historical behavior performance in the target application need to be exhaustively listed. Then, data cleaning, feature engineering, etc. are performed to generate corresponding one or more feature data. Feature correlation analysis is then performed to analyze the correlation between the one or more feature data and the business goal. Feature screening is then performed based on the analysis results. At least one feature data that has a non-causal inversion type correlation with the business goal is screened out from the one or more feature data. Then, new feature construction is performed, and corresponding indicator features are constructed for the at least one feature data according to the definition or form of the indicator, thereby obtaining one or more first indicator features corresponding to the business goal, wherein the one or more first indicator features have a correlation with the business goal, and the correlation is not of the causal inversion type, or the one or more first indicator features have a correlation with the business goal that is greater than or equal to a predetermined correlation degree, and the correlation is not of the causal inversion type.

[0034] In step S12, the computer device evaluates the one or more first indicator features based on the statistical results corresponding to multiple dimensions, and determines at least one second indicator feature from the one or more first indicator features based on the evaluation results, wherein the multiple dimensions include the user quantity scale dimension and the historical business effect dimension. In some embodiments, the usage of the one or more first indicator features is evaluated by referring to the external statistical results corresponding to multiple dimensions including the user quantity scale dimension and the historical business effect dimension, that is, for each first indicator feature, the first indicator feature is evaluated based on the user quantity scale and historical business effect corresponding to the first indicator feature to evaluate whether the first indicator feature meets the North Star indicator condition. The North Star indicator condition mainly includes two: the first is that the number of users it targets should not be too small, and the second is that the historical effect it has on the business goal should not be too small, wherein the user quantity scale can refer to the number of users involved in the first indicator feature, or it can also refer to the proportion of the number of users involved in the first indicator feature relative to the total number of users of the target application, and the historical business effect refers to the quantified average business effect that the first indicator feature has historically produced on the business goal. In some embodiments, based on the evaluation results, at least one second indicator feature that meets the North Star indicator condition is determined from the one or more first indicator features.

[0035] In step S13, the computer device inputs the at least one second indicator feature into a predetermined feature evaluation model, obtains importance information corresponding to one or more second indicator features, and determines one or more target indicator features from the one or more second indicator features based on the importance information. In some embodiments, each second indicator feature of the at least one second indicator feature is input into a trained feature evaluation model to obtain importance information corresponding to the second indicator feature output by the feature evaluation model, wherein the importance information is used to quantify the importance of the second indicator feature to the business goal. In some embodiments, the second indicator features of the one or more second indicator features whose corresponding importance is higher than or equal to a predetermined threshold can be used as target indicator features, or a predetermined number of second indicator features with the highest corresponding importance among the one or more second indicator features can also be used as target indicator features. In some embodiments, the one or more target indicator features are used as North Star indicators for business goals.

[0036] In step S14, the computer device determines the critical value information corresponding to the one or more target indicator features, and uses the critical value information as the magic number corresponding to the business goal. In some embodiments, for each target indicator feature, it is necessary to further refine the specific behavioral performance. The critical value information corresponding to the target indicator feature can be determined by a method similar to finding an inflection point. The critical value information can refer to all value information corresponding to the target indicator feature (wherein the value information can be a specific numerical value, or can also be a numerical range. For example, if the target indicator feature is the number of views of a certain page in the target application by a user, then the corresponding value information can include a number of views of 2, a number of views of 3, a number of views of 4... or the corresponding value information can also include a number of views greater than or equal to 2, a number of views greater than or equal to 3, a number of views greater than or equal to 4...) in a descending order or a descending order, starting from the critical value information, as the subsequent value information changes, the degree of change in the quantified historical business effect of the target indicator feature with respect to the business goal under the subsequent value information is less than or equal to a predetermined threshold, that is, the degree of change in the corresponding quantified historical business effect begins to slow down. For example, a curve can be constructed using the value information corresponding to the target indicator feature as the horizontal coordinate, and the quantified historical business effect of the target indicator feature on the business goal under each value information as the vertical coordinate. According to the positive direction of the horizontal coordinate, the horizontal coordinate value corresponding to the first point on the curve whose corresponding tangent direction is horizontal is used as the critical value information, or the horizontal coordinate value corresponding to the first point on the curve whose angle between the tangent direction and the horizontal direction is less than or equal to a predetermined angle threshold is used as the critical value information. For example, if the business goal is "increasing the retention rate of new users the next day after opening the app", the target indicator feature can be "video viewing time", and the critical value information corresponding to the target indicator feature can be "2 hours". The quantified historical business effect of the target indicator feature on the business goal under the critical value information is "retention rate of 64%". When the target indicator feature is greater than the critical value information, the quantified historical business effect of the target indicator feature on the business goal fluctuates around "retention rate of 64%", and the fluctuation range is within the predetermined interval threshold. This application can implement a universal data science methodology for mining business North Star indicators. All key behavioral performances can be quantified and measured, and are basically applicable to all Internet and non-Internet related businesses. It can combine the characteristics of macro analysis and micro analysis. Usually, macro analysis methods are used first to determine key indicators (to reduce the impact of inverted causality), and then micro analysis methods are used to find the magic number.

[0037] In some embodiments, step S11 includes: the computer device determines at least one first indicator feature associated with the business goal based on the product function information of the target application and the historical behavior performance information of the user; and determines one or more first indicator features corresponding to the business goal from the at least one first indicator feature. In some embodiments, first, based on the product function of the target application and the historical behavior performance of the user in the target application, one or more feature data are generated through data cleaning, feature engineering, etc., and then feature correlation analysis is performed to analyze the correlation between the one or more feature data and the business goal. Feature screening is then performed based on the analysis results to screen out at least one feature data associated with the business goal from the one or more feature data, and then new feature construction is performed. According to the definition or form of the indicator, a corresponding indicator feature is constructed for the at least one feature data to obtain at least one first indicator feature. Then, for each first indicator feature in the at least one first indicator feature, the matching degree between the first indicator feature and the business goal is measured in a quantitative manner, thereby selecting one or more first indicator features from the at least one first indicator feature whose corresponding matching degree is greater than or equal to a predetermined matching degree threshold.

[0038] In some embodiments, determining one or more first indicator features corresponding to the business objective in the at least one first indicator feature includes: for each first indicator feature in the at least one first indicator feature, determining whether the correlation between the first indicator feature and the business objective is a causal inversion, and if so, discarding the first indicator feature; and using the remaining first indicator features in the at least one first indicator feature as the one or more first indicator features corresponding to the business objective. In some embodiments, for each first indicator feature in the at least one first indicator feature, it is necessary to determine whether the correlation between the first indicator feature and the business objective is a causal inversion, that is, it is necessary to determine whether the appearance of the first indicator feature leads to the realization of the business objective or the improvement of the quantified business effect of the business objective, or whether the appearance of the first indicator feature is caused by the realization of the business objective or the improvement of the quantified business effect of the business objective. If the latter is the case, it is clearly a causal inversion correlation. If the correlation between the first indicator feature and the business objective is a causal inversion, the first indicator feature needs to be discarded; otherwise, the first indicator feature is retained. In some embodiments, the remaining first indicator features among the at least one first indicator feature may be used as one or more first indicator features corresponding to the business goal, or the first indicator features among the remaining first indicator features among the at least one first indicator feature whose corresponding matching degree is greater than or equal to a predetermined matching degree threshold may be used as one or more first indicator features corresponding to the business goal. Alternatively, several first indicator features whose corresponding matching degree is greater than or equal to a predetermined matching degree threshold may be selected from the at least one first indicator feature, and then for each of the several first indicator features, it may be determined whether the correlation between the first indicator feature and the business goal belongs to causal inversion. If so, the first indicator feature may be discarded, and the remaining first indicator features among the several first indicator features may be used as one or more first indicator features corresponding to the business goal.

[0039] In some embodiments, the taking of the remaining first indicator features in the at least one first indicator feature as one or more first indicator features corresponding to the business objective includes: dividing the remaining first indicator features in the at least one first indicator feature into one or more indicator feature sets based on the correlation between the remaining first indicator features in the at least one first indicator feature, wherein the correlation between the first indicator features in each indicator feature set is greater than or equal to a predetermined threshold; for each indicator feature set, retaining only the first indicator feature in the indicator feature set that has the highest matching degree with the business objective; and taking the remaining first indicator features in the one or more indicator feature sets as the one or more first indicator features corresponding to the business objective. In some embodiments, it is first necessary to calculate the correlation (for example, Pearson correlation) between the remaining first indicator features in the at least one first indicator feature, and divide several first indicator features whose correlation is greater than or equal to a predetermined threshold (for example, 0.8) into an indicator feature set, thereby obtaining one or more indicator feature sets, each indicator feature set includes several first indicator features, and the correlation between these several first indicator features is greater than or equal to the predetermined threshold. Then, for each indicator feature set, the matching degree between each first indicator feature in the indicator feature set and the business goal is measured in a quantitative manner, and only the first indicator feature with the highest matching degree in the indicator feature set is retained. Then, the remaining first indicator features in the one or more indicator feature sets are used as the one or more first indicator features corresponding to the business goal.

[0040] In some embodiments, the user scale corresponding to the at least one second indicator feature reaches a predetermined scale threshold, and the historical business effect corresponding to the at least one second indicator feature reaches a predetermined effect threshold. In some embodiments, each of the at least one second indicator feature needs to satisfy the following conditions: the user scale associated with the second indicator feature is greater than or equal to the predetermined scale threshold, and the quantified average business effect that the second indicator feature has historically produced on the business objective is greater than or equal to the predetermined effect threshold.

[0041] In some embodiments, the comprehensive coefficient information corresponding to the at least one second indicator feature is greater than or equal to a predetermined coefficient threshold, wherein the comprehensive coefficient information is determined based on the user quantity scale and historical business performance corresponding to the at least one second indicator feature. In some embodiments, for each of the at least one second indicator feature, the user quantity scale involved in the second indicator feature and the quantified average business performance that the second indicator feature has historically produced on the business objective can be input into a predetermined functional relationship, and the output of the functional relationship is used as the comprehensive coefficient corresponding to the second indicator feature, and the comprehensive coefficient needs to satisfy the requirement that it is greater than or equal to the predetermined coefficient threshold.

[0042] In some embodiments, the method further includes: the computer device determines the comprehensive coefficient information based on the user quantity scale and first weight information corresponding to the at least one second indicator feature, and the historical business effect and second weight information corresponding to the at least one second indicator feature. In some embodiments, for each of the at least one second indicator feature, the product of the user quantity scale involved in the second indicator feature and the predetermined first weight value can be used as the first value, the product of the quantified average business effect that the second indicator feature has historically produced on the business goal and the predetermined second weight value can be used as the second value, and then the sum of the first value and the second value is used as the comprehensive coefficient corresponding to the second indicator feature, and the comprehensive coefficient needs to satisfy that it is greater than or equal to a predetermined coefficient threshold.

[0043] In some embodiments, the one or more first indicator features are evaluated based on the statistical results corresponding to multiple dimensions, and at least one second indicator feature is determined from the one or more first indicator features based on the evaluation results, including: constructing a coordinate space based on the user quantity scale dimension and the historical business effect dimension, mapping the one or more first indicator features to at least one coordinate point in the coordinate space based on the user quantity scale and the historical business effect corresponding to the one or more first indicator features; determining the first indicator feature corresponding to the coordinate point in a predetermined area in the coordinate space as at least one second indicator feature. In some embodiments, a coordinate space can be constructed based on the user quantity scale dimension and the historical business effect dimension, and for each first indicator feature of the one or more first indicator features, the user quantity scale involved in the first indicator feature is used as a coordinate axis, and the quantified average business effect that the first indicator feature has historically produced on the business goal is used as another coordinate axis, thereby mapping the first indicator feature to a coordinate point in the coordinate space. As an example, if Figure 2As shown, at least one first indicator feature corresponding to at least one coordinate point (e.g., coordinate point-function / behavior 1, coordinate point-function / behavior 2) located in a predetermined area (e.g., the dotted area in the upper right corner) in the coordinate space can be determined as a second indicator feature.

[0044] In some embodiments, the feature evaluation model is a Shapley value model. In some embodiments, the feature evaluation model may be a Shapley value model. For each second indicator feature, the second indicator feature may be input into the Shapley value model. The Shapley value model outputs a marginal effect, and may clearly indicate whether the effect is positive or negative. The importance of the second indicator feature to the business objective may be assessed based on the magnitude of the marginal effect.

[0045] In some embodiments, there are multiple feature evaluation models; wherein the multiple models include one of the following: a Shapley value model; a feature importance tree model; and a feature relevance tree model; wherein inputting the at least one second indicator feature into a predetermined feature evaluation model to obtain importance information corresponding to one or more second indicator features includes: inputting the at least one second indicator feature into a predetermined plurality of feature evaluation models, and determining importance information corresponding to the one or more second indicator features based on output results of the plurality of feature evaluation models. In some embodiments, multiple feature evaluation models may be used, including but not limited to a Shapley value model, a feature importance tree model, and a feature relevance tree model. In some embodiments, for each second indicator feature, the second indicator feature can be input into each feature evaluation model respectively to obtain the output results of each feature evaluation model, wherein the Shapley value model outputs the marginal effect, and the first importance of the second indicator feature to the business goal can be determined based on the numerical value of the marginal effect. Then the feature correlation tree model outputs the correlation value between the second indicator feature and the business goal, and the second importance of the second indicator feature to the business goal can be determined based on the correlation value. Then the feature importance tree model directly outputs the third importance of the second indicator feature to the business goal, and then the importance of the second indicator feature to the business goal is comprehensively evaluated based on the first importance, the second importance and the third importance. For example, the average of these three importances is used as the importance information corresponding to the second indicator feature. For another example, the sum of the products of each importance and the predetermined weight value of the corresponding feature evaluation model can also be used as the importance information corresponding to the second indicator feature.

[0046] In some embodiments, determining the importance information corresponding to one or more second indicator features based on the output results of the multiple feature evaluation models includes: inputting the at least one second indicator feature into a Shapley value model to obtain the influence direction information corresponding to the at least one second indicator feature output by the Shapley value model; determining one or more third indicator features from the at least one second indicator feature based on the influence direction information, inputting the one or more third indicator features into a feature correlation tree model, and obtaining the correlation coefficients corresponding to the one or more third indicator features output by the feature correlation tree model; determining one or more second indicator features from the one or more third indicator features based on the correlation coefficients, inputting the one or more second indicator features into a feature importance tree model, and obtaining the importance information corresponding to the one or more second indicator features output by the feature importance tree model. In some embodiments, for each second indicator feature, the second indicator feature is first input into the Shapley value model, and the Shapley value model will output the direction of the impact of the second indicator feature on the business goal. If the impact direction is negative, the second indicator feature is discarded. If the impact direction is positive, the second indicator feature is further input into the correlation tree model, and the correlation tree model outputs the correlation coefficient between the second indicator feature and the business goal. If the correlation coefficient is less than a predetermined threshold, the second indicator feature is discarded. If the correlation coefficient is greater than or equal to the predetermined threshold, the second indicator feature is further input into the feature importance tree model, and the feature importance tree model outputs the importance information corresponding to the second indicator feature, that is, the importance of the second indicator feature to the business goal.

[0047] In some embodiments, the determining of the critical value information corresponding to the one or more target indicator features includes: for each target indicator feature, determining the inflection point on the historical effect curve of the business target corresponding to the target indicator feature; and using the value information corresponding to the inflection point as the critical value information corresponding to the target indicator feature. In some embodiments, for each target indicator feature, the value information corresponding to the target indicator feature can be used as the horizontal coordinate, and the quantified historical business effect of the target indicator feature on the business target under each value information can be used as the vertical coordinate to construct a historical effect curve of the business target corresponding to the target indicator feature. Then, it is necessary to determine the inflection point on the curve. The inflection point can refer to a point on the curve where the corresponding tangent direction is horizontal, or it can also refer to a point on the curve where the angle between the corresponding tangent direction and the horizontal direction is less than or equal to a predetermined angle threshold. Then, the value information corresponding to the inflection point is used as the critical value information corresponding to the target indicator feature. As an example, if Figure 3As shown, the business goal is to "increase the retention rate of new users on the next day", and the target indicator feature refers to the number of views of a page in the target application on the same day by users. The various value information of the views (for example, the number of views is greater than or equal to 2, the number of views is greater than or equal to 3, and the number of views is greater than or equal to 20) is used as the horizontal axis, and the quantified historical business effect of the target indicator feature on the business goal under each value information (for example, the retention rate is 35.0%, the retention rate is 41.8%, and the retention rate is 64.1%) is used as the vertical axis to construct a curve, and then the value information corresponding to the inflection point on the curve (for example, the horizontal axis of the inflection point is "the number of views is greater than or equal to 20", and the vertical axis of the inflection point is "the retention rate is 64.1%") is used as the critical value information corresponding to the target indicator feature.

[0048] In some embodiments, there are multiple inflection points; wherein the third importance of the second indicator feature to the business objective includes: determining a target inflection point from the multiple inflection points; and using the value information corresponding to the target inflection point as the critical value information corresponding to the target indicator feature. In some embodiments, if there are multiple inflection points on the curve, the value information corresponding to the first inflection point among the multiple inflection points can be used as the critical value information corresponding to the target indicator feature in order of the positive direction of the horizontal coordinate.

[0049] In some embodiments, determining the target inflection point from the multiple inflection points includes determining the target inflection point from the multiple inflection points based on the scale of the number of users of the value information corresponding to each inflection point. In some embodiments, the value information corresponding to the inflection point whose corresponding value information among the multiple inflection points has the largest scale of users may be used as the critical value information corresponding to the target indicator feature. Alternatively, the value information corresponding to the inflection point whose first corresponding value information among the multiple inflection points has a scale of users greater than or equal to a predetermined scale threshold may be used as the critical value information corresponding to the target indicator feature in the order of the positive direction of the horizontal axis.

[0050] In some embodiments, determining the critical value information corresponding to the one or more target indicator features includes: for each target indicator feature, determining the critical value information corresponding to the target indicator feature based on a historical business target effect curve corresponding to the target indicator feature and a user quantity scale curve corresponding to the target indicator feature. In some embodiments, for each target indicator feature, a business target historical effect curve corresponding to the target indicator feature can be constructed using the value information corresponding to the target indicator feature as the horizontal coordinate and the quantified historical business effect of the target indicator feature on the business target under each value information as the vertical coordinate. A user quantity scale curve corresponding to the target indicator feature is also constructed using the value information corresponding to the target indicator feature as the horizontal coordinate and the user quantity scale involved in each value information of the target indicator feature as the vertical coordinate, or using the value obtained by dividing the user quantity scale by a predetermined scaling factor as the vertical coordinate. Then, the critical value information corresponding to the target indicator feature is determined based on the two curves. For example, the intersection of the two curves can be used as the critical value information corresponding to the target indicator feature.

[0051] Figure 4 A computer device structure diagram for determining a magic number according to one embodiment of the present application is shown. The device includes a first module 11, a second module 12, a third module 13, and a fourth module 14. Module 11 is configured to determine one or more first indicator features corresponding to a business objective based on product function information of a target application and historical user behavior performance information. Module 12 is configured to evaluate the one or more first indicator features based on statistical results corresponding to multiple dimensions, and determine at least one second indicator feature from the one or more first indicator features based on the evaluation results, wherein the multiple dimensions include a user quantity dimension and a historical business performance dimension. Module 13 is configured to input the at least one second indicator feature into a predetermined feature evaluation model to obtain importance information corresponding to the one or more second indicator features, and determine one or more target indicator features from the one or more second indicator features based on the importance information. Module 14 is configured to determine critical value information corresponding to the one or more target indicator features, and use the critical value information as the magic number corresponding to the business objective.

[0052] Module 11 is used to determine one or more first indicator features corresponding to the business goal based on the product function information of the target application and the user's historical behavior performance information. In some embodiments, the user first needs to specify a business goal, that is, to determine the goal that the business expects to achieve, such as "improving the retention rate of new users the next day after opening the app", and then needs to exhaustively list the product functions of the target application and the user's historical behavior performance in the target application, and then perform data cleaning, feature engineering, etc. to generate corresponding one or more feature data, and then perform feature correlation analysis to analyze the correlation between the one or more feature data and the business goal, and then perform feature screening based on the analysis results, and screen out at least one feature data that has a non-causal inversion type correlation with the business goal from the one or more feature data, and then perform new feature construction, and construct corresponding indicator features for the at least one feature data according to the definition or form of the indicator, thereby obtaining one or more first indicator features corresponding to the business goal, wherein there is a correlation between the one or more first indicator features and the business goal, and the correlation is not of the causal inversion type, or there is a correlation between the one or more first indicator features and the business goal that is greater than or equal to a predetermined correlation degree, and the correlation is not of the causal inversion type.

[0053] Module 12 is configured to evaluate the one or more first indicator features based on the statistical results corresponding to multiple dimensions, and determine at least one second indicator feature from the one or more first indicator features based on the evaluation results, wherein the multiple dimensions include the user quantity scale dimension and the historical business effect dimension. In some embodiments, the usage of the one or more first indicator features is evaluated by referring to the external statistical results corresponding to multiple dimensions including the user quantity scale dimension and the historical business effect dimension, that is, for each first indicator feature, the first indicator feature is evaluated based on the user quantity scale and historical business effect corresponding to the first indicator feature to evaluate whether the first indicator feature meets the North Star indicator condition. The North Star indicator condition mainly includes two: the first is that the number of users it targets should not be too small, and the second is that the historical effect it has on the business goal should not be too small, wherein the user quantity scale can refer to the number of users involved in the first indicator feature, or it can also refer to the proportion of the number of users involved in the first indicator feature relative to the total number of users of the target application, and the historical business effect refers to the quantified average business effect that the first indicator feature has historically produced on the business goal. In some embodiments, based on the evaluation results, at least one second indicator feature that meets the North Star indicator condition is determined from the one or more first indicator features.

[0054] Module 13 is used to input the at least one second indicator feature into a predetermined feature evaluation model, obtain the importance information corresponding to the one or more second indicator features, and determine one or more target indicator features from the one or more second indicator features based on the importance information. In some embodiments, each second indicator feature of the at least one second indicator feature is input into a trained feature evaluation model to obtain the importance information corresponding to the second indicator feature output by the feature evaluation model, wherein the importance information is used to quantify the importance of the second indicator feature to the business goal. In some embodiments, the second indicator features of the one or more second indicator features whose corresponding importance is higher than or equal to a predetermined threshold can be used as target indicator features, or a predetermined number of second indicator features with the highest corresponding importance among the one or more second indicator features can also be used as target indicator features. In some embodiments, the one or more target indicator features are used as North Star indicators of business goals.

[0055] Module 14 is configured to determine critical value information corresponding to the one or more target indicator features, and use the critical value information as the magic number corresponding to the business objective. In some embodiments, for each target indicator feature, further refinement of the specific behavioral performance is required. The critical value information corresponding to the target indicator feature can be determined using a method similar to finding an inflection point. The critical value information can refer to all value information corresponding to the target indicator feature (wherein the value information can be a specific numerical value or a numerical range. For example, if the target indicator feature is the number of page views of a certain page in the target application, the corresponding value information can include a page view of 2, a page view of 3, a page view of 4, etc., or the corresponding value information can also include a page view of greater than or equal to 2, a page view of greater than or equal to 3, a page view of greater than or equal to 4, etc.) in ascending order or descending order, starting from the critical value information, as the subsequent value information changes, the degree of change in the quantified historical business effect of the target indicator feature with respect to the business objective under the subsequent value information is less than or equal to a predetermined threshold, i.e., the degree of change in the corresponding quantified historical business effect begins to slow down. For example, a curve can be constructed using the value information corresponding to the target indicator feature as the horizontal coordinate, and the quantified historical business effect of the target indicator feature on the business goal under each value information as the vertical coordinate. According to the positive direction of the horizontal coordinate, the horizontal coordinate value corresponding to the first point on the curve whose corresponding tangent direction is horizontal is used as the critical value information, or the horizontal coordinate value corresponding to the first point on the curve whose angle between the tangent direction and the horizontal direction is less than or equal to a predetermined angle threshold is used as the critical value information. For example, if the business goal is "increasing the retention rate of new users the next day after opening the app", the target indicator feature can be "video viewing time", and the critical value information corresponding to the target indicator feature can be "2 hours". The quantified historical business effect of the target indicator feature on the business goal under the critical value information is "retention rate of 64%". When the target indicator feature is greater than the critical value information, the quantified historical business effect of the target indicator feature on the business goal fluctuates around "retention rate of 64%", and the fluctuation range is within the predetermined interval threshold. This application can implement a universal data science methodology for mining business North Star indicators. All key behavioral performances can be quantified and measured, and are basically applicable to all Internet and non-Internet related businesses. It can combine the characteristics of macro analysis and micro analysis. Usually, macro analysis methods are used first to determine key indicators (to reduce the impact of inverted causality), and then micro analysis methods are used to find the magic number.

[0056] In some embodiments, the module 11 is used to: determine at least one first indicator feature associated with the business goal based on the product function information of the target application and the historical behavior performance information of the user; and determine one or more first indicator features corresponding to the business goal from the at least one first indicator feature. Figure 1 The embodiments shown are the same or similar and therefore will not be described in detail, but are incorporated herein by reference.

[0057] In some embodiments, the step of determining one or more first indicator features corresponding to the business objective from the at least one first indicator feature includes: for each first indicator feature in the at least one first indicator feature, determining whether the correlation between the first indicator feature and the business objective is a case of causal inversion; if so, discarding the first indicator feature; and using the remaining first indicator features in the at least one first indicator feature as the one or more first indicator features corresponding to the business objective. Figure 1 The embodiments shown are the same or similar and therefore will not be described in detail, but are incorporated herein by reference.

[0058] In some embodiments, the method of using the remaining first indicator features in the at least one first indicator feature as one or more first indicator features corresponding to the business objective includes: dividing the remaining first indicator features into one or more indicator feature sets according to the correlation between the remaining first indicator features in the at least one first indicator feature, wherein the correlation between the first indicator features in each indicator feature set is greater than or equal to a predetermined threshold; for each indicator feature set, retaining only the first indicator feature in the indicator feature set with the highest matching degree with the business objective; and using the remaining first indicator features in the one or more indicator feature sets as one or more first indicator features corresponding to the business objective. Here, the relevant operations are similar to Figure 1 The embodiments shown are the same or similar and therefore will not be described in detail, but are incorporated herein by reference.

[0059] In some embodiments, the user quantity scale corresponding to the at least one second indicator feature reaches a predetermined scale threshold, and the historical business effect corresponding to the at least one second indicator feature reaches a predetermined effect threshold. Figure 1 The embodiments shown are the same or similar and therefore will not be described in detail, but are incorporated herein by reference.

[0060] In some embodiments, the comprehensive coefficient information corresponding to the at least one second indicator feature is greater than or equal to a predetermined coefficient threshold, wherein the comprehensive coefficient information is determined based on the user quantity scale and historical business performance corresponding to the at least one second indicator feature. Figure 1 The embodiments shown are the same or similar and therefore will not be described in detail, but are incorporated herein by reference.

[0061] In some embodiments, the device is further configured to determine the comprehensive coefficient information based on the user quantity scale and first weight information corresponding to the at least one second indicator feature, and the historical business effect and second weight information corresponding to the at least one second indicator feature. Figure 1 The embodiments shown are the same or similar and therefore will not be described in detail, but are incorporated herein by reference.

[0062] In some embodiments, the one or more first indicator features are evaluated based on the statistical results corresponding to multiple dimensions, and at least one second indicator feature is determined from the one or more first indicator features based on the evaluation results, including: constructing a coordinate space based on the user quantity scale dimension and the historical business effect dimension, mapping the one or more first indicator features to at least one coordinate point in the coordinate space based on the user quantity scale and historical business effect corresponding to the one or more first indicator features; determining the first indicator feature corresponding to the coordinate point in a predetermined area in the coordinate space as at least one second indicator feature. Here, the relevant operations are the same as Figure 1 The embodiments shown are the same or similar and therefore will not be described in detail, but are incorporated herein by reference.

[0063] In some embodiments, the feature evaluation model is a Shapley value model. Figure 1 The embodiments shown are the same or similar and therefore will not be described in detail, but are incorporated herein by reference.

[0064] In some embodiments, there are multiple feature evaluation models; wherein the multiple models include one of the following: Shapley value model; feature importance tree model; feature correlation tree model; wherein, inputting the at least one second indicator feature into a predetermined feature evaluation model to obtain the importance information corresponding to one or more second indicator features includes: inputting the at least one second indicator feature into a predetermined plurality of feature evaluation models, and determining the importance information corresponding to one or more second indicator features based on the output results of the plurality of feature evaluation models. Here, the relevant operations are the same as Figure 1 The embodiments shown are the same or similar and therefore will not be described in detail, but are incorporated herein by reference.

[0065] In some embodiments, the determining of the importance information corresponding to one or more second indicator features based on the output results of the multiple feature evaluation models includes: inputting the at least one second indicator feature into the Shapley value model to obtain the influence direction information corresponding to the at least one second indicator feature output by the Shapley value model; determining one or more third indicator features from the at least one second indicator feature based on the influence direction information, inputting the one or more third indicator features into the feature correlation tree model, and obtaining the correlation coefficients corresponding to the one or more third indicator features output by the feature correlation tree model; determining one or more second indicator features from the one or more third indicator features based on the correlation coefficients, inputting the one or more second indicator features into the feature importance tree model, and obtaining the importance information corresponding to the one or more second indicator features output by the feature importance tree model. Here, the related operations are the same as Figure 1 The embodiments shown are the same or similar and therefore will not be described in detail, but are incorporated herein by reference.

[0066] In some embodiments, determining the critical value information corresponding to the one or more target indicator features includes: for each target indicator feature, determining the inflection point on the business target historical effect curve corresponding to the target indicator feature; and using the value information corresponding to the inflection point as the critical value information corresponding to the target indicator feature. Figure 1 The embodiments shown are the same or similar and therefore will not be described in detail, but are incorporated herein by reference.

[0067] In some embodiments, there are multiple inflection points; wherein the third importance of the second indicator feature to the business goal includes: determining a target inflection point among the multiple inflection points; and using the value information corresponding to the target inflection point as the critical value information corresponding to the target indicator feature. Figure 1 The embodiments shown are the same or similar and therefore will not be described in detail, but are incorporated herein by reference.

[0068] In some embodiments, determining the target inflection point from the plurality of inflection points includes: determining the target inflection point from the plurality of inflection points based on the number of users of the value information corresponding to each inflection point. Figure 1 The embodiments shown are the same or similar and therefore will not be described in detail, but are incorporated herein by reference.

[0069] In some embodiments, the step of determining the critical value information corresponding to the one or more target indicator features includes: for each target indicator feature, determining the critical value information corresponding to the target indicator feature based on the business target historical effect curve corresponding to the target indicator feature and the user quantity scale curve corresponding to the target indicator feature. Figure 1 The embodiments shown are the same or similar and therefore will not be described in detail, but are incorporated herein by reference.

[0070] In addition to the methods and devices described in the above embodiments, the present application also provides a computer-readable storage medium, which stores computer code. When the computer code is executed, the method described in any of the above items is executed.

[0071] The present application also provides a computer program product. When the computer program product is executed by a computer device, the method described in any one of the preceding items is executed.

[0072] The present application also provides a computer device, comprising:

[0073] one or more processors;

[0074] a memory for storing one or more computer programs;

[0075] When the one or more computer programs are executed by the one or more processors, the one or more processors are caused to implement the method as described in any one of the preceding items.

[0076] Figure 5 shows an exemplary system that can be used to implement various embodiments described in this application;

[0077] like Figure 5 In some embodiments, the system 300 can function as any of the devices described in the various embodiments. In some embodiments, the system 300 can include one or more computer-readable media (e.g., system memory or NVM / storage device 320) having instructions and one or more processors (e.g., processor(s) 305) coupled to the one or more computer-readable media and configured to execute the instructions to implement the modules and thereby perform the actions described herein.

[0078] For one embodiment, system control module 310 may include any suitable interface controller to provide any suitable interface to at least one of processor(s) 305 and / or any suitable device or component in communication with system control module 310 .

[0079] The system control module 310 may include a memory controller module 330 to provide an interface to the system memory 315. The memory controller module 330 may be a hardware module, a software module, and / or a firmware module.

[0080] System memory 315 can be used, for example, to load and store data and / or instructions for system 300. For one embodiment, system memory 315 can include any suitable volatile memory, such as a suitable DRAM. In some embodiments, system memory 315 can include double data rate type four synchronous dynamic random access memory (DDR4 SDRAM).

[0081] For one embodiment, system control module 310 may include one or more input / output (I / O) controllers to provide interfaces to NVM / storage device 320 and communication interface(s) 325 .

[0082] For example, NVM / storage 320 may be used to store data and / or instructions. NVM / storage 320 may include any suitable non-volatile memory (e.g., flash memory) and / or may include any suitable non-volatile storage device(s) (e.g., one or more hard disk drives (HDDs), one or more compact disk (CD) drives, and / or one or more digital versatile disk (DVD) drives).

[0083] NVM / storage device 320 may include storage resources that are physically part of the device on which system 300 is installed, or it may be accessible to the device without being part of the device. For example, NVM / storage device 320 may be accessed over a network via communication interface(s) 325.

[0084] Communication interface(s) 325 may provide an interface for system 300 to communicate over one or more networks and / or with any other suitable devices. System 300 may wirelessly communicate with one or more components of a wireless network in accordance with any of one or more wireless network standards and / or protocols.

[0085] For one embodiment, at least one of the processor(s) 305 may be packaged together with the logic of one or more controllers of the system control module 310 (e.g., the memory controller module 330). For one embodiment, at least one of the processor(s) 305 may be packaged together with the logic of one or more controllers of the system control module 310 to form a system-in-package (SiP). For one embodiment, at least one of the processor(s) 305 may be integrated on the same die with the logic of one or more controllers of the system control module 310. For one embodiment, at least one of the processor(s) 305 may be integrated on the same die with the logic of one or more controllers of the system control module 310 to form a system-on-chip (SoC).

[0086] In various embodiments, system 300 may be, but is not limited to, a server, a workstation, a desktop computing device, or a mobile computing device (e.g., a laptop computing device, a handheld computing device, a tablet computer, a netbook, etc.). In various embodiments, system 300 may have more or fewer components and / or a different architecture. For example, in some embodiments, system 300 includes one or more cameras, a keyboard, a liquid crystal display (LCD) screen (including a touchscreen display), a non-volatile memory port, multiple antennas, a graphics chip, an application-specific integrated circuit (ASIC), and a speaker.

[0087] It should be noted that the application can be implemented in software and / or a combination of software and hardware, for example, can be implemented using an application specific integrated circuit (ASIC), a general purpose computer or any other similar hardware device. In one embodiment, the software program of the application can be executed by a processor to realize the steps or functions described above. Similarly, the software program of the application (including relevant data structures) can be stored in a computer-readable recording medium, for example, a RAM memory, a magnetic or optical drive or a floppy disk and similar devices. In addition, some steps or functions of the application can be implemented using hardware, for example, as a circuit that cooperates with a processor to perform each step or function.

[0088] In addition, a part of the present application may be applied as a computer program product, such as a computer program instruction, which, when executed by a computer, can call or provide the method and / or technical solution according to the present application through the operation of the computer. Those skilled in the art should understand that the form in which the computer program instruction exists in a computer-readable medium includes but is not limited to a source file, an executable file, an installation package file, etc. Accordingly, the way in which the computer program instruction is executed by the computer includes but is not limited to: the computer directly executes the instruction, or the computer compiles the instruction and then executes the corresponding compiled program, or the computer reads and executes the instruction, or the computer reads and installs the instruction and then executes the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium that can be accessed by the computer.

[0089] Communication media include media by which communication signals containing, for example, computer-readable instructions, data structures, program modules, or other data are transmitted from one system to another. Communication media may include guided transmission media such as cables and wires (e.g., fiber optic, coaxial, etc.) and wireless (unguided transmission) media capable of propagating energy waves, such as acoustic, electromagnetic, RF, microwave, and infrared. Computer-readable instructions, data structures, program modules, or other data may be embodied as, for example, a modulated data signal in a wireless medium such as a carrier wave or similar mechanism such as that embodied as part of spread spectrum technology. The term "modulated data signal" refers to a signal that has one or more of its characteristics changed or set in such a manner as to encode information in the signal. Modulation may be analog, digital, or a hybrid modulation technique.

[0090] By way of example and not limitation, computer-readable storage media may include volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data. For example, computer-readable storage media include, but are not limited to, volatile memory, such as random access memory (RAM, DRAM, SRAM); and non-volatile memory, such as flash memory, various read-only memories (ROM, PROM, EPROM, EEPROM), magnetic and ferromagnetic / ferroelectric memories (MRAM, FeRAM); and magnetic and optical storage devices (hard disks, magnetic tapes, CDs, DVDs); or other media now known or later developed that can store computer-readable information / data for use by a computer system.

[0091] Here, according to one embodiment of the present application, a device is included, which includes a memory for storing computer program instructions and a processor for executing the program instructions, wherein, when the computer program instructions are executed by the processor, the device is triggered to run the methods and / or technical solutions based on the aforementioned multiple embodiments of the present application.

[0092] It is obvious to those skilled in the art that the present application is not limited to the details of the above-mentioned exemplary embodiments, and that the present application can be implemented in other specific forms without departing from the spirit or basic characteristics of the present application. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the present application is defined by the appended claims rather than the above description, and it is intended that all changes that fall within the meaning and scope of the equivalent elements of the claims are included in the present application. Any figure mark in the claims should not be regarded as limiting the claims involved. In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices stated in the device claim can also be implemented by one unit or device through software or hardware. Words such as first and second are used to indicate names and do not indicate any particular order.

Claims

1. A method for determining a magic number, wherein The method includes: Determining one or more first indicator features corresponding to the business objective based on product function information of the target application and historical user behavior performance information, wherein there is a correlation between the one or more first indicator features and the business objective, and the correlation is not a causal inversion type; Constructing a coordinate space based on a user quantity scale dimension and a historical business effect dimension, and mapping the one or more first indicator features to at least one coordinate point in the coordinate space according to the user quantity scale and the historical business effect corresponding to the one or more first indicator features; determining a first indicator feature corresponding to a coordinate point in a predetermined area in the coordinate space as at least one second indicator feature; Inputting the at least one second indicator feature into a predetermined feature evaluation model to obtain importance information corresponding to the one or more second indicator features, and determining one or more target indicator features from the one or more second indicator features based on the importance information; Critical value information corresponding to the one or more target indicator characteristics is determined, and the critical value information is used as a magic number corresponding to the business goal.

2. The method according to claim 1, wherein Determining one or more first indicator characteristics corresponding to the business goal based on the product function information of the target application and the historical behavior performance information of the user includes: Determining at least one first indicator feature associated with the business goal based on product function information of the target application and historical behavior performance information of the user; One or more first indicator features corresponding to the business objective are determined from the at least one first indicator feature.

3. The method according to claim 2, wherein: The determining of one or more first indicator features corresponding to the business objective from the at least one first indicator feature includes: For each of the at least one first indicator feature, determining whether the correlation between the first indicator feature and the business objective is a case of causal inversion; if so, discarding the first indicator feature; The remaining first indicator features in the at least one first indicator feature are used as one or more first indicator features corresponding to the business objective.

4. The method according to claim 3, wherein: The taking the remaining first indicator features of the at least one first indicator feature as the one or more first indicator features corresponding to the business objective includes: Dividing the remaining first indicator features into one or more indicator feature sets according to the correlation between the remaining first indicator features in the at least one first indicator feature, wherein the correlation between the first indicator features in each indicator feature set is greater than or equal to a predetermined threshold; For each indicator feature set, only the first indicator feature in the indicator feature set that has the highest matching degree with the business objective is retained; The remaining first indicator features in the one or more indicator feature sets are used as the one or more first indicator features corresponding to the business objectives.

5. The method according to claim 1, wherein The user quantity scale corresponding to the at least one second indicator feature reaches a predetermined scale threshold, and the historical business effect corresponding to the at least one second indicator feature reaches a predetermined effect threshold.

6. The method according to claim 1, wherein The comprehensive coefficient information corresponding to the at least one second indicator feature is greater than or equal to a predetermined coefficient threshold, wherein the comprehensive coefficient information is determined based on the user quantity scale and historical business performance corresponding to the at least one second indicator feature.

7. The method according to claim 6, wherein: The method further comprises: The comprehensive coefficient information is determined based on the user quantity scale and first weight information corresponding to the at least one second indicator feature, and the historical business performance and second weight information corresponding to the at least one second indicator feature.

8. The method according to claim 1, wherein The feature evaluation model is a Shapley value model.

9. The method according to claim 1, wherein The feature evaluation model is a plurality of models; The multiple models include: Shapley value model; Feature Importance Tree Model; Feature correlation tree model; The step of inputting the at least one second indicator feature into a predetermined feature evaluation model to obtain importance information corresponding to one or more second indicator features includes: The at least one second indicator feature is input into a plurality of predetermined feature evaluation models, and importance information corresponding to one or more second indicator features is determined based on output results of the plurality of feature evaluation models.

10. The method according to claim 9, wherein: Determining importance information corresponding to one or more second indicator features based on output results of the multiple feature evaluation models includes: Inputting the at least one second indicator feature into a Shapley value model to obtain influence direction information corresponding to the at least one second indicator feature output by the Shapley value model; Determining one or more third indicator features from the at least one second indicator feature according to the influence direction information, inputting the one or more third indicator features into a feature correlation tree model, and obtaining correlation coefficients corresponding to the one or more third indicator features output by the feature correlation tree model; One or more second indicator features are determined from the one or more third indicator features according to the correlation coefficient, and the one or more second indicator features are input into a feature importance tree model to obtain importance information corresponding to the one or more second indicator features output by the feature importance tree model.

11. The method according to claim 1, wherein Determining the critical value information corresponding to the one or more target indicator characteristics includes: For each target indicator feature, determine the inflection point on the business target historical effect curve corresponding to the target indicator feature; The value information corresponding to the inflection point is used as the critical value information corresponding to the target indicator feature.

12. The method according to claim 11, wherein There are multiple inflection points; The step of using the value information corresponding to the inflection point as the critical value information corresponding to the target indicator feature includes: determining a target inflection point among the plurality of inflection points; The value information corresponding to the target inflection point is used as the critical value information corresponding to the target indicator feature.

13. The method according to claim 12, wherein: Determining a target inflection point from the plurality of inflection points includes: A target inflection point is determined among the multiple inflection points according to the scale of the number of users of the value information corresponding to each inflection point.

14. The method according to claim 1, wherein Determining the critical value information corresponding to the one or more target indicator characteristics includes: For each target indicator feature, the critical value information corresponding to the target indicator feature is determined based on the business target historical effect curve corresponding to the target indicator feature and the user quantity scale curve corresponding to the target indicator feature.

15. A computer device for determining a magic number, comprising a memory, a processor, and a computer program stored in the memory, wherein: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 14.

16. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: The computer program / instructions, when executed, cause the system to perform the steps of the method as claimed in any one of claims 1 to 14.

17. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 14 are implemented.

Images