Model performance detection method, device, computer equipment and storage medium

Abstract

The present disclosure relates to a method, device, computer equipment and storage medium for detecting model performance. The method includes: obtaining each historical information display data stored in the information display system, wherein the historical information display data at least includes information of the information display position; determining multiple sample data from each historical information display data; retrieving display information corresponding to the information display position in each sample data through the information display system; determining whether the information combination consisting of the information display position and the retrieved display information exists in the historical information display data; if the information combination exists in the historical information display data, obtaining the interaction information corresponding to the information combination, wherein the interaction information is used to represent the interaction operation between the account and the display information; and determining the performance detection result of the model to be tested based on the interaction information. The present disclosure can improve the evaluation accuracy of the account feedback evaluation model.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a method, device, computer equipment and storage medium for detecting model performance. Background Art

[0002] At present, programmatic advertising has become the mainstream model of Internet advertising. The account feedback evaluation model based on machine learning technology is the core model of programmatic advertising. The account feedback evaluation model can estimate the account feedback rate of an advertisement when it is delivered to a certain information display position, and determine the delivery of the advertisement based on the account feedback rate. In order to improve the accuracy of the account feedback evaluation model, the information display system usually continues to train the account feedback evaluation model with new advertising data to obtain a new account feedback evaluation model. Before the new account feedback evaluation model is deployed online, it is necessary to detect whether it is more accurate than the old account feedback evaluation model. If the performance is improved, the new model will be launched online.

[0003] In the traditional evaluation scheme, the historical information display data in the information display system is divided into two parts. One part is the training set for training the account feedback evaluation model, and the other part is the test set for testing the account feedback evaluation model. After the training set and the test set are divided, the account feedback evaluation model can be trained with the historical information display data in the training set. After the training is completed, the account feedback evaluation model is tested with the historical advertisements in the test set to obtain the test results. Afterwards, the accuracy of the account feedback evaluation model can be evaluated based on the difference between the test results and the actual results.

[0004] However, traditional technologies use a large amount of historical information display data to train and test account feedback evaluation models, which has the disadvantage of inaccurate evaluation results. Summary of the invention

[0005] The present disclosure provides a method, device, computer equipment and storage medium for detecting model performance to solve the technical problem of low accuracy of model evaluation in traditional technologies. The technical solution of the present disclosure is as follows:

[0006] According to a first aspect of an embodiment of the present disclosure, a method for detecting model performance is provided, the method comprising:

[0007] Acquire various historical information display data stored in the information display system, wherein the historical information display data at least includes information of the information display position;

[0008] Determine a plurality of sample data from each of the historical information display data;

[0009] Retrieving display information corresponding to the information display position in each of the sample data through the information display system, wherein the information display system is provided with a model to be tested of the performance to be tested, and the model to be tested is used to perform one or more operations in the above retrieval process;

[0010] Determine whether an information combination consisting of the information display position and the retrieved display information exists in the historical information display data;

[0011] If the information combination exists in the historical information display data, then obtaining interaction information corresponding to the information combination, wherein the interaction information is used to represent an interaction operation between the account and the display information;

[0012] Based on the interaction information, a performance test result of the model to be tested is determined.

[0013] As an optional implementation manner, the historical information display data also includes display information, and the retrieving, through the information display system, the display information corresponding to the information display position in each of the sample data includes:

[0014] Through multiple retrieval algorithms set in the information display system, display information matching the characteristics of each information display position is retrieved from the information library; wherein one or more of the retrieval algorithms are set in the model to be tested.

[0015] As an optional implementation manner, the acquiring the interaction information corresponding to the information combination includes:

[0016] Obtaining the performance test dimension of the model to be tested;

[0017] Reading the attribute value of each information combination corresponding to the performance detection dimension;

[0018] An average value of the attribute values ​​of each information combination is determined as the attribute value of the interactive information.

[0019] As an optional implementation manner, determining the performance test result of the model to be tested based on the interaction information includes:

[0020] Determining whether the attribute value of the interactive information exceeds the target attribute value;

[0021] If the attribute value of the interactive information exceeds the target attribute value, determining that the performance test result of the model to be tested is qualified;

[0022] If the attribute value of the interactive information does not exceed the target attribute value, it is determined that the performance test result of the model to be tested is unqualified.

[0023] According to a second aspect of an embodiment of the present disclosure, a device for detecting model performance is provided, the device comprising:

[0024] A first acquisition module is configured to acquire various historical information display data stored in the information display system, wherein the historical information display data at least includes information of the information display position;

[0025] A first determining module is configured to determine a plurality of sample data from each of the historical information display data;

[0026] A retrieval module is configured to retrieve display information corresponding to the information display position in each of the sample data through the information display system, wherein a model to be tested of the performance to be tested is set in the information display system, and the model to be tested is used to perform one or more operations in the above-mentioned retrieval process;

[0027] A determination module, configured to determine whether an information combination consisting of the information display position and the retrieved display information exists in the historical information display data;

[0028] a second acquisition module, configured to acquire interaction information corresponding to the information combination if the information combination exists in the historical information display data, wherein the interaction information is used to represent an interaction operation between the account and the display information;

[0029] The second determination module is configured to determine the performance detection result of the model to be tested based on the interaction information.

[0030] As an optional implementation manner, the historical information display data also includes display information, and the retrieval module is specifically used to:

[0031] Through multiple retrieval algorithms set in the information display system, display information matching the characteristics of each information display position is retrieved from the information library; wherein one or more of the retrieval algorithms are set in the model to be tested.

[0032] As an optional implementation manner, the second acquisition module is specifically used to:

[0033] Obtaining the performance test dimension of the model to be tested;

[0034] Reading the attribute value of each information combination corresponding to the performance detection dimension;

[0035] An average value of the attribute values ​​of each information combination is determined as the attribute value of the interactive information.

[0036] As an optional implementation manner, the second determining module is specifically configured to:

[0037] Determining whether the attribute value of the interactive information exceeds the target attribute value;

[0038] If the attribute value of the interactive information exceeds the target attribute value, determining that the performance test result of the model to be tested is qualified;

[0039] If the attribute value of the interactive information does not exceed the target attribute value, it is determined that the performance test result of the model to be tested is unqualified.

[0040] According to a third aspect of an embodiment of the present disclosure, a computer device is provided, including:

[0041] processor;

[0042] a memory for storing instructions executable by the processor;

[0043] The processor is configured to execute the instructions to implement the method for detecting model performance as described in any one of the first aspects.

[0044] According to a fourth aspect of an embodiment of the present disclosure, a storage medium is provided. When instructions in the storage medium are executed by a processor of a computer device, the computer device is enabled to execute the method for detecting model performance as described in any one of the first aspects.

[0045] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, the program product comprising a computer program, the computer program being stored in a readable storage medium, at least one processor of a device reading and executing the computer program from the readable storage medium, so that the device executes the method for detecting model performance described in any one of the embodiments of the first aspect.

[0046] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0047] The computer device obtains each historical information display data stored in the information display system. The historical information display data at least includes information of the information display position. The computer device determines a plurality of sample data from each historical information display data, and retrieves the display information corresponding to the information display position in each sample data through the information display system. The information display system is provided with a model to be tested for performance to be tested, and the model to be tested is used to perform one or more operations in the above retrieval process. Then, the computer device determines whether the information combination consisting of the information display position and the retrieved display information exists in the historical information display data. If the information combination exists in the historical information display data, the computer device obtains the interaction information corresponding to the information combination, wherein the interaction information is used to represent the interaction operation between the account and the display information, and determines the performance test result of the model to be tested based on the interaction information. In this way, the model performance is tested by simulating the model online; at the same time, in the process of performance testing the model, the historical data is screened and filtered through the system containing the model to be tested, and the test results based on the data that meets the requirements are used to determine the performance of the model to be tested, which can improve the evaluation accuracy of the account feedback evaluation model.

[0048] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0050] Figure 1 is a flow chart of a method for detecting model performance according to an exemplary embodiment;

[0051] Figure 2 is a flow chart of a method for detecting model performance according to an exemplary embodiment;

[0052] Figure 3 is a flow chart of a method for detecting model performance according to an exemplary embodiment;

[0053] Figure 4 is a block diagram of a device for detecting model performance according to an exemplary embodiment;

[0054] Figure 5 The figure is a diagram showing the internal structure of a computer device according to an exemplary embodiment. DETAILED DESCRIPTION

[0055] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0056] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0057] The embodiment of the present disclosure provides a method for detecting model performance. The method for detecting model performance can be applied to an information display system to evaluate whether an account feedback evaluation model in the information display system meets a preset accuracy requirement.

[0058] Figure 1 is a flow chart of a method for detecting model performance according to an exemplary embodiment. Figure 1 As shown, the method includes the following steps.

[0059] Step 110: Acquire various historical information display data stored in the information display system.

[0060] The historical information display data at least includes information of the information display position.

[0061] In implementation, when a computer device needs to perform a performance test on a model to be tested of a performance to be tested in an information display system, the computer device can obtain historical information display data within a preset time period (such as 40 days) from the current time from the information display system. Among them, the historical information display data at least includes information of the information display position, and in the information display system, the display position is an advertising display position. Optionally, since the historical information display data is used to train the account feedback evaluation model, correspondingly, the historical information display data also includes account feedback. Among them, the account feedback can be account click feedback, account download feedback, or other types of account feedback, which are not limited in the embodiments of the present disclosure.

[0062] Step 120: determine a plurality of sample data from each historical information display data.

[0063] In implementation, after the computer device obtains the historical information display data, it can further select a preset number of sample data (i.e., a preset number of historical information display data) from the historical information display data. Optionally, the computer device can randomly select a preset number of sample data from the historical information display data, or select a preset number of sample data from the historical information display data according to a preset selection rule, or select a preset number of sample data from the historical information display data using other selection rules, which is not limited in the embodiments of the present disclosure.

[0064] For example, after the computer device obtains the historical information display data, it further selects three sample data from the historical information display data, namely sample data 1, sample data 2 and sample data 3, where the information display position in sample data 1 is P1, the information display position in sample data 2 is P2, and the information display position in sample data 1 is P3.

[0065] Step 130: retrieve the display information corresponding to the information display position in each sample data through the information display system.

[0066] Among them, a model to be tested with the performance to be tested is set in the information display system, and the model to be tested is used to perform one or more operations in the above-mentioned retrieval process.

[0067] During implementation, after the computer device determines multiple sample data from each historical information display data, for each sample data, the computer device can retrieve the display information corresponding to the information display position in the sample data from each historical information display data through the test model of the performance to be tested set in the information display system.

[0068] As an optional implementation, the historical information display data also includes display information. The specific processing process of the computer device retrieving the display information corresponding to the information display position in each sample data through the information display system is: through multiple retrieval algorithms set in the information display system, the display information matching the characteristics of each information display position is retrieved from the information library. Among them, one or more retrieval algorithms are set in the model to be tested.

[0069] In implementation, the historical information display data also includes display information. One or more retrieval algorithms are set in the model to be tested set in the information display system. After the computer device determines multiple sample data from each historical information display data, for each sample data, the computer device can retrieve the display information corresponding to the information display position in the sample data from each historical information display data through one or more retrieval algorithms in the model to be tested of the performance to be tested set in the information display system. Optionally, in addition to the information display position and display information, the historical information display data stored in the information display system may further include other types of information such as display time of the display information, account information, whether the bid is successful, whether to display, etc., which is not limited in the embodiments of the present disclosure.

[0070] For example, the historical information display data also includes display information A1, A2, and A3. The sample data selected by the computer device are sample data 1, sample data 2, and sample data 3, wherein the information display position in sample data 1 is P1, the information display position in sample data 2 is P2, and the information display position in sample data 3 is P3. For each sample data, the computer device can combine the display information (i.e., A1, A2, and A3) included in the historical information display data with the information display position in the sample data to obtain an information combination. For sample data 1, the information combination obtained is {P1, A1}, {P1, A2}, {P1, A3}, for sample data 2, the information combination obtained is {P2, A1}, {P2, A2}, {P2, A3}, and for sample data 3, the information combination obtained is {P3, A1}, {P3, A2}, {P3, A3}. After the computer device obtains multiple information combinations corresponding to each information display position, for each information display position, the computer device can input each information combination corresponding to the information display position into the model to be tested of the performance to be tested. Correspondingly, the model to be tested of the performance to be tested will output the account click probability of each information combination corresponding to the information display position. For example, the information display position in sample data 1 is information display position P1, and the information combinations corresponding to the information display position P1 are {P1, A1}, {P1, A2} and {P1, A3}, respectively. After the computer device inputs each information combination corresponding to the information display position P1 (that is, {P1, A1}, {P1, A2} and {P1, A3}) into the model to be tested of the performance to be tested, correspondingly, the model to be tested of the performance to be tested will output the account click probability of the information combination {P1, A1} corresponding to the information display position P1 as 0.6, the account click probability of the information combination {P1, A2} as 0.8, and the account click probability of the information combination {P1, A3} as 0.75. For each information display position, after the computer device obtains the account click probability of each information combination corresponding to the information display position, it can further determine the information combination with the largest account click probability among the information combinations corresponding to the information display position as the target information combination corresponding to the information display position. For example, the information display position is information display position P1, and the information combinations corresponding to the information display position are {P1, A1}, {P1, A2}, and {P1, A3}, respectively. The account click probabilities of each information combination corresponding to the information display position are 0.6, 0.8, and 0.75, respectively, and the target information combination corresponding to the information display position is {P1, A2}.

[0071] Step 140 , determining whether the information combination consisting of the information display position and the retrieved display information exists in the historical information display data.

[0072] In implementation, after the computer device retrieves the display information corresponding to the information display position in the sample data from each historical information display data, for each information display position of the sample data, the computer device can further determine whether the information combination consisting of the information display position corresponding to the sample data and the retrieved display information exists in each historical information display data. If the information combination consisting of the information display position corresponding to the sample data and the retrieved display information exists in each historical information display data, step 150 is executed.

[0073] Step 150: If the information combination exists in the historical information display data, then the interaction information corresponding to the information combination is obtained.

[0074] The interactive information is used to represent the interactive operation between the account and the displayed information.

[0075] In implementation, if the information combination consisting of the information display position corresponding to the sample data and the retrieved display information exists in each historical information display data, the computer device can further obtain the interaction information corresponding to the information combination. The interaction information is used to represent the interaction operation between the account and the display information.

[0076] As an optional implementation, Figure 2 As shown, in step 150, the processing process of the computer device acquiring the interactive information corresponding to the information combination is as follows.

[0077] Step 151, obtaining the performance test dimension of the model to be tested.

[0078] In implementation, if the information display position corresponding to the sample data and the information combination consisting of the retrieved display information exist in each historical information display data, the computer device can further obtain the performance detection dimension of the model to be tested. Among them, the performance detection dimension can be an account click rate or an account download rate, and can also be other detection dimensions, which are not limited in the embodiments of the present disclosure.

[0079] Step 152, read the attribute value of each information combination corresponding to the performance detection dimension.

[0080] In implementation, after the computer device obtains the performance detection dimension of the model to be tested, for each information combination, the computer device can further read the attribute value of the performance detection dimension corresponding to the information combination. For example, if the detection dimension is the account click rate, the computer device can read the account click rate corresponding to the information combination, and the attribute value can also be the account download rate, etc., which is not limited in the embodiments of the present disclosure.

[0081] Step 153: Determine the average value of the attribute values ​​of each information combination as the attribute value of the interactive information.

[0082] In implementation, after the computer device reads the attribute value of each information combination corresponding to the performance detection dimension, it can further determine the average value of the attribute value of each information combination as the attribute value of the interactive information.

[0083] For example, for each information display position corresponding to each sample data, after obtaining the target information combination corresponding to the information display position, the computer device can further determine whether the target information combination corresponding to the information display position exists in the historical information display data stored in the information display system.

[0084] If the target information combination corresponding to the information display position exists in the historical information display data stored in the information display system, the computer device can use the ratio of the number of the target information combination whose account click type is account clicked in the historical information display data to the total number of the target information combination as the account click probability of the target information combination corresponding to the information display position. For example, the information display position is information display position P1, and the target information combination corresponding to the information display position is {P1, A2}. The historical information display data stored in the information display system includes 1000 target information combinations {P1, A2}, of which 700 target information combinations {P1, A2} have an account click type of account clicked, and the account click probability of the target information combination {P1, A2} is 700 / 1000=0.7. For another example, the information display position is information display position P2, and the target information combination corresponding to the information display position is {P2, A1}. The historical information display data stored in the information display system includes 2000 target information combinations {P2, A1}, of which the account click type of 1000 target information combinations {P2, A1} is account clicked, then the account click probability of the target information combination {P2, A1} is 1000 / 2000=0.5.

[0085] If the target information combination corresponding to the information display position does not exist in the historical information display data stored in the information display system, the computer device may use the ratio of the number of historical information display data whose account click type is account clicked in the historical information display data to the total number of historical information display data as the account click probability of the target information combination corresponding to the information display position. For example, the information display position is information display position P1, and the target information combination corresponding to the information display position is {P1, A2}. There are 10,000 pieces of historical information display data stored in the information display system, of which 3,000 pieces of historical information display data have an account click type of account clicked, then the account click probability of the target information combination {P1, A2} is 3000 / 10,000=0.3. For another example, the information display position is information display position P2, and the target information combination corresponding to the information display position is {P2, A1}. There are 10,000 pieces of historical information display data stored in the information display system, among which the account click type of 3,000 pieces of historical information display data is account clicked, then the account click probability of the target information combination {P2, A1} is also 3000 / 10000=0.3. After the computer device obtains the account click probability of each target information combination, the average value of the account click probability of each information combination can be determined as the attribute value of the interactive information. The computer device can also only determine the average value of the account click probability corresponding to each information display position existing in the historical information display data stored in the information display system as the attribute value of the interactive information. The computer device can also determine the detection value of the model to be tested of the performance to be tested, that is, the attribute value of the interactive information, according to the preset model performance detection algorithm and the account click probability of each target information combination. Among them, the model performance detection algorithm can be a cross entropy algorithm, an AUC (Area Under Curve) algorithm, a relative information gain algorithm, etc., which is not limited in the embodiments of the present disclosure.

[0086] Step 160: Determine the performance test result of the model to be tested based on the interaction information.

[0087] In implementation, after obtaining the interaction information, the computer device may further determine the performance test result of the model to be tested based on the interaction information.

[0088] As an optional implementation, Figure 3 As shown, in step 160, the computer device determines the processing process of the performance test result of the model to be tested based on the interactive information as follows.

[0089] Step 161: determine whether the attribute value of the interactive information exceeds the target attribute value.

[0090] In implementation, a target attribute value may be pre-stored in the computer device, and the target attribute value may be set by a technician based on experience. After the computer device obtains the attribute value of the interactive information, it may further determine whether the attribute value of the interactive information exceeds the target attribute value. If the attribute value of the interactive information exceeds the target attribute value, step 162 is executed. If the attribute value of the interactive information does not exceed the target attribute value, step 163 is executed. Among them, exceeding may be greater than or greater than or equal to, which is not limited in the embodiment of the present disclosure. Correspondingly, not exceeding may be less than or equal to, or less than, which is not limited in the embodiment of the present disclosure.

[0091] Step 162: If the attribute value of the interactive information exceeds the target attribute value, it is determined that the performance test result of the model to be tested is qualified.

[0092] In implementation, if the attribute value of the interactive information exceeds the target attribute value, it means that the performance of the model to be tested meets the preset performance indicator requirements, that is, the computer device determines that the performance test result of the model to be tested is qualified.

[0093] Step 163: If the attribute value of the interactive information does not exceed the target attribute value, it is determined that the performance test result of the model to be tested is unqualified.

[0094] In implementation, if the attribute value of the interactive information does not exceed the target attribute value, it means that the performance of the model to be tested does not meet the preset performance indicator requirements, that is, the computer device determines that the performance test result of the model to be tested is unqualified.

[0095] The embodiment of the present disclosure provides a method for detecting model performance, wherein a computer device obtains each historical information display data stored in an information display system. The historical information display data at least includes information of the information display position. The computer device determines multiple sample data from each historical information display data, and retrieves the display information corresponding to the information display position in each sample data through the information display system. The information display system is provided with a model to be tested for performance detection, and the model to be tested is used to perform one or more operations in the above retrieval process. Then, the computer device determines whether the information combination consisting of the information display position and the retrieved display information exists in the historical information display data. If the information combination exists in the historical information display data, the computer device obtains the interaction information corresponding to the information combination, wherein the interaction information is used to represent the interaction operation between the account and the display information, and determines the performance detection result of the model to be tested based on the interaction information. In this way, the model performance is tested by simulating the model online; at the same time, in the process of performing performance testing on the model, the historical data is screened and filtered through the system containing the model to be tested, and the test results brought by the data that meets the requirements are used to determine the performance of the model to be tested, which can improve the evaluation accuracy of the account feedback evaluation model.

[0096] It should be understood that although Figure 1-3 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-3 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0097] Figure 4 is a block diagram of a detection device for model performance according to an exemplary embodiment. Figure 4 The device includes a first acquisition module 410, a first determination module 420, a search module 430, a judgment module 440, a second acquisition module 450 and a second determination module 460, wherein:

[0098] The first acquisition module 410 is configured to acquire various historical information display data stored in the information display system, wherein the historical information display data at least includes information of the information display position;

[0099] A first determination module 420 is configured to determine a plurality of sample data from each historical information display data;

[0100] The retrieval module 430 is configured to retrieve the display information corresponding to the information display position in each sample data through the information display system, wherein the information display system is provided with a model to be tested of the performance to be tested, and the model to be tested is used to perform one or more operations in the above retrieval process;

[0101] The determination module 440 is configured to determine whether the information combination consisting of the information display position and the retrieved display information exists in the historical information display data;

[0102] The second acquisition module 450 is configured to acquire interaction information corresponding to the information combination if the information combination exists in the historical information display data, wherein the interaction information is used to represent the interaction operation between the account and the display information;

[0103] The second determination module 460 is configured to determine the performance test result of the model to be tested based on the interaction information.

[0104] As an optional implementation, the historical information display data also includes display information, and the retrieval module 430 is specifically used to:

[0105] Through multiple retrieval algorithms set in the information display system, display information matching the characteristics of each information display position is retrieved from the information database; wherein one or more retrieval algorithms are set in the model to be tested.

[0106] As an optional implementation manner, the second acquisition module 450 is specifically configured to:

[0107] Obtain the performance test dimensions of the model to be tested;

[0108] Read the attribute values ​​of the performance detection dimensions corresponding to each information combination;

[0109] The average value of the attribute values ​​of each information combination is determined as the attribute value of the mutual information.

[0110] As an optional implementation manner, the second determining module 460 is specifically configured to:

[0111] Determine whether the attribute value of the interactive information exceeds the target attribute value;

[0112] If the attribute value of the interactive information exceeds the target attribute value, the performance test result of the model to be tested is determined to be qualified;

[0113] If the attribute value of the interactive information exceeds the target attribute value, the performance test result of the model to be tested is determined to be unqualified.

[0114] The embodiment of the present disclosure provides a device for detecting model performance, wherein a computer device obtains each historical information display data stored in an information display system. The historical information display data at least includes information of the information display position. The computer device determines a plurality of sample data from each historical information display data, and retrieves the display information corresponding to the information display position in each sample data through the information display system. The information display system is provided with a model to be tested for performance to be detected, and the model to be tested is used to perform one or more operations in the above retrieval process. Then, the computer device determines whether the information combination consisting of the information display position and the retrieved display information exists in the historical information display data. If the information combination exists in the historical information display data, the computer device obtains the interaction information corresponding to the information combination, wherein the interaction information is used to represent the interaction operation between the account and the display information, and determines the performance detection result of the model to be tested based on the interaction information. In this way, the model performance is tested by simulating the model to be put online; at the same time, in the process of performing performance testing on the model, the historical data is screened and filtered through the system containing the model to be tested, and the test results based on the data that meets the requirements are used to determine the performance of the model to be tested, which can improve the evaluation accuracy of the account feedback evaluation model.

[0115] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0116] Figure 5 is a block diagram of a computer device 500 for model evaluation according to an exemplary embodiment. For example, the device 500 may be a server. Figure 5 The device 500 includes a processing component 520, which further includes one or more processors, and a memory resource represented by a memory 522 for storing instructions executable by the processing component 520, such as an application. The application stored in the memory 522 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 520 is configured to execute the instructions to perform the above-mentioned model performance detection method.

[0117] The device 500 may also include a power component 524 configured to perform power management of the device 500, a wired or wireless network interface 526 configured to connect the device 500 to a network, and an input / output (I / O) interface 528. The device 500 may operate based on an operating system stored in the memory 522, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, or the like.

[0118] In an exemplary embodiment, a storage medium including instructions is also provided, such as a memory 522 including instructions, and the above instructions can be executed by a processor of the device 500 to complete the above-mentioned model performance detection method. The storage medium can be a non-transitory computer-readable storage medium, for example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0119] In an exemplary embodiment, a computer program product is also provided, which includes a computer program stored in a readable storage medium, and at least one processor of a device reads and executes the computer program from the readable storage medium, so that the device performs the above-mentioned model performance detection method.

[0120] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0121] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for detecting model performance, characterized in that: The method comprises: Acquire various historical information display data stored in the information display system, wherein the historical information display data at least includes information of the information display position; Determine a plurality of sample data from each of the historical information display data; Retrieving display information matching the characteristics of the information display position in each of the sample data from an information database through a retrieval algorithm set in the information display system, wherein a model to be tested of the performance to be tested is set in the information display system; Combine each of the information display positions and each of the display information retrieved to obtain a plurality of information combinations corresponding to each of the information display positions, and input each of the information combinations corresponding to each of the information display positions into the model to be tested to obtain the account click probability of each of the information combinations. For any of the information display positions, among the information combinations corresponding to the information display position, the information combination with the highest account click probability is used as the target information combination corresponding to the information display position; Determine whether the target information combination corresponding to each of the information display positions exists in the historical information display data; If the target information combination exists in the historical information display data, then obtaining interaction information corresponding to the target information combination, wherein the interaction information is used to represent an interaction operation between the account and the display information; Based on the interaction information, a performance test result of the model to be tested is determined.

2. The method according to claim 1, characterized in that The acquiring the interaction information corresponding to the target information combination includes: Obtaining the performance testing dimension of the model to be tested; Reading the attribute value of each target information combination corresponding to the performance detection dimension; An average value of the attribute values ​​of each target information combination is determined as the attribute value of the interaction information.

3. The method according to claim 1, characterized in that The step of determining the performance test result of the model to be tested based on the interaction information includes: Determining whether the attribute value of the interactive information exceeds the target attribute value; If the attribute value of the interactive information exceeds the target attribute value, determining that the performance test result of the model to be tested is qualified; If the attribute value of the interactive information does not exceed the target attribute value, it is determined that the performance test result of the model to be tested is unqualified.

4. The method according to claim 2, characterized in that: The performance detection dimension includes account click rate or account download rate.

5. A device for detecting model performance, characterized in that: The device comprises: A first acquisition module is configured to acquire various historical information display data stored in the information display system, wherein the historical information display data at least includes information of the information display position; A first determining module is configured to determine a plurality of sample data from each of the historical information display data; A retrieval module is configured to retrieve display information matching the characteristics of the information display position in each of the sample data from an information database through a retrieval algorithm set in the information display system, wherein a model to be tested of the performance to be tested is set in the information display system; The second determination module is configured to combine each of the information display positions and each of the display information retrieved to obtain a plurality of information combinations corresponding to each of the information display positions, and input each of the information combinations corresponding to each of the information display positions into the model to be tested to obtain the account click probability of each of the information combinations, and for any of the information display positions, among the information combinations corresponding to the information display position, the information combination with the largest account click probability is used as the target information combination corresponding to the information display position; A judgment module, configured to determine whether a target information combination corresponding to each of the information display positions exists in the historical information display data; a second acquisition module, configured to acquire interaction information corresponding to the target information combination if the target information combination exists in the historical information display data, wherein the interaction information is used to represent an interaction operation between the account and the display information; The third determination module is configured to determine the performance detection result of the model to be tested based on the interaction information.

6. The device according to claim 5, characterized in that The second acquisition module is specifically used for: Obtaining the performance testing dimension of the model to be tested; Reading the attribute value of each target information combination corresponding to the performance detection dimension; An average value of the attribute values ​​of each target information combination is determined as the attribute value of the interaction information.

7. The device according to claim 5, characterized in that The second determining module is specifically used to: Determining whether the attribute value of the interactive information exceeds the target attribute value; If the attribute value of the interactive information exceeds the target attribute value, determining that the performance test result of the model to be tested is qualified; If the attribute value of the interactive information does not exceed the target attribute value, it is determined that the performance test result of the model to be tested is unqualified.

8. The device according to claim 5, characterized in that The performance detection dimension includes account click rate or account download rate.

9. A computer device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the method for detecting model performance as described in any one of claims 1 to 4.

10. A storage medium, when instructions in the storage medium are executed by a processor of a computer device, enables the computer device to execute the method for detecting model performance as claimed in any one of claims 1 to 4.

Images