Information push model testing method, device, equipment, medium and product

Abstract

The present invention provides an information push model testing method, apparatus, equipment, medium, and product. The method includes: obtaining test-related information from multiple test user terminals within a preset time period; the test-related information is generated by the test user terminal based on user input operations after receiving target push information; the target push information is generated based on first push information and second push information using a hybrid interpolation method; the first push information and the second push information have the same data format; comparing the information push quality between the first information push model and the second information push model based on each of the test-related information; and determining the information push model with better information push quality between the first information push model and the second information push model as the target information push model. Compared with the A / B testing method, the information push model testing method of the present invention occupies fewer resources.

Description

Technical Field

[0001] The present invention relates to the field of information push technology, and in particular to an information push model testing method, device, equipment, medium and product. Background Art

[0002] Information push is a technology that reduces information overload by regularly delivering the information users need via the internet, using certain technical standards or protocols. In the field of information push technology, the quality of the information push model can significantly impact the overall quality of information push.

[0003] Because there are many different information push models, and the quality of newly developed information push models requires further testing, information push models are currently primarily tested through A / B testing (AB Testing).

[0004] When conducting A / B testing on information push models, a large amount of resources are consumed because of the need to compare the differences between the two information push models in real time. Summary of the Invention

[0005] The present invention provides an information push model testing method, device, equipment, medium and product to solve the problem that the current A / B test, when performing information push model testing, requires real-time comparison of the differences between two information push models, which occupies a large amount of resources.

[0006] A first aspect of the present invention provides an information push model testing method, comprising:

[0007] Obtaining test-related information for multiple test user terminals within a preset time period; the test-related information is generated by the test user terminal based on user input after receiving target push information; the target push information is generated based on first push information and second push information using a hybrid interpolation method; the first push information is generated by a first information push model; the second push information is generated by a second information push model; the first push information and the second push information have the same data format;

[0008] comparing information push quality between the first information push model and the second information push model according to the test related information;

[0009] The information push model with better information push quality between the first information push model and the second information push model is determined as the target information push model.

[0010] Furthermore, in the above method, the test-related information includes push information exposure and click-through rate;

[0011] The comparing the information push quality between the first information push model and the second information push model according to the test related information includes:

[0012] Determine, based on the test related information, a first push information exposure and a first click-through rate corresponding to the first push information;

[0013] Determining a second push information exposure and a second click-through rate corresponding to the second push information according to each of the test related information;

[0014] The first push information exposure, the first click-through rate, and the second push information exposure, the second click-through rate are compared.

[0015] Furthermore, in the above method, the comparing of the first push information exposure, the first click-through rate, and the second push information exposure, the second click-through rate, includes:

[0016] Determining the first information push quality according to the first push information exposure, the first click-through rate, and the matched first preset weights;

[0017] The second information push quality is determined according to the second push information exposure, the second click-through rate, and the matching second preset weights.

[0018] Furthermore, the method as described above, before obtaining the test related information of the plurality of test user terminals, further includes:

[0019] Obtaining identifications of multiple test user terminals;

[0020] Performing hash processing on each of the identifiers to generate a corresponding hash value;

[0021] Determining a matching hybrid interpolation order from a preset database according to each of the hash values; the preset database stores a mapping relationship between the hash values ​​and the hybrid interpolation order;

[0022] Performing mixed interpolation processing on the first push information and the second push information matching each of the identifiers according to the mixed interpolation order corresponding to each of the identifiers to generate target push information;

[0023] The target push information is sent to each of the matched test user terminals.

[0024] Furthermore, in the above method, performing hash processing on each of the identifiers to generate a corresponding hash value includes:

[0025] MD5 (full name in English: MD5 Message-Digest Algorithm, Chinese: Information Digest Algorithm) is used to perform hash processing on each of the identifiers to generate a corresponding hash value.

[0026] Furthermore, in the above method, both the first push information and the second push information include a push data sequence; the push data sequence includes a plurality of push data;

[0027] The performing mixed interpolation processing on the first push information and the second push information matching each of the identifiers according to the mixed interpolation order corresponding to each of the identifiers to generate target push information includes:

[0028] The plurality of push data corresponding to the respective identifiers are fused according to the mixed interpolation order corresponding to the respective identifiers to generate target push information.

[0029] A second aspect of the present invention provides an information push model testing device, comprising:

[0030] An acquisition module is configured to acquire test-related information of multiple test user terminals within a preset time period; the test-related information is generated by the test user terminal based on user input operations after receiving target push information; the target push information is generated based on the first push information and the second push information using a hybrid interpolation method; the first push information is generated by a first information push model; the second push information is generated by a second information push model; and the first push information and the second push information have the same data format;

[0031] A comparison module, configured to compare the information push quality between the first information push model and the second information push model according to the test related information;

[0032] The determination module is configured to determine the information push model with better information push quality between the first information push model and the second information push model as the target information push model.

[0033] Furthermore, in the above device, the test-related information includes push information exposure and click-through rate;

[0034] The comparison module is specifically used for:

[0035] Determine the first push information exposure and the first click-through rate corresponding to the first push information based on each of the test-related information; determine the second push information exposure and the second click-through rate corresponding to the second push information based on each of the test-related information; and compare the first push information exposure, the first click-through rate, and the second push information exposure, the second click-through rate.

[0036] Furthermore, in the above-mentioned device, when comparing the first push information exposure, the first click-through rate, and the second push information exposure, the second click-through rate, the comparison module is specifically configured to:

[0037] The first information push quality is determined based on the first push information exposure, the first click-through rate and the matching first preset weights; the second information push quality is determined based on the second push information exposure, the second click-through rate and the matching second preset weights.

[0038] Furthermore, the device as described above further comprises:

[0039] A push information generation module is used to obtain the identifiers of multiple test user terminals; hash each of the identifiers to generate a corresponding hash value; determine a matching hybrid interpolation order from a preset database based on each of the hash values; the preset database stores a mapping relationship between hash values ​​and hybrid interpolation orders; perform hybrid interpolation processing on the first push information and the second push information matching each of the identifiers according to the hybrid interpolation order corresponding to each of the identifiers to generate target push information; and send the target push information to each of the matching test user terminals.

[0040] Furthermore, in the above-described apparatus, when the push information generation module performs hash processing on each of the identifiers to generate a corresponding hash value, the push information generation module is specifically configured to:

[0041] The MD5 message digest algorithm is used to perform hash processing on each of the identifiers to generate a corresponding hash value.

[0042] Furthermore, in the above-mentioned device, both the first push information and the second push information include a push data sequence; the push data sequence includes a plurality of push data;

[0043] The push information generation module is specifically configured to perform mixed interpolation processing on the first push information and the second push information that match the respective identifiers according to the mixed interpolation order corresponding to the respective identifiers to generate the target push information:

[0044] The plurality of push data corresponding to the respective identifiers are fused according to the mixed interpolation order corresponding to the respective identifiers to generate target push information.

[0045] A third aspect of the present invention provides an electronic device, comprising: a memory, a processor;

[0046] Memory; a memory for storing instructions executable by the processor;

[0047] The processor is configured to execute the information push model testing method described in any one of the first aspects.

[0048] A fourth aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, they are used to implement the information push model testing method described in any one of the first aspects.

[0049] A fifth aspect of the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the information push model testing method described in any one of the first aspects.

[0050] The present invention provides a method, apparatus, device, medium, and product for testing an information push model. The method includes: obtaining test-related information from multiple test user terminals within a preset time period; the test-related information is generated by the test user terminal according to user input after receiving target push information; the target push information is generated based on first push information and second push information using a hybrid interpolation method; the first push information is generated by a first information push model; the second push information is generated by a second information push model; the first push information and the second push information have the same data format; the information push quality between the first information push model and the second information push model is compared based on each of the test-related information; and the information push model with better information push quality between the first information push model and the second information push model is determined as the target information push model. The information push model testing method of the present invention generates target push information from the first push information and the second push information in advance using a hybrid interpolation method. Then, after sending the target push information to the test user terminal, it only needs to obtain test-related information from the test user terminal within the preset time period, and can determine the information push model with better information push quality between the first information push model and the second information push model based on the test-related information. Compared with the A / B testing method, it occupies less resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0052] Figure 1 A scenario diagram for implementing the information push model testing method according to an embodiment of the present invention;

[0053] Figure 2 A schematic diagram of a flow chart of a method for testing an information push model according to a first embodiment of the present invention;

[0054] Figure 3A flowchart of a method for testing an information push model according to a second embodiment of the present invention;

[0055] Figure 4 A flowchart of a method for testing an information push model according to a third embodiment of the present invention;

[0056] Figure 5 A schematic structural diagram of an information push model testing device provided by a fourth embodiment of the present invention;

[0057] Figure 6 This is a schematic structural diagram of an electronic device provided in a fifth embodiment of the present invention.

[0058] The above drawings illustrate specific embodiments of the present invention, which will be described in more detail below. These drawings and the accompanying description are not intended to limit the scope of the present invention in any way, but rather to illustrate the concept of the present invention to those skilled in the art by reference to specific embodiments. DETAILED DESCRIPTION

[0059] Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. In the following description, when referring to the drawings, like numbers in different figures represent like or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments are not intended to represent all possible embodiments consistent with the present invention. Rather, they are merely examples of apparatus and methods consistent with certain aspects of the present invention, as detailed in the appended claims.

[0060] The technical solution of the present invention is described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below with reference to the accompanying drawings.

[0061] In order to clearly understand the technical solution of the present application, the solution of the prior art is first introduced in detail. ABTest is a commonly used control experimental method, which is mainly used for product development and model launch decision-making. First, the user terminals of the target users are grouped into experimental and control groups according to a random strategy. Under the control of a single variable, the two groups of users, experimental and control, are respectively matched to the traffic of the new and old recommendation models, and the traffic is distributed. After distribution, the effects of the two models are compared according to the evaluation of the traffic in their respective window periods, thereby providing evaluation and decision support for the recommendation model launch strategy. Due to the need to compare the differences between the two information push models in real time, A / B testing takes up a large amount of resources.

[0062] Therefore, in view of the problem that in the existing technology, when conducting A / B testing on information push models, a large amount of resources are occupied due to the need to compare the differences between the two information push models in real time, the inventors found in their research that in order to solve this problem, the test phase can be divided into two parts, one part is to generate the target push information for the test, and the other part is to collect test-related information from the test user terminal, and determine the quality of the information push model based on the test-related information to reduce the occupied resources.

[0063] Specifically, first, after sending the target push information to the test user terminal, the test-related information of multiple test user terminals within a preset time period is obtained. The test-related information is generated by the test user terminal according to the user input operation after receiving the target push information. The target push information is generated according to the first push information and the second push information using a hybrid interpolation method. The first push information is generated by the first information push model. The second push information is generated by the second information push model. The data format of the first push information and the second push information is the same. The information push quality between the first information push model and the second information push model is compared based on each test-related information. The information push model with better information push quality between the first information push model and the second information push model is determined as the target information push model.

[0064] Based on the above creative findings, the inventor proposed the technical solution of this application.

[0065] The following describes the application scenarios of the information push model testing method provided by the embodiment of the present invention. Figure 1 As shown, 1 is an electronic device and 2 is a test user terminal. The network architecture for the application scenario corresponding to the information push model testing method provided by the embodiment of the present invention includes: electronic device 1 and test user terminal 2. Test user terminal 2 includes user terminal A, user terminal B, and user terminal N used for testing.

[0066] When it is necessary to test the information push model, the electronic device 1 or other electronic device can generate corresponding target push information based on the test user terminal 2. The target push information is a fusion of the first push information generated by the first information push model to be tested and the second push information generated by the second information push model. The fusion method is mainly through a hybrid interpolation method. After generating the target push information, the electronic device 1 or other electronic device sends the target push information to the corresponding test user terminal 2. After the test user terminal 2 receives the target push information, the user will perform corresponding operations on it, such as clicking, sliding, or directly ignoring it.

[0067] After a preset time period, electronic device 1 obtains test-related information from multiple test user terminals 2, such as user terminals A through N, within the preset time period. Based on the test-related information, electronic device 1 compares the information push quality between the first information push model and the second information push model. Finally, electronic device 1 determines the information push model with the better information push quality between the first and second information push models as the target information push model. After determining the target information push model, it can be output to the model online decision-making platform for display, allowing staff to understand the test results or conduct subsequent analysis and processing.

[0068] The embodiments of the present invention are described below with reference to the accompanying drawings.

[0069] Figure 2 A flow chart of the information push model testing method provided by the first embodiment of the present invention is shown as follows: Figure 2 As shown, in this embodiment, the execution subject of the embodiment of the present invention is an information push model testing device, which can be integrated into an electronic device. The information push model testing method provided by this embodiment includes the following steps:

[0070] Step S101: Acquire test-related information for multiple test user terminals within a preset time period. The test-related information is generated by the test user terminal based on user input after receiving a target push message. The target push message is generated using a hybrid interpolation method based on first and second push messages. The first push message is generated using a first information push model. The second push message is generated using a second information push model. The first and second push messages share the same data format.

[0071] In this embodiment, the preset time period may be one day, one week, etc., and may be set according to actual needs, and this embodiment does not limit this.

[0072] The hybrid interpolation method is to mix the push information sequences output by the two information push models, and generally interpolation is performed in an alternating mixing manner.

[0073] Generally, the push information output by the first and second information push models is pre-processed to generate the target push information. For example, if the first push information outputs three products in the order A, B, and C from left to right, and the second push information outputs three products D, E, and F, the target push information obtained through mixed interpolation can be A, E, F, A, B, F, and so on. The first and second push information have the same output format as the target push information, and the data format represents the same order and quantity of the output data.

[0074] This embodiment does not limit how the first information push model and the second information push model generate push information. It can be generated based on the historical click records of the user terminal, the identification of the user terminal, etc., or it can be generated according to a preset strategy.

[0075] Step S102: comparing the information push quality between the first information push model and the second information push model according to the test related information.

[0076] In this embodiment, the test-related information may include push information exposure, click-through rate, etc. If the push information is a movie push, it may also include viewing time, etc., and adaptive adjustments can be made according to different data pushed by the push information.

[0077] The quality of information push is related to test-related information. Generally speaking, the higher the exposure and click-through rate of pushed information, the better the quality of the information push.

[0078] Step S103: Determine the information push model with better information push quality between the first information push model and the second information push model as the target information push model.

[0079] An embodiment of the present invention provides a method for testing an information push model, the method comprising: obtaining test-related information of multiple test user terminals within a preset time period. The test-related information is generated by the test user terminal according to the user input operation after receiving the target push information. The target push information is generated according to the first push information and the second push information using a hybrid interpolation method. The first push information is generated by the first information push model. The second push information is generated by the second information push model. The data format of the first push information and the second push information is the same. The information push quality between the first information push model and the second information push model is compared based on each test-related information. The information push model with better information push quality between the first information push model and the second information push model is determined as the target information push model.

[0080] The information push model testing method of the present invention generates target push information in advance based on the first push information and the second push information through a hybrid interpolation method. Then, after sending the target push information to the test user terminal, it only needs to obtain test-related information of the test user terminal within a preset time period, that is, the information push model with better information push quality between the first information push model and the second information push model can be determined based on the test-related information. Compared with the A / B testing method, there is no need to configure and intervene in the diversion strategy, there is no need to worry about the time window problem of the comparative experiment, and it occupies less resources.

[0081] Figure 3 A flow chart of the information push model testing method provided by the second embodiment of the present invention is shown as follows: Figure 3As shown, the information push model testing method provided in this embodiment is based on the information push model testing method provided in the previous embodiment of the present invention, and further refines each step. The information push model testing method provided in this embodiment includes the following steps.

[0082] Step S201: Acquire test related information of multiple test user terminals within a preset time period.

[0083] In this embodiment, the implementation of step 201 is similar to the implementation of step 101 in the previous embodiment of the present invention, and will not be described in detail here.

[0084] It should be noted that the test-related information may include push information exposure and click-through rate.

[0085] Step S202 : determining the first push information exposure and the first click-through rate corresponding to the first push information according to each test-related information.

[0086] In this embodiment, since the target push information is determined by the first push information and the second push information, the target push information may contain partial data of the first push information and / or partial data of the second push information. Therefore, it is necessary to determine the first push information exposure and the first click-through rate corresponding to the first push information based on the relevant information of each test.

[0087] Assuming that the target push information only includes the product A data in the first push information, the exposure and click-through rate of the product A data may be equivalent to the exposure and the first click-through rate of the first push information.

[0088] Step S203: determining the second push information exposure and the second click-through rate corresponding to the second push information according to the test-related information.

[0089] In this embodiment, since the target push information is determined by the first push information and the second push information, the target push information may contain partial data of the first push information and / or partial data of the second push information. Therefore, it is necessary to determine the second push information exposure and second click-through rate corresponding to the second push information based on the test-related information.

[0090] Assuming that the target push information only includes the product B data in the second push information, the exposure and click-through rate of the product B data may be equivalent to the exposure and second click-through rate of the second push information.

[0091] Step S204 : comparing the first push information exposure and the first click-through rate with the second push information exposure and the second click-through rate.

[0092] In this embodiment, the comparison of the first push information exposure, the first click-through rate, and the second push information exposure and the second click-through rate is used to determine the information push quality between the first information push model and the second information push model.

[0093] Specifically:

[0094] The first information push quality is determined according to the first push information exposure, the first click-through rate, and the matched first preset weights.

[0095] The second information push quality is determined according to the second push information exposure, the second click-through rate, and the matched second preset weights.

[0096] In this embodiment, the push information exposure and the click-through rate can be pre-set with corresponding weights. For example, if the push information exposure is more important, the information exposure weight can be set to be larger.

[0097] Step S205: Determine the information push model with better information push quality between the first information push model and the second information push model as the target information push model.

[0098] In this embodiment, the implementation of step 205 is similar to the implementation of step 103 in the previous embodiment of the present invention, and will not be described in detail here.

[0099] Figure 4 A flow chart of the information push model testing method provided by the third embodiment of the present invention is shown as follows: Figure 4 As shown, the information push model testing method provided in this embodiment is based on the information push model testing method provided in the previous embodiment of the present invention, and adds a process of sending target push information. The information push model testing method provided in this embodiment includes the following steps.

[0100] Step S301: Acquire identifications of multiple test user terminals.

[0101] In this embodiment, the identifier of the test user terminal may be used to determine the generated first push information and second push information, and may also be used to provide a basis for fusing the first push information and the second push information.

[0102] Step S302: Perform hash processing on each identifier to generate a corresponding hash value.

[0103] In this embodiment, assuming that the hash value generated after the hash processing of the user terminal identifier is 3, a hybrid interpolation order matching 3 may be selected to perform subsequent fusion processing.

[0104] At the same time, the hash processing can be performed using the MD5 information digest algorithm to hash each identifier to generate a corresponding hash value. Using the MD5 method can generate the corresponding hash value more stably and quickly.

[0105] Step S303: Determine a matching hybrid interpolation sequence from a preset database according to each hash value. The preset database stores a mapping relationship between hash values ​​and hybrid interpolation sequences.

[0106] In this embodiment, the hash value range will be different depending on the number of push data contained in the target push information. For example, if the target push information contains 3 push data and the number of pushed products is 3, the hash value range can be 0-7. If the push data is 2, the hash value range can be 0-3. The hash value range can follow 0-2. n-1 The mixed interpolation order corresponds to each hash value, and there will also be 2 n indivual.

[0107] Step S304 : performing mixed interpolation processing on the first push information and the second push information that match the respective identifiers according to the mixed interpolation order corresponding to the respective identifiers, so as to generate target push information.

[0108] The first push information and the second push information both include a push data sequence, which includes a plurality of push data.

[0109] Performing mixed interpolation processing on the first push information and the second push information matching each identifier according to the mixed interpolation order corresponding to each identifier to generate target push information, including:

[0110] The multiple push data corresponding to each identifier are fused according to the mixed interpolation order corresponding to each identifier to generate target push information.

[0111] In this embodiment, the push data in the push data sequence are generally arranged in order from left to right. For example, the first push information output is three products in the order of A, B, and C from left to right, and the first push information output is three products D, E, and F. The target push information obtained by mixed interpolation can be A, E, F, A, B, F, etc.

[0112] Step S305: Send the target push information to each matching test user terminal.

[0113] After sending the target push information to each matched test user terminal, the test related information of each test user terminal within a preset time period may be collected.

[0114] In order to describe the information push model testing method in more detail, the following will take 3 pieces of pushed data as an example for detailed explanation.

[0115] First, obtain two information push models and the first and second push messages output by the information push models. The output formats of the two information push models must be consistent. For example, if the output format is the user terminal number, product 1, product 2, product 3, that is, the information push model recommends three products to the user, and the order of recommendation for these three products is from left to right (i.e., product 1 is recommended first), then both information push models to be compared must use this output format.

[0116] To obtain evaluation indicators of information push quality, commonly used indicators include exposure rate, CTR (full name in English: Click-Through-Rate, Chinese: click-through rate), purchase success rate, etc. These indicators can also be further weighted and calculated according to actual needs.

[0117] Hybrid interpolation: Performs hybrid interpolation and truncation on the output results of the two information push models to be compared (subsequently simplified to Model A and Model B). Assume that for a certain user, the output of Model A is the user terminal number, product A1, product A2, product A3, and the output of Model B is the user terminal number, product B1, product B2, product B3. The output sequence after hybrid interpolation is user terminal number, product A1, product B1, product A2, product B2, product A3, product B3. However, since the output format must be consistent with the original output format, the above output sequence is truncated to user terminal number, product A1, product B1, product A2, as the output result.

[0118] Interpolation fairness: The interpolation described above must ensure fairness between Model A and Model B. This means that the recommendations received by Model A and Model B should be approximately the same for the end user. For large amounts of data, sequential random interpolation can be used to ensure fairness. For example, for the output of user terminal number, product 1, product 2, and product 3, the user terminal number value can be hashed using the MD5 algorithm to generate a hash value. The corresponding relationship between the hash value and the hybrid interpolation method is shown in Table 1.

[0119] Table 1 Hash value and mixed interpolation order mapping table

[0120] Hash value Mixed interpolation order 0(000) User terminal number-Product A1-Product A2-Product A3 1(001) User terminal number-Product A1-Product A2-Product B3 2(010) User terminal number-Product B1-Product A2-Product B3 3(011) User terminal number-Product A1-Product B2-Product B3 4(100) User terminal number-Product B1-Product A2-Product A3 5(101) User terminal number-Product A1-Product B2-Product A3 6(110) User terminal number-Product B1-Product B2-Product A3 7(111) User terminal number-Product B1-Product B2-Product B3

[0121] In the case of large amounts of data, the distribution of users conforms to the normal distribution, so this interpolation method can ensure the fairness of the distribution of model A and model B.

[0122] The interpolated sequence is assembled into a push message and sent to the push system, which then distributes it to the user's terminal. The interpolated sequence is transparent to the end user, non-interfering and non-intrusive, and does not degrade the end user experience.

[0123] After accumulating user behavior collection, transaction data and other test-related information for a certain time window (such as one hour, one day, one week, etc.), you can analyze the evaluation indicators. The main steps of evaluation indicator analysis are:

[0124] Data cleaning: The above-mentioned behavior collection and transaction data are cleaned according to the time window, the data that does not meet the requirements is filtered out, and the data that meets the requirements but has incomplete information is completed according to certain rules.

[0125] Deduplication and counting: Deduplication of cleaned data is performed based on two dimensions: user, time window, and recommended category. If business needs require, counting can be performed at the same time as deduplication.

[0126] Summarize: Summarize the deduplicated data according to the time window, and display the summary value as a comparison of business target data.

[0127] Finally, a multi-dimensional comparison is performed to determine the model with better information push quality from Model A and Model B.

[0128] Figure 5 This is a structural diagram of an information push model testing device provided by the fourth embodiment of the present invention, as shown in FIG. Figure 5 As shown, in this embodiment, the information push model testing device 400 includes:

[0129] Acquisition module 401 is configured to acquire test-related information from multiple test user terminals within a preset time period. The test-related information is generated by the test user terminal based on user input after receiving the target push information. The target push information is generated using a hybrid interpolation method based on the first push information and the second push information. The first push information is generated by the first information push model. The second push information is generated by the second information push model. The first and second push information have the same data format.

[0130] The comparison module 402 is configured to compare the information push quality between the first information push model and the second information push model according to the test related information.

[0131] The determination module 403 is configured to determine the information push model with better information push quality between the first information push model and the second information push model as a target information push model.

[0132] The information push model testing device provided in this embodiment can be executed Figure 2 The technical solution of the method embodiment shown in the figure has the same implementation principle and technical effect as Figure 2 The method embodiments shown are similar and will not be described in detail here.

[0133] At the same time, the information push model testing device provided by the present invention further refines the information push model testing device provided by the previous embodiment.

[0134] Optionally, in this embodiment, the test-related information includes push information exposure and click-through rate.

[0135] The comparison module 402 is specifically used for:

[0136] Determine the first push information exposure and first click-through rate corresponding to the first push information based on the test-related information. Determine the second push information exposure and second click-through rate corresponding to the second push information based on the test-related information. Compare the first push information exposure and first click-through rate with the second push information exposure and second click-through rate.

[0137] Optionally, in this embodiment, when comparing the first push information exposure, the first click-through rate, and the second push information exposure, the second click-through rate, the comparison module 402 is specifically configured to:

[0138] The first information push quality is determined based on the first push information exposure, the first click-through rate, and the matched first preset weights. The second information push quality is determined based on the second push information exposure, the second click-through rate, and the matched second preset weights.

[0139] Optionally, in this embodiment, the information push model testing device 400 further includes:

[0140] A push information generation module is configured to obtain the identifiers of multiple test user terminals. Each identifier is hashed to generate a corresponding hash value. A matching hybrid interpolation sequence is determined from a preset database based on each hash value. The preset database stores a mapping relationship between hash values ​​and hybrid interpolation sequences. A first push information and a second push information matching each identifier are hybrid interpolated according to the hybrid interpolation sequence corresponding to each identifier to generate a target push information. The target push information is then sent to each matching test user terminal.

[0141] Optionally, in this embodiment, when the push information generation module performs hash processing on each identifier to generate a corresponding hash value, it is specifically used to:

[0142] Each identifier is hashed using the MD5 message digest algorithm to generate a corresponding hash value.

[0143] Optionally, in this embodiment, the first push information and the second push information both include a push data sequence. The push data sequence includes multiple push data.

[0144] The push information generation module performs mixed interpolation processing on the first push information and the second push information matching each identifier according to the mixed interpolation order corresponding to each identifier to generate the target push information, specifically for:

[0145] The multiple push data corresponding to each identifier are fused according to the mixed interpolation order corresponding to each identifier to generate target push information.

[0146] The information push model testing device provided in this embodiment can be executed Figure 2-Figure 4 The technical solution of the method embodiment shown in the figure has the same implementation principle and technical effect as Figure 2-Figure 4 The method embodiments shown are similar and will not be described in detail here.

[0147] According to an embodiment of the present invention, the present invention further provides an electronic device, a computer-readable storage medium, and a computer program product.

[0148] like Figure 6 As shown, Figure 6 is a schematic diagram of the structure of an electronic device provided by a fifth embodiment of the present invention. The electronic device is intended to be a digital computer of various forms, such as a laptop computer, desktop computer, workstation, personal digital assistant, server, blade server, mainframe computer, and other suitable computers. The components shown herein, their connections and relationships, and their functions are provided for illustrative purposes only and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0149] like Figure 6 As shown, the electronic device includes: a processor 501 and a memory 502. Each component is connected to each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed in the electronic device.

[0150] Memory 502 is the non-transitory computer-readable storage medium provided by the present invention. The memory stores instructions executable by at least one processor, causing the at least one processor to perform the information push model testing method provided by the present invention. The non-transitory computer-readable storage medium of the present invention stores computer instructions for causing a computer to perform the information push model testing method provided by the present invention.

[0151] The memory 502 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the information push model testing method in the embodiment of the present invention (for example, the attached Figure 5 The processor 501 executes the non-transient software programs, instructions, and modules stored in the memory 502 to execute various functional applications and data processing of the server, thereby implementing the information push model testing method in the above method embodiment.

[0152] At the same time, this embodiment also provides a computer product. When the instructions in the computer product are executed by the processor of the electronic device, the electronic device can execute the information push model testing methods of the above-mentioned embodiments one to three.

[0153] Those skilled in the art will readily recognize other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art not disclosed in the present invention. The specification and examples are to be considered as exemplary only, with the true scope and spirit of the present invention being indicated by the claims.

[0154] It should be understood that the embodiments of the present invention are not limited to the precise structures described above and shown in the drawings, and various modifications and changes can be made without departing from the scope thereof. The scope of the embodiments of the present invention is limited only by the appended claims.

Claims

1. A method for testing an information push model, characterized in that: include: Obtain test related information of multiple test user terminals within a preset time period; The test-related information is generated by the test user terminal based on user input after receiving the target push information; the target push information is generated based on the first push information and the second push information using a hybrid interpolation method; the first push information is generated by the first information push model; the second push information is generated by the second information push model; the first push information and the second push information have the same data format; comparing information push quality between the first information push model and the second information push model according to the test related information; determining an information push model with better information push quality between the first information push model and the second information push model as a target information push model; the information push quality is related to the test-related information, the test-related information including push information exposure and click-through rate, and higher push information exposure and click-through rate represent better information push quality; Before acquiring the test related information of the plurality of test user terminals, the method further includes: Obtaining identifications of multiple test user terminals; Performing hash processing on each of the identifiers to generate a corresponding hash value; Determining a matching hybrid interpolation order from a preset database according to each of the hash values; the preset database stores a mapping relationship between the hash values ​​and the hybrid interpolation order; Performing mixed interpolation processing on the first push information and the second push information matching each of the identifiers according to the mixed interpolation order corresponding to each of the identifiers to generate target push information; Sending the target push information to each of the matched test user terminals; The first push information and the second push information both include a push data sequence; the push data sequence includes a plurality of push data; The performing mixed interpolation processing on the first push information and the second push information matching each of the identifiers according to the mixed interpolation order corresponding to each of the identifiers to generate target push information includes: The plurality of push data corresponding to the respective identifiers are fused according to the mixed interpolation order corresponding to the respective identifiers to generate target push information.

2. The method according to claim 1, characterized in that The test-related information includes push information exposure and click-through rate; The comparing the information push quality between the first information push model and the second information push model according to the test related information includes: Determine, based on the test related information, a first push information exposure and a first click-through rate corresponding to the first push information; Determining a second push information exposure and a second click-through rate corresponding to the second push information according to each of the test related information; The first push information exposure, the first click-through rate, and the second push information exposure, the second click-through rate are compared.

3. The method according to claim 2, characterized in that The comparing the first push information exposure, the first click-through rate, and the second push information exposure, the second click-through rate, includes: Determining the first information push quality according to the first push information exposure, the first click-through rate, and the matched first preset weights; The second information push quality is determined according to the second push information exposure, the second click-through rate, and the matching second preset weights.

4. The method according to claim 1, wherein The hashing of each identifier to generate a corresponding hash value includes: The MD5 message digest algorithm is used to perform hash processing on each of the identifiers to generate a corresponding hash value.

5. An information push model testing device, characterized in that: include: An acquisition module, configured to acquire test-related information of multiple test user terminals within a preset time period; The test-related information is generated by the test user terminal based on user input after receiving the target push information; the target push information is generated based on the first push information and the second push information using a hybrid interpolation method; the first push information is generated by the first information push model; the second push information is generated by the second information push model; the first push information and the second push information have the same data format; A comparison module, configured to compare the information push quality between the first information push model and the second information push model according to the test related information; a determination module, configured to determine an information push model having better information push quality between the first information push model and the second information push model as a target information push model; the information push quality is related to the test-related information, the test-related information including push information exposure and click-through rate, wherein higher push information exposure and click-through rate represent better information push quality; Also includes: A push information generation module is used to obtain the identifiers of multiple test user terminals, perform hash processing on each identifier to generate a corresponding hash value, determine a matching hybrid interpolation order from a preset database based on each hash value, the preset database stores a mapping relationship between hash values ​​and hybrid interpolation orders, perform hybrid interpolation processing on the first push information and the second push information that match each identifier according to the hybrid interpolation order corresponding to each identifier to generate a target push information, and send the target push information to each matching test user terminal; The first push information and the second push information both include a push data sequence, and the push data sequence includes a plurality of push data; The push information generation module performs mixed interpolation processing on the first push information and the second push information matching each identifier according to the mixed interpolation order corresponding to each identifier to generate the target push information, specifically for: The multiple push data corresponding to each identifier are fused according to the mixed interpolation order corresponding to each identifier to generate target push information.

6. An electronic device, characterized in that: include: a processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the information push model testing method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the information push model testing method according to any one of claims 1 to 4.

8. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the information push model testing method according to any one of claims 1 to 4 is implemented.

Images