Method, apparatus, device and computer program product for recommendation

Abstract

The invention provides a recommendation method, device and equipment and a computer program product, and relates to the technical field of computers and Internet. The method comprises the following steps: acquiring a plurality of object-article pairs; dividing the plurality of object-object pairs into extrapolated samples and interpolated samples, wherein the extrapolated samples include object-object pairs that are not observed; performing conversion rate prediction on the extrapolation sample by using a conversion rate prediction model to obtain a first conversion rate; obtaining a conversion rate upper limit and a conversion rate lower limit corresponding to the extrapolation sample; determining a first loss value based on the first conversion rate, the conversion rate upper limit and the conversion rate lower limit; performing conversion rate prediction on the interpolation sample by using a conversion rate prediction model, and determining a second loss value according to a conversion rate prediction result; and performing model training on the conversion rate prediction model in combination with the first loss value and the second loss value so as to perform recommendation through the trained conversion rate prediction model.

Description

Technical Field

[0001] This disclosure relates to the field of computer and Internet technology, and more particularly to a method, apparatus, device, and computer program product for recommendation. Background Technology

[0002] This section is intended to provide background or context for the embodiments of this disclosure as set forth in the claims. The description herein is not intended to be a prior art simply because it is included in this section.

[0003] Current models for predicting conversion rate (CVR) based on historical click data suffer from a serious selection bias problem. The model can only learn from limited, non-random data that users actively click on, resulting in insufficient generalization ability of the trained model on unobserved user-item pairs (such as unclicked or newly listed items). This affects the performance and robustness of the recommendation system in real-world scenarios. Summary of the Invention

[0004] The purpose of this disclosure is to provide a method, apparatus, electronic device, computer-readable storage medium, and computer program product for recommendation, which can improve the generalization ability of recommendation methods.

[0005] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0006] This disclosure provides a method for recommendation, comprising: acquiring multiple object-item pairs; dividing the multiple object-item pairs into extrapolation samples and interpolation samples, wherein the extrapolation samples include unobserved object-item pairs; predicting the conversion rate of the extrapolation samples using a conversion rate prediction model to obtain a first conversion rate; acquiring an upper limit and a lower limit of the conversion rate corresponding to the extrapolation samples; determining a first loss value based on the first conversion rate, the upper limit, and the lower limit; predicting the conversion rate of the interpolation samples using the conversion rate prediction model, and determining a second loss value based on the conversion rate prediction result; and training the conversion rate prediction model by combining the first loss value and the second loss value, so as to make recommendations using the trained conversion rate prediction model.

[0007] In some embodiments, determining a first loss value based on the first conversion rate, the upper limit of the conversion rate, and the lower limit of the conversion rate includes: obtaining the first conversion rate, the upper limit of the conversion rate, and the lower limit of the conversion rate corresponding to each extrapolated sample; determining the sample loss value of each extrapolated sample based on the first conversion rate, the upper limit of the conversion rate, and the lower limit of the conversion rate of each extrapolated sample; and determining the first loss value corresponding to the extrapolated sample based on the sample loss value of each extrapolated sample.

[0008] In some embodiments, the extrapolated sample includes a first sample, and the sample loss value includes a first sample loss value corresponding to the first sample; wherein, determining the sample loss value of each extrapolated sample based on the first conversion rate, the upper limit of the conversion rate, and the lower limit of the conversion rate of each extrapolated sample includes: determining the conversion rate difference between the first conversion rate of the first sample and the upper limit of the conversion rate of the first sample as a first difference; determining the conversion rate difference between the first conversion rate of the first sample and the lower limit of the conversion rate of the first sample as a second difference; and determining the maximum value between the first difference and the second difference as the first sample loss value of the first sample.

[0009] In some embodiments, the interpolated samples include positively biased samples with conversion labels; obtaining the upper and lower limits of the conversion rate corresponding to the extrapolated samples includes: obtaining an unbiased imputation model, which is trained based on the positively biased samples and their conversion labels; using the unbiased imputation model to predict the conversion rate of each interpolated sample to obtain a second conversion rate for each interpolated sample; and determining the upper and lower limits of the conversion rate of the extrapolated samples based on the second conversion rates of each interpolated sample.

[0010] In some embodiments, the extrapolated sample includes a second sample; wherein determining the upper and lower limits of the conversion rate of the extrapolated sample based on the second conversion rate of each interpolated sample includes: obtaining the feature distance between each interpolated sample and the second sample as a first feature distance; obtaining the first offset conversion rate corresponding to each interpolated sample, wherein the first offset conversion rate is the sum of the second conversion rate of the interpolated sample and the first feature distance; and determining the minimum value among the first offset conversion rates corresponding to each interpolated sample as the upper limit of the conversion rate of the second sample.

[0011] In some embodiments, the extrapolated sample includes a third sample; wherein determining the upper and lower limits of the conversion rate of the extrapolated sample based on the second conversion rate of each interpolated sample includes: obtaining the feature distance between each interpolated sample and the third sample as a second feature distance; obtaining the second offset conversion rate corresponding to each interpolated sample, wherein the second offset conversion rate is the difference between the second conversion rate of the interpolated sample and the second feature distance; and determining the maximum value among the first offset conversion rates corresponding to each interpolated sample as the lower limit of the conversion rate of the second sample.

[0012] In some embodiments, the interpolated samples include positively biased samples with conversion labels; wherein, predicting the conversion rate of the interpolated samples using the conversion rate prediction model and determining a second loss value based on the conversion rate prediction result includes: predicting the conversion rate of the interpolated samples using the conversion rate prediction model to obtain a second conversion rate; predicting the conversion rate label of the interpolated samples using an unbiased imputation model to obtain a pseudo-label for the interpolated samples, wherein the unbiased imputation model is trained based on the positively biased samples and the conversion labels of the positively biased samples; and determining a second loss value corresponding to the interpolated samples based on the second conversion rate and the pseudo-label.

[0013] This disclosure provides an apparatus for recommendation, comprising: an object item pair acquisition module, a sample differentiation module, a first conversion rate determination module, an upper and lower limit determination module, a first loss determination module, a second loss determination module, and a model training module.

[0014] The system includes the following modules: an object-item pair acquisition module for acquiring multiple object-item pairs; a sample differentiation module for dividing the multiple object-item pairs into extrapolated samples and interpolated samples, wherein the extrapolated samples include unobserved object-item pairs; a first conversion rate determination module for predicting the conversion rate of the extrapolated samples using a conversion rate prediction model to obtain a first conversion rate; an upper and lower limit determination module for acquiring the upper and lower limits of the conversion rate corresponding to the extrapolated samples; a first loss determination module for determining a first loss value based on the first conversion rate, the upper and lower limits of the conversion rate; a second loss determination module for predicting the conversion rate of the interpolated samples using the conversion rate prediction model and determining a second loss value based on the conversion rate prediction result; and a model training module for training the conversion rate prediction model by combining the first and second loss values, so as to make recommendations using the trained conversion rate prediction model.

[0015] This disclosure provides an electronic device comprising: a memory and a processor; the memory for storing computer program instructions; and the processor for invoking the computer program instructions stored in the memory to implement the method described above.

[0016] This disclosure provides a computer-readable storage medium having computer program instructions stored thereon to implement the recommended method as described in any of the preceding embodiments.

[0017] This disclosure provides a computer program product or computer program including computer program instructions stored in a computer-readable storage medium. The computer program instructions are read from the computer-readable storage medium, and a processor executes the computer program instructions to implement the method described above.

[0018] The recommendation method, apparatus, electronic device, computer-readable storage medium, and computer program product provided in this disclosure, through an internal and external sample collaborative training mechanism, constrains the prediction of extrapolated samples within a reasonable upper and lower limit of conversion rate. At the same time, combined with the accurate fitting of interpolated samples, they jointly improve the generalization ability and prediction accuracy of the conversion rate prediction model in unobserved areas (items that users have not clicked). This overcomes the limitations of traditional models in recommendation performance caused by data selection bias, enabling the recommendation system to more reliably discover potential interest products, promote the cold start of new products, and optimize the efficiency of global traffic allocation.

[0019] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this disclosure. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0021] Figure 1 A schematic diagram of a scenario for a recommended method or apparatus that can be applied to embodiments of this disclosure is shown.

[0022] Figure 2 This is a flowchart illustrating a recommendation method according to an exemplary embodiment.

[0023] Figure 3 This is a flowchart illustrating a method for determining a first loss value according to an exemplary embodiment.

[0024] Figure 4 This is a method for determining a sample loss value according to an exemplary embodiment.

[0025] Figure 5 This is a flowchart illustrating a method for determining the upper and lower limits of conversion rate according to an exemplary embodiment.

[0026] Figure 6 This is a method for determining the upper limit of conversion rate according to an exemplary embodiment.

[0027] Figure 7 This is a flowchart illustrating a sample partitioning method according to an exemplary embodiment.

[0028] Figure 8 This is a flowchart illustrating a second loss value determination method according to an exemplary embodiment.

[0029] Figure 9 This is a block diagram illustrating a recommended apparatus according to an exemplary embodiment.

[0030] Figure 10 A schematic diagram of the structure of an electronic device suitable for implementing embodiments of the present disclosure is shown. Detailed Implementation

[0031] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted.

[0032] Those skilled in the art will recognize that embodiments of this disclosure can be a system, apparatus, device, method, or computer program product. Therefore, this disclosure can be implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0033] The features, structures, or characteristics described in this disclosure can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more specific details omitted, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0034] In this disclosure, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0035] The accompanying drawings are merely illustrative of this disclosure, and the same reference numerals in the drawings denote the same or similar parts, thus omitting repeated descriptions of them. Some block diagrams shown in the drawings do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0036] The flowchart shown in the accompanying drawings is merely illustrative and does not necessarily include all content and steps, nor does it require execution in the described order. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0037] In the description of this disclosure, unless otherwise stated, " / " means "or," for example, A / B can mean A or B. "And / or" in this document is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. Furthermore, "at least one" means one or more, and "multiple" means two or more. The terms "first," "second," etc., do not limit the quantity or order of execution, and "first," "second," etc., do not necessarily imply differences; the terms "contains," "includes," and "has" are used to indicate an open-ended meaning of inclusion and refer to the existence of additional elements / components / etc. besides those listed.

[0038] This disclosure embodiment can be implemented by a terminal and / or a server. The terminal can obtain data from a computer device and display that data. The computer device can interact with the terminal, and can be a server hosting an application, or it can belong to the terminal (i.e., the terminal's backend), etc., without limitation.

[0039] The terminal can be a mobile phone, a laptop computer, or a playback device in a vehicle, etc., without limitation. The terminal can be considered a playback device in a vehicle, and it can display the target application. The terminal is only one example of the devices listed; the terminal in this disclosure is not limited to the listed devices. The target application in this disclosure can be any application capable of displaying multimedia information.

[0040] It is understood that the terminal mentioned in the embodiments of this disclosure can be a computer device, including but not limited to a terminal or a server. In other words, the computer device can be a server or a terminal, or a system composed of a server and a terminal. The terminal mentioned above can be an electronic device, including but not limited to mobile phones, tablets, desktop computers, laptops, handheld computers, in-vehicle devices, augmented reality / virtual reality (AR / VR) devices, head-mounted displays, smart TVs, wearable devices, smart speakers, digital cameras, webcams, and other mobile internet devices (MIDs) with network access capabilities, or terminals in scenarios such as trains, ships, and flights.

[0041] The servers mentioned above can be independent physical servers, server clusters or distributed systems composed of multiple physical servers, or cloud servers that provide basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, vehicle-road cooperation, content delivery networks (CDN), and big data and artificial intelligence platforms.

[0042] Optionally, the data involved in the embodiments of this disclosure may be stored in a computer device or may be stored based on cloud storage technology, without limitation.

[0043] To better understand the above-mentioned objectives, features and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0044] It should be noted that the collection, gathering, updating, analysis, processing, use, transmission, and storage of user personal information involved in this disclosed technical solution all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security and network security.

[0045] The following section will first explain some of the terms used in the embodiments of this disclosure so that those skilled in the art can understand them.

[0046] The Lipschitz continuity assumption is a crucial concept in mathematical analysis, machine learning, and optimization theory. It describes an upper bound on the rate of change of a function. For a given function, this means that the absolute value of the slope of the line connecting any two points on its graph will not exceed a fixed constant.

[0047] Positive biased samples (Observed Data): This refers to data that we can observe and / or that has user feedback (clicks). For example, we know a user clicked on this phone, so we have "click -> purchase" labeled data to train the CVR model. However, this data is subject to selection bias—the system is more inclined to show items that the user is likely to click on.

[0048] Unobserved Data: These are (item-user) pairs that theoretically might be shown to users, but have never actually been shown. For example, a niche phone might be perfect for user A, but the system never recommends it because it considers the click probability low. Therefore, we have no idea whether user A would click on it or make a purchase after seeing it. These samples have an "exposure bias" of 0.

[0049] Positive bias hypothesis: For every object-item pair (also known as a user-item pair), the probability of it being observed must be strictly greater than zero. In simpler terms: During data generation, there are no items that are "absolutely impossible to see." That is, for any given user, there is a tiny, however slight, chance that an item will be recommended by the system and seen by the user.

[0050] The probability of being observed is the propensity score. The probability of being observed refers to the probability that an item is shown to a specific object (such as a user), or the probability that an item is shown to a specific object and clicked by that user.

[0051] CVR prediction is the estimation of the probability that a user who sees an ad (or recommended content) will eventually place a purchase (or complete other specified conversion behavior).

[0052] The preceding text introduced some terms and concepts involved in the embodiments of this disclosure. The following text introduces the technical features involved in the embodiments of this disclosure.

[0053] With the widespread application of recommendation systems in industrial scenarios such as e-commerce, short video, and content platforms, these platforms achieve efficient traffic distribution and user experience optimization through personalized recommendations, playing a crucial role, especially in the task of predicting click-through conversion rate (CVR). CVR prediction aims to estimate the probability that an object (such as a user) will convert (e.g., purchase, registration) after clicking, and is a core indicator connecting exposure and revenue. However, due to the highly selective nature of user behavior on platforms, historical data only records a limited number of click behaviors, leading to a deviation between the training data distribution and the actual user-item distribution—a problem known as "selection bias." This bias results in insufficient generalization ability of the model in unobserved areas, severely impacting the effectiveness and robustness of the recommendation system in actual deployment.

[0054] From a business perspective, the current platform relies primarily on observed user click behavior for training when building CVR prediction models, which presents two core bottlenecks: First, data bias makes it difficult for the model to assess potential unrecommended products or new user behavior, leading to wasted traffic and lost revenue. Second, the model lacks reliable judgment capabilities for unobserved samples in key scenarios such as cold starts and cross-regional deployments, limiting the platform's refined traffic management capabilities in areas such as new product cold starts and regional operation strategy formulation. Improving the generalization ability of CVR models in non-click areas and building a conversion prediction system covering all user-item pairs has become a key path to improving the platform's traffic efficiency and monetization capabilities.

[0055] To address the selection bias problem, current mainstream methods in the industry are mostly based on causal inference frameworks to construct unbiased estimators. Representative methods include Inverse Propensity Scoring (IPS) and Doublely Robust Estimation (DR). The IPS method approximates the overall expected loss by constructing the probabilities (i.e., propensity scores) of observed user-item pairs and learning by weighting the observed samples. The DR method, building upon this, incorporates a prediction error imputation mechanism, theoretically achieving unbiased estimation if either the propensity model or the error model is accurate. These methods have been widely applied in industrial systems such as recommendation and advertising to build more robust click-through rate or conversion rate prediction models.

[0056] However, while current mainstream methods like IPS (Inverse Bias Weighted) and DR (Double Robust) perform well in debiased recommendations, they heavily rely on the "positive bias assumption," which assumes that every user-item pair has a non-zero probability of being observed. If, due to compliance, copyright, operational strategies, or cold start, some pairs are never accessed, resulting in a zero probability of observation, these methods become completely ineffective because the denominator is zero or error filling is impossible. They cannot perform any reliable risk assessment or conversion rate prediction for this "unobserved area," creating blind spots in business operations.

[0057] The exemplary embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0058] Figure 1 A schematic diagram of a scenario for a recommended method or apparatus that can be applied to embodiments of this disclosure is shown.

[0059] Please refer to Figure 1 The diagram illustrates an implementation environment provided by an exemplary embodiment of this disclosure.

[0060] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0061] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, desktop computers, wearable devices, virtual reality devices, smart home devices, etc.

[0062] Server 105 can be a server that provides various services, such as a backend management server that supports the devices operated by users using terminal devices 101, 102, and 103. The backend management server can analyze and process received requests and other data, and feed the processing results back to the terminal devices.

[0063] A server can be a standalone physical server, a server cluster or a distributed system consisting of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. This disclosure does not impose any restrictions on this.

[0064] Server 105 may, for example, acquire multiple object-item pairs; server 105 may, for example, divide the multiple object-item pairs into extrapolated samples and interpolated samples, wherein the extrapolated samples include unobserved object-item pairs; server 105 may, for example, use a conversion rate prediction model to predict the conversion rate of the extrapolated samples to obtain a first conversion rate; server 105 may, for example, acquire an upper limit and a lower limit of the conversion rate corresponding to the extrapolated samples; server 105 may, for example, determine a first loss value based on the first conversion rate, the upper limit of the conversion rate, and the lower limit of the conversion rate; server 105 may, for example, use a conversion rate prediction model to predict the conversion rate of the interpolated samples and determine a second loss value based on the conversion rate prediction result; server 105 may, for example, combine the first loss value and the second loss value to train the conversion rate prediction model so as to make recommendations using the trained conversion rate prediction model.

[0065] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Server 105 can be a single physical server or a combination of multiple servers. Depending on actual needs, it can have any number of terminal devices, networks, and servers.

[0066] Under the above system architecture, this disclosure provides a method for making recommendations, which can be executed by any electronic device with computing power.

[0067] Before explaining the embodiments of this application, the symbols and targets can be defined first: consider a set of users. A collection of items Each pair contains n users and m items, where n is an integer greater than or equal to 1, and m is an integer greater than or equal to 1. The goal of CVR (Conversion Rate After Click) prediction is to accurately predict conversion results across all user-item pairs. Therefore, the target population is defined as all user-item pairs, denoted as […]. For each Let x be the number of times x is used. u,i Let r be the characteristic of this user-item pair, such as user age and item price. u,i∈{0,1} represents the conversion label of this user-item pair. This can be represented by f(x). u,i ;θ) represents the CVR prediction model, This represents the model's predicted value for the conversion label. Ideally, if all... The transformation result r u,i As can be observed, we can train the CVR prediction model by minimizing the ideal loss function as shown in equation (1):

[0068]

[0069] in, Let l(·,·) represent the prediction error, and it can be any loss function, such as cross-entropy loss or mean squared error. However, since the conversion labels of most user-item pairs are unobservable in practice, calculating the ideal loss is not feasible.

[0070] A direct way to train a CVR model is to minimize the following naive loss function (as in Equation (2)):

[0071]

[0072] However, observational data often exhibit selection bias, which leads to a skewed distribution of the observed sample set. Distribution and target population Inconsistent. Therefore, the naive estimator's estimate of the ideal loss is biased, as shown in equation (3):

[0073]

[0074] Directly minimizing the naive loss function will lead to systematic bias in the CVR prediction model, resulting in poor performance. Therefore, the objective of this application is to construct an unbiased estimator for the ideal loss, which is crucial for achieving unbiased learning of the CVR prediction model.

[0075] It should be noted that this application will use the example of an object-item pair being a user-item pair for explanation, but this application is not limited to this.

[0076] Figure 2 This is a flowchart illustrating a recommendation method according to an exemplary embodiment. The method provided in this disclosure can be executed by any electronic device with computing power, for example, the method can be performed by the above-described... Figure 1 The execution can be performed by a server or terminal device in the embodiments, or it can be performed by both a server and a terminal device. In the following embodiments, the server is used as the execution subject for illustration, but this disclosure is not limited to this.

[0077] Reference Figure 2 The recommended method provided in this disclosure may include the following steps.

[0078] Step S202: Obtain multiple object-item pairs.

[0079] An "object-item pair" refers to a pair consisting of a "subject" and an "object". In recommendation scenarios, the "object" can refer to a user, and the "item" can refer to the recommended content or product.

[0080] Step S204: Divide multiple object-item pairs into extrapolation samples and interpolation samples, wherein the extrapolation samples include object-item pairs that have not been observed.

[0081] Extrapolated samples can refer to object-item pairs that contain objects or items that have never been observed.

[0082] Unobserved object-item pairs can refer to a given dataset (usually the training set) where there are no interaction records or associated data between a particular object (such as a user) and a particular item (such as a product).

[0083] In some embodiments, the inference sample may include both observed object-item pairs (i.e., object-item pairs with conversion tags) and unobserved object-item pairs with a high conversion tendency. This application does not limit this.

[0084] Step S206: Use the conversion rate prediction model to predict the conversion rate of the extrapolated sample and obtain the first conversion rate.

[0085] In some embodiments, a conversion rate prediction model may refer to a network model that, after training, can predict the conversion rate of object-item pairs.

[0086] Step S208: Obtain the upper limit and lower limit of the conversion rate corresponding to the extrapolated sample.

[0087] In some embodiments, the above-mentioned upper and lower limits of conversion rate may be defined based on experience or determined based on inductive samples, and this application does not impose any restrictions on this.

[0088] Step S210: Determine the first loss value based on the first conversion rate, the upper limit of the conversion rate, and the lower limit of the conversion rate.

[0089] In some embodiments, a first loss value can be calculated based on a first conversion rate, a conversion rate upper limit, and a conversion rate lower limit. For example, the loss value between the first conversion rate and the conversion rate upper limit, and the loss value between the first conversion rate and the conversion rate lower limit can be calculated; then, the larger of the two loss values ​​can be selected as the first loss value, or the average of the two loss values ​​can be used as the first loss value, or the smaller of the two loss values ​​can be selected as the first loss value.

[0090] Step S212: Use the conversion rate prediction model to predict the conversion rate of the interpolated samples, and determine the second loss value based on the conversion rate prediction results.

[0091] In some embodiments, a second loss value may be calculated using the conversion labels and / or propensity labels of the interpolated samples, as well as the conversion rate prediction results.

[0092] Step S214: Combine the first loss value and the second loss value to train the conversion rate prediction model so that the trained conversion rate prediction model can be used for recommendations.

[0093] This disclosure proposes a model training method for recommendation systems. This method explicitly divides training samples into extrapolation samples (including unknown objects or items) and interpolation samples, and designs different loss calculation strategies for each. This effectively improves the generalization ability and recommendation accuracy of the conversion rate prediction model (such as generalization ability and recommendation accuracy in cold-start scenarios). By imposing empirical or statistical upper and lower bound constraints on the predicted values ​​of extrapolation samples, this method avoids the model making overly aggressive or unreliable predictions for unknown interactions. Simultaneously, it combines the traditional supervised loss of interpolation samples to jointly optimize model parameters, thereby achieving a balance between accurate interpolation between known users and items and robust extrapolation in unknown domains. Ultimately, this significantly improves the performance of the recommendation system when facing new users or new items.

[0094] Figure 3 This is a flowchart illustrating a method for determining a first loss value according to an exemplary embodiment.

[0095] refer to Figure 3 The method for determining the first loss value may include the following steps.

[0096] Step S302: Obtain the first conversion rate, upper limit of conversion rate, and lower limit of conversion rate for each extrapolated sample.

[0097] In some embodiments, the first conversion rate, the upper limit of the conversion rate, and the lower limit of the conversion rate can be calculated separately for each extrapolated sample.

[0098] Step S304: Based on the first conversion rate, upper limit of conversion rate, and lower limit of conversion rate of each extrapolated sample, determine the sample loss value of each extrapolated sample.

[0099] Step S306: Based on the sample loss value of each extrapolated sample, determine the first loss value corresponding to the extrapolated sample.

[0100] In some embodiments, the sample loss values ​​corresponding to each extrapolated sample can be summed to determine the first loss value corresponding to the entire extrapolated sample set, such as... in This represents the sample loss corresponding to the extrapolated sample (u,i). The corresponding extrapolated sample set.

[0101] The above embodiments, through a boundary constraint mechanism, forcibly limit the conversion rate predictions of extrapolated samples (unobserved user-item pairs) to a reasonable range defined by upper and lower bounds. This ensures that the model's predictions for these unknown regions remain stable and reasonable, avoiding extreme or absurd predictions that may arise due to missing data, and effectively improving the model's generalization ability and reliability in the extrapolation region. By summarizing the individual losses of all extrapolated samples, the final first loss value provides a crucial regularization signal for model training, guiding the model to be both conservative and accurate when exploring unknown domains.

[0102] Figure 4 This is a method for determining a sample loss value according to an exemplary embodiment.

[0103] In some embodiments, the extrapolated sample may include a first sample, and the sample loss value may include the first sample loss value corresponding to the first sample.

[0104] refer to Figure 4 The method for determining the above-mentioned sample loss value may include the following steps.

[0105] Step S402: Determine the difference between the first conversion rate of the first sample and the upper limit of the conversion rate of the first sample, and use it as the first difference.

[0106] In some embodiments, the formula may be referenced. Determine the first gap mentioned above. Here, l() can represent the distance function. This can represent the upper limit of the conversion rate, f(x) u,i ;θ) represents the first conversion rate of the first sample (u,i), and θ represents the model parameters.

[0107] Step S404: Determine the difference between the first conversion rate of the first sample and the lower limit of the conversion rate of the first sample, as the second difference.

[0108] In some embodiments, the formula may be referenced. The second gap mentioned above has been determined. It can represent the lower limit of the conversion rate.

[0109] Step S406: Determine the maximum value between the first gap and the second gap as the first sample loss value for the first sample.

[0110] In some embodiments, it can be achieved through Determine the loss value for the first sample.

[0111] The method described above employs a loss calculation strategy that maximizes boundary violation. It constructs a separate loss function for each extrapolated sample (an unobserved user-item pair): by calculating the difference between its predicted conversion rate and preset upper and lower bounds, and taking the maximum of the two as the loss value for that sample. This forces the model's predictions to fall within a reasonable range defined by the upper and lower bounds. If the predicted value exceeds the boundaries, a significant loss signal is generated to penalize the model, effectively ensuring the stability and reasonableness of the model's predictions for extrapolated samples, avoiding unreliable extreme value predictions, and enhancing the model's generalization ability and reliability in sparse data regions.

[0112] Figure 5 This is a flowchart illustrating a method for determining the upper and lower limits of conversion rate according to an exemplary embodiment.

[0113] In some embodiments, the interpolated samples may include positively biased samples with conversion labels.

[0114] refer to Figure 5 The method for determining the upper and lower limits of the conversion rate mentioned above may include the following steps.

[0115] Step S502: Obtain the unbiased imputation model, which is trained based on positive bias samples and their transformed labels.

[0116] In some embodiments, the unbiased imputation model can be trained based on positive bias samples and their transformed labels to predict the bias scores of object-item pairs.

[0117] In some embodiments, the aforementioned propensity score may refer to the probability that a particular item is displayed to a particular user by the system and clicked by that user.

[0118] For positively biased samples, an unbiased imputation model g(x) can be learned using existing debiasing objectives (such as IPS loss or DS loss). u,i ;φ). The following uses IPS loss as an example: Where, λ IPS >0 represents the hyperparameter, and φ represents the parameter to be learned. The learned incomplete model g(x) u,i ;φ) can accurately predict The labels of the interpolated samples are missing. How is the propensity score in IPS loss specifically calculated? Please refer to the following embodiments; this embodiment will not be described in detail here.

[0119] in, The sample set corresponding to the interpolated sample, o u,i The observed indicator variable represents the object-item pair. When the observed indicator variable is the first value (e.g., 1), it means that the transformation label of the object-item pair has been observed (i.e., a positively biased sample). When the observed indicator variable is the second value (e.g., 0), it means that the transformation label of the object-item pair has not been observed.

[0120] The high-precision debiasing training for the interpolation region described above can utilize and optimize debiasing techniques such as IPS / DR on samples identified as interpolation regions. This involves weighting or imputing errors in the observed positive and negative feedback, training a high-precision CVR prediction auxiliary model for this purpose. This model focuses on observed data, ensuring accurate and unbiased reconstruction of the user's true conversion probability based on existing interaction samples on the platform, while providing reliable pseudo-labels and reference distributions for extrapolated samples.

[0121] Step S504: Use the unbiased imputation model to predict the conversion rate of each interpolated sample and obtain the second conversion rate of each interpolated sample.

[0122] Step S506: Determine the upper limit and lower limit of the conversion rate of the extrapolated sample based on the second conversion rate of each interpolated sample.

[0123] In some embodiments, the maximum value of the second conversion rate can be directly used as the upper limit of the conversion rate, and the minimum value of the second conversion rate can be directly used as the lower limit of the conversion rate.

[0124] In some embodiments, the upper and lower limits of the conversion rate corresponding to the extrapolated sample can be calculated by combining the feature distance between each interpolated sample and a certain extrapolated sample and the second conversion rate corresponding to each interpolated sample.

[0125] The above embodiments employ a high-precision unbiased imputation and nearest-neighbor propagation mechanism. First, an accurate unbiased imputation model is trained using interpolated samples (observed data) and debiasing techniques (such as IPS loss), thereby obtaining a reliable estimate of the true conversion rate in the interpolated region. Subsequently, these estimated values ​​(second conversion rates) or their weighted statistics based on feature distance (such as maximum / minimum values) are used as upper and lower limits of the conversion rate for extrapolated samples. This provides reasonable and reliable boundary constraints for the model's predictions in the extrapolated region, effectively generalizing the learning results from the interpolated region to the extrapolated region. This solves the prediction uncertainty caused by missing data and significantly improves the prediction stability and accuracy of the CVR model in the full sample space (especially in unobserved regions).

[0126] In some embodiments, for extrapolated samples that have never been recommended, a reasonable upper and lower bound interval for the label can be constructed for each sample based on the continuity assumption that "similar samples should have similar feedback" (Lipschitz continuity assumption). Within this interval, the model is trained to minimize worst-case risk. By explicitly incorporating the maximum potential error constraint of unobserved regions into the training objective, the final CVR model not only ensures the prediction accuracy of observed regions but also provides a controllable boundary for business risks in unknown regions. This ensures that the platform can make robust and predictable traffic allocation decisions in key business scenarios such as new product cold starts and regional expansion.

[0127] Below, this application will propose the following method for determining the upper and lower limits of conversion rate, based on the aforementioned assumption that similar samples should have continuous similar feedback.

[0128] Figure 6 This is a method for determining the upper limit of conversion rate according to an exemplary embodiment.

[0129] In some embodiments, the extrapolation sample includes a second sample.

[0130] Below, this application will use the second sample as an example to explain how to determine the upper limit of the conversion rate of the extrapolated sample.

[0131] refer to Figure 6 The method for determining the upper limit of the conversion rate may include the following steps.

[0132] Step S602: Obtain the feature distance between each interpolated sample and the second sample, and use it as the first feature distance.

[0133] In some embodiments, L|x u,i -x s,t |2 Determine each interpolated sample x s,t With the second sample x u,i The feature distance between them is used as the first feature distance. This corresponds to the interpolated sample set. This corresponds to the extrapolated sample set.

[0134] Step S604: Obtain the first shifted conversion rate corresponding to each interpolated sample. The first shifted conversion rate is the sum of the second conversion rate of the interpolated sample and the first feature distance.

[0135] Step S606: Determine the minimum value among the first offset conversion rates corresponding to each interpolated sample, and use it as the upper limit of the conversion rate for the second sample.

[0136] In some embodiments, it can be achieved through Determine the upper limit of the conversion rate, where, The corresponding first offset conversion rate.

[0137] In some embodiments, the extrapolated sample may include a third sample; then determining the upper and lower limits of the conversion rate of the extrapolated sample based on the second conversion rate of each interpolated sample may include the following steps: obtaining the feature distance between each interpolated sample and the third sample as the second feature distance; obtaining the second offset conversion rate corresponding to each interpolated sample, wherein the second offset conversion rate is the difference between the second conversion rate of the interpolated sample and the second feature distance; and determining the maximum value among the first offset conversion rates corresponding to each interpolated sample as the lower limit of the conversion rate of the second sample.

[0138] Specifically, yes, depending on... Determine the lower limit of the conversion rate for the third sample (u,i), where, The corresponding second offset conversion rate.

[0139] The above embodiment employs a dynamic boundary generation mechanism based on smoothness constraints to individually calculate the upper and lower limits of the conversion rate for each extrapolated sample. This is achieved by calculating the feature distance between the extrapolated sample and all interpolated samples, and then shifting the accurate unbiased predicted value of the interpolated sample by this distance (adding to the upper limit and subtracting from the lower limit). Finally, the minimum value of all shifted values ​​is taken as the upper limit, and the maximum value as the lower limit. This method mathematically transforms the assumption that "similar samples should have similar conversion rates" into specific boundary constraints. It ensures that the prediction range set for the extrapolated sample strictly follows the inherent laws of the data (Lipschitz continuity) while fully considering its similarity to known samples. This provides a scientific, reasonable, and personalized constraint for the model's predictions in the extrapolation region, greatly enhancing the reliability and stability of the model's predictions in unknown regions.

[0140] Figure 7 This is a flowchart illustrating a sample partitioning method according to an exemplary embodiment.

[0141] In some embodiments, the interpolated samples may include positively biased samples with conversion labels.

[0142] refer to Figure 7 The above sample partitioning method may include the following steps.

[0143] Step S702: Obtain the observation indicator variable for each object-item pair, where the observation indicator variable is the first value, indicating that the transformation label of the object-item pair has been observed, and the observation indicator variable is the second value, indicating that the transformation label of the object-item pair has not been observed.

[0144] In some embodiments, to address the missing data issues present in the formally acquired data, an observation indicator variable o can be introduced. u,i ∈{0,1}, where o u,i =1 indicates that the conversion tag r of this user-item pair is 1. u,i Observed, o u,i =0 indicates that the conversion tag r of this user-item pair is 0. u,i No observations were made. Based on this, the set of observation data can be obtained, denoted as...

[0145] Step S704: Obtain the bias rating network model. The bias rating network model is trained based on positive bias samples and can predict the exposure bias of object-item pairs.

[0146] In some embodiments, the propensity score p can be used to... u,i Defined as the probability that a user-item pair is observed, i.e., p u,i =P(o u,i =1|x u,i ).

[0147] Step S706: Use a bias scoring network model to score each object-item pair to obtain the exposure bias of each object-item pair.

[0148] In some embodiments, the propensity score model can be trained by using the common binary cross-entropy loss function to train the propensity score model π(x). u,i ;ψ): Where CE(·,·) represents the cross-entropy loss function, λ ce >0 is a hyperparameter. Let Frobenius norm represent the parameter ψ.

[0149] Step S708: If the propensity score of the object-item pair is greater than the preset threshold or the observed index of the object-item pair is the first value (as per step S702), then the object-item pair is used as an interpolation sample.

[0150] Step S710: If the propensity score of the object-item pair is less than or equal to a preset threshold or the observed indicator variable of the object-item pair is a second value, then the object-item pair is used as an extrapolation sample.

[0151] The method described above trains a lightweight propensity assessment model to quantify and rank the observable probabilities of all user-item pairs on the platform. Combined with a predefined propensity score threshold, it automatically labels a batch of potential extrapolation samples with "extremely low interaction probabilities" (samples with zero propensity scores), distinguishing them from interpolation samples that meet the positive propensity score criteria (propensity scores greater than 0). This process not only clearly delineates the "interpolation region" that traditional de-biasing methods can cover but also identifies the unknown regions that must be extrapolated, providing clear boundaries for subsequent targeted risk control and modeling.

[0152] The method provided in the above embodiments uses an adaptive sample partitioning mechanism based on propensity score to quantify the probability of each user-item pair being observed using a trained propensity score network. Combined with preset thresholds and observation indicator variables, it automatically and accurately partitions all samples into interpolated samples (high propensity score or observed) and extrapolated samples (low propensity score or unobserved). This clearly defines the known data region on which the model can rely and the unknown data region to be explored, providing a reliable data foundation for subsequent high-precision unbiased training of the interpolated region and boundary constraint prediction of the extrapolated region. This effectively enhances the pertinence and scientific nature of the overall CVR prediction framework.

[0153] Figure 8 This is a flowchart illustrating a second loss value determination method according to an exemplary embodiment.

[0154] In some embodiments, the interpolated samples may include positively biased samples with conversion labels.

[0155] refer to Figure 8 The above-mentioned method for determining the second loss value may include the following steps.

[0156] Step S802: Use the conversion rate prediction model to predict the conversion rate of the interpolated samples and obtain the second conversion rate.

[0157] Step S804: Use the unbiased imputation model to predict the conversion rate label of the interpolated sample and obtain the pseudo label of the interpolated sample. The unbiased imputation model is trained based on the positive bias sample and the conversion label of the positive bias sample.

[0158] In some embodiments, this can be applied to interpolated samples. Learn an unbiased imputation model g(x) using existing debiasing objectives (such as IPS loss). u,i;φ). The following uses IPS loss as an example: Where, λ IPS >0 is a hyperparameter. The learned imputation model g(x) u,i ;φ) can accurately predict The labels of the interpolated samples are missing.

[0159] Step S806: Determine the second loss value corresponding to the interpolated sample based on the second conversion rate and the pseudo-label.

[0160] The technical solution provided in the above embodiments uses a high-precision pseudo-label supervision mechanism. First, it uses an unbiased imputation model (such as one trained with IPS loss) to generate accurate conversion rate pseudo-labels for the interpolated samples. Then, it uses these pseudo-labels as supervision signals to calculate the loss (i.e., the second loss value) between the prediction result (second conversion rate) of the conversion rate prediction model for the interpolated samples and the pseudo-labels. This ensures that the model can overcome the influence of selection bias during the learning process in the observed data region, approximate the real user conversion preferences, effectively improve the prediction accuracy and unbiasedness of the model in the interpolation region, and provide a stable and reliable basic learning capability for the entire framework.

[0161] Below, this application will explain a recommended method in conjunction with specific embodiments.

[0162] This embodiment proposes an extrapolation debiasing method (ExtraDebias for short) aimed at extrapolating from positively biased samples to zero-biased samples. To make this extrapolation reasonable, we introduce the following Lipschitz continuity assumption regarding the results:

[0163] (Lipschitz continuity assumption) For all user-item pairs and The following formula (4) holds true.

[0164]

[0165] in, This represents the conversion probability of user u to item i. This assumption is relatively mild, requiring only that users with similar characteristics have similar conversion outcomes for their corresponding items. It allows for conversions in zero-biased samples (denoted as set). ) and positively biased samples (denoted as set) Establish a "bridge" between them and provide r for zero-biased samples. u,i The upper bound information. Based on this, a minimax optimization problem can be constructed to control the performance of the prediction model in the worst case.

[0166] Specifically, the implementation process of the above method can be as follows.

[0167] 1. Training the propensity score model: The propensity score model π(x) is trained using the common binary cross-entropy loss function. u,i For details, please refer to formula (5).

[0168]

[0169] Where CE(·,·) represents the cross-entropy loss function, λ ce >0 is a hyperparameter. Let Frobenius norm represent the parameter ψ.

[0170] 2. Extrapolation and interpolation sample identification: After training the propensity model π(x)... u,i After ;ψ), the entire set can be... For each sample (u,i), calculate the propensity score estimate. Will satisfy And o u,i User-item pairs with a value of 0 are identified as extrapolated samples, forming a set. The rest are satisfied or o u,i User-item pairs with a value of 1 are considered as interpolated samples, forming a set. The threshold C is an adjustable hyperparameter.

[0171] 3. Perform unbiased learning on interpolated samples: For interpolated samples Learn an unbiased imputation model g(x) using existing debiasing objectives (such as IPS loss). u,i ;φ). The following uses IPS loss as an example, and can be found in formula (6).

[0172]

[0173] Where, λ IPS >0 is a hyperparameter. The learned imputation model g(x) u,i ;φ) can accurately predict Missing labels on interpolated samples. Missing label imputation on interpolated samples: imputation model g(x) u,i ;φ) for all Provide pseudo-label prediction g(x) u,i ;φ).

[0174] 4. Extrapolation Sample Label Upper and Lower Bounds Estimation: Based on the Lipschitz continuity assumption, for any extrapolated sample... Calculate the immediate upper bound of its label. and the immediate lower bound Interpolated samples

[0175]

[0176] 5. Calculation of worst-case risk for extrapolated samples: For any extrapolated sample In the prediction model f(x) u,i Under θ), the maximum prediction error can be calculated by referring to formula (9).

[0177]

[0178] 6. Training the prediction model by combining interpolated and extrapolated samples: Optimize the final prediction model f(x) by combining the results of interpolated samples and the worst-case risk of extrapolated samples. u,i The loss function is shown in formula (10).

[0179]

[0180] Where, λ ext >0 is a hyperparameter.

[0181] The prediction model f(x) obtained through the above algorithm u,i ;θ), with characteristic x u,i The input is the model name, and the output is the predicted CVR value. This model can accurately predict the CVR value for both interpolated samples with a propensity score greater than 0 and extrapolated samples with a propensity score equal to 0.

[0182] It should be particularly noted that the steps in the various embodiments of the above-described method for recommendation can be overlapped, substituted, added, or deleted from each other. Therefore, these reasonable permutations and combinations of the method for recommendation should also fall within the protection scope of this disclosure, and the protection scope of this disclosure should not be limited to the embodiments.

[0183] It should be noted that the scope of protection of this application should include, but is not limited to, the specific implementation methods described in the embodiments. Any alternative solution that uses a different name but substantially performs the same function and achieves the same technical effect falls within the scope of protection defined by the claims of this application.

[0184] Based on the same inventive concept, this disclosure also provides a recommended device, as shown in the following embodiment. Since the principle by which this device embodiment solves the problem is similar to that of the method embodiment described above, the implementation of this device embodiment can refer to the implementation of the method embodiment described above, and repeated details will not be described again.

[0185] Figure 9 This is a block diagram illustrating a recommended apparatus according to an exemplary embodiment. (Refer to...) Figure 9The device 900 for recommendation provided in this embodiment may include: an object item pair acquisition module 901, a sample differentiation module 902, a first conversion rate determination module 903, an upper and lower limit determination module 904, a first loss determination module 905, a second loss determination module 906, and a model training module 909.

[0186] The object-item pair acquisition module 901 can be used to acquire multiple object-item pairs; the sample differentiation module 902 can be used to divide the multiple object-item pairs into extrapolated samples and interpolated samples, wherein the extrapolated samples include unobserved object-item pairs; the first conversion rate determination module 903 can be used to predict the conversion rate of the extrapolated samples using a conversion rate prediction model to obtain a first conversion rate; the upper and lower limit determination module 904 can be used to acquire the upper and lower limits of the conversion rate corresponding to the extrapolated samples; the first loss determination module 905 can be used to determine a first loss value based on the first conversion rate, the upper and lower limits of the conversion rate; the second loss determination module 906 can be used to predict the conversion rate of the interpolated samples using the conversion rate prediction model and determine a second loss value based on the conversion rate prediction result; the model training module 909 can be used to train the conversion rate prediction model by combining the first loss value and the second loss value, so as to make recommendations through the trained conversion rate prediction model.

[0187] It should be noted that the aforementioned object item acquisition module 901, sample differentiation module 902, first conversion rate determination module 903, upper and lower limit determination module 904, first loss determination module 905, second loss determination module 906, and model training module 909 correspond to S202 to S218 in the method embodiment. The examples and application scenarios implemented by these modules and their corresponding steps are the same, but they are not limited to the content disclosed in the above method embodiment. It should be noted that these modules, as part of a device, can be executed in a computer system such as a set of computer-executable instructions.

[0188] In some embodiments, the first loss determination module 905 may include: a sample conversion rate acquisition submodule, a sample loss value determination submodule, and a first loss value determination submodule.

[0189] The sample conversion rate acquisition submodule can be used to acquire the first conversion rate, the upper limit of the conversion rate, and the lower limit of the conversion rate for each extrapolated sample; the sample loss value determination submodule can be used to determine the sample loss value for each extrapolated sample based on the first conversion rate, the upper limit of the conversion rate, and the lower limit of the conversion rate for each extrapolated sample; and the first loss value determination submodule can be used to determine the first loss value corresponding to the extrapolated sample based on the sample loss value for each extrapolated sample.

[0190] In some embodiments, the extrapolated sample includes a first sample, and the sample loss value includes a first sample loss value corresponding to the first sample; wherein, the sample loss value determination submodule may include: a first gap determination unit, a second gap determination unit, and a first sample loss value determination unit.

[0191] The first gap determination unit can be used to determine the gap between the first conversion rate of the first sample and the upper limit of the conversion rate of the first sample, as the first gap; the second gap determination unit can be used to determine the gap between the first conversion rate of the first sample and the lower limit of the conversion rate of the first sample, as the second gap; the first sample loss value determination unit can be used to determine the maximum value between the first gap and the second gap, as the first sample loss value of the first sample.

[0192] In some embodiments, the interpolated samples include positive tendency samples with conversion labels; wherein, the upper and lower limit determination module 904 may include: a model filling acquisition submodule, a second conversion rate determination submodule, and a conversion rate upper limit determination submodule.

[0193] The imputation model acquisition submodule can be used to acquire an unbiased imputation model, which is trained based on the positive bias samples and their conversion labels. The second conversion rate determination submodule can be used to predict the conversion rate of each interpolated sample using the unbiased imputation model to acquire the second conversion rate of each interpolated sample. The conversion rate upper limit determination submodule can be used to determine the upper and lower limits of the conversion rate of the extrapolated sample based on the maximum value of the second conversion rates of each interpolated sample.

[0194] In some embodiments, the extrapolated sample includes a second sample; wherein the conversion rate upper limit determination submodule may include: a feature distance determination unit, a first offset conversion rate determination unit, and a conversion rate upper limit determination unit.

[0195] The feature distance determination unit can be used to obtain the feature distance between each interpolated sample and the second sample, as the first feature distance; the first offset conversion rate determination unit can be used to obtain the first offset conversion rate corresponding to each interpolated sample, wherein the first offset conversion rate is the sum of the second conversion rate of the interpolated sample and the first feature distance; the conversion rate upper limit determination unit can be used to determine the minimum value among the first offset conversion rates corresponding to each interpolated sample, as the conversion rate upper limit of the second sample.

[0196] In some embodiments, the extrapolation sample includes a third sample; wherein the conversion rate upper limit determination submodule may include: a second feature distance determination unit, a distance difference determination unit, and a conversion rate lower limit determination unit.

[0197] The second feature distance determination unit can be used to obtain the feature distance between each interpolated sample and the third sample, as the second feature distance; the distance difference determination unit can be used to obtain the second offset conversion rate corresponding to each interpolated sample, where the second offset conversion rate is the difference between the second conversion rate of the interpolated sample and the second feature distance; the conversion rate lower limit determination unit can be used to determine the maximum value among the first offset conversion rates corresponding to each interpolated sample, as the conversion rate lower limit of the second sample.

[0198] In some embodiments, the interpolated samples include positive bias samples with conversion labels; wherein, the sample differentiation module 902 may include: an observation indicator variable determination submodule, a bias network model acquisition submodule, a bias prediction submodule, a first judgment submodule, and a second judgment submodule.

[0199] The system includes several sub-modules: an observation indicator variable determination sub-module for obtaining observation indicator variables for each object-item pair, where a first value indicates that the conversion label of the object-item pair has been observed, and a second value indicates that the conversion label of the object-item pair has not been observed; a tendency network model acquisition sub-module for obtaining a tendency scoring network model trained on the positive tendency samples to predict the exposure tendency of object-item pairs; a tendency prediction sub-module for scoring each object-item pair using the tendency scoring network model to obtain the exposure tendency of each object-item pair; a first judgment sub-module for using the object-item pair as an interpolation sample if the tendency score of the object-item pair is greater than a preset threshold or the observation indicator variable of the object-item pair is the first value; and a second judgment sub-module for using the object-item pair as an extrapolation sample if the tendency score of the object-item pair is less than or equal to the preset threshold or the observation indicator variable of the object-item pair is the second value.

[0200] In some embodiments, the interpolated samples include positive tendency samples with conversion labels; wherein, the second loss determination module 906 may include: a second conversion rate prediction submodule, a pseudo-label prediction submodule, and a second loss value determination submodule.

[0201] The second conversion rate prediction submodule can be used to predict the conversion rate of the interpolated sample using the conversion rate prediction model to obtain a second conversion rate; the pseudo-label prediction submodule can be used to predict the conversion rate label of the interpolated sample using an unbiased imputation model to obtain a pseudo-label of the interpolated sample, wherein the unbiased imputation model is trained based on the positive bias sample and the conversion label of the positive bias sample; the second loss value determination submodule can be used to determine the second loss value corresponding to the interpolated sample based on the second conversion rate and the pseudo-label.

[0202] Since the functions of the device 900 have been described in detail in their respective method embodiments, they will not be repeated here.

[0203] The modules and / or sub-modules and / or units described in the embodiments of this disclosure can be implemented in software or hardware. The described modules and / or sub-modules and / or units can also be located in a processor. The names of these modules and / or sub-modules and / or units do not, in some cases, constitute a limitation on the module and / or sub-module and / or unit itself.

[0204] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a portion of a module or program segment containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer program instructions.

[0205] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0206] Figure 10 A schematic diagram of an electronic device suitable for implementing embodiments of the present disclosure is shown. It should be noted that... Figure 10The illustrated electronic device 1000 is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0207] like Figure 10 As shown, the electronic device 1000 includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage section 1008 into a random access memory (RAM) 1003. The RAM 1003 also stores various programs and data required for the operation of the electronic device 1000. The CPU 1001, ROM 1002, and RAM 1003 are interconnected via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0208] The following components are connected to I / O interface 1005: an input section 1006 including a keyboard, mouse, etc.; an output section 1007 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. Drive 1010 is also connected to I / O interface 1005 as needed. Removable media 1011, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 1010 as needed so that computer programs read from them can be installed into storage section 1008 as needed.

[0209] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing computer program instructions for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 1009, and / or installed from removable medium 1011. When the computer program is executed by central processing unit (CPU) 1001, it performs the functions defined above in the system of this disclosure.

[0210] It should be noted that the computer-readable storage medium disclosed herein may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable computer program instructions. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable storage medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. Computer program instructions contained on a computer-readable storage medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0211] In another aspect, this disclosure also provides a computer-readable storage medium, which may be included in the device described in the above embodiments; or it may exist independently and not assembled into the device. The computer-readable storage medium carries one or more programs that, when executed by the device, enable the device to perform the following functions: acquiring multiple object-item pairs; dividing the multiple object-item pairs into extrapolated samples and interpolated samples, wherein the extrapolated samples include unobserved object-item pairs; using a conversion rate prediction model to predict the conversion rate of the extrapolated samples, obtaining a first conversion rate; acquiring an upper limit and a lower limit of the conversion rate corresponding to the extrapolated samples; determining a first loss value based on the first conversion rate, the upper limit, and the lower limit; using the conversion rate prediction model to predict the conversion rate of the interpolated samples, and determining a second loss value based on the conversion rate prediction result; and training the conversion rate prediction model by combining the first loss value and the second loss value, so as to make recommendations using the trained conversion rate prediction model.

[0212] According to one aspect of this disclosure, a computer program product or computer program is provided, comprising computer program instructions stored in a computer-readable storage medium. The computer program instructions are read from the computer-readable storage medium, and a processor executes the computer program instructions to implement the methods provided in various optional implementations of the above embodiments.

[0213] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions of the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) and includes several computer program instructions to cause an electronic device (such as a server or terminal device, etc.) to execute the method according to the embodiments of this disclosure.

[0214] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0215] It should be understood that this disclosure is not limited to the detailed structures, drawing arrangements or implementations shown herein; rather, this disclosure is intended to cover various modifications and equivalent arrangements contained within the spirit and scope of the appended claims.

Claims

1. A method for making recommendations, characterized in that, include: Retrieve multiple object-item pairs; The multiple object-item pairs are divided into extrapolation samples and interpolation samples, wherein the extrapolation samples include unobserved object-item pairs; and the interpolation samples include positive tendency samples with conversion labels. The conversion rate of the extrapolated sample is predicted using a conversion rate prediction model to obtain a first conversion rate. Obtain the upper and lower limits of the conversion rate corresponding to the extrapolated sample; Based on the first conversion rate, the upper limit of the conversion rate, and the lower limit of the conversion rate, a first loss value is determined; The conversion rate prediction model is used to predict the conversion rate of the interpolated samples, and a second loss value is determined based on the conversion rate prediction results. The conversion rate prediction model is trained by combining the first loss value and the second loss value, so as to make recommendations using the trained conversion rate prediction model.

2. The method according to claim 1, characterized in that, Based on the first conversion rate, the upper limit of the conversion rate, and the lower limit of the conversion rate, a first loss value is determined, including: Obtain the first conversion rate, upper limit of conversion rate, and lower limit of conversion rate for each extrapolated sample; Based on the first conversion rate, upper limit of conversion rate, and lower limit of conversion rate of each extrapolated sample, determine the sample loss value of each extrapolated sample; Based on the sample loss value of each extrapolated sample, determine the first loss value corresponding to the extrapolated sample.

3. The method according to claim 2, characterized in that, The extrapolated sample includes a first sample, and the sample loss value includes the first sample loss value corresponding to the first sample; wherein, based on the first conversion rate, the upper limit of the conversion rate, and the lower limit of the conversion rate of each extrapolated sample, the sample loss value of each extrapolated sample is determined, including: The difference between the first conversion rate of the first sample and the upper limit of the conversion rate of the first sample is determined as the first difference; The difference between the first conversion rate of the first sample and the lower limit of the conversion rate of the first sample is determined as the second difference; The maximum value between the first gap and the second gap is determined as the first sample loss value of the first sample.

4. The method according to claim 1, characterized in that, The interpolated samples include positively inclined samples with conversion labels; wherein, obtaining the upper and lower limits of the conversion rate corresponding to the extrapolated samples includes: An unbiased imputation model is obtained, which is trained based on the positive bias samples and the transformed labels of the positive bias samples; The conversion rate of each interpolated sample is predicted using the unbiased imputation model to obtain the second conversion rate of each interpolated sample. The upper and lower limits of the conversion rate of the extrapolated sample are determined based on the second conversion rate of each interpolated sample.

5. The method according to claim 4, characterized in that, The extrapolated sample includes a second sample; wherein, determining the upper and lower limits of the conversion rate of the extrapolated sample based on the second conversion rate of each interpolated sample includes: Obtain the feature distance between each interpolated sample and the second sample, and use it as the first feature distance; Obtain the first offset conversion rate corresponding to each interpolated sample, where the first offset conversion rate is the sum of the second conversion rate of the interpolated sample and the first feature distance; The minimum value among the first offset conversion rates corresponding to each interpolated sample is determined as the upper limit of the conversion rate for the second sample.

6. The method according to claim 4, characterized in that, The extrapolated samples include a third sample; wherein, determining the upper and lower limits of the conversion rate of the extrapolated samples based on the second conversion rate of each interpolated sample includes: The feature distance between each interpolated sample and the third sample is obtained and used as the second feature distance. Obtain the second offset conversion rate corresponding to each interpolated sample, where the second offset conversion rate is the difference between the second conversion rate of the interpolated sample and the second feature distance; The maximum value among the first offset conversion rates corresponding to each interpolated sample is determined as the lower limit of the conversion rate for the second sample.

7. The method according to claim 1, characterized in that, The interpolated samples include positive tendency samples with conversion labels; wherein, the multiple object-item pairs are divided into extrapolated samples and interpolated samples, wherein the extrapolated samples include unobserved object-item pairs, including: Obtain observation indicator variables for each object-item pair, wherein when the observation indicator variable is a first value, it indicates that the transformation label of the object-item pair has been observed, and when the observation indicator variable is a second value, it indicates that the transformation label of the object-item pair has not been observed; Obtain a bias rating network model, which is trained based on the positive bias samples and is capable of predicting the exposure bias of object-item pairs; The bias scoring network model is used to score each object-item pair by bias, thereby obtaining the exposure bias of each object-item pair. If the propensity score of the object-item pair is greater than a preset threshold or the observed indicator variable of the object-item pair is the first value, then the object-item pair is used as an interpolation sample. If the propensity score of the object-item pair is less than or equal to the preset threshold or the observation indicator variable of the object-item pair is the second value, then the object-item pair is used as an extrapolation sample.

8. The method according to claim 1, characterized in that, The interpolated samples include positively inclined samples with conversion labels; wherein, the conversion rate prediction model is used to predict the conversion rate of the interpolated samples, and a second loss value is determined based on the conversion rate prediction result, including: The conversion rate prediction model is used to predict the conversion rate of the interpolated sample to obtain a second conversion rate. The conversion rate label is predicted for the interpolated sample using an unbiased imputation model to obtain the pseudo label of the interpolated sample. The unbiased imputation model is trained based on the positive trend sample and the conversion label of the positive trend sample. Based on the second conversion rate and the pseudo-label, determine the second loss value corresponding to the interpolated sample.

9. An electronic device, characterized in that, include: Memory and processor; The memory is used to store computer program instructions; the processor invokes the computer program instructions stored in the memory to implement the recommended method as described in any one of claims 1-7.

10. A computer program product comprising computer program instructions stored in a computer-readable storage medium, characterized in that, When the computer program instructions are executed by the processor, they implement the method according to any one of claims 1-8.

Images