Optimization method and device of recommendation system, storage medium and electronic equipment

Abstract

The application discloses an optimization method and device of a recommendation system, a storage medium and an electronic device. The method comprises: obtaining a first data set, wherein a same-interest data subset included in the first data set is a collection of first corpus data corresponding to a same-interest user; determining a bias feature according to the same-interest data subset and a preset recommendation model; performing bias elimination on the first data set to obtain a second data set according to the bias feature; determining a bias score according to the bias feature and a preset bias model; determining a second recommendation score according to the second data set and the preset recommendation model; and training the preset recommendation model to obtain a target recommendation model according to the bias score and the second recommendation score. The application solves the technical problem that the recommendation system has various biases, which greatly interferes with the model recommendation score, causes a great difference in the model recommendation score for the same data corresponding to the same interest, and thus affects the recommendation effect, reduces the user experience, and decreases the overall revenue.

Description

Technical Field

[0001] This application relates to the field of data processing, and more specifically, to an optimization method, apparatus, storage medium, and electronic device for a recommendation system. Background Technology

[0002] Current recommendation systems are always plagued by various subtle biases, such as those related to location, channel, time, audience, and exposure. These biases significantly interfere with the model's recommendation scores, leading to vastly different recommendation scores for data corresponding to the same interest. This negatively impacts recommendation performance, reduces user experience, and lowers overall revenue. However, there is currently a lack of practical methods for identifying such biases, relying heavily on human experience. Furthermore, methods for debiasing often involve manual sampling or adding manual debiasing factors to the model output, which still results in inaccurate recommendation scores that quickly drift after deployment.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This application provides an optimization method, apparatus, storage medium, and electronic device for a recommendation system, at least addressing the technical problem that various biases in the recommendation system greatly interfere with the model's recommendation score, resulting in a large difference in the model's recommendation score for data corresponding to the same interest, thereby affecting the recommendation effect, reducing user experience, and decreasing overall revenue.

[0005] According to a first aspect of the embodiments of this application, an optimization method for a recommender system is provided. The method, applied to the recommender system, includes: acquiring a first dataset, wherein the first dataset is a collection of multiple first corpus data, and the first dataset includes a subset of data with shared interests, the subset of data with shared interests being a collection of first corpus data corresponding to users with the same interest; determining deviation features based on the first dataset and a preset recommender model, wherein the preset recommender model can determine a first recommender score based on the first dataset; debiasing the first dataset based on the deviation features to obtain a second dataset; determining a deviation score based on the deviation features and a preset deviation model; determining a second recommender score based on the second dataset and the preset recommender model; and training the preset recommender model based on the deviation score and the second recommender score to obtain a target recommender model.

[0006] Optionally, based on the subset of data with shared interests and the preset recommendation model, the deviation features are determined, including: determining a third recommendation score based on the subset of data with shared interests and the preset recommendation model; eliminating any feature in the subset of data with shared interests to obtain a subset of data with shared interests after feature elimination; determining a fourth recommendation score based on the subset of data with shared interests after feature elimination and the preset recommendation model; and determining the deviation features based on the third recommendation score and the fourth recommendation score.

[0007] Optionally, based on the third and fourth recommended scores, the deviation feature is determined, including: using a first preset formula to determine the difference between the third and fourth recommended scores; comparing the difference with a preset difference value to obtain a comparison result; and in response to the comparison result indicating that the difference is greater than the preset difference value, determining the feature to be eliminated as a deviation feature.

[0008] Optionally, the first dataset is debiased to obtain the second dataset based on the bias characteristics, including: judging the number of bias characteristics to obtain a judgment result; in response to the judgment result indicating that the number of bias characteristics is a first preset number, the first dataset is debiased using a uniform distribution sampling method to obtain the second dataset.

[0009] Optionally, the optimization method of the recommendation system further includes: in response to the judgment result indicating that the number of biased features is greater than a first preset number, using an average joint sampling method to debias the first dataset to obtain a second dataset; wherein the average joint sampling method includes: performing uniform distribution sampling on the first dataset according to each biased feature to obtain multiple uniform distribution sampling results; and determining the average joint sampling result based on the multiple uniform distribution sampling results and a second preset formula.

[0010] Optionally, the target recommendation model is obtained by training the preset recommendation model based on the deviation score and the second recommendation score, including: determining the fifth recommendation score using a third preset formula based on the deviation score and the second recommendation score; and training the preset recommendation model based on the fifth recommendation score to obtain the target recommendation model.

[0011] Optionally, the optimization method for the recommender system also includes: the recommender system uses a target recommender model to complete the recommender task, wherein the recommender task includes at least recommending content that the user is interested in.

[0012] According to a second aspect of the embodiments of this application, an optimization apparatus for a recommendation system is also provided, comprising: an acquisition module, configured to acquire a first dataset, wherein the first dataset is a collection of multiple first corpus data, the first dataset including a subset of data with shared interests, the subset of data with shared interests being a collection of first corpus data corresponding to users with shared interests; a first determination module, configured to determine deviation features based on the subset of data with shared interests and a preset recommendation model, wherein the preset recommendation model can determine a first recommendation score based on the subset of data with shared interests; a debiasing module, configured to debias the first dataset based on the deviation features to obtain a second dataset; a second determination module, configured to determine a deviation score based on the deviation features and a preset deviation model; a third determination module, configured to determine a second recommendation score based on the second dataset and the preset recommendation model; and a training module, configured to train the preset recommendation model based on the deviation score and the second recommendation score to obtain a target recommendation model.

[0013] According to a third aspect of the embodiments of this application, a non-volatile storage medium is also provided, wherein a program is stored in the non-volatile storage medium, and the program controls the device where the non-volatile storage medium is located to execute the above-mentioned optimization method of the recommendation system when it runs.

[0014] According to a fourth aspect of the embodiments of this application, an electronic device is also provided, including: a memory and a processor, the processor being configured to run a program stored in the memory, wherein the program executes the optimization method of the above-described recommendation system during runtime.

[0015] In this embodiment, the following steps are taken: First dataset is obtained, which is a collection of multiple first corpus data, including a subset of data with shared interests, which is a collection of first corpus data corresponding to users with the same interests; Bias features are determined based on the subset of data with shared interests and a preset recommendation model, wherein the preset recommendation model can determine a first recommendation score based on the subset of data with shared interests; Bias is eliminated from the first dataset based on the bias features to obtain a second dataset; Bias scores are determined based on the bias features and a preset bias model; A second recommendation score is determined based on the second dataset and the preset recommendation model; The preset recommendation model is then trained based on the bias scores and the second recommendation scores. The method for obtaining the target recommendation model involves debiasing the first dataset, and then training a pre-defined recommendation model based on the debiasing score and the second recommendation score. The trained target recommendation model takes into account the influence of bias features on the model, resulting in more accurate predictions of recommendation scores. This achieves the technical effect of reducing the interference of various biases in the recommendation system on the recommendation model, making the recommendation scores predicted by the model more accurate. This solves the technical problem that various biases in the recommendation system greatly interfere with the model's recommendation scores, leading to huge differences in model recommendation scores for data corresponding to the same interest, thus affecting the recommendation effect, reducing user experience, and decreasing overall revenue. Attached Figure Description

[0016] The accompanying drawings, which are included to provide a further understanding of this application and form part of this application, illustrate exemplary embodiments and are used to explain this application, but do not constitute an undue limitation of this application. In the drawings:

[0017] Figure 1 This is a flowchart of an optimization method for a recommendation system according to one embodiment of the present invention;

[0018] Figure 2 This is a structural block diagram of an optimization apparatus for a recommendation system according to one embodiment. Detailed Implementation

[0019] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0020] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0021] According to one embodiment of the present invention, an embodiment of an optimization method for a recommendation system is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0022] This method embodiment can be executed in an electronic device, similar control device, or system that includes a memory and a processor. Taking an electronic device as an example, the electronic device may include one or more processors and a memory for storing data. Optionally, the electronic device may also include a communication device for communication functions and a display device. Those skilled in the art will understand that the above structural description is merely illustrative and does not limit the structure of the electronic device. For example, the electronic device may include more or fewer components than described above, or have a different configuration than described above.

[0023] A processor may include one or more processing units. For example, a processor may include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processing (DSP) chip, a microcontroller unit (MCU), a field-programmable gate array (FPGA), a neural network processing unit (NPU), a tensor processing unit (TPU), or an artificial intelligence (AI) processor. Different processing units may be independent components or integrated into one or more processors. In some instances, electronic devices may also include one or more processors.

[0024] The memory can be used to store computer programs, such as the computer program corresponding to the optimization method of the recommendation system in the embodiments of the present invention. The processor implements the optimization method of the recommendation system by running the computer program stored in the memory. The memory may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to electronic devices via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0025] Communication devices are used to receive or send data via a network. Specific examples of such networks may include wireless networks provided by the mobile terminal's communication provider. In one example, the communication device includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In another example, the communication device may be a radio frequency (RF) module used for wireless communication with the Internet.

[0026] The display device can be, for example, a touchscreen liquid crystal display (LCD) and a touch display (also referred to as a "touchscreen" or "touch screen"). This LCD allows the user to interact with the user interface of the mobile terminal. In some embodiments, the mobile terminal has a graphical user interface (GUI), which allows the user to interact with the GUI by touching and / or gesturing on a touch-sensitive surface. Optional human-computer interaction functions include: creating web pages, drawing, word processing, creating electronic documents, playing games, video conferencing, instant messaging, sending and receiving emails, a call interface, playing digital video, playing digital music, and / or web browsing, etc. Executable instructions for performing the above human-computer interaction functions are configured / stored in one or more processor-executable computer program products or readable storage media.

[0027] This embodiment provides an optimization method for a recommendation system running on electronic devices. Figure 1 This is a flowchart of an optimization method for a recommendation system according to one embodiment of the present invention, such as... Figure 1 As shown, the process includes the following steps:

[0028] Step S101: Obtain the first dataset, wherein the first dataset is a collection of multiple first corpus data, and the first dataset includes a subset of data with the same interest, which is a collection of first corpus data corresponding to users with the same interest.

[0029] Specifically, in a recommendation system, the system recommends content to users based on their potential interests. Different users may have the same or different interests. When executing step S101 to obtain the first dataset, it is first necessary to collect the recommendation system's logs and preprocess the data. This preprocessing may include data cleaning, data integration, and data transformation. Then, several sets of first corpus data are obtained. This set of first corpus data constitutes the first dataset. The first dataset further includes a subset of data with shared interests, which is a collection of first corpus data corresponding to multiple users with the same interests.

[0030] It should be noted that the correspondence between users and corpus data is based on liking. If the recommendation system believes that a user likes a certain corpus data, then the user is associated with that corpus data.

[0031] Step S102: Determine the deviation features based on the subset of data with the same interest and the preset recommendation model, wherein the preset recommendation model can determine the first recommendation score based on the subset of data with the same interest.

[0032] Specifically, after obtaining the first dataset in step S101, a pre-set recommendation model can be used to determine the deviation features in the subset of data with shared interests. The pre-set recommendation model can predict multiple first recommendation scores for each piece of first corpus data in the subset of data with shared interests, with one first recommendation score corresponding to one first piece of first corpus data. The deviation feature is the feature where the predicted recommendation score by the pre-set recommendation model deviates significantly before and after removing the feature.

[0033] It is important to note that the recommendation score is used to characterize the user's level of interest in the current corpus data. The higher the recommendation score, the higher the user's level of interest in the current corpus data.

[0034] Step S103: Based on the bias characteristics, the first dataset is debiased to obtain the second dataset.

[0035] Specifically, after determining the bias characteristics in the first dataset through step S102, the first dataset can be debiased based on the bias characteristics, resulting in a second dataset that does not include the bias characteristics.

[0036] It should be noted that after debiasing the first dataset, the distribution of each value corresponding to the bias feature under that feature is changed. At this point, the bias feature is no longer considered a bias feature.

[0037] Step S104: Determine the deviation score based on the deviation characteristics and the preset deviation model.

[0038] It is important to note that when determining the deviation score, the second dataset also needs to be input into the preset deviation model.

[0039] Understandably, the pre-defined bias model can determine the bias score for each piece of corpus data based on the input bias features and the second dataset. The bias score is a recommendation score that takes into account the bias features.

[0040] Step S105: Determine the second recommendation score based on the second dataset and the preset recommendation model.

[0041] Specifically, the preset recommendation model can predict multiple second recommendation scores based on each piece of data in the second dataset.

[0042] Step S106: Train the preset recommendation model to obtain the target recommendation model based on the deviation score and the second recommendation score.

[0043] Specifically, each piece of corpus data in the second dataset corresponds to a bias score and a second recommendation score. Then, based on multiple bias scores and second recommendation scores, the preset recommendation model is trained to obtain a target recommendation model with more accurate recommendations.

[0044] Understandably, a target recommendation model that takes into account bias characteristics will produce a more accurate recommendation score than a pre-defined recommendation model.

[0045] In this embodiment, the following steps are taken: First dataset is obtained, which is a collection of multiple first corpus data, including a subset of data with shared interests, which is a collection of first corpus data corresponding to users with the same interests; Bias features are determined based on the subset of data with shared interests and a preset recommendation model, wherein the preset recommendation model can determine a first recommendation score based on the subset of data with shared interests; Bias is eliminated from the first dataset based on the bias features to obtain a second dataset; Bias scores are determined based on the bias features and a preset bias model; A second recommendation score is determined based on the second dataset and the preset recommendation model; The preset recommendation model is then trained based on the bias scores and the second recommendation scores. The method for obtaining the target recommendation model involves debiasing the first dataset, and then training a pre-defined recommendation model based on the debiasing score and the second recommendation score. The trained target recommendation model takes into account the influence of bias features on the model, resulting in more accurate predictions of recommendation scores. This achieves the technical effect of reducing the interference of various biases in the recommendation system on the recommendation model, making the recommendation scores predicted by the model more accurate. This solves the technical problem that various biases in the recommendation system greatly interfere with the model's recommendation scores, leading to huge differences in model recommendation scores for data corresponding to the same interest, thus affecting the recommendation effect, reducing user experience, and decreasing overall revenue.

[0046] In some embodiments of the present invention, when performing step S102 to determine the deviation features based on the subset of data with shared interests and the preset recommendation model, the following steps may be included:

[0047] Step S1021: Determine the third recommendation score based on the subset of data with the same interest and the preset recommendation model;

[0048] Step S1022: Eliminate any feature in the subset of data with the same interest to obtain the subset of data with the same interest after feature elimination;

[0049] Step S1023: Determine the fourth recommendation score based on the subset of data with similar interests after feature removal and the preset recommendation model;

[0050] Step S1024: Determine the deviation characteristics based on the third and fourth recommendation scores.

[0051] Specifically, combining steps S1021 to S1024 above, when determining the deviation feature, firstly, the subset of data with the same interest is input into the preset recommendation model. The preset recommendation model can output a third recommendation score based on the input first dataset, where each piece of first corpus data in the subset of data with the same interest corresponds to a third recommendation score. Next, any feature in the subset of data with the same interest is eliminated to obtain the subset of data with the same interest after feature elimination. Then, the subset of data with the same interest after feature elimination is input into the preset recommendation model to obtain a fourth recommendation score, where each piece of corpus data in the subset of data with the same interest after feature elimination corresponds to a fourth recommendation score. Finally, based on the third recommendation score and the fourth recommendation score, it is determined whether the currently eliminated feature is a deviation feature.

[0052] It should be noted that, in some embodiments of the present invention, steps S1022 to S1024 are executed cyclically until it is determined whether each feature in the subset of data of the same interest is a deviation feature.

[0053] In some embodiments of the present invention, when performing step S1024, which determines the deviation characteristics based on the third recommendation score and the fourth recommendation score, the following steps may be included:

[0054] Step S1024a: Determine the difference between the third recommendation score and the fourth recommendation score using the first preset formula;

[0055] Step S1024b: Compare the difference value with the preset difference value to obtain the comparison result;

[0056] Step S1024c: In response to the comparison result showing that the difference value is greater than the preset difference value, the feature to be eliminated is determined to be a deviation feature.

[0057] The first preset formula can be: Difference value = (Fourth preset score - Third preset score) / Third preset score

[0058] For example, if the preset difference value is 50%, and the third recommendation score is 80 and the fourth recommendation score is 90, the difference between the two is 12.5%, which is less than the preset difference value, then the feature being removed is not a biased feature; if the third recommendation score is 80 and the fourth recommendation score is 130, the difference between the two is 62.5%, which is greater than the preset difference value, then the feature being removed is a biased feature.

[0059] It should be noted that the first preset formula can also be a difference formula, in which case the difference value = the fourth preset score - the third preset score. In this case, the preset difference value is defined as the difference between the two. The preset difference value can be preset according to actual needs.

[0060] It should be noted that, in some embodiments of the present invention, before removing a feature, the average of all the obtained third recommendation scores is calculated to obtain the average third recommendation score. After removing the corresponding feature, the average of all the obtained fourth recommendation scores is calculated to obtain the average fourth recommendation score. Then, the difference value is determined based on the average third recommendation score and the average fourth recommendation score, and then steps 1024b and 1024c are executed.

[0061] In some embodiments of the present invention, when performing step S103 to debias the first dataset based on the bias characteristics to obtain the second dataset, the following steps may be included:

[0062] Step S1031: Determine the number of deviation features to obtain a judgment result;

[0063] In step S1032, in response to the judgment result indicating that the number of bias features is the first preset number, the first dataset is debiased using a uniform distribution sampling method to obtain the second dataset.

[0064] It should be noted that the first preset quantity is 1.

[0065] Specifically, combining steps S1031 to S1032, when the number of biased features is determined to be 1, a uniform distribution sampling method can be used to debias (sample) the first dataset to obtain the second dataset. Uniform distribution sampling can change the distribution of each value corresponding to the biased feature in the first dataset under that feature to obtain the second dataset. At this time, the biased feature will no longer have a significant impact on the prediction of the recommendation model, that is, the feature is no longer a biased feature, and the second dataset no longer includes the biased feature.

[0066] In some embodiments of the present invention, when performing step S103 to debias the first dataset according to the bias characteristics to obtain the second dataset, the following steps may also be included:

[0067] In response to the judgment result indicating that the number of bias features is greater than the first preset number, the first dataset is debiased using an average joint sampling method to obtain the second dataset; wherein, the joint random sampling method includes: performing uniform distribution sampling on the first dataset according to each bias feature to obtain multiple uniform distribution sampling results; and determining the average joint sampling result based on the multiple uniform distribution sampling results and the second preset formula.

[0068] Specifically, when the number of biased features is greater than the first preset number, i.e., greater than 1, it is not appropriate to use the uniform distribution sampling method for sampling. Instead, the average joint sampling method is used to debias the first dataset to obtain the second dataset.

[0069] It should be noted that after uniformly distributing the sampling to the first dataset, each bias feature corresponds to a uniformly distributing sampling result. The result of uniform sampling is an expected value less than 1, which represents the discard rate of a certain value under the bias feature. Each value under each bias feature corresponds to a discard rate. The second preset formula is to multiply the uniformly distributing sampling results corresponding to multiple bias features in the same corpus data to obtain the expected value of the average joint sampling. Then, the first dataset is sampled based on the expected value of the average joint sampling to obtain the second dataset.

[0070] In some embodiments of the present invention, when performing step S106 to train the preset recommendation model to obtain the target recommendation model based on the deviation score and the second recommendation score, the following steps may be included:

[0071] Step S1061: Determine the fifth recommendation score using the third preset formula based on the deviation score and the second recommendation score;

[0072] Step S1062: Train the preset recommendation model based on the fifth recommendation score to obtain the target recommendation model.

[0073] Specifically, in steps S1061 to S1062, the fifth recommendation score is first calculated using the third preset formula through the deviation score and the second recommendation score. Then, the fifth recommendation score is used as the preset recommendation model training parameter to train the preset recommendation model to obtain the target recommendation model.

[0074] It should be noted that the third preset formula is: Fifth Recommendation Score = First Weight * Deviation Score + Second Weight * Second Recommendation Score, where the first weight and the second weight can be preset values ​​or obtained through continuous iterative training.

[0075] Optionally, the optimization method for the recommender system also includes: the recommender system uses a target recommender model to complete the recommender task, wherein the recommender task includes at least recommending content that the user is interested in.

[0076] For example, if the recommendation system is a news system, the recommendation task includes at least recommending news that the user is interested in, and the recommendation task may also include collecting user operation information on the recommended news for subsequent recommendations.

[0077] It should be noted that the process of training the preset recommendation model to obtain the target recommendation model is completed offline. Once the target recommendation model is obtained, the online service of the recommendation system uses the target recommendation model to complete the recommendation task. After the target recommendation model is launched, the preset bias model is decoupled from the recommendation system. The preset bias model is uncoupled from the target recommendation model, and the decoupling will not affect the target recommendation model.

[0078] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0079] This embodiment also provides an optimization apparatus for a recommendation system, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0080] Figure 2 This is a structural block diagram of an optimization device for a recommendation system according to one embodiment of the present invention, such as... Figure 2 As shown, an optimization device 200 for a recommendation system is used as an example. This device includes: an acquisition module 201, used to acquire a first dataset, wherein the first dataset is a collection of multiple first corpus data, and the first dataset includes a subset of data with shared interests, which is a collection of first corpus data corresponding to users with the same interests; a first determination module 202, used to determine bias features based on the subset of data with shared interests and a preset recommendation model, wherein the preset recommendation model can determine a first recommendation score based on the subset of data with shared interests; a debiasing module 203, used to debias the first dataset based on the bias features to obtain a second dataset; a second determination module 204, used to determine a bias score based on the bias features and a preset bias model; a third determination module 205, used to determine a second recommendation score based on the second dataset and the preset recommendation model; and a training module 206, used to train the preset recommendation model based on the bias score and the second recommendation score to obtain a target recommendation model.

[0081] Optionally, the first determining module 202 is further configured to: determine a third recommendation score based on a subset of data with shared interests and a preset recommendation model; eliminate any feature in the subset of data with shared interests to obtain a subset of data with shared interests after feature elimination; determine a fourth recommendation score based on the subset of data with shared interests after feature elimination and the preset recommendation model; and determine a deviation feature based on the third recommendation score and the fourth recommendation score.

[0082] Optionally, the first determining module 202 is further configured to: determine the difference between the third recommended score and the fourth recommended score using a first preset formula; compare the difference with a preset difference value to obtain a comparison result; and, in response to the comparison result indicating that the difference is greater than the preset difference value, determine that the feature to be eliminated is a deviation feature.

[0083] Optionally, the debiasing module 203 is further configured to: determine the number of deviation features to obtain a determination result; and in response to the determination result indicating that the number of deviation features is a first preset number, debias the first dataset using a uniform distribution sampling method to obtain a second dataset.

[0084] Optionally, the debiasing module 203 is further configured to: in response to the judgment result indicating that the number of bias features is greater than a first preset number, debias the first dataset using an average joint sampling method to obtain a second dataset; wherein the average joint sampling method includes: performing uniform distribution sampling on the first dataset according to each bias feature to obtain multiple uniform distribution sampling results; and determining the average joint sampling result based on the multiple uniform distribution sampling results and a second preset formula.

[0085] Optionally, the training module 206 is further configured to: determine the fifth recommendation score using a third preset formula based on the deviation score and the second recommendation score; and train the preset recommendation model based on the fifth recommendation score to obtain the target recommendation model.

[0086] Optionally, the optimization device of the recommendation system further includes a recommendation module connected to the training module 206. The recommendation module is used to complete the recommendation task using the target recommendation model, wherein the recommendation task includes at least recommending content that the user is interested in.

[0087] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0088] Embodiments of the present invention also provide a non-volatile storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above method embodiments when run on a computer or processor.

[0089] Optionally, in this embodiment, the non-volatile storage medium described above can be configured to store a computer program for performing the following steps:

[0090] Step S101: Obtain the first dataset, wherein the first dataset is a collection of multiple first corpus data, and the first dataset includes a subset of data with the same interest, which is a collection of first corpus data corresponding to users with the same interest.

[0091] Step S102: Determine the deviation features based on the subset of data with the same interest and the preset recommendation model, wherein the preset recommendation model can determine the first recommendation score based on the subset of data with the same interest.

[0092] Step S103: Based on the bias characteristics, the first dataset is debiased to obtain the second dataset.

[0093] Step S104: Determine the deviation score based on the deviation characteristics and the preset deviation model.

[0094] Step S105: Determine the second recommendation score based on the second dataset and the preset recommendation model.

[0095] Step S106: Train the preset recommendation model to obtain the target recommendation model based on the deviation score and the second recommendation score.

[0096] Optionally, in this embodiment, the aforementioned non-volatile storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0097] Embodiments of the present invention also provide an electronic device, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the steps in any of the above method embodiments.

[0098] Optionally, in this embodiment, the processor in the above-described electronic device may be configured to run a computer program to perform the following steps:

[0099] Step S101: Obtain the first dataset, wherein the first dataset is a collection of multiple first corpus data, and the first dataset includes a subset of data with the same interest, which is a collection of first corpus data corresponding to users with the same interest.

[0100] Step S102: Determine the deviation features based on the subset of data with the same interest and the preset recommendation model, wherein the preset recommendation model can determine the first recommendation score based on the subset of data with the same interest.

[0101] Step S103: Based on the bias characteristics, the first dataset is debiased to obtain the second dataset.

[0102] Step S104: Determine the deviation score based on the deviation characteristics and the preset deviation model.

[0103] Step S105: Determine the second recommendation score based on the second dataset and the preset recommendation model.

[0104] Step S106: Train the preset recommendation model to obtain the target recommendation model based on the deviation score and the second recommendation score.

[0105] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0106] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0107] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0108] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0109] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0110] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0111] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0112] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. An optimization method for a recommender system, characterized in that, The method is applied to a recommendation system, and comprises: obtaining a first data set, wherein the first data set is a collection of a plurality of first corpus data, and the first data set comprises a same-interest data subset, which is a collection of the first corpus data corresponding to a same-interest user; determining a bias feature according to the same-interest data subset and a preset recommendation model, wherein the preset recommendation model can determine a first recommendation score according to the same-interest data subset; de-biasing the first data set according to the bias feature to obtain a second data set; determining a bias score according to the bias feature and a preset bias model; determining a second recommendation score according to the second data set and the preset recommendation model; training the preset recommendation model to obtain a target recommendation model according to the bias score and the second recommendation score; wherein the determining of the bias feature according to the same-interest data subset and the preset recommendation model comprises: determining a third recommendation score according to the same-interest data subset and the preset recommendation model; de-biasing any feature in the same-interest data subset to obtain a same-interest data subset after de-biasing; determining a fourth recommendation score according to the same-interest data subset after de-biasing and the preset recommendation model; determining a difference value of the third recommendation score and the fourth recommendation score by using a first preset formula; comparing the difference value with a preset difference value to obtain a comparison result; and in response to the comparison result indicating that the difference value is greater than the preset difference value, determining that the currently de-biased feature is the bias feature.

2. The method of claim 1, wherein, The de-biasing of the first data set according to the bias feature to obtain the second data set comprises: judging the number of the bias features to obtain a judgment result; in response to the judgment result indicating that the number of the bias features is a first preset number, de-biasing the first data set by using a uniform distribution sampling method to obtain the second data set.

3. The method of claim 2, wherein, Further comprising: in response to the judgment result indicating that the number of the bias features is greater than the first preset number, de-biasing the first data set by using an average joint sampling method to obtain the second data set; wherein the average joint sampling method comprises: de-biasing the first data set according to each bias feature to obtain a plurality of uniform distribution sampling results; determining an average joint sampling result according to the plurality of uniform distribution sampling results and a second preset formula.

4. The method of claim 1, wherein, The training of the preset recommendation model to obtain the target recommendation model according to the bias score and the second recommendation score comprises: determining a fifth recommendation score by using a third preset formula according to the bias score and the second recommendation score; training the preset recommendation model to obtain the target recommendation model according to the fifth recommendation score.

5. The method of claim 1, wherein, Further comprising: the recommendation system completes a recommendation task by using the target recommendation model, wherein the recommendation task at least comprises recommending content interested by a user.

6. An optimization apparatus of a recommendation system characterized by comprising: comprises: An acquisition module is configured to acquire a first data set, wherein the first data set is a collection of a plurality of first corpus data, and the first data set includes a same-interest data subset, and the same-interest data subset is a collection of the first corpus data corresponding to a same-interest user. A first determination module is configured to determine a bias feature according to the same-interest data subset and a preset recommendation model, wherein the preset recommendation model can determine a first recommendation score according to the same-interest data subset. A bias elimination module is configured to eliminate bias from the first data set to obtain a second data set according to the bias feature. A second determination module is configured to determine a bias score according to the bias feature and a preset bias model. A third determination module is configured to determine a second recommendation score according to the second data set and the preset recommendation model. A training module is configured to train the preset recommendation model to obtain a target recommendation model according to the bias score and the second recommendation score. The first determination module is further configured to determine a third recommendation score according to the same-interest data subset and the preset recommendation model, eliminate any feature in the same-interest data subset to obtain an eliminated same-interest data subset, determine a fourth recommendation score according to the eliminated same-interest data subset and the preset recommendation model, determine a difference value of the third recommendation score and the fourth recommendation score by using a first preset formula, compare the difference value with a preset difference value to obtain a comparison result, and determine that a currently eliminated feature is the bias feature in response to the comparison result indicating that the difference value is greater than the preset difference value.

7. A non-volatile storage medium, characterized by The non-volatile storage medium stores a program, wherein the program controls a device in which the non-volatile storage medium is located to perform the optimization method of the recommendation system in any one of claims 1 to 5 when the program is running.

8. An electronic device, comprising: The device comprises: a memory and a processor, and the processor is configured to run a program stored in the memory, wherein the program performs the optimization method of the recommendation system in any one of claims 1 to 5 when the program is running.

Images