Data processing method and device, equipment and storage medium

Abstract

The embodiment of the invention relates to a data processing method and device, equipment and a storage medium. The method comprises the steps that multiple object recommendation models are utilized, multiple recommendation metrics of candidate objects to be recommended for a target entity are determined respectively, and the recommendation metrics indicate the degree that the candidate objects are recommended to the target entity; obtaining corresponding model features of a plurality of object recommendation models, wherein the model feature of each object recommendation model is used for representing the ability of the object recommendation model in the aspect of object recommendation; determining a target recommendation metric of the candidate object for the target entity based on the corresponding model features of the plurality of object recommendation models and the plurality of recommendation metrics; and generating an object recommendation result for the target entity based on the target recommendation metric. Therefore, the recommendation capability of the fused object recommendation model can be improved, and the quality of the object recommendation service can be improved.

Description

Technical Field

[0001] Example embodiments of the present disclosure generally relate to the field of computers, and more particularly, to methods, apparatuses, devices, and computer-readable storage media for data processing. Background Art

[0002] With the rapid development of Internet technology, more and more applications and websites are designed to have functions related to object recommendation. Such applications and websites can provide object recommendation services with the help of object recommendation models. For the same entity, the recommendation results of different object recommendation models may be different. Therefore, people expect to be able to merge the recommendation results of multiple object recommendation models to improve the quality of object recommendation services. Summary of the invention

[0003] In a first aspect of the present disclosure, a method for data processing is provided. The method includes: using multiple object recommendation models, respectively determining multiple recommendation metrics of candidate objects to be recommended for a target entity, the recommendation metrics indicating the degree to which the candidate objects are recommended to the target entity; obtaining corresponding model features of multiple object recommendation models, the model features of each object recommendation model being used to characterize the ability of the object recommendation model in object recommendation; determining a target recommendation metric of the candidate object for the target entity based on the corresponding model features of the multiple object recommendation models and the multiple recommendation metrics; and generating an object recommendation result for the target entity based on the target recommendation metric.

[0004] In a second aspect of the present disclosure, a device for data processing is provided. The device includes: a recommendation metric determination module, configured to use multiple object recommendation models to respectively determine multiple recommendation metrics of candidate objects to be recommended for a target entity, the recommendation metrics indicating the degree to which the candidate objects are recommended to the target entity; a model feature acquisition module, configured to acquire corresponding model features of multiple object recommendation models, the model features of each object recommendation model being used to characterize the ability of the object recommendation model in object recommendation; a target metric determination module, configured to determine the target recommendation metric of the candidate object for the target entity based on the corresponding model features of the multiple object recommendation models and the multiple recommendation metrics; and a recommendation result generation module, configured to generate an object recommendation result for the target entity based on the target recommendation metric.

[0005] In a third aspect of the present disclosure, an electronic device is provided. The device includes at least one processing unit; and at least one memory, the at least one memory is coupled to the at least one processing unit and stores instructions for execution by the at least one processing unit. When the instructions are executed by the at least one processing unit, the device executes the method of the first aspect.

[0006] In a fourth aspect of the present disclosure, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and the computer program can be executed by a processor to implement the method of the first aspect.

[0007] It should be understood that the contents described in this content section are not intended to limit the key features or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. In the accompanying drawings, the same or similar reference numerals represent the same or similar elements, wherein:

[0009] Figure 1 A schematic diagram showing an example environment in which embodiments of the present disclosure can be implemented;

[0010] Figure 2 A flowchart showing a process for data processing according to some embodiments of the present disclosure is shown;

[0011] Figure 3 A schematic diagram showing an example process of data processing according to some embodiments of the present disclosure;

[0012] Figure 4 A schematic diagram showing an example of determining a target recommendation metric according to some embodiments of the present disclosure;

[0013] Figure 5 A schematic structural block diagram showing an apparatus for data processing according to some embodiments of the present disclosure; and

[0014] Figure 6 A block diagram of an electronic device is shown in which one or more embodiments of the present disclosure may be implemented. DETAILED DESCRIPTION

[0015] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0016] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0017] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0018] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0019] It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0020] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0021] In the description of the embodiments of the present disclosure, the term "including" and similar terms should be understood as open inclusion, that is, "including but not limited to". The term "based on" should be understood as "based at least in part on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The term "some embodiments" should be understood as "at least some embodiments". Other explicit and implicit definitions may be included below. The terms "first", "second", etc. may refer to different or the same objects. Other explicit and implicit definitions may be included below.

[0022] As used herein, the term "model" can learn the association between the corresponding input and output from the training data, so that the corresponding output can be generated for a given input after the training is completed. The generation of the model can be based on machine learning technology. Deep learning is a machine learning algorithm that processes inputs and provides corresponding outputs by using multiple layers of processing units. In this article, "model" may also be referred to as "machine learning model", "machine learning network" or "network", which are used interchangeably in this article. A model may also include different types of processing units or networks.

[0023] Generally, machine learning can be roughly divided into three stages, namely the training stage, the testing stage, and the application stage (also called the inference stage). In the training stage, a given model can be trained using a large amount of training data, and the parameter values ​​are continuously updated iteratively until the model can obtain consistent inferences that meet the expected goals from the training data. Through training, the model can be considered to be able to learn the association from input to output (also called the mapping of input to output) from the training data. The parameter values ​​of the trained model are determined. In the testing stage, the test input is applied to the trained model to test whether the model can provide the correct output, thereby determining the performance of the model. In the application stage, the model can be used to process the actual input based on the parameter values ​​obtained from the training to determine the corresponding output.

[0024] Embodiments of the present disclosure relate to recommending objects to entities. In some cases, an entity may perform a conversion or transformation operation on an object. The entity here may include, but is not limited to, an individual, an organization, an institution, a user, etc. The object may include, but is not limited to, an item, media content (e.g., a video), etc. The conversion is a resource demand behavior that an entity may perform on an object, such as, but is not limited to, purchase, click, browse, register, download, etc.

[0025] As briefly mentioned above, for the same entity, the recommendation results of different object recommendation models may be different. Therefore, people expect to be able to merge the recommendation results of multiple object recommendation models to improve the quality of object recommendation services.

[0026] Generally speaking, when it is necessary to compare and verify the recommendation effects of multiple object recommendation models, these models are often connected to a certain amount of traffic online, and various indicators of these models in actual traffic are observed and compared. Since each model needs to be launched separately and allocated traffic for verification, there may not be so much experimental traffic that can be connected in the early stage of access to the business or when the number of experiments is too large, which may cause large errors in the final verification results. In addition, different models may have their own advantages and disadvantages in different scenarios. It is difficult for a model to achieve the best results in all scenarios, and different models cannot be directly integrated to further improve the recommendation effect, which will affect the quality of the recommendation.

[0027] To this end, an embodiment of the present disclosure proposes an improved scheme for data processing. According to various embodiments of the present disclosure, multiple object recommendation models are used to respectively determine multiple recommendation metrics for candidate objects to be recommended for a target entity. The recommendation metric indicates the degree to which the candidate object is recommended to the target entity. The corresponding model features of the multiple object recommendation models are obtained, and the model features of each object recommendation model are used to characterize the ability of the object recommendation model in object recommendation. Based on the corresponding model features of the multiple object recommendation models and the multiple recommendation metrics, the target recommendation metric of the candidate object for the target entity is determined. Based on the target recommendation metric, an object recommendation result for the target entity is generated.

[0028] Thus, multiple model features and multiple recommendation metrics corresponding to multiple object recommendation models can be fused, and the final target recommendation metric can be determined based on the fusion result. The object recommendation result for the target entity can be determined based on the fused target recommendation metric. In this way, compared with a single object recommendation model, the recommendation ability of the fused object recommendation model is improved, thereby improving the quality of the object recommendation service.

[0029] Figure 1 1 is a schematic diagram of an example environment 100 in which embodiments of the present disclosure can be implemented. Figure 1 As shown, example environment 100 may include electronic device 110 .

[0030] In the example environment 100, the electronic device 110 can determine the object recommendation result 104 of the object 102 for the target entity. It can be understood that although Figure 1 Only one object 102 and one object recommendation result 104 are shown, but the object 102 may include multiple objects, and the object recommendation result 104 may also include multiple object recommendation results corresponding to the multiple objects. The present disclosure does not limit the number of objects and object recommendation results.

[0031] The electronic device 110 can use the object recommendation model 115 to determine the object recommendation result 104 of the object 102 for the target entity. The object recommendation model 115 can be, for example, any neural network that can perform object recommendation, including but not limited to a fully convolutional network (FCN), a convolutional neural network (CNN), a recurrent neural network (RNN), etc., and the embodiments of the present disclosure are not limited in this respect. In some embodiments, the object recommendation model 115 can be stored locally on the electronic device 110, and the electronic device 110 can directly use the local object recommendation model 115 to implement object recommendation. In some embodiments, the object recommendation model 115 can also be a model stored in the cloud, and the electronic device 110 can call the object recommendation model 115 stored in the cloud to implement object recommendation when it needs to perform related tasks.

[0032] The electronic device 110 may include any computing system with computing capabilities, such as various computing devices / systems, terminal devices, server devices, etc. The terminal device may be any type of mobile terminal, fixed terminal or portable terminal, etc., including a mobile phone, a desktop computer, a laptop computer, a notebook computer, a netbook computer, a tablet computer, a media computer, a multimedia tablet, a personal communication system (PCS) device, a personal navigation device, a personal digital assistant (PDA), an audio / video player, a digital camera / camcorder, a positioning device, a television receiver, a radio broadcast receiver, an e-book device, a virtual reality (VR) all-in-one machine, a game console, a game book, or any combination of the foregoing, including accessories and peripherals of these devices or any combination thereof.

[0033] The server-side device can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content distribution networks, and big data and artificial intelligence platforms. Server-side devices can include computing systems / servers such as mainframes, edge computing nodes, computing devices in cloud environments, and so on.

[0034] It should also be understood that the environment 100 and its structure and functionality are described for exemplary purposes only and do not imply any limitation on the scope of the present disclosure.

[0035] Figure 2 1 shows a flowchart of a process 200 for data processing according to some embodiments of the present disclosure. The process 200 may be implemented at the electronic device 110. For ease of discussion, reference will be made to Figure 1 The process 200 is described based on the environment 100. It should be noted that the operations performed by the electronic device 110 described above and the operations performed by the electronic device 110 described later may be performed by related applications installed on the electronic device 110.

[0036] In block 210, the electronic device 110 uses multiple object recommendation models to respectively determine multiple recommendation metrics of the candidate objects to be recommended for the target entity, where the recommendation metrics indicate the degree to which the candidate objects are recommended to the target entity. The recommendation metrics may include, but are not limited to, recommendation scores, recommendation probabilities, etc. It is understood that the higher the degree to which the candidate objects are recommended to the target entity, the greater the recommendation metrics.

[0037] The electronic device 110 may determine a model input based at least on the candidate object to be recommended. The electronic device 110 further provides the model input to the trained multiple object recommendation models respectively. The multiple object recommendation models may further output multiple model outputs that may indicate the recommendation metrics of the candidate object for the target entity based on the model input respectively. The electronic device 110 may obtain the multiple model outputs from the multiple object recommendation models and determine the recommendation metrics indicated by each of the multiple model outputs.

[0038] Regarding the training of the multiple object recommendation models, in some embodiments, the multiple object recommendation models may be trained by the electronic device 110. In other embodiments, the multiple object recommendation models may also be trained by other electronic devices, and the electronic device 110 may, for example, obtain the trained multiple object recommendation models via a communication connection with other electronic devices. It should be noted that the multiple object recommendation models may be trained by the same electronic device or by different electronic devices (for example, the electronic device 110 may obtain different trained object recommendation models from different other electronic devices).

[0039] The following combination Figure 3 The process of training and applying the multiple object recommendation models at the electronic device 110 to obtain multiple recommendation metrics is described. Figure 3 FIG. 3 is a schematic diagram showing an example process 300 of data processing according to some embodiments of the present disclosure. Figure 3 As shown, the electronic device 110 may acquire multiple models 320 (eg, may include model 320 - 1, ..., model 320 -N, where N is a positive integer).

[0040] In the case where the multiple models 320 are all untrained models, the electronic device 110 can perform (310) model training on the multiple models 320. For example, the electronic device 110 can perform (310-1) model training 1 on model 1, perform (310-N) model training N on model N, and so on.

[0041] In response to the completion of the training, the electronic device 110 may determine multiple model parameters of the multiple trained models 320. The electronic device 110 may then fix the model parameters of the multiple models 320, and perform (330) model estimation using the multiple trained models 320. Similar to performing model training, the electronic device 110 may perform (330-1) model estimation 1 on model 1, perform (330-N) model estimation N on model N, and so on. The electronic device 110 may, for example, obtain entity features of a target entity to be recommended, and object features of a candidate object to be recommended. The electronic device 110 may determine a model input based at least on the entity features and the object features. The electronic device 110 may then perform model estimation on the multiple models by respectively inputting the determined model inputs into the multiple models 320, and obtaining multiple model outputs output by the multiple models 320 that may indicate the recommendation metrics of the candidate object for the target entity.

[0042] The electronic device 110 may determine (340) the recommendation metrics of each of the multiple models 320 based on the model outputs corresponding to each of the multiple models 320. Exemplarily, the electronic device 110 may determine (340-1) the recommendation metric 1 corresponding to model 1, determine (340-N) the recommendation metric N corresponding to model N, and so on. It is understood that if the recommendation metric is a specific recommendation score, the electronic device 110 may obtain the recommendation score indicating the degree to which the candidate object is recommended to the target entity determined by each model in the multiple models 320. The greater the degree of recommendation, the greater the value of the recommendation score.

[0043] Return to reference Figure 2 In box 220, the electronic device 110 obtains corresponding model features of multiple object recommendation models, and the model features of each object recommendation model are used to characterize the ability of the object recommendation model in object recommendation. The model features of the object recommendation model may include, for example, metric features (also referred to as scoring features) and prediction features (also referred to as reflux features). The metric features of each object recommendation model may be used to characterize the recommendation metric determined using the object recommendation model. The specific determination method of the metric features and the prediction features is described below using the first object recommendation model as an example. It can be understood that the first object recommendation model can be any object recommendation model among the multiple object recommendation models.

[0044] Regarding the metric feature, the electronic device 110 may, for example, determine the metric feature of the first object recommendation model based on the recommendation metric corresponding to the first object recommendation model. In some embodiments, the electronic device 110 may determine the metric feature of the first object recommendation model using the value of the first recommendation metric determined by the first object recommendation model. Exemplarily, taking the recommendation metric as a recommendation score as an example, the electronic device 110 may determine the specific value of the recommendation score determined by the first object recommendation model, and determine the metric feature of the first object recommendation model based on the value.

[0045] In some embodiments, the electronic device 110 may also use the ranking of the first recommendation metric among multiple recommendation metrics to determine the metric characteristics of the first object recommendation model. Exemplarily, the electronic device 110 may obtain multiple recommendation metrics corresponding to multiple object recommendation models. The electronic device 110 may, for example, sort the multiple recommendation metrics from large to small / from small to large based on the specific numerical values ​​corresponding to each of the multiple recommendation metrics. The electronic device 110 may further determine the metric characteristics of the first object recommendation model based on the ranking of the first recommendation metric among multiple recommendation metrics (for example, in the case of a large to small sorting method, the first recommendation metric ranks XXth).

[0046] In some embodiments, the electronic device 110 may also use the ranking of the first recommendation metric in a group of recommendation metrics to determine the metric characteristics of the first object recommendation model. Exemplarily, the electronic device 110 may obtain a group of objects to be recommended, including candidate objects. The electronic device 110 may use the first object recommendation model to determine the recommendation metrics corresponding to each of the group of objects, that is, the electronic device 110 may obtain a group of recommendation metrics indicating the degree to which each object in the group of objects is recommended to the target entity. The electronic device 110 may sort the group of recommendation metrics from large to small / from small to large based on the specific numerical values ​​corresponding to each of the group of recommendation metrics. The electronic device 110 may further determine the metric characteristics of the first object recommendation model based on the ranking of the first recommendation metric in a group of recommendation metrics (for example, in the case of a large to small ranking method, the first recommendation metric ranks XXth).

[0047] It can be understood that the electronic device 110 can determine the metric characteristics of the first object recommendation model by utilizing a combination of one or more of the value of the first recommendation metric determined by the first object recommendation model, the ranking of the first recommendation metric among multiple recommendation metrics, and the ranking of the first recommendation metric among a set of recommendation metrics.

[0048] Regarding the estimated features, the electronic device 110 may, for example, determine the estimated features of the first object recommendation model based on historical estimated information of the first object recommendation model. The historical estimated information of the first object recommendation model may, for example, include a recommendation metric determined historically by the first object recommendation model for the object to be recommended and a first conversion result of whether the object to be recommended is converted.

[0049] In some embodiments, the electronic device 110 may determine the estimated features of the first object recommendation model using the average ranking of the corresponding recommendation metrics determined by the first object recommendation model for multiple reference objects. The entities to which these multiple reference objects are recommended perform conversion operations. For example, these multiple reference objects are all objects recommended to the target entity, and these multiple reference objects are all objects for which the target entity performs conversion operations.

[0050] As described above, the electronic device 110 can obtain a group of objects to be recommended each time, that is, the electronic device 110 can obtain multiple groups of objects in history. The electronic device 110 can use the first object recommendation model to respectively determine the recommendation metrics corresponding to each of the group of objects, that is, the electronic device 110 can obtain a set of recommendation metrics indicating the degree to which each object in the group of objects is recommended to the target entity. The electronic device 110 can sort the group of recommendation metrics from large to small / small to large based on the specific numerical values ​​corresponding to each of the group of recommendation metrics. The electronic device 110 can obtain multiple groups of rankings corresponding to multiple groups of objects in history. For the convenience of expression, the following is explained by taking sorting from large to small as an example. In the case of sorting from large to small, the higher the ranking, the greater the degree to which the object recommendation model determines that the corresponding object is recommended to the target entity.

[0051] The electronic device 110 can determine the ranking of the recommended metric of each reference object in a corresponding group of objects among multiple reference objects. The electronic device 110 can obtain that the recommended metric ranking corresponding to the first reference object in the first group of objects is 3rd, the recommended metric ranking corresponding to the second reference object in the second group of objects is 2nd, and so on. The multiple reference objects may include different objects from the same group of objects. For example, the electronic device 110 can obtain that the recommended metric ranking corresponding to the third reference object in the third group of objects is 3rd, and the recommended metric ranking corresponding to the fourth reference object in the third group of objects is 5th. The multiple reference objects may also include different objects from the same group of objects. For example, the electronic device 110 can obtain that the recommended metric ranking corresponding to the fifth reference object in the fourth group of objects is 1st, and the recommended metric ranking corresponding to the fifth reference object in the fifth group of objects is 3rd.

[0052] The electronic device 110 may determine multiple rankings corresponding to the multiple reference objects, and determine the average ranking of the multiple rankings. For example, if the multiple rankings include 3 rankings, namely the 3rd, 4th, and 5th, the electronic device 110 may determine the average ranking to be the 4th. This average ranking may, for example, reflect the accuracy of the recommendation metric determined by the first object recommendation model. The electronic device 110 may further determine the estimated features of the first object recommendation model based on the determined average ranking.

[0053] In some embodiments, the electronic device 110 may also determine the estimated features of the first object recommendation model by using the proportion of reference objects ranked before the first threshold rank in the corresponding recommendation metrics of the multiple reference objects. Exemplarily, after the electronic device 110 determines the multiple rankings corresponding to the multiple reference objects, each of the multiple rankings may be compared with the first threshold rank to determine whether the corresponding ranking is a ranking before the first threshold rank. For example, if 5 reference objects are included, and the 5 rankings corresponding to the 5 reference objects are the 3rd, 4th, 2nd, 4th and 1st, respectively, and the first threshold rank is 3, then the electronic device 110 may determine that only the third reference object ranked 2nd and 1st and the fifth reference object corresponding to the ranking are before the first threshold rank. The electronic device 110 may further determine that the proportion of reference objects ranked before the first threshold rank in the 5 reference objects is 40%. This proportion may also reflect the accuracy of the recommendation metric determined by the first object recommendation model, for example. The electronic device 110 may determine the estimated features of the first object recommendation model based on this proportion.

[0054] In some embodiments, the electronic device 110 may also determine the estimated features of the first object recommendation model using the type of objects ranked before the second threshold rank in the historical recommendation metric generated by the first object recommendation model. Exemplarily, if 5 reference objects are included, and the 5 rankings corresponding to these 5 reference objects are 3rd, 4th, 2nd, 4th and 1st, respectively, and the second threshold rank is 3, then the electronic device 110 may determine that only the third reference object ranked 2nd and 1st and the fifth reference object corresponding to the ranking are before the second threshold rank. The electronic device 110 may determine the type of objects corresponding to the third reference object and the fifth reference object respectively. The electronic device 110 may then determine the estimated features of the first object recommendation model based on the determined types. It is understandable that the types corresponding to different objects may be different. The number of types may, for example, reflect the diversity of the first object recommendation model, and the larger the number, the stronger the diversity of the model. The electronic device 110 may, for example, determine the estimated features of the first object recommendation model based on the determined number of multiple types.

[0055] Similarly, the electronic device 110 may determine the estimated features of the first object recommendation model by utilizing a combination of an average ranking of corresponding recommendation metrics determined by the first object recommendation model for multiple reference objects, a proportion of reference objects ranked before a first threshold ranking in the corresponding recommendation metrics of the multiple reference objects, and one or more of the types of objects ranked before a second threshold ranking in the historical recommendation metrics generated by the first object recommendation model.

[0056] The electronic device 110 can determine the metric features and estimated features of the first object recommendation model based on the above method, and determine the metric features and estimated features of other object recommendation models in the multiple object recommendation models based on a similar method to obtain the metric features and estimated features corresponding to each of the multiple object recommendation models. Thus, the electronic device 110 can obtain multiple model features corresponding to the multiple object recommendation models.

[0057] The historical prediction information here can be updated based on the result of each object recommended by the electronic device 110 and whether the object is converted. Specifically, the electronic device 110 can recommend the candidate object to the target entity based on the object recommendation result of the candidate object to be recommended. The determination of the object recommendation result will be described in detail later and will not be repeated here. The electronic device 110 can obtain a second conversion result of whether the candidate object is converted by the target entity. The electronic device 110 can then update the corresponding historical prediction information of multiple object recommendation models based on the second conversion result and multiple recommendation metrics. That is, the electronic device 110 can update the historical prediction information based on the conversion results of whether the candidate objects recommended in history are converted by the target entity and the multiple recommendation metrics corresponding to the multiple object recommendation models. The conversion results and multiple recommendation metrics here can also be referred to as reflow information, reflow data, etc.

[0058] Continue to refer Figure 3 ,like Figure 3As shown, the electronic device 110 can perform (350) result fusion on the obtained multiple recommendation metrics (for example, it can include recommendation metric 1 corresponding to model 1, recommendation metric N corresponding to model N, and so on). The electronic device 110 can determine (360) the object recommendation result for the target entity based on the fusion result. The specific method of performing fusion and determining the object recommendation result will be described in detail below, and will not be repeated here. The electronic device 110 can recommend the candidate object to the target entity based on the object recommendation result. The electronic device 110 can recommend the candidate object to the target entity by performing (370) online display (that is, by presenting the media content corresponding to the candidate object in the corresponding user interface). The electronic device 110 can obtain the reflow data 380 (for example, it can include the conversion result of whether the candidate object is converted by the target entity and the multiple recommendation metrics corresponding to the multiple object recommendation models) to update the historical estimation information. The electronic device 110 can update the corresponding estimation features of the multiple object recommendation models based on the updated historical estimation information, and perform result fusion based on at least the updated corresponding estimation features.

[0059] In block 230, the electronic device 110 determines the target recommendation metric of the candidate object for the target entity based on the corresponding model features of the multiple object recommendation models and the multiple recommendation metrics. That is, the electronic device 110 can obtain the corresponding metric features and the corresponding estimated features of the multiple object recommendation models and the multiple recommendation metrics corresponding to the multiple object recommendation models to determine the target recommendation metric of the candidate object for the target entity.

[0060] In some embodiments, the electronic device 110 may also obtain pattern features. Pattern features may, for example, indicate a pattern of object recommendation performed for a target entity. The electronic device 110 may, for example, perform object recommendation in different models in different scenarios. For example, the electronic device 110 may perform object recommendation in mode A in scenario A, in mode B in scenario B, and so on. The electronic device 110 may then generate a target recommendation metric based on the pattern features, corresponding model features of multiple object recommendation models (including corresponding metric features and corresponding estimated features), and multiple recommendation metrics.

[0061] The electronic device 110 may fuse the pattern features, the corresponding model features of the multiple object recommendation models, and the multiple recommendation metrics in any appropriate manner to generate the target recommendation metric. For example, the electronic device 110 may fuse the pattern features, the corresponding model features of the multiple object recommendation models, and the multiple recommendation metrics using a pre-acquired fusion rule to generate the target recommendation metric.

[0062] In some embodiments, the electronic device 110 may use a fusion model to fuse pattern features, corresponding model features of multiple object recommendation models, and multiple recommendation metrics. Similar to the object recommendation model, the fusion model may be, for example, any neural network that can perform data fusion (also referred to as feature fusion), including but not limited to a fully convolutional network (FCN), a convolutional neural network (CNN), a recurrent neural network (RNN), etc., and the embodiments of the present disclosure are not limited in this respect. In some embodiments, the fusion model may be a logistic regression (LR) model. In some embodiments, the fusion model may be stored locally in the electronic device 110, and the electronic device 110 may directly use the local fusion model to implement data fusion. In some embodiments, the fusion model may also be a model stored in the cloud, and the electronic device 110 may call the fusion model stored in the cloud to implement data fusion when it is necessary to perform related tasks. The fusion model may be trained by the electronic device 110, or it may be trained by other electronic devices. The fusion model may be trained by the electronic device 110 using acquired historical data, for example.

[0063] Figure 4 FIG. 4 is a schematic diagram showing an example 400 of determining a target recommendation metric according to some embodiments of the present disclosure. Figure 4 As shown, the electronic device 110 can obtain the pattern feature 410, the recommendation metrics 421 corresponding to each of the multiple object recommendation models (i.e., the models shown in the figure) (e.g., the recommendation metric 421-1 corresponding to model 1, the recommendation metric 421-N corresponding to model N, etc.) and the model feature 422 (e.g., the model feature 422-1 corresponding to model 1, the model feature 422-N corresponding to model N, etc.). The electronic device 110 can provide the pattern feature 410 and the multiple recommendation metrics 421 and the multiple model features 422 corresponding to the multiple object recommendation models as model inputs to the fusion model 430. The fusion model 430 can generate a model output indicating a target recommendation metric 435 based on the acquired model input. The electronic device 110 can obtain the model output and determine the target recommendation metric 435 generated using the fusion model 430 based on the model output.

[0064] In block 240 , the electronic device 110 generates an object recommendation result for the target entity based on the target recommendation metric.

[0065] The electronic device 110 may, for example, determine whether to recommend a candidate object to a target entity based on a target recommendation metric corresponding to the candidate object. In the case where there are multiple candidate objects to be recommended to a target entity, the electronic device 110 may, for example, determine a recommendation priority of the first candidate object among the multiple candidate objects based on the target recommendation metric corresponding to the first candidate object. It is understood that the higher the recommendation priority, the more preferentially the corresponding candidate object may be recommended to the target entity. For example, the electronic device 110 may preferentially display candidate objects with high corresponding priorities in the user interface.

[0066] In summary, according to various embodiments of the present disclosure, multiple model features and multiple recommendation metrics corresponding to multiple object recommendation models can be fused, and the final target recommendation metric can be determined based on the fusion result. The object recommendation result for the target entity can be determined based on the fused target recommendation metric. In this way, the recommendation capability of the fused object recommendation model can be improved and the quality of the object recommendation service can be improved.

[0067] The embodiments of the present disclosure also provide corresponding devices for implementing the above methods or processes. Figure 5 A schematic structural block diagram of an apparatus 500 for data processing according to some embodiments of the present disclosure is shown. The apparatus 500 may be implemented as or included in the electronic device 110. Each module / component in the apparatus 500 may be implemented by hardware, software, firmware or any combination thereof.

[0068] As shown in the figure, the device 500 includes a recommendation metric determination module 510, which is configured to use multiple object recommendation models to respectively determine multiple recommendation metrics for the candidate objects to be recommended for the target entity, and the recommendation metrics indicate the degree to which the candidate objects are recommended to the target entity. The device 500 also includes a model feature acquisition module 520, which is configured to obtain corresponding model features of multiple object recommendation models, and the model features of each object recommendation model are used to characterize the ability of the object recommendation model in object recommendation. The device 500 also includes a target metric determination module 530, which is configured to determine the target recommendation metric of the candidate object for the target entity based on the corresponding model features of the multiple object recommendation models and the multiple recommendation metrics. The device 500 also includes a recommendation result generation module 540, which is configured to generate an object recommendation result for the target entity based on the target recommendation metric.

[0069] In some embodiments, the model feature acquisition module 520 is further configured to: determine corresponding metric features of multiple object recommendation models based on multiple recommendation metrics, and the metric feature of each object recommendation model is used to characterize the recommendation metric determined using the object recommendation model.

[0070] In some embodiments, the model feature acquisition module 520 is further configured to: for a first object recommendation model among multiple object recommendation models, determine a metric feature of the first object recommendation model based on at least one of the following: a value of a first recommendation metric determined using the first object recommendation model, a ranking of the first recommendation metric among multiple recommendation metrics, or a ranking of the first recommendation metric among a group of recommendation metrics, a group of recommendation metrics being determined by the first object recommendation model for a group of objects to be recommended, the group of objects including candidate objects.

[0071] In some embodiments, the model feature acquisition module 520 is further configured to: determine the corresponding estimated features of multiple object recommendation models based on the corresponding historical estimation information of multiple object recommendation models, the historical estimation information of each object recommendation model including the recommendation metric historically determined by the object recommendation model for the object to be recommended and the first conversion result of whether the object to be recommended has been converted.

[0072] In some embodiments, the model feature acquisition module 520 is further configured to: for a second object recommendation model among multiple object recommendation models, determine the estimated features of the second object recommendation model based on at least one of the following: the average ranking of corresponding recommendation metrics determined by the second object recommendation model for multiple reference objects respectively, the conversion operations performed on the entities to which the multiple reference objects are recommended, the proportion of reference objects ranked before a first threshold ranking in the corresponding recommendation metrics of the multiple reference objects among the multiple reference objects, or the type of objects ranked before a second threshold ranking in the historical recommendation metrics generated by the second object recommendation model.

[0073] In some embodiments, the device 500 also includes: a second result acquisition module, configured to obtain a second conversion result of whether the candidate object is converted by the target entity if the candidate object is recommended to the target entity based on the object recommendation result; and an information update module, configured to update the corresponding historical prediction information of multiple object recommendation models based on the second conversion result and multiple recommendation metrics.

[0074] In some embodiments, the device 500 also includes: a pattern feature acquisition module, configured to acquire pattern features, the pattern of object recommendation performed for the target entity, and the target metric determination module 530 is further configured to: generate a target recommendation metric based on the pattern features, corresponding model features of multiple object recommendation models and multiple recommendation metrics using a fusion model.

[0075] The modules and / or units included in the device 500 can be implemented in various ways, including software, hardware, firmware, or any combination thereof. In some embodiments, one or more modules and / or units can be implemented using software and / or firmware, such as machine executable instructions stored on a storage medium. In addition to or as an alternative to machine executable instructions, some or all of the modules and / or units in the device 500 can be implemented at least in part by one or more hardware logic components. As an example and not limitation, exemplary types of hardware logic components that can be used include field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0076] Figure 6 1 shows a block diagram of an electronic device 600 in which one or more embodiments of the present disclosure may be implemented. It should be understood that Figure 6 The electronic device 600 shown is merely exemplary and should not constitute any limitation on the functionality and scope of the embodiments described herein. Figure 6 The electronic device 600 shown can be used to implement Figure 1 An electronic device 110.

[0077] like Figure 6 As shown, the electronic device 600 is in the form of a general electronic device. The components of the electronic device 600 may include, but are not limited to, one or more processors or processing units 610, a memory 620, a storage device 630, one or more communication units 640, one or more input devices 650, and one or more output devices 660. The processing unit 610 may be an actual or virtual processor and is capable of performing various processes according to a program stored in the memory 620. In a multi-processor system, multiple processing units execute computer executable instructions in parallel to improve the parallel processing capability of the electronic device 600.

[0078] The electronic device 600 typically includes a plurality of computer storage media. Such media may be any accessible media that is accessible to the electronic device 600, including but not limited to volatile and non-volatile media, removable and non-removable media. The memory 620 may be a volatile memory (e.g., registers, caches, random access memory (RAM)), a non-volatile memory (e.g., a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), flash memory), or some combination thereof. The storage device 630 may be a removable or non-removable medium, and may include a machine-readable medium, such as a flash drive, a disk, or any other medium, which may be capable of being used to store information and / or data and may be accessed within the electronic device 600.

[0079] The electronic device 600 may further include additional removable / non-removable, volatile / non-volatile storage media. Figure 6 As shown in , a disk drive for reading or writing from a removable, non-volatile disk (e.g., a "floppy disk") and an optical drive for reading or writing from a removable, non-volatile optical disk may be provided. In these cases, each drive may be connected to the bus (not shown) by one or more data media interfaces. The memory 620 may include a computer program product 625 having one or more program modules that are configured to perform various methods or actions of various embodiments of the present disclosure.

[0080] The communication unit 640 implements communication with other electronic devices through a communication medium. Additionally, the functions of the components of the electronic device 600 can be implemented with a single computing cluster or multiple computing machines that can communicate through a communication connection. Therefore, the electronic device 600 can operate in a networked environment using a logical connection with one or more other servers, a network personal computer (PC), or another network node.

[0081] The input device 650 may be one or more input devices, such as a mouse, a keyboard, a tracking ball, etc. The output device 660 may be one or more output devices, such as a display, a speaker, a printer, etc. The electronic device 600 may also communicate with one or more external devices (not shown) through the communication unit 640 as needed, such as a storage device, a display device, etc., communicate with one or more devices that allow a user to interact with the electronic device 600, or communicate with any device that allows the electronic device 600 to communicate with one or more other electronic devices (e.g., a network card, a modem, etc.). Such communication may be performed via an input / output (I / O) interface (not shown).

[0082] According to an exemplary implementation of the present disclosure, a computer-readable storage medium is provided, on which computer-executable instructions are stored, wherein the computer-executable instructions are executed by a processor to implement the method described above. According to an exemplary implementation of the present disclosure, a computer program product is also provided, which is tangibly stored on a non-transitory computer-readable medium and includes computer-executable instructions, and the computer-executable instructions are executed by a processor to implement the method described above.

[0083] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices, equipment, and computer program products implemented according to the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0084] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processing unit of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0085] Computer-readable program instructions can be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operational steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, so that the instructions executed on the computer, other programmable data processing apparatus, or other device implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0086] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple implementations of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of a module, program segment or instruction includes one or more executable instructions for realizing the logical function of the specification. In some implementations as replacements, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be realized by a special hardware-based system that performs the function or action of the specification, or can be realized by a combination of special hardware and computer instructions.

[0087] The above descriptions of various implementations of the present disclosure are exemplary, non-exhaustive, and not limited to the disclosed implementations. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described implementations. The selection of terms used herein is intended to best explain the principles of the implementations, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the various implementations disclosed herein.

Claims

1. A data processing method, comprising: Using multiple object recommendation models, respectively determine multiple recommendation metrics of the candidate objects to be recommended for the target entity, the recommendation metrics indicating the degree to which the candidate objects are recommended to the target entity; Acquire corresponding model features of the multiple object recommendation models, where the model feature of each object recommendation model is used to characterize the capability of the object recommendation model in object recommendation; Determining a target recommendation metric for the candidate object with respect to the target entity based on corresponding model features of the multiple object recommendation models and the multiple recommendation metrics; as well as Based on the target recommendation metric, an object recommendation result for the target entity is generated.

2. The method according to claim 1, wherein obtaining corresponding model features of the plurality of object recommendation models comprises: Based on the multiple recommendation metrics, corresponding metric features of the multiple object recommendation models are determined, and the metric feature of each object recommendation model is used to characterize the recommendation metric determined by using the object recommendation model.

3. The method according to claim 2, wherein determining corresponding metric features of the plurality of object recommendation models comprises: For a first object recommendation model among the multiple object recommendation models, a metric feature of the first object recommendation model is determined based on at least one of the following: a value of a first recommendation metric determined using the first object recommendation model, the ranking of the first recommendation metric among the plurality of recommendation metrics, or The ranking of the first recommendation metric in a set of recommendation metrics, the set of recommendation metrics being respectively determined by the first object recommendation model for a set of objects to be recommended, the set of objects including the candidate object.

4. The method according to claim 1, wherein obtaining corresponding model features of the plurality of object recommendation models comprises: Based on the corresponding historical prediction information of the multiple object recommendation models, the corresponding prediction features of the multiple object recommendation models are determined, and the historical prediction information of each object recommendation model includes the recommendation metric historically determined by the object recommendation model for the object to be recommended and a first conversion result of whether the object to be recommended is converted.

5. The method according to claim 4, wherein determining the corresponding estimated features of the plurality of object recommendation models comprises: For a second object recommendation model among the multiple object recommendation models, determining an estimated feature of the second object recommendation model based on at least one of the following: The second object recommendation model determines an average ranking of corresponding recommendation metrics for a plurality of reference objects, respectively, and the entities to which the plurality of reference objects are recommended perform conversion operations, the proportion of reference objects ranked before a first threshold rank in the recommendation metrics corresponding to the plurality of reference objects among the plurality of reference objects, or The type of the object ranked before the second threshold in the historical recommendation metric generated by the second object recommendation model.

6. The method according to claim 4, further comprising: If the candidate object is recommended to the target entity according to the object recommendation result, obtaining a second conversion result of whether the candidate object is converted by the target entity; as well as Based on the second conversion result and the multiple recommendation metrics, corresponding historical prediction information of the multiple object recommendation models is updated.

7. The method according to claim 1, further comprising: Acquire a pattern feature, wherein the pattern feature is a pattern of object recommendation executed for the target entity, and Determining the target recommendation metric of the candidate object for the target entity includes: The target recommendation metric is generated by utilizing a fusion model based on the pattern features, the corresponding model features of the multiple object recommendation models and the multiple recommendation metrics.

8. A device for data processing, comprising: A recommendation metric determination module is configured to use multiple object recommendation models to respectively determine multiple recommendation metrics of the candidate objects to be recommended for the target entity, wherein the recommendation metrics indicate the degree to which the candidate objects are recommended to the target entity; A model feature acquisition module is configured to acquire corresponding model features of the plurality of object recommendation models, wherein the model feature of each object recommendation model is used to characterize the capability of the object recommendation model in object recommendation; a target metric determination module configured to determine a target recommendation metric of the candidate object for the target entity based on corresponding model features of the plurality of object recommendation models and the plurality of recommendation metrics; as well as The recommendation result generating module is configured to generate an object recommendation result for the target entity based on the target recommendation metric.

9. An electronic device, comprising: at least one processing unit; as well as At least one memory, the at least one memory being coupled to the at least one processing unit and storing instructions for execution by the at least one processing unit, the instructions causing the electronic device to perform the method according to any one of claims 1 to 7 when executed by the at least one processing unit.

10. A computer-readable storage medium having a computer program stored thereon, wherein the computer program can be executed by a processor to implement the method according to any one of claims 1 to 7.

Images