A data processing method, apparatus and device, and a storage medium

Abstract

The embodiment of the application provides a data processing method, device and equipment and a storage medium, the method comprises the following steps: mapping original object features in an object feature set through at least two nonlinear functions in an object relationship identification network of an initial prediction model, obtaining at least two nonlinear object features, and correlating and fusing the at least two nonlinear object features through a fusion function to obtain fused object relationship features. Similarly, the media relationship identification network can be used to determine the fused media relationship features based on a media feature set. According to the fused media relationship features and the fused object relationship features, the prediction interest degree of a sample object for sample multimedia data is determined. According to the labeled interest degree and the predicted interest degree, the parameters of the at least two nonlinear functions respectively corresponding to the object relationship identification network and the media relationship identification network are adjusted to obtain a media prediction model. The prediction accuracy of the interest degree is improved through the application.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a data processing method, apparatus, device, and storage medium. Background Technology

[0002] With the development of internet technology, multimedia platforms (such as video platforms, novel platforms, and shopping platforms) are becoming increasingly popular among users (i.e., target audiences). A related challenge is how to filter out multimedia data of interest to target audiences from the vast amounts of data available on these platforms. Currently, predictive models are typically used to predict the target audience's level of interest in multimedia data, allowing for the filtering of relevant data from a large dataset. However, in practice, it has been found that current predictive models struggle to uncover the relationships between different features, resulting in relatively low accuracy in predicting interest levels. Summary of the Invention

[0003] This application provides a data processing method, apparatus, device, and storage medium that can improve the accuracy of interest prediction.

[0004] One embodiment of this application provides a data processing method, including:

[0005] The process involves obtaining the object feature set of the sample objects, the media feature set of the sample multimedia data, the initial prediction model, and the labeled interest level of the sample objects in relation to the sample multimedia data. The initial prediction model includes an object relationship recognition network and a media relationship recognition network.

[0006] By using at least two nonlinear functions in the object relationship recognition network, the original object features in the object feature set are mapped to obtain at least two nonlinear object features. By using the fusion function in the object relationship recognition network, the at least two nonlinear object features are associated and fused to obtain fused object relationship features.

[0007] By using at least two nonlinear functions in the media relationship identification network, the original media features in the media feature set are mapped to obtain at least two nonlinear media features. By using the fusion function in the media relationship identification network, the at least two nonlinear media features are correlated and fused to obtain fused media relationship features.

[0008] Based on the aforementioned characteristics of converged media relationships and the aforementioned characteristics of converged objects relationships, the predicted interest degree of the aforementioned sample objects in relation to the aforementioned sample multimedia data is determined.

[0009] Based on the above-mentioned labeled interest and predicted interest, the parameters of at least two nonlinear functions corresponding to the above-mentioned object relationship recognition network and media relationship recognition network are adjusted to obtain the media prediction model; the above-mentioned media prediction model is used to predict the interest of the target object in the multimedia data to be recommended.

[0010] One embodiment of this application provides a data processing apparatus, including:

[0011] The acquisition module is used to acquire the object feature set of the sample objects, the media feature set of the sample multimedia data, the initial prediction model, and the labeled interest degree of the sample objects in relation to the sample multimedia data; the initial prediction model includes an object relationship recognition network and a media relationship recognition network.

[0012] The first identification module is used to map the original object features in the object feature set to at least two nonlinear functions in the object relationship identification network to obtain at least two nonlinear object features, and to perform association and fusion of the at least two nonlinear object features to obtain fused object relationship features through the fusion function in the object relationship identification network.

[0013] The second identification module is used to map the original media features in the media feature set to at least two nonlinear functions in the media relationship identification network to obtain at least two nonlinear media features, and to fuse the at least two nonlinear media features to obtain fused media relationship features by using the fusion function in the media relationship identification network.

[0014] The prediction module is used to determine the predicted interest degree of the sample objects in the sample multimedia data based on the above-mentioned integrated media relationship characteristics and the above-mentioned integrated object relationship characteristics.

[0015] The training module is used to adjust the parameters of at least two nonlinear functions in the object relationship recognition network and the media relationship recognition network based on the above-mentioned labeled interest and predicted interest, so as to obtain a media prediction model; the media prediction model is used to predict the interest of the target object in the multimedia data to be recommended.

[0016] One aspect of this application provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0017] One embodiment of this application provides a computer storage medium storing a computer program, which, when executed by a processor, performs the steps of the above-described method.

[0018] One aspect of this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0019] This application proposes a dual-tower model (initial prediction model), which includes an object relationship recognition network and a media relationship recognition network. Each network comprises multiple nonlinear functions and fusion functions. The object relationship recognition network uses these functions to mine the relationships between original object features (i.e., fused object relationship features), while the media relationship recognition network uses these functions to mine the relationships between original media features (i.e., fused media relationship features). Since the multiple nonlinear functions in both networks are learnable, training the media prediction model with adjusted parameters allows it to accurately mine the relationships between different features using the adjusted nonlinear functions and fusion functions. This enhances the expressive power of the media prediction model and improves the accuracy of interest prediction. Attached Figure Description

[0020] Figure 1 This is a schematic diagram of the structure of a data processing system provided in this application;

[0021] Figure 2 This is a flowchart illustrating a data processing method provided in this application. Figure 1 ;

[0022] Figure 3 This is a schematic diagram of a scenario for generating fused object relationship features provided in this application;

[0023] Figure 4 This is a schematic diagram illustrating a scenario for generating converged media relationship features, as provided in this application.

[0024] Figure 5 This is a flowchart illustrating a data processing method provided in this application. Figure 2 ;

[0025] Figure 6 This is a schematic diagram of the structure of an initial prediction model provided in this application;

[0026] Figure 7 This is a schematic diagram illustrating a product recommendation scenario provided in this application;

[0027] Figure 8 This is a schematic diagram of the structure of a data processing device provided in this application;

[0028] Figure 9This is a schematic diagram of the structure of a computer device provided in this application. Detailed Implementation

[0029] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0030] First, let's introduce Figure 1 , Figure 1 This is a schematic diagram of the structure of a data processing system provided in an embodiment of this application, such as... Figure 1 As shown in the schematic diagram, the data processing system may include a server 10 and a terminal cluster, which may include multiple terminals. Figure 1 As shown, taking a terminal cluster consisting of three terminals as an example, the terminal cluster can specifically include a first terminal, a second terminal, and a third terminal. It can be understood that the first terminal, the second terminal, and the third terminal can all connect to the server via the network so that each terminal can interact with the server through the network connection.

[0031] Understandably, a server can be a single physical server, a server cluster or distributed system consisting of at least two physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud knowledge bases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. A terminal can specifically refer to in-vehicle terminals, smartphones, tablets, laptops, desktop computers, smart speakers, speakers with screens, smart TVs, smartwatches, etc., but is not limited to these. Various terminals and servers can be directly or indirectly connected via wired or wireless communication. Furthermore, the number of terminals and servers can be one or at least two; this application does not impose any restrictions.

[0032] In this application, server 10 can be used to train a media prediction model. This media prediction model can be obtained by adjusting the parameters of at least two nonlinear functions corresponding to the object relationship recognition network and the media relationship recognition network included in the initial prediction model, using the object feature set of the sample object, the media feature set of the sample multimedia data, and the labeled interest degree of the sample object towards the sample multimedia data as training samples. The initial prediction model can refer to the prediction model to be trained.

[0033] It should be noted that the training samples mentioned above can be obtained by server 10 from any terminal in the terminal cluster, or by server 10 from the network, or by server 10 from local storage.

[0034] In one embodiment, after training the media prediction model, server 10 can store the model in its local memory. Any terminal in the terminal cluster can then use the media prediction model stored in server 10's local memory to predict the target object's interest in the multimedia data to be recommended. For example, any terminal in the terminal cluster can send a prediction request to server 10 regarding the target object's interest in the multimedia data to be recommended. Upon receiving the prediction request, server 10 predicts the target object's interest in the multimedia data to be recommended and returns the predicted interest to the terminal.

[0035] In one embodiment, after training the media prediction model, the server 10 can send the media prediction model to each terminal in the terminal cluster. Each terminal can store the media prediction model in its local memory. Each terminal can use the media prediction model in its local memory to predict the target object's interest in the multimedia data to be recommended.

[0036] In one embodiment, after the server 10 trains the media prediction model, it can send the media prediction model to the cloud server. The cloud server can store the media prediction model in its local storage. Each terminal can call the media prediction model in the local storage of the cloud server to predict the target object's interest in the multimedia data to be recommended.

[0037] It should be noted that the process of predicting the target audience's interest in the multimedia data to be recommended based on the media prediction model, as well as the training process of the media prediction model, can be found below.

[0038] Please see Figure 2 , Figure 2 This is a flowchart illustrating a data processing method provided in this application. Figure 1 .like Figure 2 As shown, this method can be derived from... Figure 1 It can be executed by any terminal in the terminal cluster, or by... Figure 1 The server in the middle can be used to execute it, or it can be executed by... Figure 1 The terminal cluster in the application uses terminals and servers to collaboratively execute the data processing method. The device used to execute this data processing method in this application can be collectively referred to as a computer device. The method may include the following steps:

[0039] S101. Obtain the object feature set of the sample object, the media feature set of the sample multimedia data, the initial prediction model, and the labeled interest degree of the sample object in relation to the sample multimedia data.

[0040] In this application, the computer device can acquire an initial prediction model and training samples, and deploy the initial prediction model in the computer device; after deployment, the computer device can acquire from the training samples the object feature set of the sample objects, the media feature set of the sample multimedia data, and the labeled interest degree of the sample objects for the sample multimedia data.

[0041] It should be noted that the training samples mentioned above may be obtained from media platforms; or, they may be downloaded from the internet; or, they may be sample data used to train the initial prediction model. Media platforms can refer to video applications, live streaming applications, novel applications, music applications, social applications, etc.

[0042] It should be noted that the number of sample objects can be one or more; the sample objects can be users of the media platform; or, the sample objects can be objects used to train the initial prediction model.

[0043] Each sample object's set of object features can include one or more original object features of the sample object; different original object features can reflect different object attribute information of the sample object. Object attribute information can include object name, object level, browsing history, etc. Browsing history can be used to indicate the multimedia data that the sample object has viewed on the media platform, i.e., the multimedia data that the object is interested in.

[0044] It should be noted that the number of sample multimedia data can be one or more; sample multimedia data can include at least one of video, audio, text, and images.

[0045] Each sample multimedia data set can include one or more original media features of the sample multimedia data; different original media features can reflect different media attribute information of the sample multimedia data. Media attribute information can include media name, creator information, release time, media content, historical operation records, etc.

[0046] The historical operation record can reflect the interactive operations performed on the sample multimedia data within a historical time period; the historical time period is the time period before the current time; the interactive operation includes at least one of the following: viewing operation, sharing operation, liking operation, commenting operation, etc.

[0047] Creator information may include creator identifiers (such as creator name, creator nickname, creator account, etc.), user level, etc.; media content may reflect the content in the sample multimedia data.

[0048] Specifically, when the sample multimedia data is video, the media content of the video can refer to one or more frames of video images (such as the video's cover image), video segments, etc. When the sample multimedia data is audio, the media content of the audio can refer to the audio itself. When the sample multimedia data is an image, the media content of the image can refer to the image itself. When the sample multimedia data is text, the media content of the text can refer to the main content of the text, etc.

[0049] It should be noted that the labeled interest level can be used to reflect the actual probability that the sample object will perform interactive operations on the sample multimedia data; the labeled interest level can be determined based on the sample object's historical browsing records, or the labeled interest level can be obtained by manual labeling.

[0050] It should be noted that the initial prediction model mentioned above includes an object relationship recognition network and a media relationship recognition network. The object relationship recognition network can be used to mine the relationships between original object features in an object feature set; the media relationship recognition network can be used to mine the relationships between original media features in a media feature set.

[0051] S102. By using at least two nonlinear functions in the object relationship recognition network, the original object features in the object feature set are mapped to obtain at least two nonlinear object features. By using the fusion function in the object relationship recognition network, the at least two nonlinear object features are associated and fused to obtain fused object relationship features.

[0052] In this application, the computer device can input the original object features from the object feature set into the object relationship recognition network of the initial prediction model. The computer device can then use a nonlinear function in the object relationship recognition network to map one of the original object features in the object feature set, obtaining a nonlinear object feature. This process continues until the computer device obtains at least two nonlinear object features corresponding to at least two nonlinear functions in the object relationship recognition network. Next, the computer device can input these at least two nonlinear object features into the object relationship recognition network, and through the fusion function of the object relationship recognition network, it can correlate and fuse these at least two nonlinear object features to obtain fused object relationship features. By using multiple nonlinear functions and the fusion function, the correlation between the original object features is fully explored, which helps improve the prediction accuracy of the initial prediction model.

[0053] It's important to note that the nonlinear function in the object relationship recognition network is used to map the original object features from the low-dimensional space to a high-dimensional space. In other words, nonlinear object features belong to the high-dimensional space, which can refer to a space conducive to capturing the relationships between different object features. Put simply, nonlinear object features reflect more complex object features of the sample objects, which helps the fusion function to uncover the relationships between different object features. The fused object relationship features can reflect the relationships between different object features.

[0054] It should be noted that the at least two nonlinear functions in the object relationship recognition network can be the same or different, and this application does not impose any restrictions on this. The nonlinear functions in this application can include polynomial functions, sine functions, cosine functions, spline functions, etc., which can avoid the situation of gradient value explosion (i.e., too large) or vanishing (too small) during the training phase of the initial prediction model, which is conducive to improving the training effect of the initial prediction model and improving the prediction accuracy of the initial prediction model.

[0055] Specifically, the training performance of the initial prediction model is related to its gradient values. If the gradient values ​​explode or vanish, the training performance of the initial prediction model is poor; if the gradient values ​​are within a stable threshold, the training performance is good. Simultaneously, the gradient values ​​are related to the nonlinear function. If the derivative range of the nonlinear function is uncontrollable, it will cause the gradient values ​​to explode or vanish; if the derivative range of the nonlinear function is controllable, the gradient values ​​will remain within a stable threshold. Since the derivative range of the nonlinear function in this application is controllable, the situation of gradient value explosion or vanishing during the training phase of the initial prediction model can be avoided, which is beneficial to improving the training performance and prediction accuracy of the initial prediction model.

[0056] The fusion function in the object relation recognition network can include summation functions, weighted summation functions, etc. The object relation recognition network can be obtained by initializing a Kolmogorov-Arnold Network (KAN).

[0057] Please refer to Figure 3 , Figure 3 This is a schematic diagram illustrating a scenario for generating fused object relationship features, as provided in this application. For example... Figure 3 As shown, the object relationship recognition network 30 includes three nonlinear functions and one fusion function. The three nonlinear functions are nonlinear function 30a, nonlinear function 30b and nonlinear function 30c.

[0058] Specifically, the computer device can obtain three object attribute information of the sample object: the object name (e.g., "Xiao Li"), the object level (e.g., "Member Level"), and the browsing history. The computer device can extract original object feature 31a from the object name, original object feature 31b from the object level, and original object feature 31c from the browsing history. Using a nonlinear function 30a, original object feature 31a can be mapped to obtain nonlinear object feature 30d; using a nonlinear function 30b, original object feature 31b can be mapped to obtain nonlinear object feature 30e; and using a nonlinear function 30c, original object feature 31c can be mapped to obtain nonlinear object feature 30f. Furthermore, the computer device can use a fusion function to correlate and fuse nonlinear object features 30d, 30e, and 30f to obtain the fused object relationship features of the sample object.

[0059] In one embodiment, the computer device can use a nonlinear function in an object relationship recognition network to map multiple original object features to obtain a nonlinear object feature. Specifically, the computer device can concatenate multiple original object features to obtain a concatenated original object feature, and then use a nonlinear function in the object relationship recognition network to map the concatenated original object feature to obtain a nonlinear object feature. This facilitates the fusion function in uncovering more correlations between original object features, thereby improving the prediction accuracy of the initial prediction model.

[0060] S103. By using at least two nonlinear functions in the media relationship identification network, the original media features in the media feature set are mapped to obtain at least two nonlinear media features. By using the fusion function in the media relationship identification network, the at least two nonlinear media features are correlated and fused to obtain fused media relationship features.

[0061] In this application, the computer device can input the original media features from the media feature set into the media relationship recognition network of the initial prediction model. The computer device can then use a nonlinear function in the media relationship recognition network to map one of the original media features in the media feature set, obtaining a nonlinear media feature. This process continues until the computer device obtains at least two nonlinear media features corresponding to at least two nonlinear functions in the media relationship recognition network. Next, the computer device can input these at least two nonlinear media features into the media relationship recognition network, and through the fusion function of the media relationship recognition network, it can correlate and fuse these at least two nonlinear media features to obtain a fused media relationship feature. By using multiple nonlinear functions and a fusion function, the correlation between the original media features is fully explored, which helps improve the prediction accuracy of the initial prediction model.

[0062] It should be noted that the nonlinear function in the media relationship recognition network is used to map the original media features in the low-dimensional space to a high-dimensional space. That is, nonlinear media features belong to the high-dimensional space, which can refer to a space conducive to capturing the relationships between different media features. In other words, nonlinear media features reflect the more complex media features of the sample media, which is beneficial for assisting the fusion function in mining the relationships between different media features. The fused media relationship features can reflect the relationships between different original media features.

[0063] It should be noted that at least two nonlinear functions in the media relationship identification network may be the same or different, and this application does not impose any restrictions on this. The fusion function in the media relationship identification network may include a summation function, a weighted summation function, etc.

[0064] It should be noted that the media relationship identification network can be obtained by initializing the Kolmogorov-Arnold Network (KAN).

[0065] Please refer to Figure 4 , Figure 4 This is a schematic diagram illustrating a scenario for generating converged media relationship features, as provided in this application. For example... Figure 4 As shown, the media relationship identification network 40 includes three nonlinear functions and one fusion function. The three nonlinear functions are nonlinear function 40a, nonlinear function 40b and nonlinear function 40c.

[0066] Specifically, computer devices can acquire video images, video names, and video text; among these, the video images can be obtained by extracting frames from the video. Figure 4The following example uses three video images. The computer device can extract the image features corresponding to the three video images respectively, and concatenate the three image features to obtain the original media feature 41a; extract the original media feature 41b from the video name, and extract the original media feature 41c from the video text.

[0067] Furthermore, the computer device can map the original media feature 41a using nonlinear function 40a to obtain nonlinear media feature 40d; it can map the original media feature 41b using nonlinear function 40b to obtain nonlinear media feature 40e; and it can map the original media feature 41c using nonlinear function 40c to obtain nonlinear media feature 40f. By employing a fusion function to correlate and fuse nonlinear media features 40d, 40e, and 40f, the fused media relationship features of the advertising video can be obtained.

[0068] In one embodiment, the computer device can use a nonlinear function in a media relationship recognition network to map multiple original media features to obtain a nonlinear media feature. Specifically, the computer device can concatenate multiple original media features to obtain a concatenated original media feature, and then use a nonlinear function in a media relationship recognition network to map the concatenated original media feature to obtain a nonlinear media feature. This facilitates the fusion function in uncovering more correlations between original media features, thereby improving the prediction accuracy of the initial prediction model.

[0069] S104. Based on the characteristics of the relationship between the converged media and the relationship between the converged objects, determine the predicted interest of the sample objects in the sample multimedia data.

[0070] In this application, the computer device can determine the degree of matching between the relationship features of the fused media and the relationship features of the fused objects, and determine the predicted interest degree of the sample objects in the sample multimedia data based on the degree of matching.

[0071] It should be noted that the predicted interest level can be used to reflect the predicted probability that a sample object will perform an interactive operation in response to the sample multimedia data. The higher the predicted interest level, the greater the probability that the sample object will perform an interactive operation in response to the sample multimedia data; the lower the predicted interest level, the lower the probability that the sample object will perform an interactive operation in response to the sample multimedia data.

[0072] It should be noted that the matching degree can be obtained based on the matching degree formula, which includes the cosine matching degree formula, the Euclidean distance formula, etc.

[0073] S105. Based on the labeled interest degree and the predicted interest degree, adjust the parameters of at least two nonlinear functions corresponding to the object relationship recognition network and the media relationship recognition network respectively to obtain the media prediction model.

[0074] In this application, a computer device can determine the prediction loss of an initial prediction model based on labeled interest and predicted interest. A large prediction loss indicates low prediction accuracy, while a small prediction loss indicates high prediction accuracy. When the prediction loss is large, the parameters of at least two nonlinear functions in the object relationship recognition network and media relationship recognition network of the initial prediction model can be adjusted to obtain a media prediction model. When the prediction loss is small, the computer device can directly determine the initial prediction model as the media prediction model. By adjusting the parameters of the nonlinear functions, the initial prediction model's ability to mine and identify nonlinear features can be enriched, resulting in a media prediction model with higher prediction accuracy. Predicting the target object's interest in the recommended multimedia data based on this media prediction model helps improve the prediction accuracy of interest.

[0075] It should be noted that a media prediction model can refer to a prediction model with high accuracy, that is, a model that accurately predicts the interest of the target audience in the multimedia data to be recommended.

[0076] It should be noted that "smaller prediction loss" can mean that the prediction loss is less than the prediction loss threshold; "larger prediction loss" can mean that the prediction loss is greater than or equal to the prediction loss threshold. The prediction loss threshold can be a default value in the computer equipment, or it can be manually set.

[0077] This application proposes a dual-tower model (initial prediction model), which includes an object relationship recognition network and a media relationship recognition network. Each network comprises multiple nonlinear functions and fusion functions. The object relationship recognition network uses these functions to mine the relationships between original object features (i.e., fused object relationship features), while the media relationship recognition network uses these functions to mine the relationships between original media features (i.e., fused media relationship features). Since the multiple nonlinear functions in both networks are learnable, training the media prediction model with adjusted parameters allows it to accurately mine the relationships between different features using the adjusted nonlinear functions and fusion functions. This enhances the expressive power of the media prediction model and improves the accuracy of interest prediction.

[0078] Please see Figure 5 , Figure 5 This is a flowchart illustrating a data processing method provided in this application. Figure 2 .like Figure 5 As shown, this method can be derived from... Figure 1 It can be executed by any terminal in the terminal cluster, or by... Figure 1 The server in the middle can be used to execute it, or it can be executed by... Figure 1 The terminal cluster in the application uses terminals and servers to collaboratively execute the data processing method. The device used to execute this data processing method in this application can be collectively referred to as a computer device. The method may include the following steps:

[0079] S201. Obtain the object feature set of the sample object, the media feature set of the sample multimedia data, the initial prediction model, and the labeled interest degree of the sample object in relation to the sample multimedia data.

[0080] In one embodiment, the process of generating the object feature set of the sample object and the media feature set of the sample multimedia data may include the following steps: A computer device can acquire object attribute information of the sample object and media attribute information of the sample multimedia data; the computer device can acquire an object feature index table, which may include at least one object feature index pair, each object feature index pair including an object attribute information and an original object feature. The computer device can query the object feature index table for object feature index pairs whose included object attribute information is the same as the object attribute information of the sample object. The computer device can determine the original object feature included in the queried object feature index pair as the original object feature of the sample object; and add the original object feature to the object feature set of the sample object.

[0081] Furthermore, the computer device can obtain a media feature index table, which may include at least one media feature index pair. Each media feature index pair includes a media attribute information and an original media feature. The computer device can query the media feature index pair from the media feature index table that contains media attribute information identical to the media attribute information of the sample media. The computer device can then determine the original media feature included in the queried media feature index pair as the original media feature of the sample media and add the original media feature to the object feature set of the sample object. The index table facilitates convenient querying and retrieval of the corresponding original feature based on attribute information, improving data processing efficiency.

[0082] In one embodiment, a computer device can input object attribute information and media attribute information into a feature extractor, which extracts original object features from the object attribute information and original media features from the media attribute information. The feature extractor may include a transformer (a model structure), Generative Adversarial Networks (GANs), etc.

[0083] S202. Obtain the P candidate object relationship features obtained from the i-th object relationship recognition layer.

[0084] It should be noted that the aforementioned object relationship recognition network may include M object relationship recognition layers, each of which includes at least two nonlinear functions and a fusion function. Here, M is an integer greater than 1; that is, the object relationship recognition network includes at least two object relationship recognition layers.

[0085] It should be noted that the candidate object relationship features obtained by each object relationship recognition layer can reflect the correlation between different original object features of the sample objects obtained by each object relationship recognition layer.

[0086] In this application, a computer device can input the original object features from the object feature set into a first object relationship recognition layer to obtain candidate object relationship features corresponding to the first object relationship recognition layer. The candidate object relationship features obtained from the first object relationship recognition layer are then input into a second object relationship recognition layer. Based on at least two nonlinear functions and a fusion function in the second object relationship recognition layer, the candidate object relationship features obtained from the first object relationship recognition layer are mapped and fused to obtain the candidate object relationship features obtained from the second object relationship recognition layer. This process is repeated sequentially for the above M object relationship recognition layers to obtain fused object relationship features. Using multiple object relationship recognition layers increases the number of nonlinear functions in the object relationship recognition network, which is beneficial for improving the network's ability to mine the correlation between different original object features, improving its data fitting ability, and consequently, enhancing the performance and prediction accuracy of the initial prediction model.

[0087] Specifically, taking the (i+1)th object relationship recognition layer as an example, the generation process of the candidate object relationship features corresponding to each relationship recognition layer is explained. The computer device can obtain P fused object relationship features based on the fusion function in the i-th object relationship recognition layer.

[0088] It should be noted that the candidate object relationship features obtained by the first object relationship recognition layer are determined based on the original object features in the object feature set. That is, the computer device can determine the original object features in the object feature set as the candidate object relationship features obtained by the first object relationship recognition layer; where i is a positive integer less than M and P is an integer greater than 1.

[0089] S203. Using the Q nonlinear functions in the (i+1)th object relationship recognition layer, P candidate object relationship features are mapped to obtain Q nonlinear object features.

[0090] In this application, a computer device can select a nonlinear function from the Q nonlinear functions in the (i+1)th object relationship recognition layer that is associated with the first candidate object relationship feature among the P candidate object relationship features. This first candidate object relationship feature is then input into the selected nonlinear function to calculate the corresponding nonlinear object feature. By sequentially performing the above operation on the P candidate object relationship features, the Q nonlinear object features corresponding to the (i+1)th object relationship recognition layer can be obtained.

[0091] It should be noted that Q is an integer greater than or equal to P. A candidate object relationship feature in the i-th object relationship recognition layer can be associated with one or more nonlinear functions in the (i+1)-th object relationship recognition layer, but the nonlinear functions associated with different candidate object relationship features are different.

[0092] For example, the selected nonlinear functions are polynomial functions, as shown in formula (1).

[0093]

[0094] Among them, in formula (1) For nonlinear object characteristics; a in formula (1) c The c-th parameter in the selected nonlinear function can be adjusted based on the prediction loss; x in formula (1) c In formula (1), x represents the relational feature of the candidate object. c In this context, 'c' represents the exponential coefficient.

[0095] In one implementation, the i-th object relationship identification layer includes P first nodes for reflecting the P candidate object relationship features, with one first node corresponding to one candidate object relationship feature; the (i+1)-th object relationship identification layer includes K second nodes, and the P first nodes and the K second nodes are connected by Q edges; one edge corresponds to one nonlinear function.

[0096] The above-mentioned mapping of P candidate object relationship features to obtain Q nonlinear object features through Q nonlinear functions in the (i+1)th object relationship recognition layer can include the following steps: The computer device obtains the target edge connected to the first node 'a' among the P first nodes from the Q edges connecting the P first nodes and the aforementioned K second nodes; where 'a' is a positive integer less than or equal to P. The computer device can input the candidate object relationship feature corresponding to the first node 'a' into the nonlinear function reflected by the target edge, and calculate the nonlinear object feature corresponding to the target edge based on the nonlinear function reflected by the target edge. This operation is performed sequentially for the Q edges until the nonlinear object features corresponding to the Q edges are obtained, i.e., the Q nonlinear object features corresponding to the (i+1)th object relationship recognition layer. The nonlinear functions in the object relationship recognition layer correspond to edges, rather than nodes, which helps improve the interpretability of the object relationship recognition network.

[0097] It should be noted that one of the P first nodes mentioned above is connected to at least one of the K second nodes, that is, a first node can be connected to a second node, and a first node can also be connected to multiple second nodes.

[0098] It should be noted that when the first node is connected to an edge, it indicates that there is a correlation between the relational features of the candidate object corresponding to the first node and the nonlinear function reflected by the edge.

[0099] S204. Using the fusion function in the (i+1)th object relationship recognition layer, fuse every two non-linear object features with a relationship among the Q non-linear object features to obtain K candidate object relationship features.

[0100] It should be noted that having a correlation relationship can mean that the first correlation degree between two nonlinear object features among the Q nonlinear object features is greater than the correlation degree threshold. The first correlation degree can be determined based on at least one of the feature matching degree, correlation state, etc. between the two nonlinear object features.

[0101] The association status includes an associated status and an unassociated status; the association status can be manually labeled.

[0102] It should be noted that the first correlation degree being greater than the correlation degree threshold can include at least one of the following situations: 1. The feature matching degree between two non-linear object features is greater than the feature matching degree threshold; 2. The correlation state between two non-linear object features is an already correlated state.

[0103] In this application, the computer device can determine the first correlation degree between any two nonlinear object features among Q nonlinear object features, input the two nonlinear object features whose corresponding first correlation degree is greater than the correlation degree threshold into the fusion function in the (i+1)th object relationship recognition layer, calculate the first candidate object relationship feature based on the first fusion function in the (i+1)th object relationship recognition layer, until K candidate object relationship features are obtained.

[0104] In one embodiment, each of the K second nodes is connected to at least two of the Q edges. When at least two of the Q edges are connected to a second node, it indicates that the association state between the nonlinear object features corresponding to the at least two edges is an associated state, that is, the nonlinear object features corresponding to the at least two edges have an association relationship.

[0105] The above-mentioned fusion function in the (i+1)th object relationship recognition layer fuses every two nonlinear object features with a correlation among the Q nonlinear object features to obtain K candidate object relationship features. This process may include the following steps: The computer device can filter out at least two edges from the Q edges that connect to a second node b among the K second nodes; where b is a positive integer less than or equal to K. The computer device can input the nonlinear object features corresponding to the at least two edges connected to the second node b into the fusion function in the (i+1)th object relationship recognition layer, and calculate the candidate object relationship features corresponding to the second node b based on the fusion function in the (i+1)th object relationship recognition layer.

[0106] S205. The fused object relationship features obtained from the Mth object relationship recognition layer are determined as fused object relationship features.

[0107] In this application, the above operations are performed sequentially for the above M object relationship identification layers. The computer device can obtain the candidate object relationship features obtained by the Mth object relationship identification layer and determine the candidate object relationship features obtained by the Mth object relationship identification layer as the fused object relationship features.

[0108] Among them, the Mth object relationship identification layer is the object relationship identification layer that is executed last among the M object relationship identification layers.

[0109] S206. By using at least two nonlinear functions in the media relationship identification network, the original media features in the media feature set are mapped to obtain at least two nonlinear media features. By using the fusion function in the media relationship identification network, the at least two nonlinear media features are correlated and fused to obtain fused media relationship features.

[0110] In one embodiment, the media relationship identification network may include M media relationship identification layers, each media relationship identification layer including at least two nonlinear functions and a fusion function. Here, M is an integer greater than 1; that is, the media relationship identification network includes at least two media relationship identification layers.

[0111] Step 206 above may include: the computer device inputs the original media features from the media feature set into the first media relationship identification layer, and maps and fuses the original media features based on at least two nonlinear functions and a fusion function in the first media relationship identification layer to obtain candidate media relationship features corresponding to the first media relationship identification layer. The candidate media relationship features obtained from the first media relationship identification layer are then input into the second media relationship identification layer, and the candidate media relationship features obtained from the first media relationship identification layer are mapped and fused based on at least two nonlinear functions and a fusion function in the second media relationship identification layer to obtain candidate media relationship features obtained from the second media relationship identification layer. This operation is performed sequentially for the above M media relationship identification layers, and the computer device can obtain fused media relationship features. Using multiple media relationship identification layers increases the number of nonlinear functions in the media relationship identification network, which is beneficial for improving the network's ability to mine the correlation between different original media features, improving its data fitting ability, and consequently, improving the performance and prediction accuracy of the initial prediction model.

[0112] In one embodiment, the process of generating candidate object relationship features corresponding to each relationship identification layer may include: taking the (i+1)th media relationship identification layer as an example, the computer device can obtain P fused media relationship features based on the fusion function in the i-th media relationship identification layer. The computer device can select the nonlinear function associated with the first candidate media relationship feature from the Q nonlinear functions in the (i+1)th media relationship identification layer, input the first candidate media relationship feature into the selected nonlinear function, and calculate the nonlinear media feature corresponding to the first candidate media relationship feature. By sequentially performing the above operations on the P candidate media relationship features, Q nonlinear media features corresponding to the (i+1)th media relationship identification layer can be obtained. The computer device can determine the second correlation degree between any two nonlinear media features among Q nonlinear media features. The two nonlinear media features whose corresponding second correlation degree is greater than the correlation degree threshold are input into the fusion function in the (i+1)th media relationship identification layer. The first candidate media relationship feature of the (i+1)th media relationship identification layer is calculated based on the first fusion function in the (i+1)th media relationship identification layer, until the K candidate media relationship features of the (i+1)th media relationship identification layer are obtained.

[0113] It should be noted that the candidate object relationship features obtained by each media relationship identification layer can reflect the correlation between multiple original object features of the sample multimedia data obtained by each media relationship identification layer.

[0114] It should be noted that a candidate media relationship feature in the i-th media relationship identification layer may be associated with one or more nonlinear functions in the (i+1)-th media relationship identification layer, but the nonlinear functions associated with different candidate media relationship features are different.

[0115] It should be noted that the aforementioned second correlation degree can be determined based on at least one of the feature matching degree, correlation status, etc., between the two non-linear media features. The aforementioned second correlation degree being greater than the correlation degree threshold can include at least one of the following situations: 1. The feature matching degree between the two non-linear media features is greater than the feature matching threshold; 2. The correlation status between the two non-linear media features is an already correlated state.

[0116] It should be noted that the Mth media relationship identification layer is the last media relationship identification layer to be executed among the M media relationship identification layers.

[0117] In one implementation, the i-th media relationship identification layer includes P third nodes that reflect the P converged media relationship features, with one third node corresponding to one converged media relationship feature; the (i+1)-th media relationship identification layer includes K fourth nodes, and the P third nodes and the K fourth nodes are connected by Q edges; each edge reflects a nonlinear function.

[0118] The generation process of the aforementioned Q nonlinear media features may include the following steps: A computer device selects target edges from the Q edges connecting the P third nodes to the K fourth nodes, each edge connecting to a third node *a* among the P third nodes; where *a* is a positive integer less than or equal to P. The computer device can input the candidate media relationship features corresponding to the third node *a* into the nonlinear function reflected by the target edge, and calculate the nonlinear media features corresponding to the target edge based on the nonlinear function. This process is repeated for each of the Q edges until the nonlinear media features corresponding to the Q edges are obtained, i.e., the Q nonlinear media features corresponding to the (i+1)th media relationship recognition layer. The nonlinear function in the media relationship recognition layer corresponds to edges, not nodes, which improves the interpretability of the media relationship recognition network.

[0119] It should be noted that any of the P third nodes mentioned above is connected to at least one of the K fourth nodes, that is, a third node can be connected to a fourth node, and a third node can also be connected to multiple fourth nodes.

[0120] It should be noted that when a third node is connected to an edge, it indicates that there is a correlation between the fusion media relationship characteristics corresponding to the third node and the nonlinear function reflected by the edge.

[0121] In one embodiment, each of the K fourth nodes is connected to at least two of the Q edges. When at least two of the Q edges are connected to a fourth node, it indicates that the nonlinear media features corresponding to the at least two edges are associated, meaning that the nonlinear media features corresponding to the at least two edges are related.

[0122] The generation process of the aforementioned K candidate media relationship features may include the following steps: The computer device can select at least two edges from the aforementioned Q edges that connect to the fourth node b among the K fourth nodes; where b is a positive integer less than or equal to K. The computer device can input the nonlinear media features corresponding to the at least two edges connected to the fourth node b into the fusion function in the aforementioned (i+1)th media relationship identification layer, and calculate the candidate media relationship features corresponding to the fourth node b based on the fusion function in the aforementioned (i+1)th media relationship identification layer.

[0123] S207. Based on the characteristics of the relationship between the converged media and the relationship between the converged objects, determine the predicted interest of the sample objects in the sample multimedia data.

[0124] Optionally, step 207 above may include the following steps: The computer device can input the converged media relationship features and the converged object relationship features into the matching degree formula; calculate the matching degree between the converged media relationship features and the converged object relationship features based on the matching degree formula, and determine the matching degree between the converged media relationship features and the converged object relationship features as the relationship matching degree between the sample object and the sample multimedia data. The computer device can use a regularization formula to regularize the relationship matching degree between the sample object and the sample multimedia data to obtain the processed relationship matching degree, and determine the processed relationship matching degree as the predicted interest degree of the sample object in the sample multimedia data. By using the matching degree formula, the similarity between the converged media relationship features and the converged object relationship features is quantified into the relationship matching degree between the sample object and the sample multimedia data, which is beneficial for determining the interest degree of the sample object in the sample multimedia data.

[0125] It should be noted that the matching degree formula may include the cosine matching degree formula, the Euclidean distance formula, etc.

[0126] S208. Based on the labeled interest degree and the predicted interest degree, adjust the parameters of at least two nonlinear functions corresponding to the object relationship recognition network and the media relationship recognition network respectively to obtain the media prediction model.

[0127] Optionally, step 208 above may include: the computer device determines the prediction loss of the initial prediction model based on the labeled interest and predicted interest. When the prediction loss is greater than or equal to the prediction loss threshold, the initial prediction model is determined to be in a non-converged state. Based on this, when the initial prediction model is in a non-converged state, the computer device can adjust the parameters of at least two nonlinear functions corresponding to the object relationship recognition network and the media relationship recognition network, respectively, based on the prediction loss, to obtain an adjusted initial prediction model. The adjusted initial prediction model is iteratively trained until it meets the stopping iterative training condition, thus obtaining a media prediction model. Further, when the prediction loss is less than the prediction loss threshold, the initial prediction model is determined to be in a converged state, and the initial prediction model can be directly identified as a media prediction model. By using the prediction loss threshold as the basis for judging the model's convergence state, it is possible to effectively determine whether the initial prediction model needs training, and to a certain extent, avoid overfitting or underfitting of the initial prediction model.

[0128] It should be noted that a larger prediction loss indicates lower prediction accuracy of the initial prediction model; conversely, a smaller prediction loss indicates higher prediction accuracy. When the prediction loss is greater than or equal to the prediction loss threshold, the initial prediction model is in a non-converged state, indicating low prediction accuracy. When the prediction loss is less than the prediction loss threshold, the initial prediction model is in a converged state, indicating high prediction accuracy.

[0129] In one embodiment, the number of sample multimedia data is N, where N is an integer greater than 1. Determining the prediction loss of the initial prediction model based on the labeled interest and the predicted interest can include the following steps: The computer device can calculate the error between the labeled interest and the predicted interest of the sample object for the j-th sample multimedia data, and determine the calculated error as the initial media prediction loss corresponding to the j-th sample multimedia data. The computer device can add the initial media prediction losses corresponding to the N sample multimedia data respectively to obtain the total loss; divide the total loss by N to obtain the prediction loss of the initial prediction model. Here, j is a positive integer less than or equal to M.

[0130] For example, the computer device can sequentially input the labeled interest degree and predicted interest degree of the sample object for M sample multimedia data into formula (2) to obtain the prediction loss of the initial prediction model.

[0131]

[0132] Wherein, L in formula (2) C To predict the loss; y in formula (2) j The annotation of the interest level of the sample object for the multimedia data of the j-th sample; y j It can be 1 or 0; when y j When y is 1, it indicates that the sample object has performed an interactive operation on the multimedia data of the j-th sample; when y is 1, it indicates that the sample object has performed an interactive operation on the multimedia data of the j-th sample. j When the value is 0, it indicates that the sample object has not performed any interactive operations on the j-th sample multimedia data.

[0133] Among them, in formula (2) The predicted interest level of the sample object for the multimedia data of the j-th sample; in formula (2) The calculated result is the initial media prediction loss corresponding to the multimedia data of the j-th sample. The base of log in formula (2) can be either the default value in the computer equipment or a manually set value; there are no restrictions on this.

[0134] In one embodiment, the computer device can adjust the parameters of at least two nonlinear functions corresponding to the object relationship recognition network and the media relationship recognition network, respectively, based on the prediction loss, to obtain an adjusted initial prediction model. The object feature set of the sample objects and the media feature set of the sample multimedia data are input into the adjusted initial prediction model. Based on the object feature set and the media feature set, the adjusted initial prediction model determines the predicted interest degree of the sample objects towards the sample multimedia data. Based on the predicted interest degree obtained from the adjusted initial prediction model and the labeled interest degree of the sample objects towards the sample multimedia data, the convergence state of the adjusted initial prediction model is determined. When the convergence state of the adjusted initial prediction model is converged, the computer device can determine the adjusted initial prediction model as the media prediction model.

[0135] In one embodiment, the process of determining the convergence state of the adjusted initial prediction model may include the following steps: After obtaining the adjusted initial prediction model, the computer device may download test samples from the network or obtain test samples from local storage; from the test samples, the object feature set of the test object, the media feature set of the test multimedia data, and the labeled interest degree of the test object for the test multimedia data are obtained. Based on the object feature set of the test object and the media feature set of the test multimedia data, the predicted interest degree of the test object for the test multimedia data is determined using the adjusted initial prediction model. Based on the labeled interest degree and predicted interest degree of the test object for the test multimedia data, the prediction loss of the adjusted initial prediction model is determined using the above formula (2); when the prediction loss of the adjusted initial prediction model is less than the prediction loss threshold, the convergence state of the adjusted initial prediction model is determined to be a converged state; when the prediction loss of the adjusted initial prediction model is greater than or equal to the prediction loss threshold, the convergence state of the adjusted initial prediction model is determined to be a non-converged state. Using the test data set helps to avoid the situation where the adjusted initial prediction model only overfits to the training samples and has poor prediction accuracy in actual applications.

[0136] It should be noted that test samples are used to test the trained prediction model (i.e., the adjusted initial prediction model), and training samples are used to train the initial prediction model; the test objects in the test samples are different from the sample objects in the training samples, or the test multimedia data in the test samples are different from the sample objects in the training samples.

[0137] In one embodiment, during the process of predicting interest using the adjusted initial prediction model, the computer device can obtain the model performance parameters of the adjusted initial prediction model. When the prediction loss of the adjusted initial prediction model is less than the prediction loss threshold and the model performance parameters of the adjusted initial prediction model meet the model performance standard, the computer device can determine that the convergence state of the adjusted initial prediction model is converged. The computer device can then identify the adjusted initial prediction model as a media prediction model.

[0138] It should be noted that model performance parameters may include at least one of the following: memory usage, prediction duration, resource utilization, area under the ROC curve (AUC), and recall.

[0139] Specifically, when the model performance parameter is memory utilization, meeting the model performance standard means that the memory utilization is less than the memory utilization threshold; when the model performance parameter is prediction duration, meeting the model performance standard means that the prediction duration is less than the prediction duration threshold; when the model performance parameter is resource utilization, meeting the model performance standard means that the resource utilization is less than the resource utilization threshold; when the model performance parameter is AUC, meeting the model performance standard means that the AUC is greater than the AUC threshold; and when the model performance parameter is recall, meeting the model performance standard means that the recall is greater than the recall threshold.

[0140] In one embodiment, when the convergence state of the adjusted initial prediction model is non-converged, the computer device can adjust the parameters of at least two nonlinear functions corresponding to the object relationship recognition network and the media relationship recognition network in the adjusted initial prediction model to obtain a readjusted initial prediction model; until the computer device determines that the convergence state of the readjusted initial prediction model is converged, the computer device can determine the readjusted initial prediction model as the media prediction model.

[0141] Please refer to Figure 6 , Figure 6 This is a schematic diagram of the structure of an initial prediction model provided in this application, such as... Figure 6 As shown, the initial prediction model includes an object relationship recognition network 60, a media relationship recognition network 61, a feature extraction network 62, and a feature extraction network 63.

[0142] It should be noted that the object relationship recognition network 60 includes three object relationship recognition layers: object relationship recognition layer 60a, object relationship recognition layer 60b, and object relationship recognition layer 60c. Specifically, object relationship recognition layer 60a includes 5 nodes, object relationship recognition layer 60b includes 3 nodes, and object relationship recognition layer 60c includes 2 nodes.

[0143] The object relationship recognition layer 60a can be the input layer of the object relationship recognition network 60. The input layer of the object relationship recognition network 60 may not include nonlinear functions and fusion functions. Specifically, each node in the object relationship recognition layer 60a can be used to receive one or more original object features from the object feature set, and determine the received original object features as the candidate object relationship features obtained by that node.

[0144] If the connection between any two adjacent object relationship recognition layers in the object relationship recognition network is fully connected (meaning any node in each object relationship recognition layer is connected to every node in the adjacent object relationship recognition layer via an edge), and similarly, if the connection between any two adjacent media relationship recognition layers in the media relationship recognition network is fully connected (meaning any node in each media relationship recognition layer is connected to every node in the adjacent media relationship recognition layer), this will result in an excessively large number of edges in the initial prediction model. In other words, the number of parameters in the nonlinear functions corresponding to the edges in the initial prediction model will be too high.

[0145] It should be noted that the connection between any two adjacent object relationship identification layers in the initial prediction model of this application is a non-fully connected method. That is, any node in each object relationship identification layer is connected to at least one node in the adjacent object relationship identification layer through an edge. This reduces the storage space required by the initial prediction model of this application, which helps to alleviate the storage pressure on the device. Furthermore, if the nodes in the prediction model correspond to nonlinear functions and the edges correspond to fusion functions, the mapping process of the nonlinear functions is not easily visualized intuitively. Since the nodes in the initial prediction model of this application correspond to fusion functions and the edges correspond to nonlinear functions, the mapping process of the nonlinear functions is easily visualized intuitively and can easily interact with objects, thereby improving the interpretability of the initial prediction model.

[0146] Specifically, the three nodes in object relationship recognition layer 60b are connected to the five nodes in object relationship recognition layer 60a by six edges; object relationship recognition layer 60b includes nonlinear functions corresponding to these six edges. For example, node 60f in object relationship recognition layer 60b is connected to node 60d in object relationship recognition layer 60a by edge 60e, and edge 60e corresponds to a nonlinear function.

[0147] In this layer, two nodes in object relationship recognition layer 60c are connected to three nodes in object relationship recognition layer 60b by four edges; object relationship recognition layer 60c includes nonlinear functions corresponding to these four edges.

[0148] It should be noted that the media relationship identification network 61 includes three media relationship identification layers: media relationship identification layer 61a, media relationship identification layer 61b, and media relationship identification layer 61c. Specifically, media relationship identification layer 61a includes 5 nodes, media relationship identification layer 61b includes 3 nodes, and media relationship identification layer 61c includes 2 nodes.

[0149] The media relationship identification layer 61a can be the input layer of the media relationship identification network 61. The input layer of the media relationship identification network 61 may not include nonlinear functions and fusion functions. Specifically, each node in the media relationship identification layer 61a can be used to receive one or more original media features from the media feature set, and determine the received original media features as the candidate media relationship features obtained by that node.

[0150] It should be noted that the connection between any two adjacent media relationship identification layers in the initial prediction model is not fully connected. That is, any node in each media relationship identification layer is connected to at least one node in an adjacent media relationship identification layer via an edge. Specifically, the three nodes in media relationship identification layer 61b are connected to the five nodes in media relationship identification layer 61a via six edges; media relationship identification layer 61b includes nonlinear functions corresponding to these six edges. For example, node 61f in media relationship identification layer 61b is connected to node 61d in media relationship identification layer 61a via edge 61e. Edge 61e corresponds to a nonlinear function.

[0151] In this layer, two nodes in media relationship identification layer 61c are connected to three nodes in media relationship identification layer 61b by four edges; media relationship identification layer 61c includes nonlinear functions corresponding to these four edges.

[0152] Furthermore, the prediction process of the initial prediction model is explained. Specifically, the computer device can acquire five object attribute information of the sample object, namely object attribute information B1, object attribute information B2, object attribute information B3, object attribute information B4, and object attribute information B5, and input these five object attribute information into the feature extraction network 62. Through the feature extraction network 62, original object feature A1 is generated based on object attribute information B1, original object feature A2 is generated based on object attribute information B2, original object feature A3 is generated based on object attribute information B3, original object feature A4 is generated based on object attribute information B4, and original object feature A5 is generated based on object attribute information B5. The computer device can generate an object feature set 64 based on the above five original object features; the five original object features in the object feature set 64 are then input into five nodes in the object relationship recognition layer 60a.

[0153] For example, the original object feature A5 is input into node 60d in the object relationship recognition layer 60a, and the original object feature A5 is determined as the candidate object relationship feature corresponding to node 60d; the candidate object relationship feature corresponding to node 60d is then input into the object relationship recognition layer 60b. The nonlinear function corresponding to the edge 60e connecting nodes 60f and 60d is used to map the candidate object relationship feature corresponding to node 60d, obtaining the nonlinear object feature reflected by edge 60e. Next, the fusion function in the object relationship recognition layer 60b can be used to fuse the nonlinear object features reflected by the two edges connected to node 60f, obtaining the candidate object relationship feature corresponding to node 60f. Referring to the above operations, until the candidate object relationship features corresponding to nodes 60h and 60m are obtained, the fusion function in the object relationship recognition layer 60c is used to fuse the candidate object relationship features corresponding to nodes 60h and 60m, obtaining the fused object relationship feature 66.

[0154] Furthermore, the computer device can acquire five media attribute information from the sample multimedia data, namely media attribute information D1, media attribute information D2, media attribute information D3, media attribute information D4, and media attribute information D5, and input these five media attribute information into the feature extraction network 63. Through the feature extraction network 63, original media feature C1 is generated based on media attribute information D1, original media feature C2 is generated based on media attribute information D2, original media feature C3 is generated based on media attribute information D3, original media feature C4 is generated based on media attribute information D4, and original media feature C5 is generated based on media attribute information D5. The computer device can then generate a media feature set 65 based on these five original media features; and input the five original media features from the media feature set 65 into five nodes in the media relationship recognition layer 61a, respectively.

[0155] For example, the original media feature C5 is input into node 61d in the media relationship identification layer 61C, and the original media feature C5 is determined as the candidate media relationship feature corresponding to node 61d. The candidate media relationship feature corresponding to node 61d is input into the media relationship identification layer 61b. The nonlinear function corresponding to the edge 61e connecting nodes 61f and 61d is used to map the candidate media relationship feature corresponding to node 61d, resulting in the nonlinear media feature reflected by edge 61e. Next, the fusion function in the media relationship identification layer 61b can be used to fuse the nonlinear media features reflected by the two edges connected to node 61f, respectively, to obtain the candidate media relationship feature corresponding to node 61f. Referring to the above operations, until the candidate media relationship features corresponding to nodes 61h and 61m are obtained, the fusion function in the media relationship identification layer 61c is used to fuse the candidate media relationship features corresponding to nodes 61h and 61m, respectively, to obtain the fused media relationship feature 67. Next, based on the fusion object relationship feature 66 and the fusion media relationship feature 67, the predicted interest degree 68 of the sample object in the sample multimedia data can be determined.

[0156] In one embodiment, after training a media prediction model, the interest level of a target object in the multimedia data to be recommended can be predicted based on the media prediction model. Specifically, the computer device can obtain the target object feature set and the media feature sets corresponding to the S multimedia data to be recommended. Using the object relationship recognition network in the media prediction model, the fused object relationship features of the target object are determined based on the target object feature set; using the media relationship recognition network in the media prediction model, the fused media relationship features corresponding to the S multimedia data to be recommended are determined based on the media feature sets corresponding to the S multimedia data to be recommended. Based on the fused object relationship features of the target object and the fused media relationship features corresponding to the S multimedia data to be recommended, the interest level of the target object in the S multimedia data to be recommended can be predicted. The computer device can then select target multimedia data from the S candidate multimedia data to be recommended for promotion to the target object based on the target object's interest level in the S multimedia data to be recommended.

[0157] It should be noted that the selection of target multimedia data for promotion to the target audience can include any of the following methods: 1. From S candidate multimedia data to be recommended, select multimedia data with an interest level greater than the interest level threshold, and determine the selected multimedia data to be recommended as the target multimedia data; 2. According to the target audience's interest level in the S multimedia data to be recommended, sort the S multimedia data to be recommended to obtain a recommendation sequence, and obtain the part of the multimedia data to be recommended that appears at the beginning of the recommendation sequence, and determine the obtained part of the multimedia data to be recommended as the target multimedia data; the part of the multimedia data to be recommended may include one or more multimedia data to be recommended.

[0158] The interest threshold can be a default value in the computer device, or it can be manually set.

[0159] In this sequence, multimedia data appearing earlier in the order of recommendations corresponds to greater interest; conversely, multimedia data appearing later in the order of recommendations corresponds to lower interest. N can be a default value from the computer device, or N can be manually set.

[0160] Please see Figure 7 , Figure 7 This is a schematic diagram of a product recommendation scenario provided in this application. The media prediction model in this application can be applied to product recommendation scenarios; specifically, the three products include facial cleanser 71, facial cleanser 72, and facial cleanser 73. The multimedia data to be recommended for facial cleanser 71 is promotional video 71a, the multimedia data to be recommended for facial cleanser 72 is promotional video 72a, and the multimedia data to be recommended for facial cleanser 73 is promotional text 73a.

[0161] like Figure 7 As shown, the computer device can acquire the object feature set 70a of the target object 70, and input the object feature set 70a into the media prediction model 74 to obtain the fused object relationship feature 70b of the target object 70a. The computer device can acquire the media feature set 71b of the promotional video 71a, the media feature set 72b of the promotional video 72a, and the media feature set 73b of the promotional text 73a; input the media feature set 71b into the media prediction model 74 to obtain the fused media relationship feature 71c of the promotional video 71a; input the media feature set 72b into the media prediction model 74 to obtain the fused media relationship feature 72c of the promotional video 72a; input the media feature set 73b into the media prediction model 74 to obtain the fused media relationship feature 73c of the promotional text 73a.

[0162] Furthermore, based on the fusion object relationship feature 70b and the fusion media relationship feature 71c, the interest level (i.e., 75) of the target object 70a towards the promotional video 71a can be determined; based on the fusion object relationship feature 70b and the fusion media relationship feature 72c, the interest level (i.e., 80) of the target object 70a towards the promotional video 72a can be determined; and based on the fusion object relationship feature 70b and the fusion media relationship feature 73c, the interest level (i.e., 63) of the target object 70a towards the promotional text 73a can be determined. Taking an interest threshold of 78 as an example, since only the target object 70a's interest level in the promotional video 72a is greater than the interest threshold, the computer device can determine that the promotional video 72a needs to be promoted to the target object 70a. In this way, the recommended multimedia data (i.e., the promotional video 72a) that the target object is interested in can be filtered from the recommended multimedia data corresponding to the three products, which is beneficial to improving the conversion rate of the target object towards the recommended multimedia data corresponding to the product in the product recommendation scenario.

[0163] The object relationship recognition network in this application may include multiple object relationship recognition layers, each of which includes multiple nonlinear functions and fusion functions. By utilizing the multiple nonlinear functions and fusion functions included in each of the multiple object relationship recognition layers, the object relationship recognition network can accurately uncover more complex relationships between different features (i.e., highly accurate fusion of object relationship features), which is beneficial for improving the data fitting ability of the initial prediction model, and consequently, improving the prediction accuracy of the trained media prediction model.

[0164] Please see Figure 8 , Figure 8 This is a schematic diagram of the structure of a data processing device provided in this application. Figure 8 As shown, the data processing apparatus may include:

[0165] The acquisition module 811 is used to acquire the object feature set of the sample object, the media feature set of the sample multimedia data, the initial prediction model, and the labeled interest degree of the sample object in relation to the sample multimedia data; the initial prediction model includes an object relationship recognition network and a media relationship recognition network.

[0166] The first identification module 812 is used to map the original object features in the object feature set through at least two nonlinear functions in the object relationship identification network to obtain at least two nonlinear object features, and to perform association and fusion of the at least two nonlinear object features through the fusion function in the object relationship identification network to obtain fused object relationship features.

[0167] The second identification module 813 is used to map the original media features in the media feature set through at least two nonlinear functions in the media relationship identification network to obtain at least two nonlinear media features, and to perform correlation and fusion of the at least two nonlinear media features through the fusion function in the media relationship identification network to obtain fused media relationship features.

[0168] The prediction module 814 is used to determine the predicted interest degree of the sample object in the sample multimedia data based on the above-mentioned integrated media relationship characteristics and the above-mentioned integrated object relationship characteristics.

[0169] Training module 815 is used to adjust the parameters of at least two nonlinear functions in the object relationship recognition network and the media relationship recognition network according to the above-mentioned labeled interest degree and the above-mentioned predicted interest degree, so as to obtain a media prediction model; the above-mentioned media prediction model is used to predict the interest degree of the target object in the multimedia data to be recommended.

[0170] Optionally, the above object relationship recognition network includes M object relationship recognition layers, each of which includes at least two nonlinear functions and a fusion function; M is an integer greater than 1.

[0171] The first identification module 812 described above can also be used to perform the following operations:

[0172] Obtain P fused object relationship features from the i-th object relationship recognition layer; i is a positive integer less than M, and P is an integer greater than 1; the fused object relationship features obtained from the 1st object relationship recognition layer are generated based on the original object features in the above object feature set;

[0173] The above P fused object relationship features are mapped by Q nonlinear functions in the (i+1)th object relationship recognition layer to obtain Q nonlinear object features; Q is an integer greater than or equal to P.

[0174] The fusion function in the (i+1)th object relationship recognition layer is used to fuse every two non-linear object features that are related among the Q non-linear object features to obtain K fused object relationship features; K is a positive integer.

[0175] The fused object relationship features obtained from the Mth object relationship recognition layer are determined as fused object relationship features.

[0176] Optionally, the i-th object relationship identification layer includes P first nodes for reflecting the P fused object relationship features, and the (i+1)-th object relationship identification layer includes K second nodes. The P first nodes and the K second nodes are connected by Q edges; each edge reflects a nonlinear function.

[0177] The first identification module 812 described above can also be used to perform the following operations:

[0178] From the above Q edges, obtain the target edge that connects to the first node a among the above P first nodes; a is a positive integer less than or equal to P;

[0179] By mapping the nonlinear function reflected by the target edge, the relationship features of the fusion object corresponding to the first node a are obtained, and the nonlinear object features corresponding to the target edge are obtained.

[0180] This continues until the Q nonlinear object features corresponding to the aforementioned Q edges are obtained.

[0181] Optionally, each of the K second nodes is connected to at least two of the Q edges.

[0182] The first identification module 812 described above can also be used to perform the following operations:

[0183] From the above Q edges, obtain at least two edges that connect to the second node b among the above K second nodes; b is a positive integer less than or equal to K;

[0184] By using the fusion function in the (i+1)th object relationship recognition layer, the nonlinear object features corresponding to at least two edges connected to the second node b are fused to obtain the fused object relationship features corresponding to the second node b.

[0185] Optionally, the training module 815 described above can also be used to perform the following operations:

[0186] Based on the above labeled interest and the above predicted interest, determine the prediction loss of the above initial prediction model;

[0187] Based on the predicted loss described above, the convergence state of the initial prediction model is determined.

[0188] When the convergence state of the above initial prediction model is non-converged, the parameters of at least two nonlinear functions corresponding to the above object relationship recognition network and the above media relationship recognition network are adjusted according to the above prediction loss to obtain the media prediction model.

[0189] Optionally, the number of multimedia data samples mentioned above is N; N is an integer greater than 1.

[0190] The training module 815 described above can also be used to perform the following operations:

[0191] Based on the labeled interest and predicted interest of the sample objects for the j-th sample multimedia data, determine the initial media prediction loss corresponding to the j-th sample multimedia data; j is a positive integer less than or equal to M;

[0192] The average value of the initial media prediction loss corresponding to the above N sample multimedia data is applied to obtain the prediction loss of the above initial prediction model.

[0193] Optionally, the training module 815 described above can also be used to perform the following operations:

[0194] Based on the above prediction loss, the parameters of at least two nonlinear functions corresponding to the above object relationship recognition network and the above media relationship recognition network are adjusted to obtain the adjusted initial prediction model;

[0195] Obtain the set of object features of the test object, the set of media features of the test multimedia data, and the labeled interest degree of the test object in relation to the test multimedia data;

[0196] Based on the object feature set of the test objects, the media feature set of the test multimedia data, and the labeled interest of the test objects in relation to the test multimedia data, the convergence state of the adjusted initial prediction model is tested.

[0197] When the test shows that the above-adjusted initial prediction model has converged, the above-adjusted initial prediction model is determined as the media prediction model.

[0198] Optionally, the prediction module 814 described above can also be used to perform the following operations:

[0199] The matching degree is calculated by performing a matching degree operation on the above-mentioned integrated media relationship characteristics and the above-mentioned integrated object relationship characteristics to obtain the relationship matching degree between the above-mentioned sample objects and the above-mentioned sample multimedia data.

[0200] The matching degree between the above sample objects and the above sample multimedia data is regularized to obtain the predicted interest degree of the above sample objects in relation to the above sample multimedia data.

[0201] Optionally, the acquisition module 811 described above can also be used for:

[0202] Obtain the object attribute information of the above sample objects, and the media attribute information of the above sample multimedia data;

[0203] From the object feature index table, find the original object features that have an index relationship with the above object attribute information, and generate the object feature set of the above sample objects based on the found original object features.

[0204] From the media feature index table, find the original media features that have an index relationship with the above media attribute information, and generate the media feature set of the above sample multimedia data based on the found original media features.

[0205] Optionally, the above-mentioned data processing apparatus can also be used for:

[0206] Obtain the target object feature set and the media feature set corresponding to the S multimedia data to be recommended; S is a positive integer;

[0207] Using the media prediction model described above, based on the target object feature set and the candidate media feature sets corresponding to the S multimedia to be recommended, the interest of the target object in the S multimedia to be recommended data is predicted.

[0208] Based on the interest of the target audience in the S multimedia data to be recommended, target multimedia data for promotion to the target audience is selected from the S candidate multimedia data to be recommended.

[0209] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0210] This application proposes a dual-tower model (initial prediction model), which includes an object relationship recognition network and a media relationship recognition network. Each network comprises multiple nonlinear functions and fusion functions. The object relationship recognition network uses these functions to mine the relationships between original object features (i.e., fused object relationship features), while the media relationship recognition network uses these functions to mine the relationships between original media features (i.e., fused media relationship features). Since the multiple nonlinear functions in both networks are learnable, training the media prediction model with adjusted parameters allows it to accurately mine the relationships between different features using the adjusted nonlinear functions and fusion functions. This enhances the expressive power of the media prediction model and improves the accuracy of interest prediction.

[0211] Please see Figure 9 , Figure 9This is a schematic diagram of the structure of a computer device provided in this application. For example... Figure 9 As shown, the aforementioned computer device 900 can refer to a server or terminal, including: a processor 901, a network interface 904, and a memory 905. Furthermore, the aforementioned computer device 900 may also include: a user interface 903, and at least one communication bus 902. The communication bus 902 is used to implement communication between these components. In some embodiments, the user interface 903 may include a display screen and a keyboard; optionally, the user interface 903 may also include a standard wired interface or a wireless interface. The network interface 904 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 905 may be high-speed RAM or non-volatile memory, such as at least one disk storage device. The memory 905 may also optionally be at least one storage device located remotely from the aforementioned processor 901. Figure 9 As shown, the memory 905, which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and computer programs.

[0212] exist Figure 9 In the computer device 900 shown, the network interface 904 provides network communication functionality; the user interface 903 is mainly used to provide an input interface; and the processor 901 can be used to call computer programs stored in the memory 905 to execute:

[0213] The process involves obtaining the object feature set of the sample objects, the media feature set of the sample multimedia data, the initial prediction model, and the labeled interest level of the sample objects in relation to the sample multimedia data. The initial prediction model includes an object relationship recognition network and a media relationship recognition network.

[0214] By using at least two nonlinear functions in the object relationship recognition network, the original object features in the object feature set are mapped to obtain at least two nonlinear object features. By using the fusion function in the object relationship recognition network, the at least two nonlinear object features are associated and fused to obtain fused object relationship features.

[0215] By using at least two nonlinear functions in the media relationship identification network, the original media features in the media feature set are mapped to obtain at least two nonlinear media features. By using the fusion function in the media relationship identification network, the at least two nonlinear media features are correlated and fused to obtain fused media relationship features.

[0216] Based on the aforementioned characteristics of converged media relationships and the aforementioned characteristics of converged objects relationships, the predicted interest degree of the aforementioned sample objects in relation to the aforementioned sample multimedia data is determined.

[0217] Based on the above-mentioned labeled interest and predicted interest, the parameters of at least two nonlinear functions corresponding to the above-mentioned object relationship recognition network and media relationship recognition network are adjusted to obtain the media prediction model; the above-mentioned media prediction model is used to predict the interest of the target object in the multimedia data to be recommended.

[0218] Optionally, the above object relationship recognition network includes M object relationship recognition layers, each of which includes at least two nonlinear functions and a fusion function; M is an integer greater than 1.

[0219] The processor 901 described above can be used to call computer programs stored in memory 905 to execute:

[0220] Obtain P fused object relationship features from the i-th object relationship recognition layer; i is a positive integer less than M, and P is an integer greater than 1; the fused object relationship features obtained from the 1st object relationship recognition layer are generated based on the original object features in the above object feature set;

[0221] The above P fused object relationship features are mapped by Q nonlinear functions in the (i+1)th object relationship recognition layer to obtain Q nonlinear object features; Q is an integer greater than or equal to P.

[0222] The fusion function in the (i+1)th object relationship recognition layer is used to fuse every two non-linear object features that are related among the Q non-linear object features to obtain K fused object relationship features; K is a positive integer.

[0223] The fused object relationship features obtained from the Mth object relationship recognition layer are determined as fused object relationship features.

[0224] Optionally, the i-th object relationship identification layer includes P first nodes for reflecting the P fused object relationship features, and the (i+1)-th object relationship identification layer includes K second nodes. The P first nodes and the K second nodes are connected by Q edges; each edge reflects a nonlinear function.

[0225] The processor 901 described above can be used to call computer programs stored in memory 905 to execute:

[0226] From the above Q edges, obtain the target edge that connects to the first node a among the above P first nodes; a is a positive integer less than or equal to P;

[0227] By mapping the nonlinear function reflected by the target edge, the relationship features of the fusion object corresponding to the first node a are obtained, and the nonlinear object features corresponding to the target edge are obtained.

[0228] This continues until the Q nonlinear object features corresponding to the aforementioned Q edges are obtained.

[0229] Optionally, each of the K second nodes is connected to at least two of the Q edges.

[0230] The processor 901 described above can be used to call computer programs stored in memory 905 to execute:

[0231] From the above Q edges, obtain at least two edges that connect to the second node b among the above K second nodes; b is a positive integer less than or equal to K;

[0232] By using the fusion function in the (i+1)th object relationship recognition layer, the nonlinear object features corresponding to at least two edges connected to the second node b are fused to obtain the fused object relationship features corresponding to the second node b.

[0233] Optionally, the processor 901 described above can be used to call a computer program stored in memory 905 to execute:

[0234] Based on the above labeled interest and the above predicted interest, determine the prediction loss of the above initial prediction model;

[0235] Based on the predicted loss described above, the convergence state of the initial prediction model is determined.

[0236] When the convergence state of the above initial prediction model is non-converged, the parameters of at least two nonlinear functions corresponding to the above object relationship recognition network and the above media relationship recognition network are adjusted according to the above prediction loss to obtain the media prediction model.

[0237] Optionally, the number of multimedia data sets in the above sample is N; N is an integer greater than 1.

[0238] The processor 901 described above can be used to call computer programs stored in memory 905 to execute:

[0239] Based on the labeled interest and predicted interest of the sample objects for the j-th sample multimedia data, determine the initial media prediction loss corresponding to the j-th sample multimedia data; j is a positive integer less than or equal to M;

[0240] The average value of the initial media prediction loss corresponding to the above N sample multimedia data is applied to obtain the prediction loss of the above initial prediction model.

[0241] Optionally, the processor 901 described above can be used to call a computer program stored in memory 905 to execute:

[0242] Based on the above prediction loss, the parameters of at least two nonlinear functions corresponding to the above object relationship recognition network and the above media relationship recognition network are adjusted to obtain the adjusted initial prediction model;

[0243] Obtain the set of object features of the test object, the set of media features of the test multimedia data, and the labeled interest degree of the test object in relation to the test multimedia data;

[0244] Based on the object feature set of the test objects, the media feature set of the test multimedia data, and the labeled interest of the test objects in relation to the test multimedia data, the convergence state of the adjusted initial prediction model is tested.

[0245] When the test shows that the above-adjusted initial prediction model has converged, the above-adjusted initial prediction model is determined as the media prediction model.

[0246] Optionally, the processor 901 described above can be used to call a computer program stored in memory 905 to execute:

[0247] The matching degree is calculated by performing a matching degree operation on the above-mentioned integrated media relationship characteristics and the above-mentioned integrated object relationship characteristics to obtain the relationship matching degree between the above-mentioned sample objects and the above-mentioned sample multimedia data.

[0248] The matching degree between the above sample objects and the above sample multimedia data is regularized to obtain the predicted interest degree of the above sample objects in relation to the above sample multimedia data.

[0249] Optionally, the processor 901 described above can be used to call a computer program stored in memory 905 to execute:

[0250] Obtain the object attribute information of the above sample objects, and the media attribute information of the above sample multimedia data;

[0251] From the object feature index table, find the original object features that have an index relationship with the above object attribute information, and generate the object feature set of the above sample objects based on the found original object features.

[0252] From the media feature index table, find the original media features that have an index relationship with the above media attribute information, and generate the media feature set of the above sample multimedia data based on the found original media features.

[0253] Optionally, the processor 901 described above can be used to call a computer program stored in memory 905 to execute:

[0254] Obtain the target object feature set and the media feature set corresponding to the S multimedia data to be recommended; S is a positive integer;

[0255] Using the media prediction model described above, based on the target object feature set and the candidate media feature sets corresponding to the S multimedia to be recommended, the interest of the target object in the S multimedia to be recommended data is predicted.

[0256] Based on the interest of the target audience in the S multimedia data to be recommended, target multimedia data for promotion to the target audience is selected from the S candidate multimedia data to be recommended.

[0257] This application proposes a dual-tower model (initial prediction model), which includes an object relationship recognition network and a media relationship recognition network. Each network comprises multiple nonlinear functions and fusion functions. The object relationship recognition network uses these functions to mine the relationships between original object features (i.e., fused object relationship features), while the media relationship recognition network uses these functions to mine the relationships between original media features (i.e., fused media relationship features). Since the multiple nonlinear functions in both networks are learnable, training the media prediction model with adjusted parameters allows it to accurately mine the relationships between different features using the adjusted nonlinear functions and fusion functions. This enhances the expressive power of the media prediction model and improves the accuracy of interest prediction.

[0258] Furthermore, it should be noted that this application also provides a computer-readable storage medium storing a computer program executed by the aforementioned data processing apparatus. This computer program includes program instructions, which, when executed by the processor, enable the execution of the data processing method described in the corresponding embodiments above. Therefore, further details will not be repeated here. Additionally, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the embodiments of the computer-readable storage medium involved in this application, please refer to the description of the method embodiments of this application.

[0259] As an example, the above program instructions can be deployed and executed on a single computer device, or deployed and executed on at least two computer devices in one location, or executed on at least two computer devices distributed in at least two locations and interconnected via a communication network. At least two computer devices distributed in at least two locations and interconnected via a communication network can form a blockchain network.

[0260] The aforementioned computer-readable storage medium may be a data processing apparatus provided in any of the foregoing embodiments or a central storage unit of the aforementioned computer device, such as a hard disk or central storage of the computer device. The computer-readable storage medium may also be an external storage device of the computer device, such as a plug-in hard disk, smart memory card (SMC), secure digital (SD) card, flash card, etc., provided on the computer device. Furthermore, the computer-readable storage medium may include both the central storage unit and external storage devices of the computer device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0261] The terms "first," "second," etc., in the specification, claims, and drawings of this application are used to distinguish content in different media, rather than to describe a specific order. Furthermore, the term "comprising," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, apparatus, product, or device that includes a series of steps or units is not limited to the listed steps or modules, but may optionally include steps or modules not listed, or may optionally include other step units inherent to these processes, methods, apparatuses, products, or devices.

[0262] In practice, the collection and processing of data in this application should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the data subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the data subject.

[0263] This application also provides a computer program product, including a computer program. When executed by a processor, the computer program implements the data processing method and decoding method described in the preceding embodiments, and therefore will not be repeated here. Furthermore, the beneficial effects of using the same method will also not be repeated. For technical details not disclosed in the embodiments of the computer program product involved in this application, please refer to the description of the method embodiments of this application.

[0264] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0265] The methods and related apparatus provided in this application are described with reference to the method flowcharts and / or structural diagrams provided in this application. Specifically, each block of the method flowchart and / or structural diagram, as well as combinations of blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable network-connected device to create a machine, such that the instructions, which execute via the processor of the computer or other programmable network-connected device, generate instructions for implementing the process. Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable network-connected device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 A schematic diagram of one or more processes and / or structures. Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable network-connected device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 A process or multiple processes and / or structures illustrate the steps of the functions specified in one or more boxes.

[0266] The above-disclosed embodiments are merely preferred embodiments of this application and should not be construed as limiting the scope of this application. Therefore, any equivalent variations made in accordance with the claims of this application shall still fall within the scope of this application.

Claims

1. A data processing method, characterized in that, include: Obtain the object feature set of the sample object, the media feature set of the sample multimedia data, the initial prediction model, and the labeled interest degree of the sample object in relation to the sample multimedia data; The initial prediction model includes an object relationship recognition network and a media relationship recognition network; At least two nonlinear functions in the object relationship recognition network are used to map the original object features in the object feature set to obtain at least two nonlinear object features. The fusion function in the object relationship recognition network is used to associate and fuse the at least two nonlinear object features to obtain fused object relationship features. At least two nonlinear functions in the media relationship identification network are used to map the original media features in the media feature set to obtain at least two nonlinear media features. The fusion function in the media relationship identification network is used to associate and fuse the at least two nonlinear media features to obtain fused media relationship features. Based on the fused media relationship characteristics and the fused object relationship characteristics, the predicted interest degree of the sample object in the sample multimedia data is determined; Based on the labeled interest degree and the predicted interest degree, the parameters of at least two nonlinear functions corresponding to the object relationship recognition network and the media relationship recognition network are adjusted to obtain the media prediction model; the media prediction model is used to predict the interest degree of the target object in the multimedia data to be recommended.

2. The method according to claim 1, characterized in that, The object relationship recognition network includes M object relationship recognition layers, each of which includes at least two nonlinear functions and a fusion function; M is an integer greater than 1. The process of mapping the original object features in the object feature set to at least two nonlinear object features using at least two nonlinear functions in the object relationship recognition network, and then fusing these at least two nonlinear object features using a fusion function in the object relationship recognition network to obtain fused object relationship features, includes: Obtain P candidate object relationship features from the i-th object relationship recognition layer; i is a positive integer less than M, and P is an integer greater than 1; the candidate object relationship features obtained from the first object relationship recognition layer are generated based on the original object features in the object feature set; The P candidate object relationship features are mapped using Q nonlinear functions in the (i+1)th object relationship recognition layer to obtain Q nonlinear object features; Q is an integer greater than or equal to P. The fusion function in the (i+1)th object relationship recognition layer is used to fuse every two non-linear object features that are related among the Q non-linear object features to obtain K candidate object relationship features; K is a positive integer. The candidate object relationship features obtained from the Mth object relationship recognition layer are determined as the fused object relationship features.

3. The method according to claim 2, characterized in that, The i-th object relationship recognition layer includes P first nodes for reflecting the relationship features of the P candidate objects, and the (i+1)-th object relationship recognition layer includes K second nodes. The P first nodes and the K second nodes are connected by Q edges; each edge reflects a nonlinear function. The process of mapping the P candidate object relationship features using Q nonlinear functions in the (i+1)th object relationship recognition layer to obtain Q nonlinear object features includes: From the Q edges, obtain the target edge that connects to the first node a among the P first nodes; a is a positive integer less than or equal to P; By mapping the candidate object relationship features corresponding to the first node a through the nonlinear function reflected by the target edge, the nonlinear object features corresponding to the target edge are obtained. This continues until the Q nonlinear object features corresponding to the Q edges are obtained.

4. The method according to claim 3, characterized in that, Each of the K second nodes is connected to at least two of the Q edges; The fusion function in the (i+1)th object relationship recognition layer is used to fuse every two non-linear object features with a relationship among the Q non-linear object features to obtain K candidate object relationship features, including: From the Q edges, obtain at least two edges that connect to the second node b among the K second nodes; b is a positive integer less than or equal to K; The nonlinear object features corresponding to at least two edges connected to the second node b are fused using the fusion function in the (i+1)th object relationship recognition layer to obtain the candidate object relationship features corresponding to the second node b.

5. The method according to claim 1, characterized in that, The step of adjusting the parameters of at least two nonlinear functions corresponding to the object relationship recognition network and the media relationship recognition network based on the labeled interest degree and the predicted interest degree to obtain the media prediction model includes: The prediction loss of the initial prediction model is determined based on the labeled interest degree and the predicted interest degree. Based on the predicted loss, determine the convergence state of the initial prediction model; When the initial prediction model is in a non-convergent state, the parameters of at least two nonlinear functions corresponding to the object relationship recognition network and the media relationship recognition network are adjusted according to the prediction loss to obtain the media prediction model.

6. The method according to claim 5, characterized in that, The number of sample multimedia data is N; N is an integer greater than 1; The step of determining the prediction loss of the initial prediction model based on the labeled interest degree and the predicted interest degree includes: Based on the labeled interest degree and predicted interest degree of the sample object for the j-th sample multimedia data, determine the initial media prediction loss corresponding to the j-th sample multimedia data; j is a positive integer less than or equal to M; The prediction loss of the initial prediction model is obtained by averaging the initial media prediction losses corresponding to the N sample multimedia data.

7. The method according to claim 5, characterized in that, The step of adjusting the parameters of at least two nonlinear functions corresponding to the object relationship recognition network and the media relationship recognition network based on the prediction loss to obtain the media prediction model includes: Based on the prediction loss, the parameters of at least two nonlinear functions corresponding to the object relationship recognition network and the media relationship recognition network are adjusted to obtain the adjusted initial prediction model; Obtain the object feature set of the test object, the media feature set of the test multimedia data, and the labeled interest degree of the test object in relation to the test multimedia data; Based on the object feature set of the test object, the media feature set of the test multimedia data, and the labeled interest degree of the test object towards the test multimedia data, the convergence state of the adjusted initial prediction model is tested. When the test shows that the adjusted initial prediction model has converged, the adjusted initial prediction model is determined as the media prediction model.

8. The method according to claim 1, characterized in that, The step of determining the predicted interest degree of the sample object in the sample multimedia data based on the fused media relationship characteristics and the fused object relationship characteristics includes: A matching degree calculation is performed on the fused media relationship features and the fused object relationship features to obtain the relationship matching degree between the sample object and the sample multimedia data; The matching degree between the sample object and the sample multimedia data is regularized to obtain the predicted interest degree of the sample object in relation to the sample multimedia data.

9. The method according to claim 1, characterized in that, The method further includes: Obtain the object attribute information of the sample object and the media attribute information of the sample multimedia data; From the object feature index table, find the original object features that have an index relationship with the object attribute information, and generate the object feature set of the sample object based on the found original object features; From the media feature index table, find the original media features that have an index relationship with the media attribute information, and generate the media feature set of the sample multimedia data based on the found original media features.

10. The method according to claim 1, characterized in that, The method further includes: Obtain the target object feature set and the media feature set corresponding to the S multimedia data to be recommended; S is a positive integer; Using the media prediction model, based on the target object feature set and the candidate media feature sets corresponding to the S multimedia to be recommended, the interest of the target object in the S multimedia to be recommended data is predicted. Based on the target object's interest in the S multimedia data to be recommended, target multimedia data for promotion to the target object is selected from the S candidate multimedia data to be recommended.

11. A data processing apparatus, characterized in that, include: The acquisition module is used to acquire the object feature set of the sample object, the media feature set of the sample multimedia data, the initial prediction model, and the labeled interest degree of the sample object in relation to the sample multimedia data; the initial prediction model includes an object relationship recognition network and a media relationship recognition network. The first identification module is used to map the original object features in the object feature set through at least two nonlinear functions in the object relationship identification network to obtain at least two nonlinear object features, and to perform association and fusion of the at least two nonlinear object features through the fusion function in the object relationship identification network to obtain fused object relationship features. The second identification module is used to map the original media features in the media feature set through at least two nonlinear functions in the media relationship identification network to obtain at least two nonlinear media features, and to perform association and fusion of the at least two nonlinear media features through the fusion function in the media relationship identification network to obtain fused media relationship features. The prediction module is used to determine the predicted interest degree of the sample object in the sample multimedia data based on the fused media relationship characteristics and the fused object relationship characteristics; The training module is used to adjust the parameters of at least two nonlinear functions in the object relationship recognition network and the media relationship recognition network according to the labeled interest degree and the predicted interest degree, so as to obtain a media prediction model; the media prediction model is used to predict the interest degree of the target object in the multimedia data to be recommended.

12. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the method of any one of claims 1 to 10.

13. A computer storage medium, characterized in that, The computer storage medium stores a computer program, which, when executed by a processor, performs the method as described in any one of claims 1 to 10.

14. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 10.

Images