Business data processing method and device, equipment and medium

Abstract

The embodiment of the invention provides a business data processing method and device, equipment and a medium. The method comprises the steps that sample multi-modal features of a sample information sequence are acquired from a sample multi-modal feature library; obtaining sample position coding features of the N pieces of sample service information in the sample information sequence, and obtaining sample information input features according to the sample information sequence, the sample position coding features and the sample multi-modal features; performing feature extraction processing on the sample information input features through a feature extractor in the initial prediction model to obtain sample information recommendation features, and obtaining information recommendation probability distribution corresponding to the sample information recommendation features through an output layer in the initial prediction model; and correcting network parameters of the initial prediction model according to the information recommendation probability distribution, and determining a feature extractor in the initial prediction model containing the corrected network parameters as an information prediction model. Through the embodiment of the invention, the accuracy of information recommendation can be improved.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a business data processing method, device, equipment and medium. Background Art

[0002] With the rapid development of Internet technology, people can obtain various information (e.g., video, audio, news, etc.) through the Internet to meet different needs. However, in the environment of large-scale data, most of the data is redundant for users, and users may only be interested in certain information. Therefore, providing personalized information for each user has become a daily need.

[0003] In current information recommendation scenarios, the exposure rate and click-through rate of the information to be recommended are usually used as evaluation indicators for recommendation. However, sorting the information to be recommended in the resource pool by indicators such as exposure rate and click-through rate may result in information that has been exposed and clicked many times recently being more likely to be recommended again, while some high-quality resources in the resource pool with low exposure rate and click-through rate that users may be interested in cannot be recommended, resulting in low accuracy of information recommendation. Summary of the invention

[0004] The embodiments of the present application provide a business data processing method, apparatus, device and medium, which can improve the accuracy of information recommendation.

[0005] On the one hand, an embodiment of the present application provides a method for processing business data, including:

[0006] Obtain a sample information sequence including N sample business information, and obtain a sample multimodal feature corresponding to each sample business information in the sample information sequence in a sample multimodal feature library; the sample multimodal feature is determined according to text representation information and image representation information corresponding to the sample business information; N is an integer greater than 1;

[0007] Obtain sample position coding features of N sample business information in the sample information sequence, and obtain sample information input features according to the sample information sequence, the sample position coding features, and the sample multimodal features corresponding to the N sample business information;

[0008] Input the sample information input features into the initial prediction model, perform feature extraction processing on the sample information input features through the feature extractor in the initial prediction model, obtain the sample information recommendation features corresponding to the sample information sequence, and obtain the information recommendation probability distribution corresponding to the sample information recommendation features through the output layer in the initial prediction model;

[0009] The network parameters of the initial prediction model are modified according to the information recommendation probability distribution, and the feature extractor in the initial prediction model containing the modified network parameters is determined as the information prediction model; the information prediction model is used to output the business information recommendation features corresponding to the business recommendation information contained in the business recommendation information set, and the business information recommendation features are used to determine the object recommendation information corresponding to the business object.

[0010] On the one hand, an embodiment of the present application provides a method for processing business data, including:

[0011] Obtain a business recommendation information set, and obtain a business information sequence corresponding to the business recommendation information x in the business recommendation information set; the length of the business information sequence corresponding to the business recommendation information x is the same as the length of the sample information sequence;

[0012] Obtaining a business multimodal feature corresponding to the business recommendation information x in a business multimodal feature library; the business multimodal feature is determined according to text representation information and image representation information corresponding to the business recommendation information;

[0013] Obtaining a business position coding feature of the business recommendation information x in the business information sequence, and obtaining a business information input feature corresponding to the business recommendation information x according to the business information sequence corresponding to the business recommendation information x, the business position coding feature corresponding to the business recommendation information x, and the business multimodal feature corresponding to the business recommendation information x;

[0014] The business information input features corresponding to the business recommendation information x are input into the information prediction model, and feature extraction processing is performed on the business information input features corresponding to the business recommendation information x through the information prediction model to obtain the business information recommendation features corresponding to the business recommendation information x; the business information recommendation features corresponding to the business recommendation information x are used to determine the object recommendation information corresponding to the business object.

[0015] On the one hand, an embodiment of the present application provides a business data processing device, including:

[0016] The first feature acquisition module is used to acquire a sample information sequence including N sample business information, and acquire a sample multimodal feature corresponding to each sample business information in the sample information sequence in a sample multimodal feature library; the sample multimodal feature is determined according to the text representation information and the image representation information corresponding to the sample business information; N is an integer greater than 1;

[0017] The first feature combination module is used to obtain sample position coding features of N sample business information in the sample information sequence, and obtain sample information input features according to the sample information sequence, the sample position coding features, and the sample multimodal features corresponding to the N sample business information;

[0018] A probability distribution acquisition module is used to input sample information input features into the initial prediction model, perform feature extraction processing on the sample information input features through the feature extractor in the initial prediction model, obtain sample information recommendation features corresponding to the sample information sequence, and obtain information recommendation probability distribution corresponding to the sample information recommendation features through the output layer in the initial prediction model;

[0019] The prediction model training module is used to correct the network parameters of the initial prediction model according to the information recommendation probability distribution, and determine the feature extractor in the initial prediction model containing the corrected network parameters as the information prediction model; the information prediction model is used to output the business information recommendation features corresponding to the business recommendation information contained in the business recommendation information set, and the business information recommendation features are used to determine the object recommendation information corresponding to the business object.

[0020] The service data processing device further includes:

[0021] A sample data acquisition module is used to acquire a sample business data set, and acquire sample text data and sample image data corresponding to the sample business information contained in the sample business data set;

[0022] A text encoding module, used to perform text encoding processing on the sample text data corresponding to the sample business information a in the sample business data set to obtain text representation information corresponding to the sample business information a;

[0023] An image encoding module, used for performing image encoding processing on sample image data corresponding to sample service information a in the sample service data set to obtain image representation information corresponding to the sample service information a;

[0024] A feature fusion module is used to perform feature fusion on the text representation information corresponding to the sample business information a and the image representation information corresponding to the sample business information a to obtain a sample multimodal feature corresponding to the sample business information a;

[0025] The feature adding module is used to obtain the sample multimodal features corresponding to the sample business information contained in the sample business data set, and add the sample multimodal features corresponding to each sample business information in the sample business data set to the sample multimodal feature library.

[0026] Among them, the text encoding module includes:

[0027] A character division unit is used to divide the sample text data corresponding to the sample business information a in the sample business data set into D unit characters, and obtain the unit word vectors corresponding to the D unit characters respectively; D is a positive integer;

[0028] A text vector acquisition unit, used to acquire text vectors corresponding to the D unit characters respectively according to semantic information of the D unit characters in the sample text data corresponding to the sample business information a;

[0029] A position vector acquisition unit, used to acquire position vectors corresponding to the D unit characters respectively according to the text positions of the D unit characters in the sample text data corresponding to the sample business information a;

[0030] A vector superposition unit, used to obtain text input features corresponding to sample business information a according to the unit word vector, the text vector and the position vector;

[0031] The text feature processing unit is used to input the text input features into the text encoder, perform text encoding processing on the text input features through the text encoder to obtain the text encoding features corresponding to the sample business information a, perform full connection processing on the text encoding features to obtain the text representation information corresponding to the sample business information a.

[0032] Among them, the image encoding module includes:

[0033] A first convolution processing unit, used to input the sample image data corresponding to the sample business information a in the sample business data set into the image encoder, perform convolution processing on the sample image data corresponding to the sample business information a through the convolution layer in the image encoder to obtain a first sample convolution feature, and perform maximum pooling processing on the first sample convolution feature to obtain a first sample pooling feature;

[0034] The first input feature acquisition unit is used to acquire the input feature of the yth residual block structure in the image encoder; when y is 1, the input feature of the yth residual block structure is the first sample pooling feature; when y is not 1, the input feature of the yth residual block structure is the output feature of the y-1th residual block structure; y is a positive integer less than or equal to P, and P represents the number of residual block structures included in the image encoder;

[0035] A second convolution processing unit is used to perform convolution processing on the input features of the yth residual block structure to obtain a second sample convolution feature of the yth residual block structure, and obtain an output feature of the yth residual block structure according to the second sample convolution feature of the yth residual block structure and the input features of the yth residual block structure;

[0036] The fully connected processing unit is used to perform average pooling processing on the output features of the P-th residual block structure to obtain a second sample pooling feature, and perform full connection processing on the second sample pooling feature to obtain image representation information corresponding to the sample business information a.

[0037] Wherein, the first feature combination module includes:

[0038] A sinusoidal position coding unit, used for dividing the positions of the N sample service information in the sample information sequence into even positions and odd positions, performing sinusoidal position coding on the even positions in the N sample service information, and obtaining sinusoidal coding information corresponding to the even positions;

[0039] A cosine position coding unit, used for performing cosine position coding on odd positions in the N sample business information to obtain cosine coding information corresponding to the odd positions;

[0040] The position coding feature acquisition unit is used to determine the sine coding information and the cosine coding information as sample position coding features of N sample business information in the sample information sequence.

[0041] Wherein, the first feature combination module includes:

[0042] A vector conversion unit, used to perform vector conversion on each sample service information in the sample information sequence to obtain a sample sequence vector corresponding to the sample information sequence;

[0043] The second input feature acquisition unit is used to determine the product of the sample sequence vector and the sample multimodal feature and the sum of the sample position encoding feature as the sample information input feature.

[0044] The feature extractor includes M attention components, one attention component includes K self-attention layers, and M and K are both positive integers; the probability distribution acquisition module includes:

[0045] The third input feature acquisition unit is used to acquire the input feature of the i-th attention component among the M attention components; when i is 1, the input feature of the i-th attention component is the sample information input feature; when i is not 1, the input feature of the i-th attention component is the output feature of the i-1-th attention component; i is a positive integer less than or equal to M;

[0046] The first feature transformation unit is used to obtain the transformation parameter matrix of the jth self-attention layer in the i-th attention component, and perform feature transformation processing on the input features of the i-th attention component through the transformation parameter matrix to obtain sample transformation features; j is a positive integer less than or equal to K;

[0047] An information mask processing unit is used to obtain an information mask matrix of the j-th self-attention layer, and perform information mask processing on the sample transformation features through the information mask matrix to obtain output features of the j-th self-attention layer; the information mask matrix is ​​used to indicate the sample business information used by the initial prediction model when determining the sample business information features corresponding to the t+1-th sample business information in the sample information sequence, where t is a positive integer less than N;

[0048] A feature concatenation unit is used to concatenate the output features of the K self-attention layers in the i-th attention component to obtain the sample mask concatenation feature of the i-th attention component;

[0049] A first normalization unit, used for obtaining a first sample joint feature according to the sample mask splicing feature and the input feature of the i-th attention component, and normalizing the first sample joint feature to obtain a sample normalized feature;

[0050] A second feature transformation unit is used to perform feature transformation processing on the sample normalized features through the feedforward network layer in the i-th attention component to obtain an attention transformation feature;

[0051] The second normalization unit is used to obtain a second sample joint feature based on the attention transformation feature and the sample normalization feature, normalize the second sample joint feature to obtain the output feature of the i-th attention component, and determine the output feature of the M-th attention component as the sample business recommendation feature corresponding to the sample information sequence.

[0052] The sample transformation features include a query matrix Q, a key matrix K and a value matrix V; and the information mask processing unit includes:

[0053] A first point multiplier unit is used to determine the product of the transposed matrix of the key matrix K and the query matrix Q as the first candidate matrix, and determine the product of the first candidate matrix and the information mask matrix as the second candidate matrix;

[0054] The second point multiplier unit is used to obtain the number of columns corresponding to the query matrix Q, normalize the ratio between the second candidate matrix and the square root of the number of columns, obtain the attention weight matrix, and determine the product of the attention weight matrix and the value matrix V as the output feature of the j-th self-attention layer.

[0055] The sample information recommendation feature corresponding to the sample information sequence includes the sample information recommendation feature associated with each sample business information in the sample information sequence; and the probability distribution acquisition module includes:

[0056] a third feature conversion unit, configured to perform feature conversion processing on the sample information recommendation feature associated with the t+1th sample service information in the sample information sequence through the output layer in the initial prediction model, to obtain a candidate service recommendation feature corresponding to the sample information recommendation feature associated with the t+1th sample service information; t is a positive integer less than N;

[0057] An index operation unit, used to perform an index operation on the candidate service recommendation feature corresponding to the t+1th sample service information to obtain a recommendation index value corresponding to the t+1th sample service information;

[0058] A first recommendation probability value acquisition unit is used to accumulate the recommendation index values ​​corresponding to each sample service information in the sample information sequence to obtain a total recommendation index value, and determine the ratio between the recommendation index value corresponding to the t+1th sample service information and the total recommendation index value as the recommendation probability value corresponding to the t+1th sample service information;

[0059] The probability distribution acquisition unit determines the information recommendation probability distribution according to the recommendation probability value corresponding to each sample business information in the sample information sequence.

[0060] Among them, the prediction model training module includes:

[0061] A second recommendation probability value acquisition unit is used to acquire the recommendation probability values ​​of the first t sample business information in the sample information sequence in the information recommendation probability distribution; one sample business information corresponds to one recommendation probability value, and t is a positive integer less than N;

[0062] A sample prediction loss acquisition unit is used to perform a logarithmic operation on the recommended probability value of each sample information in the first t sample business information to obtain a logarithmic probability value corresponding to each sample information in the first t sample business information, and to accumulate the logarithmic probability value corresponding to each sample business information in the first t sample business information to obtain a sample prediction loss corresponding to the t+1th sample business information;

[0063] The model iteration training unit is used to iteratively train the network parameters of the initial prediction model according to the sample prediction loss corresponding to the t+1th sample business information until the updated network parameters meet the training end conditions, stop the training, and determine the feature extractor in the initial prediction model at the end of the training as the information prediction model.

[0064] On the one hand, an embodiment of the present application provides a business data processing device, including:

[0065] A business information sequence acquisition module is used to acquire a business recommendation information set, and acquire a business information sequence corresponding to business recommendation information x in the business recommendation information set; the length of the business information sequence corresponding to the business recommendation information x is the same as the length of the sample information sequence;

[0066] The second feature acquisition module acquires the business multimodal features corresponding to the business recommendation information x in the business multimodal feature library; the business multimodal features are determined according to the text representation information and the image representation information corresponding to the business recommendation information;

[0067] The second feature combination module is used to obtain the business position coding feature of the business recommendation information x in the business information sequence, and obtain the business information input feature corresponding to the business recommendation information x according to the business information sequence corresponding to the business recommendation information x, the business position coding feature corresponding to the business recommendation information x, and the business multimodal feature corresponding to the business recommendation information x;

[0068] The third feature acquisition module is used to input the business information input features corresponding to the business recommendation information x into the information prediction model, and perform feature extraction processing on the business information input features corresponding to the business recommendation information x through the information prediction model to obtain the business information recommendation features corresponding to the business recommendation information x; the business information recommendation features corresponding to the business recommendation information x are used to determine the object recommendation information corresponding to the business object.

[0069] Among them, the business information sequence acquisition module includes:

[0070] An initial information sequence acquisition unit is used to acquire an initial information sequence having the same length as the sample information sequence; the elements in the initial information sequence are all mask filling values;

[0071] The identification information replacement unit is used to replace any element in the initial information sequence with the identification information corresponding to the service recommendation information x in the service recommendation information set to obtain the service information sequence corresponding to the service recommendation information x.

[0072] The service data processing device further includes:

[0073] A recommendation information pairing module, used to pair the business recommendation information in the business recommendation information set to obtain a plurality of recommendation information pairs;

[0074] A first similarity evaluation value acquisition module is used to determine a similarity evaluation value between two pieces of business recommendation information included in each pair of recommendation information according to business information recommendation features corresponding to the two pieces of business recommendation information included in each pair of recommendation information;

[0075] The information relationship structure generating module is used to generate the information relationship structure corresponding to the business recommendation information set according to the business recommendation information set and the similarity evaluation value associated with each recommendation information pair.

[0076] The service data processing device further includes:

[0077] A historical browsing information acquisition module is used to obtain a collection of historical browsing information of a business object on the information recommendation platform;

[0078] A recommendation information adding module, configured to add the service recommendation information x to the candidate recommendation information set if the similarity evaluation value between the service recommendation information x in the information relationship structure and the historical browsing information in the historical browsing information set is greater than a similarity threshold;

[0079] The recommendation information statistics module is used to count the frequencies corresponding to each business recommendation information in the candidate recommendation information set, and determine the object recommendation information corresponding to the business object in the candidate recommendation information set according to the frequencies corresponding to each business recommendation information in the candidate recommendation information set.

[0080] The service data processing device further includes:

[0081] A fourth feature acquisition module is used to obtain the business multimodal features corresponding to the business recommendation information b according to the text representation information and image representation information corresponding to the business recommendation information b if the newly added business recommendation information b in the information recommendation platform is detected, and add the business multimodal features corresponding to the business recommendation information b to the business multimodal feature library;

[0082] A third feature combination module is used to obtain the business information input feature corresponding to the business recommendation information b according to the business information sequence corresponding to the business recommendation information b, the business position coding feature corresponding to the business recommendation information b, and the business multimodal feature corresponding to the business recommendation information b;

[0083] A fifth feature acquisition module is used to perform feature extraction processing on the business information input features of the business recommendation information b through the information prediction model to obtain initial information recommendation features corresponding to the business recommendation information b, and perform pooling processing on the initial information recommendation features corresponding to the business recommendation information b to obtain business information recommendation features corresponding to the business recommendation information b;

[0084] A second similarity evaluation value acquisition module is used to acquire a similarity evaluation value between the business information recommendation feature corresponding to the business recommendation information b and the business information recommendation feature corresponding to each business recommendation information in the business recommendation information set;

[0085] The information relationship structure updating module is used to update the information relationship structure according to the business recommendation information b and the similarity evaluation value between the business recommendation information b and each business recommendation information in the business recommendation information set.

[0086] On one hand, an embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory is connected to the processor, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method provided in the above aspect of the embodiment of the present application.

[0087] On one hand, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program is suitable for being loaded and executed by a processor, so that a computer device with a processor executes the method provided in the above aspect of the embodiment of the present application.

[0088] According to one aspect of the present application, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the method provided in the above aspect.

[0089] In an embodiment of the present application, after obtaining the sample multimodal features associated with the sample information sequence in the sample multimodal feature library, the sample information sequence, the sample position coding features associated with the sample information sequence, and the sample multimodal features can be combined into sample information input features; then, the sample information input features are subjected to feature extraction processing by the feature extractor in the initial prediction model pair to obtain the sample information recommendation features corresponding to the sample information sequence, the initial prediction model is trained by the information recommendation probability distribution corresponding to the sample information recommendation features, and the feature extractor in the trained initial prediction model is determined as the information prediction model. Among them, the sample multimodal features integrate the text representation information and image representation information of the sample business information, and the sample multimodal features are used to represent the sample business information for the training of the initial prediction model, which can better reflect the substantive content of the sample business information. The information prediction model obtained by training the sample multimodal features can improve the information representation ability of the business information recommendation features, and thus improve the accuracy of information recommendation. BRIEF DESCRIPTION OF THE DRAWINGS

[0090] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0091] Figure 1 It is a structural diagram of a network architecture provided by an embodiment of the present application;

[0092] Figure 2 It is a flowchart of a business data processing method provided by an embodiment of the present application;

[0093] Figure 3 is a schematic diagram of a training initial prediction model provided in an embodiment of the present application;

[0094] Figure 4 It is a flowchart of another business data processing method provided in an embodiment of the present application;

[0095] Figure 5 is a schematic diagram of obtaining multimodal features of a sample provided by an embodiment of the present application;

[0096] Figure 6 is a structural schematic diagram of a feature extractor in an initial prediction model provided in an embodiment of the present application;

[0097] Figure 7 It is a flowchart of another business data processing method provided in an embodiment of the present application;

[0098] Figure 8 This is a schematic diagram of obtaining a service information recommendation feature provided by an embodiment of the present application;

[0099] Fig. 9 This is a schematic diagram of a video recommendation scenario provided by an embodiment of the present application;

[0100] Fig.10 It is a structural diagram of a business data processing device provided in an embodiment of the present application;

[0101] Fig.11 is a structural diagram of another business data processing device provided in an embodiment of the present application;

[0102] Fig.12 It is a structural diagram of another business data processing device provided in an embodiment of the present application;

[0103] Fig.13 It is a structural diagram of another business data processing device provided in an embodiment of the present application;

[0104] Fig.14 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0105] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0106] See also Figure 1 , Figure 11 is a schematic diagram of a network architecture provided by an embodiment of the present application. The network architecture may include a server 10d and a terminal cluster. The terminal cluster may include one or more terminal devices. The network architecture may also include one or more servers. The number of terminal devices and servers included in the network architecture is not limited here. Figure 1 As shown, the terminal cluster may specifically include terminal device 10a, terminal device 10b and terminal device 10c, etc.; all terminal devices in the terminal cluster (for example, terminal device 10a, terminal device 10b and terminal device 10c, etc.) may be connected to the server 10d through a network, and the network may be a wide area network, or a local area network, or a combination of a wide area network and a local area network, and data transmission between each terminal device and the server 10d may be achieved using wired or wireless methods.

[0107] in, Figure 1 The server 10d shown can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms. This application does not limit the type of server.

[0108] Figure 1 The terminal devices in the terminal cluster shown may include but are not limited to: smart phones, tablet computers, laptops, desktop computers, mobile Internet devices (MID), wearable devices (such as smart watches, smart bracelets, etc.), smart voice interaction devices, smart home appliances (such as smart TVs, etc.), vehicle-mounted devices, aircraft, and other electronic devices with information recommendation functions. This application does not limit the type of terminal devices. It is understandable that, if Figure 1 Each terminal device in the terminal cluster shown in the figure can be installed with an application client. When the application client runs in each terminal device, it can be respectively connected to the above Figure 1 The server 10d shown in the figure performs data interaction. The application client running in each terminal device may be an independent client or an embedded sub-client integrated in a certain client, which is not limited in the present application.

[0109] Among them, Figure 1As shown, the terminal devices in the terminal cluster can integrate an application client with an information recommendation function, which can also be called an information recommendation platform. The information recommendation platform can recommend information of objects of interest to business objects; the application client can include but is not limited to: multimedia clients (for example, short video clients, live broadcast clients, long video clients), entertainment clients (for example, game clients), social clients (for example, instant messaging application clients, office clients), information clients, etc. The business recommendation information in the application client can be a video in a multimedia client, a commodity in an e-commerce client, a game prop in a game client, or information content in an information client, etc., which is not limited in the embodiments of the present application.

[0110] Below Figure 1 Taking the terminal device 10a in the terminal cluster shown as an example, the application client of the terminal device 10a can integrate the trained initial prediction model, wherein the initial prediction model can include a feature extractor, an output layer, etc. In the embodiment of the present application, the feature extractor in the trained initial prediction model can be determined as an information prediction model. The information prediction model can be used to output the business information recommendation feature corresponding to the business recommendation information contained in the business recommendation information set; wherein the business recommendation information set can include multiple business recommendation information; the business information recommendation feature can be used to determine the object recommendation information corresponding to the business object; the object recommendation information refers to the business recommendation information in the business recommendation information set that is about to be pushed to the business object.

[0111] Specifically, the terminal device 10a can obtain the business multimodal features corresponding to the business recommendation information in the business recommendation information set from the business multimodal library, and combine the business position coding features, business information sequence, and business multimodal features associated with the business recommendation information into business information input features; perform feature extraction processing on the business information input features through the information prediction model to obtain the business information recommendation features corresponding to each business recommendation information in the business recommendation information set; the terminal device 10a can establish an information relationship structure associated with the business recommendation information set through the business information recommendation features. When the business object enters the application client in the terminal device 10a, the terminal device 10a can obtain the historical browsing information of the business object, and quickly find the business recommendation information with a large similar evaluation value to the historical browsing information from the information relationship structure and recommend it to the business object. It can be seen that the embodiment of the present application is suitable for the recommendation scenario of item to item (I2I) with a lighter quantity, which can improve the efficiency and accuracy of information recommendation.

[0112] It is understandable that the information prediction model mentioned above includes the feature extractor in the trained initial prediction model, that is, before using the information prediction model to output the business information recommendation features corresponding to the business recommendation information, the initial prediction model needs to be trained. Figures 2 to 6 The training process of the initial prediction model is described.

[0113] See also Figure 2 , Figure 2 1 is a flow chart of a business data processing method provided by an embodiment of the present application. It can be understood that the business data processing method can be executed by a computer device, and the computer device can be a terminal device (for example, Figure 1 Any terminal device in the terminal cluster of the corresponding embodiment), or it can be a server (for example, Figure 1 The server 10d shown in the figure), the embodiment of the present application takes the service data processing method executed by the terminal device as an example for explanation; wherein, the service data processing method may include the following steps S101 to S104:

[0114] Step S101: obtaining a sample information sequence including N sample business information, and obtaining a sample multimodal feature corresponding to each sample business information in the sample information sequence in a sample multimodal feature library.

[0115] In an embodiment of the present application, N sample business information can be selected from the sample business data set to form a sample information sequence, and the sample information sequence can be recorded as {sample information sequence 1, sample information sequence 2, ..., sample information sequence N}. Wherein, N is an integer greater than 1. Wherein, the sample business data set refers to a set of representative sample business information collected from one or more fields. For example, in the field of audio and video, the sample business data set may include some audio and video data of the audio and video playback platform online; in the field of news and information, the sample business data set may include some articles or videos on the news and information platform; in the field of shopping, the sample business data set may include some products that are being traded or have been removed from the shopping platform.

[0116] The N sample business information in the sample information sequence may be sample business information of the same field, or may be a random combination of sample business information of different fields; the N sample business information in the sample information sequence may also be sample business information of the same category, or may be a random combination of sample business information of multiple categories. For example, the sample sequence information may include V (V is a positive integer less than N) videos and NV audios, etc., may include N videos containing the same object, may include N clothing products, or may include V social news information and NV entertainment information, etc.

[0117] In an embodiment of the present application, the sample multimodal features corresponding to the sample information sequence can be found in the sample multimodal feature library. That is to say, the sample multimodal features corresponding to the sample business information in the sample business data set can be extracted in advance by a multimodal encoder and stored in the sample multimodal feature library. It can be understood that the sample multimodal features may refer to features that fuse different types of representation information of the sample business information. For example, the sample multimodal features may refer to features that fuse multiple representation information such as text representation information and image representation information (including video representation information) corresponding to the sample business information. The method for obtaining the sample multimodal features will be described in detail below.

[0118] It can be understood that the sample multimodal features integrate the text representation information and image representation information of the sample business information. Using the sample multimodal features to represent the sample business information for training the initial prediction model can better reflect the essential content of the sample business information. The information prediction model trained by the sample multimodal features can improve the information representation ability of the business information recommendation features, thereby improving the accuracy of information recommendation.

[0119] See also Figure 3 , Figure 3 Schematic diagram of a training initial prediction model provided in an embodiment of the present application. Figure 3 As shown, the sample information sequence can be input into the initial prediction model 20b, wherein the initial prediction model 20b can be a natural language processing model. Figure 3As shown, the network structure of the initial prediction model 20b may include a feature extractor 20d and an output layer 20f. The feature extractor 20d may be a Transformer (a network model using an attention mechanism), a GPT (Generative Pre-trained Transformer, a pre-trained generative converter), a BERT (Bidirectional Encoder Representation from Transformers, a natural language processing model) structure, a BERT4Rec (a natural language processing model improved on the basis of BERT), etc., or may be a deformation of any of the above natural language processing models. It is understandable that the natural language processing model can well mine the global information of the sample business information, thereby enhancing the quality of feature extraction and helping to improve the accuracy of subsequent information recommendation. In addition, the natural language processing model should be able to naturally learn the dependencies between the various sample business information in the sample information sequence, thereby improving the feature expression capability. It can be seen that the embodiment of the present application comprehensively utilizes the sample multimodal features and the powerful inference capability of the language model. The information prediction model obtained by training the sample multimodal features can improve the information representation capability of the business information recommendation feature, thereby improving the accuracy of the information recommendation.

[0120] like Figure 3 As shown, the sample multimodal feature library can be stored in the multimodal embedding layer in the embedding layer 20c in the initial prediction model 20b. After obtaining the sample information sequence, it can be input into the initial prediction model 20b, and the sample multimodal features corresponding to each sample business information in the sample information sequence can be found from the multimodal embedding layer in the initial prediction model 20b through the identification information corresponding to the sample business information. The identification information of the sample business information can be information used to uniquely identify or distinguish different sample business information, and can specifically include but is not limited to: serial number, QR code, barcode, or can be information obtained by numbers, letters, symbols or a combination thereof. In an embodiment of the present application, since the sample multimodal features are obtained in advance according to the multimodal encoder, the sample multimodal features remain unchanged during the training process of the initial prediction model, and the weights will not change with the back propagation algorithm.

[0121] Step S102: Obtain sample position coding features of N sample business information in the sample information sequence, and obtain sample information input features according to the sample information sequence, the sample position coding features, and the sample multimodal features corresponding to the N sample business information.

[0122] like Figure 3As shown, the sample information input feature can be used as the input feature of the feature extractor 20d in the initial prediction model 20b. The sample information input feature can be a feature determined according to the sample information sequence, the sample position coding feature, and the sample multimodal features corresponding to the N sample business information.

[0123] It can be understood that since the initial prediction model 20b can be a natural language processing model, the initial prediction model 20b usually includes a Transformer structure, and the Transformer structure is usually used to process and model serialized data (such as natural language processing tasks such as machine translation). Since the sample business information may contain image or video data, the structured information (spatial information) in the image or video data cannot be encoded in a serialized manner, that is, the Transformer structure cannot be directly applied to non-serialized image data. Therefore, in image-related tasks, position encoding (Position Embedding) can be used to position encode the two-dimensional spatial information of the sample business information to improve the processing effect of the Transformer structure on images or videos.

[0124] In the embodiment of the present application, in order to retain the two-dimensional spatial information in the sample business information, the position encoding method can be used to perform position encoding processing on the sample information sequence to obtain the sample position encoding features of the N sample business information in the sample information sequence. Among them, the position encoding method involved in the embodiment of the application may include but is not limited to: sine cosine position encoding (2D sine position embedding), learnable position encoding (learnable position embedding), etc. For ease of understanding, the following takes sine cosine position encoding as an example to describe the position encoding processing in the sample information sequence.

[0125] The sine-cosine position coding process of the sample information sequence may include: dividing the positions of the N sample service information in the sample information sequence into even positions and odd positions, performing sine position coding on the even positions in the N sample service information to obtain sine coding information corresponding to the even positions; performing cosine position coding on the odd positions in the N sample service information to obtain cosine coding information corresponding to the odd positions; and then determining the sine coding information and the cosine coding information as sample position coding features of the N sample service information in the sample information sequence. The sine-cosine position coding process may be shown in the following formulas (1) to (2):

[0126]

[0127]

[0128] Wherein, in the above formula (1) and formula (2), pos represents the position of the sample business information in the sample information sequence, 2t represents the even position, and 2t+1 represents the odd position; each position in the sample information sequence can be regarded as a sine curve, and the wavelength is a geometric series from 2π to 1000*2π; d2 represents the dimension of a single sample business information in the sample information sequence, such as it can be considered as the height in the dimension of the sample information sequence, or it can be understood as the number of columns of the sample information sequence. Formula (1) can represent the sine coding information of the sample business information at the even position, and formula (2) can represent the cosine coding information of the sample business information at the odd position. It can be understood that the sample position coding feature associated with the sample information sequence can be stored in the position embedding layer in the embedding layer 20c in the initial prediction model 20b, and the sample position coding feature can continuously adjust the weight during the training process of the initial prediction model.

[0129] Furthermore, the sample information sequence, the sample position coding features, and the sample multimodal features corresponding to the N sample business information can be combined into a sample information input feature. The embodiments of the present application do not limit the specific combination method. In one possible implementation, each sample business information in the sample information sequence can be vectorized to obtain a sample sequence vector corresponding to the sample information sequence; for example, one-hot (unique hot code) can be used to vectorize each sample business information in the sample information sequence to obtain a sample business vector corresponding to each sample business information, and combine them into a sample sequence vector. Then, the product of the sample sequence vector and the sample multimodal feature and the sum of the sample position coding features can be determined as the sample information input feature. Among them, the sample information input feature can be shown as the following formula (3):

[0130] h 0 =UW e +W p (3)

[0131] Among them, h in formula (3) 0 represents the sample information input feature, that is, the input feature of the feature extractor 20d in the initial prediction model 20b; U represents the sample sequence vector; W e Represents the sample multimodal features corresponding to the sample information sequence; W p Represents the sample position encoding feature.

[0132] Optionally, the sum of the sample sequence vector, the sample multimodal feature, and the sample position encoding feature may be determined as the sample information input feature. The sample information input feature may be as shown in the following formula (4):

[0133] h 0 =U+W e+W p (4)

[0134] Step S103: Input the sample information input features into the initial prediction model, perform feature extraction processing on the sample information input features through the feature extractor in the initial prediction model, obtain the sample information recommendation features corresponding to the sample information sequence, and obtain the information recommendation probability distribution corresponding to the sample information recommendation features through the output layer in the initial prediction model.

[0135] The sample information recommendation feature is an embedding representation obtained by processing the sample information input feature through the feature extractor in the initial prediction model; the sample information recommendation feature can capture the key semantics, themes, emotions and other aspects of the sample business information, and has high expressiveness and richness. The information recommendation probability distribution includes the recommendation probability value corresponding to each sample business information in the sample information sequence. The recommendation probability values ​​of the first t sample business information contained in the information recommendation probability distribution can be used to predict the probability of generating the t+1th sample business information. Where t is a positive integer less than N.

[0136] In an embodiment of the present application, the sample information input features can be used as input features in a feature extractor in an initial prediction model. The feature extractor may include one or more attention components. The sample information input features can be subjected to attention encoding processing through the attention components to obtain sample information recommendation features corresponding to the sample information sequence. The specific attention encoding processing process will be described in detail below and will not be repeated here.

[0137] like Figure 3 As shown, after the sample information recommendation feature 20e corresponding to the sample information sequence is obtained by the feature extractor 20d, the output layer 20f can obtain the information recommendation probability distribution 20g corresponding to the sample information recommendation feature 20e. Specifically, the sample information recommendation feature 20e can be subjected to feature transformation processing by the output layer 20f to obtain the candidate information recommendation feature corresponding to the sample information recommendation feature 20e, and the candidate information recommendation feature can be normalized to obtain the information recommendation probability distribution.

[0138] Step S104: modifying the network parameters of the initial prediction model according to the information recommendation probability distribution, and determining the feature extractor in the initial prediction model including the modified network parameters as the information prediction model.

[0139] like Figure 3As shown, the network parameters of the initial prediction model 20b can be modified according to the information recommendation probability distribution 20g, so as to obtain the information prediction model. Among them, the information prediction model can be considered as the feature extractor in the trained initial prediction model, so the information prediction model and the feature extractor have the same network structure. The information prediction model can be used to output the business information recommendation features corresponding to the business recommendation information contained in the business recommendation information set, and the business information recommendation features are used to determine the object recommendation information corresponding to the business object. The business information recommendation features can capture the key semantics, themes, emotions and other aspects of the business recommendation information, and have high expressiveness and richness, so that it can be applied to the item-to-item (I2I) recommendation scenario, or it can also be applied to the object-to-item (U2I) recommendation scenario, which helps to improve the accuracy of information recommendation.

[0140] It is understandable that in the embodiment of the present application, one sample business information may correspond to one recommendation probability value. The training process of the initial prediction model may be unsupervised training, that is, the sample business information in the sample information sequence is unlabeled sample data. The recommendation probability value of the t+1th sample business information may be predicted based on the given recommendation probability values ​​of the first t sample business information. Wherein, t is a positive integer less than N. The specific value of T may be 1, 2, ... N.

[0141] Specifically, the recommendation probability values ​​of the first t sample business information in the sample information sequence can be obtained in the information recommendation probability distribution; the recommendation probability value of each sample information in the first t sample business information is logarithmically calculated to obtain the logarithmic probability value corresponding to each sample information in the first t sample business information, and the logarithmic probability value corresponding to each sample business information in the first t sample business information is accumulated to obtain the sample prediction loss corresponding to the t+1th sample business information. The sample prediction loss corresponding to the t+1th sample business information can be shown as the following formula (5):

[0142]

[0143] Among them, in formula (5) represents the sample prediction loss corresponding to the t+1th sample business information; θ represents the network parameters in the initial prediction model, x represents the sample business information in the sample information sequence; x t+1 represents the t+1th sample service information in the sample information sequence, where t is a positive integer smaller than the sample information sequence (which can be denoted as N) contained in the sample information sequence; x ＜tt1 represents the sample service information before the t+1th sample service information in the sample information sequence, that is, the first t sample service information; It represents the recommendation probability value of the t+1th sample business information, and log() represents logarithmic processing.

[0144] After obtaining the sample prediction loss corresponding to the t+1th sample business information, the network parameters of the initial prediction model can be iteratively trained according to the sample prediction loss corresponding to the t+1th sample business information until the updated network parameters meet the training end conditions, and the training is stopped, and the feature extractor in the initial prediction model at the end of the training is determined as the information prediction model. Among them, the training end conditions of the initial prediction model may include but are not limited to: the number of training times of the initial prediction model reaches the preset maximum number of iterations, or the sample prediction loss corresponding to the t+1th sample business information is less than or equal to the preset error value, or the update amplitude of the network parameters of the initial prediction model is less than the preset error value, etc., and the embodiments of the present application do not limit this.

[0145] For example, in the training phase of the initial prediction model, AdamW (an optimization method) or optimization methods such as gradient descent with momentum can be used as optimizers for training. In the training phase of the initial prediction model, the initial prediction model can be set to train for 10 epochs (iterative rounds), where one epoch represents a complete training of the sample information sequence once, and epoch=10 represents a complete training of the sample information sequence 10 times, and the feature extractor in the initial prediction model at the end of the 10th epoch training is determined as the information prediction model.

[0146] In an embodiment of the present application, after obtaining the sample multimodal features associated with the sample information sequence in the sample multimodal feature library, the sample information sequence, the sample position coding features associated with the sample information sequence, and the sample multimodal features can be combined into sample information input features; then, the sample information input features are subjected to feature extraction processing by the feature extractor in the initial prediction model pair to obtain the sample information recommendation features corresponding to the sample information sequence, the initial prediction model is trained by the information recommendation probability distribution corresponding to the sample information recommendation features, and the feature extractor in the trained initial prediction model is determined as the information prediction model. Among them, the sample multimodal features integrate the text representation information and image representation information of the sample business information, and the sample multimodal features are used to represent the sample business information for the training of the initial prediction model, which can better reflect the substantive content of the sample business information. The information prediction model obtained by training the sample multimodal features can improve the information representation ability of the business information recommendation features, and thus improve the accuracy of information recommendation.

[0147] See also Figure 4 , Figure 4FIG. 1 is a flow chart of another method for processing business data provided by an embodiment of the present application. It is understandable that the method for processing business data can be executed by a computer device, which can be a terminal device (for example, Figure 1 Any terminal device in the terminal cluster of the corresponding embodiment), or it can be a server (for example, Figure 1 The server 10d shown in the figure), the embodiment of the present application takes the service data processing method executed by the terminal device as an example for explanation; wherein, the service data processing method may include the following steps S201 to S216:

[0148] Step S201: Acquire a sample business data set, and acquire sample text data and sample image data corresponding to sample business information contained in the sample business data set.

[0149] After obtaining the sample business data set, if the sample business information includes video data, the video data can be framed to obtain sample image data; if the image data in the sample business information contains text information, optical character recognition (OCR) technology can be used to extract the text information in the image data to obtain sample text data.

[0150] See also Figure 5 , Figure 5 is a schematic diagram of obtaining multimodal features of a sample provided by an embodiment of the present application. Figure 5 As shown, the multimodal encoder 40d can be used to extract features from the sample business information in the sample business data set to obtain the sample multimodal features 40h. For ease of description, the embodiment of the present application takes the sample business information a in the sample business information a in the sample business data set as an example to describe in detail the process of obtaining the sample multimodal features of the sample business information. The sample business information a is any sample business information in the sample business data set, and the sample multimodal features of other sample business information in the sample business data set can be obtained in the same way.

[0151] Step S202: Perform text encoding processing on the sample text data corresponding to the sample business information a in the sample business data set to obtain text representation information corresponding to the sample business information a.

[0152] like Figure 5As shown, the sample text data corresponding to the sample business information a can be subjected to text encoding processing by the text encoder in the multimodal encoder 40a to obtain the text representation information 40f corresponding to the sample business information a. Specifically, the sample text data corresponding to the sample business information a in the sample business data set can be divided into D unit characters (the unit characters here can be single characters or single words in the sample text data corresponding to the sample business information a), and the unit word vectors corresponding to the D unit characters are obtained; for example, one-hot (unique hot code) can be used to represent each unit character by vector to obtain a unit word vector; or word2vec (a word vector model) can be used to represent each unit character by vector to obtain a unit word vector; each unit character in the sample text data corresponding to the sample business information a can also be converted into a word vector (i.e., a unit word vector) by querying a word vector table, wherein the word vector table can contain word vectors corresponding to all commonly used characters, and the word vector table can be understood as a "dictionary" containing all commonly used character vectors, and the unit character can refer to a single character or a single word in the sample text data corresponding to the sample business information a. Wherein, D is a positive integer, such as D can take the value of 1, 2, ...

[0153] In an embodiment of the present application, the text vectors corresponding to the D unit characters can be obtained according to the semantic information of the D unit characters in the sample text data corresponding to the sample business information a; the position vectors corresponding to the D unit characters can be obtained according to the text positions of the D unit characters in the sample text data corresponding to the sample business information a. Among them, the text vector can be used to characterize the global semantic information of the sample text data corresponding to the sample business information a, and can be fused with the unit word vector; because the semantic information carried by the unit characters appearing at different positions in the sample text data corresponding to the sample business information a is different (for example, the semantic information carried by "I wait for you" and "You wait for me" is different), a different position vector can be attached to each of the D unit characters in the sample text data corresponding to the sample business information a for distinction.

[0154] Furthermore, the unit word vector, text vector and position vector can be superimposed to obtain the text input feature corresponding to the sample business information a. That is, the sum of the unit word vector, text vector and position vector can be determined as the text input feature corresponding to the sample business information a. Figure 5As shown, the text input feature can be input into the text encoder, and the text encoder performs text encoding processing on the text input feature to obtain the text encoding feature corresponding to the sample business information a, and the text encoding feature is fully connected to obtain the text representation information corresponding to the sample business information a. The network structure of the text encoder may include but is not limited to the current text processing model, such as an encoder with a Transformer structure, a BERT structure, GPT, a BART structure (a natural language processing model improved on the basis of BERT), RNN (Recurrent Neural Network), LSTM (Long Short-Term Memory, etc., or it may be a deformation of any of the above natural language processing networks, or a combination of any two or more of the above networks, etc. The embodiment of the present application does not limit the network structure of the text encoder.

[0155] Specifically, when the text encoder is a BERT structure, the main structure of the text encoder is a transformer, and the text encoder may include an attention encoding component (which may be a multi-head self-attention component), a normalized network layer, a multi-layer perceptron (or a feedforward network layer), etc. After the text input features are input into the text encoder, the attention encoding component in the text encoder may be used to process the text input features; or the text input features may be position-encoded to obtain position encoding features corresponding to the text input features, and the text input features and their position encoding features may be added to obtain the initial representation information of the text, and then the attention encoding component in the text encoder may be used to process the text input features.

[0156] The features obtained after the text input features pass through the attention encoding component in the text encoder can be called text attention encoding features, and then the text attention encoding features can be normalized according to the first normalization network layer in the text encoder to obtain the first normalized text features, and the first normalized text features and the text input features are combined into text joint features. According to the feedforward network layer (or multi-layer perceptron) in the text encoder, the text joint features are transformed to obtain text transformation features, and then the text transformation features can be normalized according to the second normalization network layer in the text encoder to obtain the second normalized text features, and the second normalized text features and the text joint features are combined into text encoding features.

[0157] The text encoding feature can be understood as the output feature of the text encoder. The text encoding feature usually has a higher dimension. In order to make the subsequent sample multimodal features match the dimension of the initial prediction model, the text encoding feature can be input into Figure 5The representation layer 40d shown in the figure may include one or more fully connected layers. The text encoding features are fully connected through the representation layer 40d to obtain text representation information 40f corresponding to the sample business information a to reduce the data dimension. The text representation information can be used to characterize the text features in the sample text data corresponding to the sample business information a.

[0158] Step S203: performing image coding processing on the sample image data corresponding to the sample service information a in the sample service data set to obtain image representation information corresponding to the sample service information a.

[0159] like Figure 5 As shown, the sample image data corresponding to the sample business information a can be subjected to image encoding processing by the image encoder in the multimodal encoder 40a to obtain the image representation information 40g corresponding to the sample business information a. The image representation information 40g can be used to characterize the image features in the sample image data corresponding to the sample business information a. The network structure of the image encoder may include but is not limited to ResNet (a residual network model), DenseNet (a densely connected network model), Transformer network model, Conformer (a network model composed of a convolutional neural network and a transformer), Swin Transformer (a network model based on a transformer structure), and any deformation of the above feature extraction network, or a combination of any two or more of the above networks, etc. The present application does not limit the network structure of the image encoder.

[0160] In a possible implementation, the network structure of the image encoder may include a convolution layer, a maximum pooling layer, one or more residual block structures, and an average pooling layer. In an embodiment of the present application, the sample image data corresponding to the sample business information a in the sample business data set may be input into the image encoder, and the sample image data corresponding to the sample business information a may be convolved through the convolution layer in the image encoder to obtain a first sample convolution feature; the embodiment of the present application does not limit the size and number of convolution kernels in the convolution layer, for example, the number of convolution kernels may be one or more, and the size of the convolution kernel may be 1×1 or 7×7, etc. Then, the first sample convolution feature may be subjected to maximum pooling through the maximum pooling layer in the image encoder to obtain a first sample pooling feature, so as to reduce the dimension of the data, reduce the number of parameters, and simplify the subsequent calculation burden.

[0161] Among them, the image encoder may include P residual block structures, P is a positive integer, and the value of P can be 1, 2, 3... Residual connections can be performed between the P residual block structures. For example, the feature after adding the input feature of the previous residual block structure and the output feature of the previous residual block structure can be used as the input feature of the next residual block structure. Each residual block structure in the image encoder may include one or more convolutional layers. The embodiment of the present application does not limit the size and number of convolution kernels in the convolutional layer. For example, the number of convolution kernels can be one or more, and the size of the convolution kernel can be 3×3 or 7×7, etc. Optionally, the residual block structure may also include an activation layer, and the activation layer may include one or more activation functions such as ReLU function, Sigmoid function, and Tanh function.

[0162] In the embodiment of the present application, the specific process of performing convolution processing on the first sample pooling feature through the residual block structure may include: obtaining the input feature of the yth residual block structure among the P residual block structures in the image encoder; when y is 1, the input feature of the yth residual block structure is the first sample pooling feature; when y is not 1, the input feature of the yth residual block structure is the output feature of the y-1th residual block structure; y is a positive integer less than or equal to P, and the specific value of y can be 1, 2, ... P. Then, the input feature of the yth residual block structure can be convolved through one or more convolution layers in the yth residual block structure to obtain the second sample convolution feature of the yth residual block structure; the second sample convolution feature of the yth residual block structure can be expressed as: Z y =w y *x y +b y , where Z y It can represent the second sample convolution feature output by the convolution layer in the yth residual block structure, w y It can represent the weight of the convolutional layer in the yth residual block structure, x y It can represent the input features in the y-th residual block structure, b y It can represent the bias vector of the convolutional layer in the y-th residual block structure.

[0163] In an embodiment of the present application, the second sample convolution feature of the yth residual block structure and the input feature of the yth residual block structure can be combined into the output feature of the yth residual block structure; the combination here can be feature addition processing; the output feature of the yth residual block structure can be used as the input feature of the y+1th residual block structure; wherein the yth residual block structure and the y+1th residual block structure are connected in the image encoder. Until the output feature of the Pth residual block structure (the last residual block structure) is obtained; and then the output feature of the Pth residual block structure can be averagely pooled through the average pooling layer in the image encoder to obtain the second sample pooling feature, so as to reduce the dimension of the data, reduce the number of parameters, and simplify the subsequent calculation burden; the second sample pooling feature can be considered as the output feature of the image encoder.

[0164] The second sample pooling feature output by the image encoder usually has a higher dimension. In order to make the subsequent sample multimodal features match the dimension of the initial prediction model, the image encoding feature can be input into Figure 5 The representation layer 40e shown may include one or more fully connected layers. The second sample pooling features are fully connected through the representation layer 40e to obtain image representation information 40g corresponding to the sample business information a, so as to further reduce the data dimension.

[0165] Step S204: feature fusion is performed on the text representation information corresponding to the sample business information a and the image representation information corresponding to the sample business information a to obtain sample multimodal features corresponding to the sample business information a.

[0166] The feature fusion method may include but is not limited to: ① performing feature splicing processing on the text representation information corresponding to the sample business information a and the image representation information corresponding to the sample business information a, and determining the spliced ​​features as the sample multimodal features corresponding to the sample business information a; ② calculating the average value between the text representation information corresponding to the sample business information a and the image representation information corresponding to the sample business information a, and determining the average value as the sample multimodal features corresponding to the sample business information a. The sample multimodal features corresponding to the sample business information a may be obtained by using one or more of the above feature fusion methods.

[0167] Step S205: Acquire sample multimodal features corresponding to the sample business information contained in the sample business data set, and add the sample multimodal features corresponding to each sample business information in the sample business data set to the sample multimodal feature library.

[0168] It is understandable that the sample multimodal features corresponding to other sample business information in the sample business data set can be obtained in the above manner. After obtaining the sample multimodal features of all sample business information contained in the sample business data set, they can be added to the sample multimodal feature library, so that the corresponding sample multimodal features can be quickly found from the sample multimodal feature library based on the identification information of the sample business information. In other words, when the sample multimodal features are used to train the initial prediction model, the extracted sample multimodal features can be directly obtained from the sample multimodal feature library, thereby saving the training time of the initial prediction model.

[0169] Step S206: Obtain a sample information sequence including N sample business information, and obtain a sample multimodal feature corresponding to each sample business information in the sample information sequence in a sample multimodal feature library.

[0170] Step S207: Obtain sample position coding features of the N sample business information in the sample information sequence, and obtain sample information input features according to the sample information sequence, the sample position coding features, and the sample multimodal features corresponding to the N sample business information.

[0171] The specific implementation process of step S206 and step S207 can be found in the above Figure 3 The description of step S101 to step S102 in the corresponding embodiment will not be repeated here.

[0172] Step S208: Obtain the input features of the i-th attention component among the M attention components.

[0173] See also Figure 6 , Figure 6 is a schematic diagram of the structure of a feature extractor in an initial prediction model provided in an embodiment of the present application. The feature extractor may include one or more attention components, such as Figure 6 As shown, the feature extractor may include M attention components, and the M attention components have the same network structure. M may be a positive integer, such as M may be 1, 2, ... The M attention components may be connected in sequence, that is, the output feature of the previous attention component may be used as the input feature of the next attention component. At this time, when i is 1, the input feature of the i-th attention component among the M attention components is the sample information input feature; when i is not 1, the input feature of the i-th attention component is the output feature of the i-1-th attention component; i is a positive integer less than or equal to M, such as i may be 1, 2, ... M.

[0174] Step S209: Obtain the transformation parameter matrix of the jth self-attention layer in the i-th attention component, perform feature transformation processing on the input features of the i-th attention component through the transformation parameter matrix, and obtain sample transformation features.

[0175] like Figure 6 The attention component 30e shown can be the network structure of the i-th attention component among the M attention components. The attention component 30e can include a masked multi-head attention structure 30a (Mask Multi-HeadAttention), a combination + normalization layer 30b (Add&Norm), a feed-forward network layer 30c, and a combination + normalization layer 30d and other structures; wherein the masked multi-head attention structure 30a can include multiple self-attention layers, such as the number of self-attention layers included in the masked multi-head attention structure 30a is K, and the input features of the K self-attention layers can be the same, K is a positive integer, such as K can be 1,2,3...

[0176] The transformation parameter matrix corresponding to any one of the K self-attention subcomponents included in the masked multi-head attention structure 30a (for example, the j-th self-attention layer, where j is a positive integer less than or equal to T) can be obtained, and the input features of the i-th attention component are transformed into sample transformation features based on the transformation parameter matrix of the j-th self-attention layer. The sample transformation features may include a query matrix Q, a key matrix K, and a value matrix V.

[0177] In the embodiment of the present application, the transformation parameter matrix of the j-th self-attention subcomponent may include three parameter matrices: the first transformation matrix W q , the second transformation matrix W k And the third transformation matrix W v It can be understood that the transformation parameter matrix is ​​the parameters learned during the training process of the feature extractor in the initial prediction model. The input feature of the i-th attention component can be recorded as X i ∈R L×d , the input feature X of the i-th attention component i And the first parameter matrix W in the transformation parameter matrix q By performing a dot multiplication operation, we can get the query matrix Q, that is, Q = X i W q ; The input feature X of the i-th attention component i And the second parameter matrix W in the transformation parameter matrix k Perform a dot multiplication operation to obtain the key matrix K, that is, K = X i W k ; The input feature X of the i-th attention component iAnd the third parameter matrix W in the transformation parameter matrix v Perform a dot multiplication operation to obtain the value matrix V, that is, V = X i W v . Among them, each query vector in the query matrix Q can be used to encode the similarity relationship between each feature and other features, and the similarity relationship can determine the dependency information between the feature and the previous feature.

[0178] Step S210: Obtain the information mask matrix of the j-th self-attention layer, perform information masking processing on the sample transformation features through the information mask matrix, and obtain the output features of the j-th self-attention layer.

[0179] Among them, the information mask matrix is ​​used to indicate the sample business information used by the initial prediction model when determining the sample business information features corresponding to the t+1th sample business information in the sample information sequence, and t is a positive integer less than N. In other words, the information mask matrix can be used to specify the sample business information features at which positions in the sample information sequence should be ignored or masked so as to exclude them when calculating the attention weight matrix. The size of the information mask matrix is ​​the same as the size of the sample transformation feature. Assuming that the sample transformation feature is a representation vector containing N sample business information, the information mask matrix is ​​an N×N matrix.

[0180] For example, when predicting the sample business information feature corresponding to the t+1th sample business information in the sample information sequence, the representation vector corresponding to the first t sample business information in the sample transformation feature can be retained, and the representation vector corresponding to the t+1th to Nth sample business information in the sample transformation feature can be masked and filled (masked padding); for example, the representation vector corresponding to the t+1th to Nth sample business information can be replaced with 0 or an infinitesimal number to obtain the information mask matrix associated with the t+1th sample business information. In this case, after the sample transformation feature is subjected to information masking, when predicting the sample business information feature corresponding to the t+1th sample business information in the sample information sequence, only the representation vectors of the first t sample business information can be used, thereby preventing information leakage. Similarly, when predicting the sample business information features corresponding to the t+2th sample business information in the sample information sequence, the representation vector corresponding to the first t+1 sample business information in the sample transformation features is retained, and the representation vectors corresponding to the t+2th to Nth sample business information in the sample transformation features are replaced with 0 or an infinitesimal number to obtain the information mask matrix associated with the t+2th sample business information.

[0181] Specifically, the transposed matrix of the key matrix K in the sample transformation feature and the query matrix Q in the sample transformation feature can be multiplied by a dot operation, and then the product of the transposed matrix of the key matrix K and the query matrix Q is determined as the first candidate matrix (which can be expressed as QK T ); The first candidate matrix can be considered as the inner product (also called dot product, dot product) of each row vector in the query matrix Q and the key matrix K. T Perform a dot multiplication operation with the information mask matrix (which can be expressed as Mask), and then convert the first candidate matrix QK T The product of the information mask matrix Mask is determined as the second candidate matrix (which can be expressed as MaskQK T ).

[0182] In order to prevent the inner product of the matrix from being too large, the number of columns corresponding to the query matrix Q can be obtained (the query matrix Q and the key matrix K have the same number of columns, which can also be called vector dimension), and the second candidate matrix MaskQK T and the square root of the number of columns (which can be expressed as ) is normalized to obtain the attention weight matrix. The attention weight matrix can be expressed as Among them, the attention weight matrix can be considered as a kind of "dynamic weight", which can be used to characterize the information transfer between the sample information input features of the sample information sequence; the softmax function is a function used for normalization processing, and the softmax function can be used to calculate the self-attention coefficient of a single feature for other features. Each row in is softmaxed. The dot product between the attention weight matrix and the value matrix V is determined as the output feature corresponding to the jth self-attention layer. The output feature here can be expressed as

[0183] Step S211: Concatenate the output features of the K self-attention layers in the i-th attention component to obtain the sample mask concatenation features of the i-th attention component.

[0184] Since the masked multi-head attention structure 30a includes K self-attention layers, the output features of the K self-attention layers can be obtained, which are recorded as output features O 1 , output feature O 2 , ..., output feature O K , and then the output features corresponding to the K self-attention layers (i.e., output features O 1 , output feature O 2 , ..., output feature O K) are concatenated (concat) to obtain the sample mask concatenation feature of the i-th attention component; that is, the sample mask concatenation feature is the output feature of the masked multi-head attention structure 30a. It can be understood that the multi-head self-attention calculation can be performed on the sample information sequence through the masked multi-head attention structure 30a to capture the dependency between different positions in the sample information sequence, which can enhance the feature expression ability of the sample mask concatenation feature.

[0185] Step S212: Obtain a first sample joint feature based on the sample mask splicing feature and the input feature of the i-th attention component, and normalize the first sample joint feature to obtain a sample normalized feature.

[0186] like Figure 6 As shown, the input feature X of the i-th attention component can be obtained by the combination + normalization layer 30b in the attention component 30e. i The combination (Add) in the combination + normalization layer 30b can be used to convert the input feature X of the i-th attention component into i The sample mask splicing feature is combined into a first sample joint feature. The normalization in the combination + normalization layer 30b can be used to normalize the first sample joint feature to obtain a sample normalized feature. The normalization process here can refer to converting the first sample joint feature into a feature with the same mean variance. The combination + normalization layer 30b can combine and normalize the features, which can improve the training efficiency of the initial prediction model and the performance of the initial prediction model.

[0187] Step S213: Perform feature transformation processing on the sample normalized features through the feedforward network layer in the i-th attention component to obtain attention transformation features.

[0188] Step S214: Obtain a second sample joint feature based on the attention transformation feature and the sample normalization feature, normalize the second sample joint feature to obtain the output feature of the i-th attention component, and determine the output feature of the M-th attention component as the sample business recommendation feature corresponding to the sample information sequence.

[0189] like Figure 6As shown, the sample normalized features can be subjected to feature transformation processing through the feedforward network layer 30c in the attention component 30e to obtain the attention transformation feature. It can be understood that the feedforward network layer 30c can include one or more fully connected layers, and the feedforward network layer 30c can perform fully connected processing on the vector representation of the sample business information at each position in the sample information sequence, which helps to improve the feature expression ability of the attention transformation feature. Then, the attention transformation feature and the sample normalized feature can be combined into a second sample joint feature through the combination + normalization layer 30d in the attention component 30e, and the second sample joint feature is normalized, and the normalized second sample joint feature is determined as the output feature of the i-th attention component. Similarly, the output features of other attention components in the M attention components can be obtained in the same way. After obtaining the output features of the M-th attention component (the last attention component of the feature extractor), it can be determined as the sample business recommendation feature corresponding to the sample information sequence.

[0190] In an embodiment of the present application, feature extraction is performed on sample information input features by a feature extractor in an initial prediction model, thereby enhancing the quality of feature extraction so that the extracted sample business recommendation features can more comprehensively characterize the global information of the sample business information in the sample information sequence, thereby helping to improve the prediction accuracy of the information prediction model, and further improving the accuracy of information recommendation.

[0191] Step S215: Obtaining the information recommendation probability distribution corresponding to the sample information recommendation feature through the output layer in the initial prediction model.

[0192] The sample information recommendation feature corresponding to the sample information sequence includes the sample information recommendation feature associated with each sample service information in the sample information sequence. For ease of description, the embodiment of the present application takes the t+1th sample service information in the sample information sequence as an example to describe in detail the method for obtaining the recommendation probability value of the sample service information in the sample information sequence. t is a positive integer less than N.

[0193] Specifically, the sample information recommendation features associated with the t+1th sample business information in the sample information sequence can be transformed by the output layer in the initial prediction model to obtain the candidate business recommendation features corresponding to the sample information recommendation features associated with the t+1th sample business information. The output layer in the initial prediction model may include a multilayer perceptron (MLP) and a Softmax function. The sample information recommendation features associated with the t+1th sample business information can be transformed by the multilayer perceptron in the output layer to obtain the candidate business recommendation features corresponding to the sample information recommendation features associated with the t+1th sample business information. The feature change can be a linear transformation, and the sample information recommendation features can be mapped to a vector space with a dimension equal to the vocabulary size by the linear transformation. The multilayer perceptron may include an input layer, one or more hidden layers, and an output layer, and the layers are fully connected (Fully Connected), that is, the output features of the nodes in the previous layer can be all connected together in the multilayer perceptron as the input features of the next layer. Optionally, the multilayer perceptron may further include one or more activation layers, which may include one or more activation functions such as ReLU function, Sigmoid function and Tanh function, for performing nonlinear change processing on the recommended features of the sample information.

[0194] After obtaining the candidate service recommendation feature associated with the t+1th sample service information, each element in the candidate service recommendation feature can be converted into a recommendation probability value through the Softmax function contained in the output layer, thereby obtaining the recommendation probability value corresponding to the t+1th sample service information. Specifically, the Softmax function can be used to perform an exponential operation on the candidate service recommendation feature corresponding to the t+1th sample service information to obtain the recommendation index value corresponding to the t+1th sample service information; the recommendation index value corresponding to each sample service information in the sample information sequence is accumulated to obtain the total recommendation index value, and the ratio between the recommendation index value corresponding to the t+1th sample service information and the total recommendation index value is determined as the recommendation probability value corresponding to the t+1th sample service information. Among them, the recommendation probability value corresponding to the t+1th sample service information can be shown as the following formula (6):

[0195]

[0196] Among them, α in formula (6) t+1 Indicates the recommendation probability value corresponding to the t+1th sample business information; α t ' +1 represents the candidate service recommendation feature associated with the t+1th sample service information; exp() represents exponential operation; exp(α t' +1 ) represents the recommendation index value corresponding to the t+1th sample service information; Indicates the total value of the recommendation index.

[0197] Similarly, the recommendation probability values ​​of other sample service information in the sample information sequence can be obtained in the same manner as above. After obtaining the recommendation probability values ​​corresponding to the N sample service information in the sample information sequence, the information recommendation probability distribution corresponding to the sample information sequence can be generated according to the recommendation probability value corresponding to each sample service information in the sample information sequence.

[0198] Step S216: modifying the network parameters of the initial prediction model according to the information recommendation probability distribution, and determining the feature extractor in the initial prediction model including the modified network parameters as the information prediction model.

[0199] The specific implementation process of step S216 can be found in the above Figure 2 The description of step S104 in the corresponding embodiment will not be repeated here.

[0200] In an embodiment of the present application, after obtaining the sample multimodal features associated with the sample information sequence in the sample multimodal feature library, the sample information sequence, the sample position coding features associated with the sample information sequence, and the sample multimodal features can be combined into sample information input features; then, the sample information input features are subjected to feature extraction processing by the feature extractor in the initial prediction model pair to obtain the sample information recommendation features corresponding to the sample information sequence, the initial prediction model is trained by the information recommendation probability distribution corresponding to the sample information recommendation features, and the feature extractor in the trained initial prediction model is determined as the information prediction model. Among them, the sample multimodal features integrate the text representation information and image representation information of the sample business information, and the sample multimodal features are used to represent the sample business information for the training of the initial prediction model, which can better reflect the substantive content of the sample business information. The information prediction model obtained by training the sample multimodal features can improve the information representation ability of the business information recommendation features, and thus improve the accuracy of information recommendation.

[0201] In an embodiment of the present application, the trained information prediction model can be used to extract the business information recommendation features of the business recommendation information contained in the business recommendation information set, and the business information recommendation features can be applied in the information recommendation scenario to improve the accuracy of the information recommendation. Among them, the information recommendation scenario can include but is not limited to the video recommendation scenario, audio recommendation scenario, advertising recommendation scenario, information recommendation scenario, or product recommendation scenario, etc.

[0202] Specifically, see Figure 7 , Figure 7FIG. 1 is a flow chart of another method for processing business data provided by an embodiment of the present application. It is understandable that the method for processing business data can be executed by a computer device, which can be a terminal device (for example, Figure 1 Any terminal device in the terminal cluster of the corresponding embodiment), or it can be a server (for example, Figure 1 The server 10d shown in the figure, the embodiment of the present application takes the service data processing method executed by the terminal device as an example for explanation. Figure 7 As shown, the business data processing method may include the following steps S301 to S304:

[0203] Step S301: Acquire a service recommendation information set, and acquire a service information sequence corresponding to the service recommendation information x in the service recommendation information set.

[0204] The service recommendation information set may include multiple service recommendation information, and the service recommendation information set may include service recommendation information covering one or more fields. For example, the service recommendation information set may include audio data or video data, or may include commodities and advertisements, etc.

[0205] For ease of understanding, the embodiment of the present application takes the business recommendation information x in the business recommendation information set as an example to describe in detail the method for obtaining the business information recommendation feature. Among them, the business recommendation information x is any business recommendation information in the business recommendation information set; in order to ensure that the dimension of the business recommendation information x is consistent with the information prediction model, the length of the business information sequence corresponding to the business recommendation information x can be set to be the same as the length of the sample information sequence mentioned above. For example, when the length of the sample information sequence is N, the length of the business information sequence of the business recommendation information x is also N.

[0206] In a possible implementation, a method for acquiring a service information sequence corresponding to the service recommendation information x may include: acquiring an initial information sequence having the same length as the sample information sequence; wherein the elements in the initial information sequence are all mask filling values; that is, the N positions of the initial information sequence are all mask filling values, for example, the initial information sequence may be expressed as {mask 1 , mask 2 , ..., mask N}. Specifically, mask 1 Indicates the mask fill value of the first position in the initial information sequence, mask 2 Indicates the mask fill value of the second position in the initial information sequence, and so on. N Indicates the mask filling value of the Nth position in the initial information sequence. The mask filling value can be 0, or an infinitesimal number, which is not limited in the embodiment of the present application.

[0207] Furthermore, any element in the initial information sequence may be replaced with the identification information corresponding to the service recommendation information x to obtain the service information sequence corresponding to the service recommendation information x. For example, the first element in the initial information sequence may be replaced with the identification information corresponding to the service recommendation information x. In this case, the service information sequence corresponding to the service recommendation information x may be represented as {ID x , mask 2 , ..., mask N}; Alternatively, the second element in the initial information sequence may be replaced with the identification information corresponding to the service recommendation information x. In this case, the service information sequence corresponding to the service recommendation information x may be represented as {mask 1 , ID x , ..., mask N}. Among them, ID x Indicates the identification information corresponding to the business recommendation information x.

[0208] Step S302: Obtain the business multimodal features corresponding to the business recommendation information x in the business multimodal feature library.

[0209] Step S303: Obtain the business position coding feature of the business recommendation information x in the business information sequence, and obtain the business information input feature corresponding to the business recommendation information x according to the business information sequence corresponding to the business recommendation information x, the business position coding feature corresponding to the business recommendation information x, and the business multimodal feature corresponding to the business recommendation information x.

[0210] Step S304: input the business information input features corresponding to the business recommendation information x into the information prediction model, perform feature extraction processing on the business information input features corresponding to the business recommendation information x through the information prediction model, and obtain the business information recommendation features corresponding to the business recommendation information x.

[0211] Among them, the business multimodal feature library includes multimodal features corresponding to the business recommendation information in the business recommendation information set, and the business multimodal features can be determined according to the text representation information and image representation information corresponding to the business recommendation information; the business information recommendation feature corresponding to the business recommendation information x is used to determine the object recommendation information corresponding to the business object. The method for obtaining the business multimodal feature can refer to the method for obtaining the sample multimodal feature above. Similarly, the method for obtaining the business multimodal feature library can refer to the method for obtaining the sample multimodal feature library above, which will not be repeated here. The business recommendation information x can be position-encoded according to the position of the identification information corresponding to the business recommendation information x in the business information sequence to obtain the business position coding feature corresponding to the business recommendation information x. The specific position coding method can refer to the description above, which will not be repeated here.

[0212] Specifically, see Figure 8 , Figure 8 FIG. 1 is a schematic diagram of obtaining a service information recommendation feature provided by an embodiment of the present application. Figure 8 As shown, according to the identification information corresponding to the business recommendation information x, the business multimodal feature 50c corresponding to the business recommendation information x can be searched from the business multimodal feature library 50a; the business position coding feature 50b of the business recommendation information x in the business information sequence is obtained, and then the business multimodal feature 50c, the business position coding feature 50b and the business information sequence 50b can be combined to obtain the business information input feature 50e corresponding to the business recommendation information x. For specific combination methods, see Figure 2 The relevant description in step S102 of the corresponding embodiment will not be repeated here.

[0213] like Figure 8 As shown, after obtaining the business information input feature 50e, the business information input feature 50e can be input into the information prediction model 50f, and the business information input feature corresponding to the business recommendation information x is subjected to feature extraction processing by the information prediction model 50f to obtain the business information recommendation feature corresponding to the business recommendation information x. For specific feature extraction methods, see Figure 4 The corresponding descriptions of steps S208 to S214 are not repeated here.

[0214] Similarly, the above method can be used to obtain the business information recommendation features corresponding to other business recommendation information in the business recommendation information set, and then the Approximate Nearest Neighbor (ANN) algorithm can be used to establish the information relationship structure corresponding to the business recommendation information set through the business information recommendation features, and apply it to the item-to-item (I2I) recommendation scenario with a lighter volume, thereby improving the efficiency and accuracy of information recommendation. The information relationship structure may include but is not limited to a clustering graph, a kd-tree (a tree structure), a hash table, and the like.

[0215] For ease of description, the business information recommendation features associated with the business recommendation information set are referred to as a business information recommendation feature set. In a possible implementation, the information relationship structure may be generated by: using a clustering method to divide the business information recommendation feature set to obtain L feature subsets, and determining the L feature subsets as the information relationship structure associated with the business recommendation information set; wherein L is a positive integer, which is used to represent the number of feature subsets, that is, the number of divisions of the business information recommendation features associated with the business recommendation information set.

[0216] Specifically, the central feature t can be determined from the business information recommendation feature setu , obtain each business information recommendation feature and the central feature t in the business information recommendation feature set u The similarity between them; u is a positive integer less than or equal to L; if the vth business information recommendation feature in the business information recommendation feature set is similar to the central feature t u The similarity is the largest, then the vth business information recommendation is added to the central feature t u The pending cluster C u ; v is a positive integer less than or equal to the number of recommended business information features contained in the recommended business information feature set; according to the cluster to be processed C u The business information recommendation features contained in the update center feature t u , until the cluster to be processed C u The updated central feature t u When the update stop condition is met, the pending cluster C u Determined as a feature subset. The update stop condition can be the central feature t u The update amplitude is less than or equal to the preset error value, or the central feature t u The number of updates reaches the preset maximum number of updates.

[0217] In the application embodiment, the number of clusters (also referred to as the number of clusters) of the business information recommendation features included in the business information recommendation feature set can be preset. When the number of clusters is L, L business information recommendation features can be randomly selected from the business information recommendation feature set as the initial central features, that is, the central feature t u ; The number of clusters set in the clustering algorithm can affect the final clustering effect of the business information recommendation features. If the number of clusters is too large, similar business information recommendation features may be divided into different clusters. If the number of clusters is too small, dissimilar business information recommendation features may be divided into the same cluster. In the embodiment of the present application, the number of clusters can be set based on experience, such as L=500. Of course, the number of clusters can also be determined by cross-validation and other methods.

[0218] The distance between each business information recommendation feature and each central feature in the business information recommendation feature set can be calculated (the distance between two business information recommendation features can be measured by the similarity between the two. The greater the similarity, the shorter the distance, and the smaller the similarity, the greater the distance). That is, the distance between each business information recommendation feature and each central feature in the business information recommendation feature set, and the business information recommendation feature (such as the vth business information recommendation feature) is divided into the central feature t closest to the business information recommendation feature. u The pending cluster C u , we can preliminarily get L clusters to be processed. Among them, each central feature t uEach of them corresponds to a cluster to be processed, and each cluster to be processed may include multiple business information recommendation features. The number of business information recommendation features in each cluster to be processed may be the same or different. Among them, the method used to calculate the distance between two business information recommendation features may include but is not limited to: Euclidian distance, Manhattan distance, Minkowski distance, and cosine similarity.

[0219] For the L clusters to be processed obtained above, the mean of all the business information recommendation features contained in each cluster to be processed can be calculated, and the central feature corresponding to the cluster to be processed can be updated according to the mean to obtain a new central feature, and then the business information recommendation features contained in each cluster to be processed can be updated according to the distance between each business information recommendation feature in the business information recommendation feature set and each new central feature. The above process is repeated continuously. When the business information recommendation features contained in each cluster to be processed meet the update stop condition, the L clusters to be processed at this time can be determined as the final clustering result of the business information recommendation feature set, that is, L feature subsets, and the L feature subsets are determined as the information relationship structure associated with the business recommendation information set.

[0220] In a possible implementation, the method for generating the information relationship structure may also include: the business recommendation information in the business recommendation information set may be paired to obtain multiple recommendation information pairs. Assume that the business recommendation information set includes business recommendation information 1, business recommendation information 2, business recommendation information 3, and business recommendation information 4, wherein business recommendation information 1 and business recommendation information 2 belong to the same category of recommendation information, for example, business recommendation information 1 and business recommendation information 2 are both documentaries; business recommendation information 3 and business recommendation information 4 belong to the same category of recommendation information, for example, business recommendation information 3 and business recommendation information 4 are both costume dramas. In a possible implementation, the business recommendation information in the business recommendation information set may be combined in pairs to obtain 6 recommendation information pairs; or two business recommendation information of the same category in the business recommendation information combination may be combined to obtain a recommendation information pair including business recommendation information 1 and business recommendation information 2, and a recommendation information pair including business recommendation information 3 and business recommendation information 4.

[0221] Further, the similarity evaluation value between the two business recommendation information contained in each recommendation information pair can be determined according to the business information recommendation features corresponding to the two business information contained in each information recommendation pair. In the embodiment of the present application, the similarity evaluation value can be understood as the similarity between the two business recommendation information, and the similarity can be expressed by the distance between the two business information recommendation features associated with the recommendation information pair. The larger the distance between the two, the smaller the corresponding similarity; conversely, the smaller the distance between the two, the greater the corresponding similarity. The calculation method used for the distance between the two business information recommendation features can refer to the above description, which will not be repeated here. Optionally, the similarity evaluation value can be numerically displayed in the form of a numerical value. For example, the similarity evaluation value can be a numerical value between 0 and 1, or it can also be a numerical value between 0 and 100. The larger the numerical value, the larger the similarity evaluation value, and the greater the similarity between the two business information recommendation features. Then, according to the business recommendation information set and the similarity evaluation value associated with each recommendation information pair, the information relationship structure corresponding to the business recommendation information set is generated. That is to say, the information relationship structure may include part or all of the business recommendation information in the business recommendation information set, as well as the similar evaluation values ​​associated with the business recommendation information, and the information relationship structure may be used to quickly recommend information. Optionally, field prompt labels may be added to each business recommendation information in the information relationship structure according to the field to which the business recommendation information set belongs, thereby further improving the accuracy of information recommendation. For example, if the field to which the business recommendation information x belongs is the short video field, a "short video" field prompt label may be added to the business recommendation information x in the information relationship structure; if the field to which the business recommendation information y belongs is the documentary field, a "documentary" field prompt label may be added to the business recommendation information y in the information relationship structure.

[0222] Furthermore, the information relationship structure can be used to quickly search for similar business recommendation information, thereby improving the efficiency of information recommendation. Specifically, a collection of historical browsing information of a business object on an information recommendation platform can be obtained; the information recommendation platform can be an application client with an information recommendation function, specifically an audio and video application, an instant messaging application, a shopping application, etc.; the information recommendation platform can store a pre-built information relationship structure. The historical browsing information contained in the historical browsing information collection is the business recommendation information that the business object has browsed on the information recommendation platform.

[0223] If the information relationship structure is obtained based on a clustering algorithm, the similarity between the historical browsing information contained in the historical browsing information set and the central feature corresponding to each feature subset in the information relationship structure can be calculated, and all business recommendation information associated with the feature subset corresponding to the central feature with the greatest similarity to the historical browsing information can be determined as the object recommendation information corresponding to the business object, and the object recommendation information can be pushed to the business object on the information recommendation platform.

[0224] Assuming that the information relationship structure is obtained based on the similarity evaluation value, when the similarity evaluation value between the business recommendation information x in the information relationship structure and the historical browsing information in the historical browsing information set is greater than the similarity threshold, the business recommendation information x can be added to the candidate recommendation information set; then, the frequency corresponding to each business recommendation information in the candidate recommendation information set can be counted, and based on the frequency corresponding to each business recommendation information in the candidate recommendation information set, the object recommendation information corresponding to the business object in the candidate recommendation information set can be determined.

[0225] Among them, the similarity threshold refers to a boundary or limit value set for the similarity evaluation value. Assuming that the similarity evaluation value between business recommendation information 1 and business recommendation information 2 is greater than the similarity threshold, in this case, if the business object is interested in business recommendation information 1, it is very likely to be interested in business recommendation information 2. Therefore, when the similarity evaluation value between business recommendation information x and the historical browsing information in the historical browsing information set is greater than the similarity threshold, business recommendation information x can be added to the candidate recommendation information set. Optionally, the first few business recommendation information with the highest frequency of occurrence in the candidate recommendation information set can be used as object recommendation information of the business object, and it can be pushed to the business object in the information recommendation platform; for example, the business recommendation information with the highest frequency of occurrence in the candidate recommendation information set can be pushed to the business object. Optionally, when the information relationship structure contains a domain prompt tag, the business recommendation information in the candidate recommendation information set that has the same or similar domain prompt tag as the historical browsing information can also be determined as object recommendation information, and it can be pushed to the business object.

[0226] It is understandable that the information relationship structure generated based on the business information recommendation feature can be applied to any information recommendation scenario. For example, a video recommendation scenario, an audio recommendation scenario, an advertisement recommendation scenario, an information recommendation scenario, or a product recommendation scenario, etc. For ease of understanding, the present application embodiment takes the video recommendation scenario as an example to describe the application process of the information relationship structure.

[0227] Specifically, see Fig. 9 , Fig. 9 is a schematic diagram of a video recommendation scenario provided by an embodiment of the present application. Fig. 9As shown, when the business object B performs a trigger operation on the video application A in the terminal device 10a, the terminal device 10a can respond to the trigger operation on the video application A, start the video application A in the terminal device 10a, and enter the homepage of the video application A. The terminal device 10a can obtain the historical browsing videos of the business object B in the video application A. For example, the business object B watched the video 60a and the video 60b in the video application A, indicating that the business object B may be interested in videos with a high similarity to the video 60a and the video 60b.

[0228] like Fig. 9 As shown, the information relationship structure 61a can be used to quickly search for videos that may be of interest to business object B. The information relationship structure 61a may include all videos launched by video application A, as well as similarity evaluation values ​​associated with each video and other videos. For ease of understanding, the present embodiment of the application is described by setting the similarity threshold to 60. Fig. 9 As shown, videos 60c and 60e whose similarity evaluation values ​​with video 60a are greater than a similarity threshold can be screened out from the information relationship structure 61a, and videos 60c and 60e can be added to the candidate recommended video set; similarly, videos 60d and 60e whose similarity evaluation values ​​with video 60b are greater than a similarity threshold can be screened out from the information relationship structure 61a, and videos 60c and 60e can be added to the candidate recommended video set. Then, the frequency corresponding to each video in the candidate recommended video set is counted, and the video with the highest frequency is used as the object recommended video (i.e., video 60e), which is pushed to business object B in video application A, thereby improving the efficiency and accuracy of video recommendation. Optionally, when business object B is not interested in video 60e, a trigger operation can be performed on the "change a batch" control in video application A, and terminal device 10a can respond to the trigger operation on the "change a batch" control and re-recommend videos for business object B, thereby further improving the accuracy of video recommendation.

[0229] It is understandable that, since the embodiment of the present application does not directly model the identification information of the sample business information, but trains the model by extracting the sample multimodal features of the sample business information, the information prediction model trained by the embodiment of the present application has a strong generalization ability, and will not cause the prediction accuracy of the information prediction model to decrease due to the change of the status of the business recommendation information (for example, online or off the shelf), thereby requiring the model to be trained multiple times. In addition, the cold start problem caused by the lack of interactive data on the new business recommendation information can also be effectively avoided. Therefore, the embodiment of the present application is not only applicable to conventional recommendation tasks, but also to cold start tasks. Among them, conventional recommendation tasks refer to recommending business recommendation information that has been online on the information recommendation platform for a period of time; cold start tasks refer to recommending business recommendation information newly launched on the information recommendation platform.

[0230] For ease of description, the embodiment of the present application takes the business recommendation information b as the newly launched business recommendation information on the information recommendation platform as an example, and describes in detail the processing process of the item cold start task. In the embodiment of the present application, if the newly added business recommendation information b in the information recommendation platform is detected, the business multimodal features corresponding to the business recommendation information b can be obtained according to the text representation information and image representation information corresponding to the business recommendation information b, and the business multimodal features corresponding to the business recommendation information b can be added to the business multimodal feature library; the specific acquisition process of the business multimodal features corresponding to the business recommendation information b can refer to the description in the previous text. Then, the business information sequence corresponding to the business recommendation information b, the business position coding features corresponding to the business recommendation information b, and the business multimodal features corresponding to the business recommendation information b can be combined into the business information input features corresponding to the business recommendation information b. The specific combination method can refer to the description in the previous text.

[0231] After obtaining the business information input features corresponding to the business recommendation information b, the business information input features of the business recommendation information b can be subjected to feature extraction processing through the information prediction model to obtain the initial information recommendation features corresponding to the business recommendation information b. The specific implementation process can refer to the acquisition process of the business information recommendation features corresponding to the business recommendation information a mentioned above. Further, the initial information recommendation features corresponding to the business recommendation information b can be subjected to pooling processing (which can be average pooling processing or maximum pooling processing) to obtain the business information recommendation features corresponding to the business recommendation information b; the business information recommendation features corresponding to the business recommendation information b obtained at this time can well represent the information of the business recommendation information b.

[0232] Furthermore, the business information recommendation features corresponding to the business recommendation information b and the similarity evaluation values ​​between the business information recommendation features corresponding to each business recommendation information in the business recommendation information set can be obtained; based on the business recommendation information b and the similarity evaluation values ​​between the business recommendation information b and each business recommendation information in the business recommendation information set, the information relationship structure is updated, and a recommendation relationship is established between the business recommendation information b and the business recommendation information that has been launched on the information recommendation platform, so that the recommendation task of new business recommendation information can be carried out.

[0233] Optionally, the embodiments of the present application are also applicable to the user to item (U2I) recommendation scenario. Specifically, the trained initial prediction model can be determined as the information recommendation model, wherein the information recommendation model has the same network structure as the initial prediction model. A historical browsing information set of the business object on the information recommendation platform can be obtained, and a historical browsing information sequence can be generated based on the historical browsing information in the historical browsing information set. The length of the historical browsing information sequence is consistent with the length of the sample information sequence, that is, the length of the historical browsing information sequence is also N; when the number of historical browsing information in the historical browsing information set is less than N, mask filling is performed at the corresponding position of the historical browsing information sequence.

[0234] Furthermore, the business information sequence corresponding to the historical browsing information sequence, the business position coding features corresponding to the historical browsing information sequence, and the business multimodal features corresponding to the historical browsing information sequence can be combined into the business information input features corresponding to the historical browsing information sequence. The specific combination method can refer to the description in the previous text. The business information input features corresponding to the historical browsing information sequence can be subjected to feature extraction processing by the feature extractor in the information recommendation model to obtain the business information recommendation features corresponding to the historical browsing information sequence, and the information recommendation probability distribution corresponding to the business information recommendation features corresponding to the historical browsing information sequence can be obtained by the output layer in the information recommendation model, and the business information recommendation features corresponding to the largest recommendation probability value in the information recommendation probability distribution are determined as the object recommendation features; the similarity between the object recommendation features and the business information recommendation features corresponding to the business recommendation information in the business recommendation information set is calculated, and the business recommendation information associated with the business information recommendation features having the greatest similarity to the object recommendation features is determined as the object recommendation information, and it is pushed to the business object on the information recommendation platform, thereby improving the accuracy of information recommendation.

[0235] In an embodiment of the present application, the sample multimodal features integrate the text representation information and image representation information of the sample business information. The sample multimodal features are used to represent the sample business information for training the initial prediction model, which can better reflect the essential content of the sample business information. The information prediction model obtained by training the sample multimodal features can improve the information representation ability of the business information recommendation features, thereby improving the accuracy of information recommendation.

[0236] It is understandable that in the specific implementation of the present application, information such as the user's historical browsing history in the application client may be involved. When the above embodiments of the present application are applied to specific products or technologies, the user's permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0237] See also Fig.10 , Fig.10 Schematic diagram of a business data processing device provided in an embodiment of the present application. Fig.10 As shown, the business data processing device 1 may include: a first feature acquisition module 11, a first feature combination module 12, a probability distribution acquisition module 13 and a prediction model training module 14, wherein:

[0238] The first feature acquisition module 11 is used to acquire a sample information sequence including N sample business information, and acquire a sample multimodal feature corresponding to each sample business information in the sample information sequence in a sample multimodal feature library; the sample multimodal feature is determined according to the text representation information and the image representation information corresponding to the sample business information; N is an integer greater than 1;

[0239] The first feature combination module 12 is used to obtain the sample position coding features of the N sample business information in the sample information sequence, and obtain the sample information input features according to the sample information sequence, the sample position coding features, and the sample multimodal features corresponding to the N sample business information;

[0240] The probability distribution acquisition module 13 is used to input the sample information input features into the initial prediction model, perform feature extraction processing on the sample information input features through the feature extractor in the initial prediction model, obtain the sample information recommendation features corresponding to the sample information sequence, and obtain the information recommendation probability distribution corresponding to the sample information recommendation features through the output layer in the initial prediction model;

[0241] The prediction model training module 14 is used to correct the network parameters of the initial prediction model according to the information recommendation probability distribution, and determine the feature extractor in the initial prediction model containing the corrected network parameters as the information prediction model; the information prediction model is used to output the business information recommendation features corresponding to the business recommendation information contained in the business recommendation information set, and the business information recommendation features are used to determine the object recommendation information corresponding to the business object.

[0242] The specific implementation functions of the first feature acquisition module 11, the first feature combination module 12, the probability distribution acquisition module 13 and the prediction model training module 14 can be found in the aforementioned Figure 2 The description of steps S101 to S104 in the corresponding embodiment will not be repeated here.

[0243] In an embodiment of the present application, the sample multimodal features integrate the text representation information and image representation information of the sample business information. The sample multimodal features are used to represent the sample business information for training the initial prediction model, which can better reflect the essential content of the sample business information. The information prediction model obtained by training the sample multimodal features can improve the information representation ability of the business information recommendation features, thereby improving the accuracy of information recommendation.

[0244] See also Fig.11 , Fig.11 Schematic diagram of another business data processing device provided in an embodiment of the present application. Fig.11 As shown, the business data processing device 2 may include: a first feature acquisition module 21, a first feature combination module 22, a probability distribution acquisition module 23 and a prediction model training module 24, wherein:

[0245] The first feature acquisition module 21 is used to acquire a sample information sequence including N sample business information, and acquire a sample multimodal feature corresponding to each sample business information in the sample information sequence in the sample multimodal feature library; the sample multimodal feature is determined according to the text representation information and the image representation information corresponding to the sample business information; N is an integer greater than 1;

[0246] The first feature combination module 22 is used to obtain the sample position coding features of the N sample business information in the sample information sequence, and obtain the sample information input features according to the sample information sequence, the sample position coding features, and the sample multimodal features corresponding to the N sample business information;

[0247] The probability distribution acquisition module 23 is used to input the sample information input features into the initial prediction model, perform feature extraction processing on the sample information input features through the feature extractor in the initial prediction model, obtain the sample information recommendation features corresponding to the sample information sequence, and obtain the information recommendation probability distribution corresponding to the sample information recommendation features through the output layer in the initial prediction model;

[0248] The prediction model training module 24 is used to correct the network parameters of the initial prediction model according to the information recommendation probability distribution, and determine the feature extractor in the initial prediction model containing the corrected network parameters as the information prediction model; the information prediction model is used to output the business information recommendation features corresponding to the business recommendation information contained in the business recommendation information set, and the business information recommendation features are used to determine the object recommendation information corresponding to the business object.

[0249] In a possible implementation, the business data processing device 2 may further include: a sample data acquisition module 25, a text encoding module 26, an image encoding module 27, a feature fusion module 28 and a feature adding module 29, wherein:

[0250] The sample data acquisition module 25 is used to acquire a sample business data set, and acquire sample text data and sample image data corresponding to the sample business information contained in the sample business data set;

[0251] A text encoding module 26, configured to perform text encoding processing on the sample text data corresponding to the sample business information a in the sample business data set to obtain text representation information corresponding to the sample business information a;

[0252] An image encoding module 27, configured to perform image encoding processing on sample image data corresponding to sample service information a in the sample service data set to obtain image representation information corresponding to the sample service information a;

[0253] A feature fusion module 28 is used to perform feature fusion on the text representation information corresponding to the sample business information a and the image representation information corresponding to the sample business information a to obtain a sample multimodal feature corresponding to the sample business information a;

[0254] The feature adding module 29 is used to obtain sample multimodal features corresponding to the sample business information contained in the sample business data set, and add the sample multimodal features corresponding to each sample business information in the sample business data set to the sample multimodal feature library.

[0255] In a possible implementation, the text encoding module 26 may include: a character segmentation unit 261, a text vector acquisition unit 262, a position vector acquisition unit 263, a vector superposition unit 264 and a text feature processing unit 265, wherein:

[0256] The character division unit 261 is used to divide the sample text data corresponding to the sample business information a in the sample business data set into D unit characters, and obtain the unit word vectors corresponding to the D unit characters respectively; D is a positive integer;

[0257] The text vector acquisition unit 262 is used to acquire text vectors corresponding to the D unit characters respectively according to the semantic information of the D unit characters in the sample text data corresponding to the sample business information a;

[0258] The position vector acquisition unit 263 is used to acquire the position vectors corresponding to the D unit characters respectively according to the text positions of the D unit characters in the sample text data corresponding to the sample business information a;

[0259] A vector superposition unit 264, used to obtain text input features corresponding to the sample business information a according to the unit word vector, the text vector and the position vector;

[0260] The text feature processing unit 265 is used to input the text input features into the text encoder, perform text encoding processing on the text input features through the text encoder to obtain the text encoding features corresponding to the sample business information a, perform full connection processing on the text encoding features to obtain the text representation information corresponding to the sample business information a.

[0261] In a possible implementation, the image encoding module 27 may include: a first convolution processing unit 271, a first input feature acquisition unit 272, a second convolution processing unit 273 and a full connection processing unit 274, wherein:

[0262] The first convolution processing unit 271 is used to input the sample image data corresponding to the sample business information a in the sample business data set into the image encoder, perform convolution processing on the sample image data corresponding to the sample business information a through the convolution layer in the image encoder to obtain a first sample convolution feature, and perform maximum pooling processing on the first sample convolution feature to obtain a first sample pooling feature;

[0263] The first input feature acquisition unit 272 is used to acquire the input feature of the yth residual block structure in the image encoder; when y is 1, the input feature of the yth residual block structure is the first sample pooling feature; when y is not 1, the input feature of the yth residual block structure is the output feature of the y-1th residual block structure; y is a positive integer less than or equal to P, and P represents the number of residual block structures included in the image encoder;

[0264] A second convolution processing unit 273 is used to perform convolution processing on the input features of the yth residual block structure to obtain a second sample convolution feature of the yth residual block structure, and obtain an output feature of the yth residual block structure according to the second sample convolution feature of the yth residual block structure and the input features of the yth residual block structure;

[0265] The fully connected processing unit 274 is used to perform average pooling processing on the output features of the P-th residual block structure to obtain second sample pooling features, and perform full connection processing on the second sample pooling features to obtain image representation information corresponding to the sample business information a.

[0266] In a possible implementation, the first feature combination module 22 may include: a sine position encoding unit 221, a cosine position encoding unit 222, and a position encoding feature acquisition unit 223, wherein:

[0267] A sinusoidal position coding unit 221 is used to divide the positions of the N sample service information in the sample information sequence into even positions and odd positions, and perform sinusoidal position coding on the even positions in the N sample service information to obtain sinusoidal coding information corresponding to the even positions;

[0268] A cosine position coding unit 222, configured to perform cosine position coding on odd positions in the N sample service information to obtain cosine coding information corresponding to the odd positions;

[0269] The position coding feature acquisition unit 223 is used to determine the sine coding information and the cosine coding information as sample position coding features of the N sample service information in the sample information sequence.

[0270] In a possible implementation, the first feature combination module 22 may include: a vector conversion unit 224 and a second input feature acquisition unit 225, wherein:

[0271] A vector conversion unit 224 is used to perform vector conversion on each sample service information in the sample information sequence to obtain a sample sequence vector corresponding to the sample information sequence;

[0272] The second input feature acquisition unit 225 is used to determine the product of the sample sequence vector and the sample multimodal feature and the sum of the sample position encoding feature as the sample information input feature.

[0273] In a possible implementation, the feature extractor includes M attention components, one attention component includes K self-attention layers, and M and K are both positive integers; the probability distribution acquisition module 23 may include:

[0274] The third input feature acquisition unit 231 is used to acquire the input feature of the i-th attention component among the M attention components; when i is 1, the input feature of the i-th attention component is the sample information input feature; when i is not 1, the input feature of the i-th attention component is the output feature of the i-1-th attention component; i is a positive integer less than or equal to M;

[0275] The first feature transformation unit 232 is used to obtain the transformation parameter matrix of the j-th self-attention layer in the i-th attention component, and perform feature transformation processing on the input features of the i-th attention component through the transformation parameter matrix to obtain sample transformation features; j is a positive integer less than or equal to K;

[0276] The information mask processing unit 233 is used to obtain the information mask matrix of the j-th self-attention layer, and perform information mask processing on the sample transformation features through the information mask matrix to obtain the output features of the j-th self-attention layer; the information mask matrix is ​​used to indicate the sample business information used by the initial prediction model when determining the sample business information features corresponding to the t+1-th sample business information in the sample information sequence, where t is a positive integer less than N;

[0277] A feature concatenation unit 234 is used to concatenate the output features of the K self-attention layers in the i-th attention component to obtain a sample mask concatenation feature of the i-th attention component;

[0278] A first normalization unit 235 is used to obtain a first sample joint feature according to the sample mask splicing feature and the input feature of the i-th attention component, and normalize the first sample joint feature to obtain a sample normalized feature;

[0279] A second feature transformation unit 236 is used to perform feature transformation processing on the sample normalized features through the feedforward network layer in the i-th attention component to obtain an attention transformation feature;

[0280] The second normalization unit 237 is used to obtain a second sample joint feature based on the attention transformation feature and the sample normalization feature, normalize the second sample joint feature to obtain the output feature of the i-th attention component, and determine the output feature of the M-th attention component as the sample business recommendation feature corresponding to the sample information sequence.

[0281] In a possible implementation, the sample transformation feature includes a query matrix Q, a key matrix K, and a value matrix V; the information mask processing unit 233 may include: a first point multiplier unit 2331 and a second point multiplier unit 2332, wherein:

[0282] A first point multiplier unit 2331 is used to determine the product of the transposed matrix of the key matrix K and the query matrix Q as the first candidate matrix, and determine the product of the first candidate matrix and the information mask matrix as the second candidate matrix;

[0283] The second point multiplier unit 2332 is used to obtain the number of columns corresponding to the query matrix Q, normalize the ratio between the second candidate matrix and the square root of the number of columns, obtain the attention weight matrix, and determine the product of the attention weight matrix and the value matrix V as the output feature of the j-th self-attention layer.

[0284] The sample information recommendation feature corresponding to the sample information sequence includes the sample information recommendation feature associated with each sample business information in the sample information sequence; the probability distribution acquisition module 23 includes: a third feature conversion unit 238, an exponential operation unit 239, a first recommendation probability value acquisition unit 2310 and a probability distribution acquisition unit 2311, wherein:

[0285] The third feature conversion unit 238 is used to perform feature conversion processing on the sample information recommendation feature associated with the t+1th sample service information in the sample information sequence through the output layer in the initial prediction model to obtain a candidate service recommendation feature corresponding to the sample information recommendation feature associated with the t+1th sample service information; t is a positive integer less than N;

[0286] An index operation unit 239 is used to perform an index operation on the candidate service recommendation feature corresponding to the t+1th sample service information to obtain a recommendation index value corresponding to the t+1th sample service information;

[0287] The first recommendation probability value acquisition unit 2310 is used to accumulate the recommendation index values ​​corresponding to each sample service information in the sample information sequence to obtain a total recommendation index value, and determine the ratio between the recommendation index value corresponding to the t+1th sample service information and the total recommendation index value as the recommendation probability value corresponding to the t+1th sample service information;

[0288] The probability distribution acquiring unit 2311 determines the information recommendation probability distribution according to the recommendation probability value corresponding to each sample business information in the sample information sequence.

[0289] In a possible implementation, the prediction model training module 24 may include: a second recommendation probability value acquisition unit 241, a sample prediction loss acquisition unit 242, and a model iteration training unit 243, wherein:

[0290] The second recommendation probability value acquisition unit 241 is used to acquire the recommendation probability values ​​of the first t sample business information in the sample information sequence in the information recommendation probability distribution; one sample business information corresponds to one recommendation probability value, and t is a positive integer less than N;

[0291] The sample prediction loss acquisition unit 242 is used to perform a logarithmic operation on the recommended probability value of each sample information in the first t sample business information to obtain a logarithmic probability value corresponding to each sample information in the first t sample business information, and accumulate the logarithmic probability value corresponding to each sample business information in the first t sample business information to obtain a sample prediction loss corresponding to the t+1th sample business information;

[0292] The model iteration training unit 243 is used to iteratively train the network parameters of the initial prediction model according to the sample prediction loss corresponding to the t+1th sample business information until the updated network parameters meet the training end conditions, stop the training, and determine the feature extractor in the initial prediction model at the end of the training as the information prediction model.

[0293] According to an embodiment of the present application, the steps involved in the business data processing method shown above can be Fig.11 The business data processing device 2 shown in the figure is executed by each module and each unit. For example, Figure 2 Step S101 shown can be performed by Fig.11 The first feature acquisition module 21 shown is used to execute, Figure 2 Step S102 shown can be performed by Fig.11 The first feature combination module 22 shown is executed, Figure 2 Step S103 shown can be performed by Fig.11 The probability distribution acquisition module 23 shown is used to execute, Figure 2 Step S104 shown can be performed by Fig.11 The predictive model training module 24 shown is used to perform the following.

[0294] According to one embodiment of the present application, Fig.11The various modules (units) in the business data processing device 2 shown can be respectively or all combined into one or several modules (units) to constitute, or some (some) modules (units) thereof can also be split into at least two smaller sub-units in function, and the same operation can be achieved without affecting the realization of the technical effects of the embodiments of the present application. The above modules (units) are divided based on logical functions. In practical applications, the function of a module (unit) can also be realized by at least two modules (units), or the function of at least two modules (units) can be realized by one module (unit). In other embodiments of the present application, the business data processing device 2 can also include other modules (units). In practical applications, these functions can also be implemented with the assistance of other modules (units), and can be implemented by the collaboration of at least two modules (units).

[0295] In an embodiment of the present application, the sample multimodal features integrate the text representation information and image representation information of the sample business information. The sample multimodal features are used to represent the sample business information for training the initial prediction model, which can better reflect the essential content of the sample business information. The information prediction model obtained by training the sample multimodal features can improve the information representation ability of the business information recommendation features, thereby improving the accuracy of information recommendation.

[0296] See also Fig.12 , Fig.12 Schematic diagram of the structure of another business data processing device provided in the embodiment of the present application. Fig.12 As shown, the service data processing device 3 may include: a service information sequence acquisition module 31, a second feature acquisition module 32, a second feature combination module 33 and a third feature acquisition module 34, wherein:

[0297] The service information sequence acquisition module 31 is used to acquire a service recommendation information set, and acquire a service information sequence corresponding to the service recommendation information x in the service recommendation information set; the length of the service information sequence corresponding to the service recommendation information x is the same as the length of the sample information sequence;

[0298] The second feature acquisition module 32 acquires the business multimodal feature corresponding to the business recommendation information x from the business multimodal feature library; the business multimodal feature is determined according to the text representation information and the image representation information corresponding to the business recommendation information;

[0299] The second feature combination module 33 is used to obtain the service position coding feature of the service recommendation information x in the service information sequence, and obtain the service information input feature corresponding to the service recommendation information x according to the service information sequence corresponding to the service recommendation information x, the service position coding feature corresponding to the service recommendation information x, and the service multimodal feature corresponding to the service recommendation information x;

[0300] The third feature acquisition module 34 is used to input the business information input features corresponding to the business recommendation information x into the information prediction model, and perform feature extraction processing on the business information input features corresponding to the business recommendation information x through the information prediction model to obtain the business information recommendation features corresponding to the business recommendation information x; the business information recommendation features corresponding to the business recommendation information x are used to determine the object recommendation information corresponding to the business object.

[0301] The specific implementation functions of the business information sequence acquisition module 31, the second feature acquisition module 32, the second feature combination module 33 and the third feature acquisition module 34 can be referred to in the aforementioned Figure 7 The description of steps S301 to S304 in the corresponding embodiment will not be repeated here.

[0302] In an embodiment of the present application, the sample multimodal features integrate the text representation information and image representation information of the sample business information. The sample multimodal features are used to represent the sample business information for training the initial prediction model, which can better reflect the essential content of the sample business information. The information prediction model obtained by training the sample multimodal features can improve the information representation ability of the business information recommendation features, thereby improving the accuracy of information recommendation.

[0303] See also Fig.13 , Fig.13 Schematic diagram of the structure of another service data processing device provided in the embodiment of the present application. Fig.13 As shown, the service data processing device 4 may include: a service information sequence acquisition module 41, a second feature acquisition module 42, a second feature combination module 43 and a third feature acquisition module 44, wherein:

[0304] The service information sequence acquisition module 41 is used to acquire a service recommendation information set, and acquire a service information sequence corresponding to the service recommendation information x in the service recommendation information set; the length of the service information sequence corresponding to the service recommendation information x is the same as the length of the sample information sequence;

[0305] The second feature acquisition module 42 acquires the business multimodal feature corresponding to the business recommendation information x from the business multimodal feature library; the business multimodal feature is determined according to the text representation information and the image representation information corresponding to the business recommendation information;

[0306] The second feature combination module 43 is used to obtain the service position coding feature of the service recommendation information x in the service information sequence, and obtain the service information input feature corresponding to the service recommendation information x according to the service information sequence corresponding to the service recommendation information x, the service position coding feature corresponding to the service recommendation information x, and the service multimodal feature corresponding to the service recommendation information x;

[0307] The third feature acquisition module 44 is used to input the business information input features corresponding to the business recommendation information x into the information prediction model, and perform feature extraction processing on the business information input features corresponding to the business recommendation information x through the information prediction model to obtain the business information recommendation features corresponding to the business recommendation information x; the business information recommendation features corresponding to the business recommendation information x are used to determine the object recommendation information corresponding to the business object.

[0308] In a possible implementation, the service information sequence acquisition module 41 may include: an initial information sequence acquisition unit 411 and an identification information replacement unit 412, wherein:

[0309] The initial information sequence acquisition unit 411 is used to acquire an initial information sequence with the same length as the sample information sequence; the elements in the initial information sequence are all mask filling values;

[0310] The identification information replacing unit 412 is used to replace any element in the initial information sequence with the identification information corresponding to the service recommendation information x in the service recommendation information set, so as to obtain the service information sequence corresponding to the service recommendation information x.

[0311] In a possible implementation, the business data processing device 4 may further include: a recommendation information pairing module 45, a first similarity evaluation value acquisition module 46 and an information relationship structure generation module 47, wherein:

[0312] The recommendation information pairing module 45 is used to pair the business recommendation information in the business recommendation information set to obtain a plurality of recommendation information pairs;

[0313] A first similarity evaluation value acquisition module 46 is used to determine a similarity evaluation value between two pieces of business recommendation information included in each pair of recommendation information according to business information recommendation features corresponding to the two pieces of business recommendation information included in each pair of recommendation information;

[0314] The information relationship structure generating module 47 is used to generate an information relationship structure corresponding to the business recommendation information set according to the business recommendation information set and the similarity evaluation value associated with each recommendation information pair.

[0315] In a possible implementation, the business data processing device 4 may further include: a history browsing information acquisition module 48, a recommendation information adding module 49 and a recommendation information statistics module 410, wherein:

[0316] A historical browsing information acquisition module 48 is used to acquire a collection of historical browsing information of a business object on the information recommendation platform;

[0317] A recommendation information adding module 49, configured to add the service recommendation information x to the candidate recommendation information set if the similarity evaluation value between the service recommendation information x in the information relationship structure and the historical browsing information in the historical browsing information set is greater than a similarity threshold;

[0318] The recommendation information statistics module 4010 is used to count the frequencies corresponding to each business recommendation information in the candidate recommendation information set, and determine the object recommendation information corresponding to the business object in the candidate recommendation information set according to the frequencies corresponding to each business recommendation information in the candidate recommendation information set.

[0319] In a possible implementation, the business data processing device 4 may further include: a fourth feature acquisition module 4011, a third feature combination module 4012, a fifth feature acquisition module 4013, a second similarity evaluation value acquisition module 4014 and an information relationship structure update module 4015, wherein:

[0320] The fourth feature acquisition module 4011 is used to obtain the business multimodal features corresponding to the business recommendation information b according to the text representation information and image representation information corresponding to the business recommendation information b if the newly added business recommendation information b in the information recommendation platform is detected, and add the business multimodal features corresponding to the business recommendation information b to the business multimodal feature library;

[0321] The third feature combination module 4012 is used to obtain the business information input feature corresponding to the business recommendation information b according to the business information sequence corresponding to the business recommendation information b, the business position coding feature corresponding to the business recommendation information b, and the business multimodal feature corresponding to the business recommendation information b;

[0322] The fifth feature acquisition module 4013 is used to perform feature extraction processing on the business information input features of the business recommendation information b through the information prediction model to obtain the initial information recommendation features corresponding to the business recommendation information b, and perform pooling processing on the initial information recommendation features corresponding to the business recommendation information b to obtain the business information recommendation features corresponding to the business recommendation information b;

[0323] The second similarity evaluation value acquisition module 4014 is used to acquire the similarity evaluation value between the business information recommendation feature corresponding to the business recommendation information b and the business information recommendation feature corresponding to each business recommendation information in the business recommendation information set;

[0324] The information relationship structure updating module 4015 is used to update the information relationship structure according to the business recommendation information b and the similarity evaluation value between the business recommendation information b and each business recommendation information in the business recommendation information set.

[0325] In an embodiment of the present application, the sample multimodal features integrate the text representation information and image representation information of the sample business information. The sample multimodal features are used to represent the sample business information for training the initial prediction model, which can better reflect the essential content of the sample business information. The information prediction model obtained by training the sample multimodal features can improve the information representation ability of the business information recommendation features, thereby improving the accuracy of information recommendation.

[0326] According to an embodiment of the present application, the steps involved in the business data processing method shown above can be Fig.13 The business data processing device 4 shown in the figure is executed by each module and each unit. For example, Figure 7 Step S301 shown can be performed by Fig.13 The business information sequence acquisition module 41 shown is executed. Figure 7 Step S302 shown can be performed by Fig.13 The second feature acquisition module 42 shown is used to execute, Figure 7 Step S303 shown can be performed by Fig.13 The second feature combination module 43 is shown to execute, Figure 7 Step S304 shown can be performed by Fig.13 The third feature acquisition module 44 shown is used to perform the above.

[0327] According to one embodiment of the present application, Fig.13 The various modules (units) in the business data processing device 4 shown can be respectively or all merged into one or several modules (units) to constitute, or some (some) modules (units) thereof can also be split into at least two smaller sub-units in function, and the same operation can be achieved without affecting the realization of the technical effects of the embodiments of the present application. The above modules (units) are divided based on logical functions. In practical applications, the function of a module (unit) can also be realized by at least two modules (units), or the function of at least two modules (units) is realized by one module (unit). In other embodiments of the present application, the business data processing device 4 can also include other modules (units). In practical applications, these functions can also be implemented with the assistance of other modules (units), and can be implemented by the collaboration of at least two modules (units).

[0328] See also Fig.14 , Fig.14 Schematic diagram of the structure of a computer device provided in an embodiment of the present application. Fig.14 As shown, the computer device 1000 can be Figure 1 Any terminal device in the terminal cluster in the corresponding embodiment (for example, terminal device 10a), or a server (for example Figure 1The server 10d shown in the figure will not be limited here. For ease of understanding, the embodiment of the present application takes the computer device as a terminal device as an example. The computer device 1000 may include: a processor 1001, a network interface 1004 and a memory 1005. In addition, the computer device 1000 may also include: a user interface 1003, and at least one communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. Among them, the user interface 1003 may include a display screen (Display), a keyboard (Keyboard), and optionally, the user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or it may be a non-volatile memory (non-volatile memory), such as at least one disk storage. The memory 1005 may optionally also be at least one storage device located away from the aforementioned processor 1001. As Fig.14 As shown, the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and a device control application program.

[0329] in, Fig.14 The network interface 1004 in the computer device 1000 shown can also provide network communication functions; and the user interface 1003 is mainly used to provide an input interface for the user; it can be understood that the processor 1001 can be used to call the device control application stored in the memory 1005.

[0330] It should be understood that the computer device 1000 described in the embodiment of the present application can execute the above Figure 2 , Figure 4 and Figure 7 The description of the business data processing method in any corresponding embodiment can also be performed as described above. Fig.10 The description of the service data processing device 1 in the corresponding embodiment is as follows: Fig.11 The description of the business data processing device 2 in the corresponding embodiment is as follows: Fig.12 The description of the service data processing device 3 in the corresponding embodiment, or the above-mentioned Fig.13 The description of the service data processing device 4 in the corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated here either.

[0331] In addition, it should be pointed out here that: the embodiment of the present application also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program executed by the business data processing device 1, the business data processing device 2, the business data processing device 3 or the business data processing device 4 mentioned above, and the computer program includes computer instructions. When the processor executes the computer instructions, it can execute the above Figure 2 , Figure 4 and Figure 7 The description of the business data processing method in any corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc. For technical details not disclosed in the computer-readable storage medium embodiment involved in this application, please refer to the description of the method embodiment of this application. As an example, computer instructions can be deployed on a computer device for execution, or on multiple computer devices located at one location, or on multiple computer devices distributed at multiple locations and interconnected by a communication network. Multiple computer devices distributed at multiple locations and interconnected by a communication network can form a blockchain system.

[0332] In addition, it should be noted that: the embodiment of the present application also provides a computer program product or a computer program, which may include computer instructions, and the computer instructions may be stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor may execute the computer instructions, so that the computer device executes the above Figure 2 , Figure 4 and Figure 7 The description of the business data processing method in any corresponding embodiment will not be repeated here. In addition, the description of the beneficial effects of the same method will not be repeated. For technical details not disclosed in the computer program product or computer program embodiment involved in this application, please refer to the description of the method embodiment of this application.

[0333] It should be noted that, for the above-mentioned various method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.

[0334] The steps in the method of the embodiment of the present application can be adjusted in order, combined and deleted according to actual needs.

[0335] The modules in the device of the embodiment of the present application can be merged, divided and deleted according to actual needs.

[0336] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0337] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A business data processing method, characterized in that: include: Obtain a sample information sequence including N sample business information, and obtain a sample multimodal feature corresponding to each sample business information in the sample information sequence in a sample multimodal feature library; The sample multimodal feature is determined according to the text representation information and the image representation information corresponding to the sample business information; N is an integer greater than 1; Obtaining sample position coding features of the N sample business information in the sample information sequence, and obtaining sample information input features according to the sample information sequence, the sample position coding features, and sample multimodal features corresponding to the N sample business information; Inputting the sample information input features into the initial prediction model, performing feature extraction processing on the sample information input features through the feature extractor in the initial prediction model to obtain sample information recommendation features corresponding to the sample information sequence, and obtaining information recommendation probability distribution corresponding to the sample information recommendation features through the output layer in the initial prediction model; The network parameters of the initial prediction model are modified according to the information recommendation probability distribution, and the feature extractor in the initial prediction model containing the modified network parameters is determined as the information prediction model; the information prediction model is used to output the business information recommendation features corresponding to the business recommendation information contained in the business recommendation information set, and the business information recommendation features are used to determine the object recommendation information corresponding to the business object.

2. The method according to claim 1, characterized in that The method further comprises: Acquire the sample business data set, and acquire sample text data and sample image data corresponding to the sample business information contained in the sample business data set; Performing text encoding processing on the sample text data corresponding to the sample business information a in the sample business data set to obtain text representation information corresponding to the sample business information a; Performing image coding processing on sample image data corresponding to sample service information a in the sample service data set to obtain image representation information corresponding to the sample service information a; Performing feature fusion on the text representation information corresponding to the sample business information a and the image representation information corresponding to the sample business information a to obtain a sample multimodal feature corresponding to the sample business information a; The sample multimodal features corresponding to the sample business information contained in the sample business data set are obtained, and the sample multimodal features corresponding to each piece of sample business information in the sample business data set are added to the sample multimodal feature library.

3. The method according to claim 2, characterized in that The step of performing text encoding processing on the sample text data corresponding to the sample business information a in the sample business data set to obtain text representation information corresponding to the sample business information a includes: Divide the sample text data corresponding to the sample business information a in the sample business data set into D unit characters, and obtain the unit word vectors corresponding to the D unit characters respectively; D is a positive integer; According to the semantic information of the D unit characters in the sample text data corresponding to the sample business information a, obtaining text vectors corresponding to the D unit characters respectively; According to the text positions of the D unit characters in the sample text data corresponding to the sample business information a, obtaining the position vectors corresponding to the D unit characters respectively; Obtaining text input features corresponding to the sample business information a according to the unit word vector, the text vector, and the position vector; The text input feature is input into a text encoder, the text encoder performs text encoding processing on the text input feature to obtain the text encoding feature corresponding to the sample business information a, and the text encoding feature is fully connected to obtain the text representation information corresponding to the sample business information a.

4. The method according to claim 2, characterized in that: The step of performing image coding processing on the sample image data corresponding to the sample service information a in the sample service data set to obtain the image representation information corresponding to the sample service information a includes: Inputting the sample image data corresponding to the sample service information a in the sample service data set into the image encoder, performing convolution processing on the sample image data corresponding to the sample service information a through the convolution layer in the image encoder to obtain a first sample convolution feature, and performing maximum pooling processing on the first sample convolution feature to obtain a first sample pooling feature; Obtaining an input feature of the yth residual block structure in the image encoder; when y is 1, the input feature of the yth residual block structure is the first sample pooling feature; when y is not 1, the input feature of the yth residual block structure is the output feature of the y-1th residual block structure; y is a positive integer less than or equal to P, P is a positive integer, and P represents the number of residual block structures included in the image encoder; Performing convolution processing on the input features of the yth residual block structure to obtain a second sample convolution feature of the yth residual block structure, and obtaining an output feature of the yth residual block structure according to the second sample convolution feature of the yth residual block structure and the input features of the yth residual block structure; The output features of the P-th residual block structure are average pooled to obtain second sample pooled features, and the second sample pooled features are fully connected to obtain image representation information corresponding to the sample business information a.

5. The method according to claim 1, characterized in that The obtaining of sample position coding features of the N sample service information in the sample information sequence includes: Dividing the positions of the N sample service information in the sample information sequence into even positions and odd positions, performing sinusoidal position coding on the even positions in the N sample service information to obtain sinusoidal coding information corresponding to the even positions; Performing cosine position coding on the odd positions in the N sample service information to obtain cosine coding information corresponding to the odd positions; The sine coding information and the cosine coding information are determined as sample position coding features of the N sample business information in the sample information sequence.

6. The method according to claim 1, characterized in that The obtaining the sample information input feature according to the sample information sequence, the sample multimodal feature and the sample position coding feature comprises: Performing vector conversion on each sample service information in the sample information sequence to obtain a sample sequence vector corresponding to the sample information sequence; The product of the sample sequence vector and the sample multimodal feature and the sum of the sample position coding feature are determined as the sample information input feature.

7. The method according to claim 1, characterized in that The feature extractor includes M attention components, one attention component includes K self-attention layers, and M and K are both positive integers; The step of performing feature extraction processing on the sample information input features by the feature extractor in the initial prediction model to obtain sample service recommendation features corresponding to the sample information sequence includes: Obtain the input feature of the i-th attention component among the M attention components; when i is 1, the input feature of the i-th attention component is the sample information input feature; when i is not 1, the input feature of the i-th attention component is the output feature of the i-1-th attention component; i is a positive integer less than or equal to M; Obtain a transformation parameter matrix of the jth self-attention layer in the i-th attention component, and perform feature transformation processing on the input features of the i-th attention component through the transformation parameter matrix to obtain sample transformation features; j is a positive integer less than or equal to K; Obtaining an information mask matrix of the j-th self-attention layer, performing information masking processing on the sample transformation features through the information mask matrix, and obtaining output features of the j-th self-attention layer; the information mask matrix is ​​used to indicate the sample business information used by the initial prediction model when determining the sample business information features corresponding to the t+1-th sample business information in the sample information sequence, where t is a positive integer less than N; The output features of the K self-attention layers in the i-th attention component are concatenated to obtain the sample mask concatenated features of the i-th attention component; Obtaining a first sample joint feature according to the sample mask splicing feature and the input feature of the i-th attention component, and normalizing the first sample joint feature to obtain a sample normalized feature; Performing feature transformation processing on the sample normalized features through the feedforward network layer in the i-th attention component to obtain attention transformation features; According to the attention transformation feature and the sample normalization feature, a second sample joint feature is obtained, the second sample joint feature is normalized to obtain the output feature of the i-th attention component, and the output feature of the M-th attention component is determined as the sample business recommendation feature corresponding to the sample information sequence.

8. The method according to claim 7, characterized in that The sample transformation features include a query matrix Q, a key matrix K and a value matrix V; The information masking matrix is ​​used to perform information masking on the sample transformation features to obtain the output features of the j-th self-attention layer, including: Determine the product of the transposed matrix of the key matrix K and the query matrix Q as a first candidate matrix, and determine the product of the first candidate matrix and the information mask matrix as a second candidate matrix; Obtain the number of columns corresponding to the query matrix Q, normalize the ratio between the second candidate matrix and the square root of the number of columns to obtain an attention weight matrix, and determine the product of the attention weight matrix and the value matrix V as the output feature of the j-th self-attention layer.

9. The method according to claim 1, characterized in that: The sample information recommendation feature corresponding to the sample information sequence includes the sample information recommendation feature associated with each sample business information in the sample information sequence; The obtaining, through the output layer in the initial prediction model, the information recommendation probability distribution corresponding to the sample information recommendation feature includes: Performing feature transformation processing on the sample information recommendation feature associated with the t+1th sample service information in the sample information sequence through the output layer in the initial prediction model to obtain a candidate service recommendation feature corresponding to the sample information recommendation feature associated with the t+1th sample service information; t is a positive integer less than N; Performing an exponential operation on the candidate service recommendation feature corresponding to the t+1th sample service information to obtain a recommendation index value corresponding to the t+1th sample service information; Accumulate the recommendation index values ​​corresponding to each sample service information in the sample information sequence to obtain a total recommendation index value, and determine the ratio between the recommendation index value corresponding to the t+1th sample service information and the total recommendation index value as the recommendation probability value corresponding to the t+1th sample service information; The information recommendation probability distribution is determined according to the recommendation probability value corresponding to each sample business information in the sample information sequence.

10. The method according to claim 1, characterized in that The step of modifying the network parameters of the initial prediction model according to the information recommendation probability distribution and determining the feature extractor in the initial prediction model including the modified network parameters as the information prediction model comprises: Obtaining the recommendation probability values ​​of the first t sample service information in the sample information sequence in the information recommendation probability distribution; one sample service information corresponds to one recommendation probability value, and t is a positive integer less than N; Performing a logarithmic operation on the recommended probability value of each sample information in the first t sample service information to obtain a logarithmic probability value corresponding to each sample information in the first t sample service information, and accumulating the logarithmic probability value corresponding to each sample service information in the first t sample service information to obtain a sample prediction loss corresponding to the t+1th sample service information; The network parameters of the initial prediction model are iteratively trained according to the sample prediction loss corresponding to the t+1th sample business information until the updated network parameters meet the training end condition, the training is stopped, and the feature extractor in the initial prediction model at the end of the training is determined as the information prediction model.

11. A business data processing method, characterized in that: include: Acquire a service recommendation information set, and acquire a service information sequence corresponding to the service recommendation information x in the service recommendation information set; The length of the service information sequence corresponding to the service recommendation information x is the same as the length of the sample information sequence; Acquire a business multimodal feature corresponding to the business recommendation information x in a business multimodal feature library; the business multimodal feature is determined according to text representation information and image representation information corresponding to the business recommendation information; Obtaining a service position coding feature of the service recommendation information x in the service information sequence, and obtaining a service information input feature corresponding to the service recommendation information x according to the service information sequence corresponding to the service recommendation information x, the service position coding feature corresponding to the service recommendation information x, and the service multimodal feature corresponding to the service recommendation information x; The business information input features corresponding to the business recommendation information x are input into the information prediction model, and the business information input features corresponding to the business recommendation information x are subjected to feature extraction processing by the information prediction model to obtain the business information recommendation features corresponding to the business recommendation information x; the business information recommendation features corresponding to the business recommendation information x are used to determine the object recommendation information corresponding to the business object; the information prediction model is trained using the business data processing method described in any one of claims 1 to 10.

12. The method according to claim 11, characterized in that The acquiring the service information sequence corresponding to the service recommendation information x in the service recommendation information set includes: Obtaining an initial information sequence having the same length as the sample information sequence; wherein the elements in the initial information sequence are all mask filling values; Any element in the initial information sequence is replaced with identification information corresponding to the service recommendation information x in the service recommendation information set to obtain a service information sequence corresponding to the service recommendation information x.

13. The method according to claim 11, characterized in that The method further comprises: Pairing the business recommendation information in the business recommendation information set to obtain a plurality of recommendation information pairs; Determine a similarity evaluation value between two pieces of business recommendation information included in each pair of recommendation information according to the business information recommendation features corresponding to the two pieces of business recommendation information included in each pair of recommendation information; An information relationship structure corresponding to the business recommendation information set is generated according to the business recommendation information set and the similarity evaluation value associated with each pair of recommendation information.

14. The method according to claim 13, characterized in that The method further comprises: Obtaining a collection of historical browsing information of the business object on the information recommendation platform; If the similarity evaluation value between the business recommendation information x in the information relationship structure and the historical browsing information in the historical browsing information set is greater than a similarity threshold, then adding the business recommendation information x to the candidate recommendation information set; The frequencies corresponding to each piece of business recommendation information in the candidate recommendation information set are counted, and according to the frequencies corresponding to each piece of business recommendation information in the candidate recommendation information set, the object recommendation information corresponding to the business object is determined in the candidate recommendation information set.

15. The method according to claim 13, characterized in that The method further comprises: If newly added business recommendation information b is detected in the information recommendation platform, the business multimodal features corresponding to the business recommendation information b are obtained according to the text representation information and the image representation information corresponding to the business recommendation information b, and the business multimodal features corresponding to the business recommendation information b are added to the business multimodal feature library; Obtaining a business information input feature corresponding to the business recommendation information b according to the business information sequence corresponding to the business recommendation information b, the business position coding feature corresponding to the business recommendation information b, and the business multimodal feature corresponding to the business recommendation information b; Performing feature extraction processing on the business information input features of the business recommendation information b through the information prediction model to obtain initial information recommendation features corresponding to the business recommendation information b, and performing pooling processing on the initial information recommendation features corresponding to the business recommendation information b to obtain business information recommendation features corresponding to the business recommendation information b; Obtaining a similarity evaluation value between a business information recommendation feature corresponding to the business recommendation information b and a business information recommendation feature corresponding to each business recommendation information in the business recommendation information set; The information relationship structure is updated according to the business recommendation information b and the similarity evaluation values ​​between the business recommendation information b and each business recommendation information in the business recommendation information set.

16. A business data processing device, characterized in that: include: A first feature acquisition module, used to acquire a sample information sequence including N sample business information, and acquire a sample multimodal feature corresponding to each sample business information in the sample information sequence in a sample multimodal feature library; the sample multimodal feature is determined according to text representation information and image representation information corresponding to the sample business information; N is an integer greater than 1; A first feature combination module is used to obtain sample position coding features of the N sample business information in the sample information sequence, and obtain sample information input features according to the sample information sequence, the sample position coding features, and sample multimodal features corresponding to the N sample business information; A probability distribution acquisition module, used to input the sample information input features into an initial prediction model, perform feature extraction processing on the sample information input features through a feature extractor in the initial prediction model, obtain sample information recommendation features corresponding to the sample information sequence, and obtain information recommendation probability distribution corresponding to the sample information recommendation features through an output layer in the initial prediction model; A prediction model training module is used to correct the network parameters of the initial prediction model according to the information recommendation probability distribution, and determine the feature extractor in the initial prediction model containing the corrected network parameters as the information prediction model; the information prediction model is used to output the business information recommendation features corresponding to the business recommendation information contained in the business recommendation information set, and the business information recommendation features are used to determine the object recommendation information corresponding to the business object.

17. A business data processing device, characterized in that: include: A service information sequence acquisition module is used to acquire a service recommendation information set, and acquire a service information sequence corresponding to the service recommendation information x in the service recommendation information set; the length of the service information sequence corresponding to the service recommendation information x is the same as the length of the sample information sequence; A second feature acquisition module acquires a business multimodal feature corresponding to the business recommendation information x from a business multimodal feature library; the business multimodal feature is determined according to text representation information and image representation information corresponding to the business recommendation information; A second feature combination module is used to obtain a service position coding feature of the service recommendation information x in the service information sequence, and obtain a service information input feature corresponding to the service recommendation information x according to the service information sequence corresponding to the service recommendation information x, the service position coding feature corresponding to the service recommendation information x, and the service multimodal feature corresponding to the service recommendation information x; A third feature acquisition module is used to input the business information input features corresponding to the business recommendation information x into the information prediction model, and perform feature extraction processing on the business information input features corresponding to the business recommendation information x through the information prediction model to obtain the business information recommendation features corresponding to the business recommendation information x; the business information recommendation features corresponding to the business recommendation information x are used to determine the object recommendation information corresponding to the business object; the information prediction model is trained using the business data processing method described in any one of claims 1 to 10.

18. A computer device, characterized in that: including memory and processor; The memory is connected to the processor, the memory is used to store a computer program, and the processor is used to call the computer program so that the computer device executes the method described in any one of claims 1 to 10, or executes the method described in any one of claims 11 to 15.

19. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which is suitable for being loaded and executed by a processor, so that a computer device having the processor executes the method according to any one of claims 1 to 10, or executes the method according to any one of claims 11 to 15.

20. A computer program product, characterized in that The method comprises a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 10, or implements the method according to any one of claims 11 to 15.

Images