Information recommendation method and device, electronic equipment, computer readable storage medium and computer program product

Abstract

The invention provides an information recommendation method and device, electronic equipment, a computer readable storage medium and a computer program product. The method comprises the steps of obtaining a vector combination; performing dimension reduction based on the vector combination to obtain a first output vector; performing attention coding based on the first multi-modal feature vector to obtain a first weight coefficient set corresponding to the first multi-modal vector, and determining a second output vector representing the individual interest of the target object based on the first weight coefficient set; performing feature extraction on the first multi-modal feature vector to obtain a third output vector representing the group interest of the group to which the target object belongs; determining recommendation parameters based on the first output vector, the second output vector and the third output vector; and determining a target recommendation object for the target object from the plurality of candidate objects based on the plurality of recommendation parameters corresponding to the plurality of candidate objects. According to the method and the device, the target recommendation object can be determined from multiple dimensions, and the recommendation accuracy is improved.

Description

Technical Field

[0001] This application relates to artificial intelligence technology, and more particularly to an information recommendation method, apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0002] With the rapid development of the internet and information technology, people are increasingly inclined to use the internet for social activities. Through user analysis on various platforms, recommendations of potential users with direct or indirect connections to the target audience can be made.

[0003] Related technologies start from the characteristics of the target object and use the characteristics of the target user to select the target recommendation object. However, due to the limitations of the characteristics used, the accuracy and relevance of the recommendation are not high. Summary of the Invention

[0004] This application provides an information recommendation method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can determine the target recommendation object from multiple dimensions and improve the accuracy of the recommendation.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides an information recommendation method, the method comprising:

[0007] Obtain a vector combination, wherein the vector combination includes a first multimodal feature vector of the target object and a second multimodal feature vector of the candidate object;

[0008] Dimensionality reduction is performed based on the vector combination to obtain the first output vector;

[0009] Attention encoding is performed based on the first multimodal feature vector to obtain a first set of weight coefficients corresponding to the first multimodal vector, and a second output vector representing the individual interest of the target object is determined based on the first set of weight coefficients.

[0010] Feature extraction is performed on the first multimodal feature vector to obtain a third output vector representing the group interest of the group to which the target object belongs;

[0011] Recommendation parameters are determined based on the first output vector, the second output vector, and the third output vector;

[0012] Based on the multiple recommendation parameters corresponding to the multiple candidate objects, a target recommendation object for the target object is determined from the multiple candidate objects.

[0013] This application provides an information recommendation device, the device comprising:

[0014] An acquisition module is used to acquire a vector combination, wherein the vector combination includes a first multimodal feature vector of the target object and a second multimodal feature vector of the candidate object;

[0015] The dimension reduction module is used to perform dimension reduction based on the vector combination to obtain a first output vector;

[0016] The encoding module is used to perform attention encoding based on the first multimodal feature vector to obtain a first set of weight coefficients corresponding to the first multimodal vector, and to determine a second output vector representing individual interest based on the first set of weight coefficients.

[0017] The convolution module is used to extract features from the first multimodal feature vector to obtain a third output vector representing the group's interest.

[0018] The determining module is configured to determine recommendation parameters based on the first output vector, the second output vector, and the third output vector; and to determine a target recommendation object for the target object from among the multiple candidate objects based on the multiple recommendation parameters corresponding to the multiple candidate objects respectively.

[0019] This application provides an electronic device, the electronic device comprising:

[0020] Memory is used to store executable instructions for a computer;

[0021] The processor, when executing computer-executable instructions stored in the memory, implements the information recommendation method provided in the embodiments of this application.

[0022] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the information recommendation method provided in this application when executed by a processor.

[0023] This application provides a computer program product, including a computer program or computer executable instructions, which, when executed by a processor, implements the information recommendation method provided in this application.

[0024] The embodiments of this application have the following beneficial effects:

[0025] Based on the first multimodal feature vector of the target object and the second multimodal feature vector of the candidate object, three different dimensions of output vectors are determined. After fusing the three output vectors, recommendation parameters are determined. The target recommendation object is determined according to the recommendation parameters. While exploring the relationship between the target object and the candidate object, the individual interests of the target object and the group interests of the target object's group are also considered. The target recommendation object is determined by information from multiple dimensions, making the target recommendation object more in line with the target object's interests and improving the accuracy and relevance of the recommendation. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the architecture of the information recommendation system 100 provided in an embodiment of this application;

[0027] Figure 2 This is a schematic diagram of the structure of the server 200 provided in the embodiments of this application;

[0028] Figure 3A This is a first flowchart illustrating the information recommendation method provided in this application embodiment;

[0029] Figure 3B This is a second flowchart illustrating the information recommendation method provided in the embodiments of this application;

[0030] Figure 3C This is a schematic diagram of the third process of the information recommendation method provided in the embodiments of this application;

[0031] Figure 3D This is a schematic diagram of the fourth process of the information recommendation method provided in the embodiments of this application;

[0032] Figure 3E This is a schematic diagram of the fifth process of the information recommendation method provided in the embodiments of this application;

[0033] Figure 3F This is a schematic diagram of the sixth process of the information recommendation method provided in the embodiments of this application;

[0034] Figure 3G This is a schematic diagram of the seventh process of the information recommendation method provided in the embodiments of this application;

[0035] Figure 3H This is the eighth flowchart of the information recommendation method provided in the embodiments of this application;

[0036] Figure 3I This is a ninth flowchart illustrating the information recommendation method provided in the embodiments of this application;

[0037] Figure 3J This is a schematic diagram of the tenth process of the information recommendation method provided in the embodiments of this application;

[0038] Figure 3K This is a schematic diagram of the eleventh step of the information recommendation method provided in the embodiments of this application;

[0039] Figure 3L This is a schematic diagram of the twelfth step of the information recommendation method provided in the embodiments of this application;

[0040] Figure 3M This is a schematic diagram of the thirteenth step of the information recommendation method provided in the embodiments of this application;

[0041] Figure 3N This is a schematic diagram of the fourteenth process of the information recommendation method provided in the embodiments of this application;

[0042] Figure 4 This is a schematic diagram illustrating the training process of the language understanding model provided in the embodiments of this application;

[0043] Figure 5 This is a schematic diagram of the Transformer encoder provided in the embodiments of this application;

[0044] Figure 6 This is a schematic diagram illustrating the principle of training the first machine learning model provided in an embodiment of this application;

[0045] Figure 7 This is a schematic diagram illustrating the principle of training a second machine learning model according to an embodiment of this application;

[0046] Figure 8 This is a schematic diagram illustrating the principle of training a third machine learning model according to an embodiment of this application;

[0047] Figure 9A This is a schematic diagram of a user's personal information page provided in an embodiment of this application;

[0048] Figure 9B This is a schematic diagram of the user's friend recommendation interface provided in an embodiment of this application;

[0049] Figure 10 This is a flowchart of the multimodal friend recommendation system provided in the embodiments of this application;

[0050] Figure 11 This is an overall framework diagram of the multimodal recommendation model provided in the embodiments of this application;

[0051] Figure 12A This is a schematic diagram illustrating the changes in click-through rate over 7 days, provided in an embodiment of this application.

[0052] Figure 12B This is a schematic diagram illustrating the change in the 7-day click pass rate provided in the embodiments of this application.

[0053] It should be noted that the terms "first" and "second" mentioned above are only used to distinguish between different options and do not represent the degree of superiority or inferiority of the options or their priority in the implementation process. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0055] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0056] In the following description, the terms "first / second / third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first / second / third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0057] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0058] Unless otherwise specified, "at least one" as used below refers to one or more cases, and "multiple" can refer to two or more cases.

[0059] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0060] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0061] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0062] 1) The target object refers to the object that the recommendation system wants to serve, such as the users for whom the system needs to recommend information.

[0063] 2) Candidate objects refer to objects that the recommendation system considers recommending to the target object, such as potential friends. Candidate objects can be selected by the recommendation system based on at least one factor among the target object's social network, interests, and behavioral habits, or they can be objects that the target object already knows but is not yet a friend of. Candidate objects are the input to the recommendation system, from which the system selects one or more of the most suitable target objects to recommend to the target object.

[0064] 3) Individual interests refer to the content or activities preferred by an individual target user. These interests may be explicit, such as a user's public preferences, favorites, and comments on social media; or they may be implicit, such as a user's historical behavioral data, search history, and click behavior. The individual interests of the target user are the basis for the recommendation system to make personalized recommendations. The recommendation system will filter and recommend content or friends that may match the target user's personal preferences based on these interests.

[0065] 4) Group interests refer to the content or activities generally preferred by the social group or user group to which the target audience belongs. Group interests may be jointly determined by the target audience's social network, community, and organization, reflecting the common preferences within the target audience's social network. Group interests can be used to supplement individual interests, helping the recommendation system better understand the target audience's social background and environment, thereby making more accurate information recommendations.

[0066] 5) Recommendation parameters are metrics used to describe and evaluate the recommendation value of candidate objects to the target object. For example, when both the candidate object and the target object are users, recommendation parameters can help the recommendation system understand the similarity or relevance between the candidate object and the target object; as another example, when the candidate object is various types of information such as news, music, and videos, and the target object is users, recommendation parameters can be metrics such as click-through rate and conversion rate.

[0067] 6) Nonlinear mapping refers to a mathematical operation or computational process used to map points in a nonlinear system or input space to points in a linear or nonlinear output space. In machine learning and deep learning, nonlinear mapping is typically implemented through neural networks. Neural networks consist of many layers, each of which can be viewed as a nonlinear mapping. These layers introduce nonlinearity through activation functions, mappings such as the sigmoid curve, the rectified linear unit (ReLU), and the hyperbolic tangent (Tanh), enabling the entire network to learn complex input-output relationships.

[0068] 7) Mean pooling averages the pixels within each local region of the feature map to generate a new, smaller feature map. Mean pooling is used to reduce the dimensionality of feature maps in convolutional neural networks. Pooling layers can be placed after convolutional layers, and their purpose is to reduce the dimensionality of the data while preserving important feature information. This process can be viewed as a compression of the feature map, reducing the spatial dimensionality of the data and also reducing the number of parameters the model needs to learn.

[0069] When making information recommendations, related technologies usually use multimodal features to determine the target recommendation object. However, when extracting multimodal features of the target object or candidate object, redundant information is often introduced. At the same time, related technologies only consider the individual interests of the target object, which limits the scope of recommendations.

[0070] Based on the above analysis, the applicant found that the information recommendation methods of related technologies cannot reduce the noise introduced by multimodal analysis, nor do they combine the individual and collective interests of the target object for friend recommendation. In response to the above problems, this application provides an information recommendation method that can determine the target recommendation object from multiple dimensions and improve the accuracy of the recommendation.

[0071] This application provides an information recommendation method, apparatus, electronic device, computer-readable storage medium, and computer program product, which can determine target recommendation objects from multiple dimensions and improve the accuracy of recommendations. The following describes exemplary applications of the electronic device provided in this application. The electronic device provided in this application can be implemented as various types of terminals such as laptops, tablets, desktop computers, set-top boxes, smartphones, smart speakers, smartwatches, smart TVs, and in-vehicle terminals, or it can be implemented as a server. The following will describe exemplary applications when the electronic device is implemented as a server.

[0072] See Figure 1 , Figure 1This is a schematic diagram of the architecture of the information recommendation system 100 provided in the embodiment of this application. In order to support an information recommendation application, the terminal 400 connects to the server 200 through the network 300. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0073] Server 200 calculates multiple recommendation parameters corresponding to multiple candidate objects to determine the target recommended object from the multiple candidate objects, and sends it to terminal 400. Terminal 400 recommends the target recommended object in the human-computer interaction interface 410.

[0074] Taking a scenario of friend recommendation in a game as an example, server 200 calculates multiple recommendation parameters corresponding to multiple candidate objects, sorts the multiple candidate objects in descending order according to the recommendation parameters, selects a preset number of candidate objects from the descending sort results as target recommendation objects for the target object, and sends the relevant information of the target recommendation objects to terminal 400 so that the recommendations can be made on the human-computer interaction interface 410 in descending order, making it convenient for the target object to add friends (i.e., target recommendation objects) according to the recommendation parameters.

[0075] Taking the scenario of news recommendation in news software or website as an example, server 200 calculates recommendation parameters corresponding to multiple candidate news items, sorts the multiple candidate news items in descending order according to the recommendation parameters, selects a preset number of candidate news items from the descending sort result as target recommended news for the target audience, and sends them to terminal 400 so that they can be displayed on the human-computer interaction interface 410 in descending order. This makes it convenient to recommend news to the target audience according to the recommendation parameters, so that the recommended news is more in line with the user's habits and preferences.

[0076] Taking a video recommendation scenario in video software or website as an example, server 200 calculates recommendation parameters corresponding to multiple candidate videos, sorts the multiple candidate videos in descending order according to the recommendation parameters, selects a preset number of candidate videos from the descending sort results as target recommendation videos for the target audience, and sends them to terminal 400 so that they can be displayed on the human-computer interaction interface 410 in descending order. This facilitates recommending videos to the target audience according to the recommendation parameters, making the recommended videos more in line with the user's habits and preferences.

[0077] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminals and servers can be connected directly or indirectly via wired or wireless communication, which is not limited in this embodiment.

[0078] The embodiments of this application can be implemented using artificial intelligence (AI) technology. AI is the theory, methods, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new type of intelligent machine that can react in a way similar to human intelligence. AI also studies the design principles and implementation methods of various intelligent machines, enabling them to possess perception, reasoning, and decision-making capabilities.

[0079] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, pre-trained model technology, operating / interactive systems, and mechatronics. Among these, pre-trained models, also known as large-scale models or foundational models, can be widely applied to downstream tasks across various AI fields after fine-tuning. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0080] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the server 200 provided in the embodiments of this application. Figure 2 The server 200 shown includes at least one processor 210, memory 230, and at least one network interface 220. The various components of server 200 are coupled together via a bus system 240. It is understood that the bus system 240 is used to implement communication between these components. In addition to a data bus, the bus system 240 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 240.

[0081] The processor 210 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0082] The memory 230 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 230 may optionally include one or more storage devices physically located away from the processor 210.

[0083] The memory 230 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 230 described in this application embodiment is intended to include any suitable type of memory.

[0084] In some embodiments, memory 230 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0085] Operating system 231 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0086] The network communication module 232 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 220, exemplary network interfaces 220 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0087] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 An information recommendation device 233 stored in memory 230 is shown. This device can be software in the form of programs and plug-ins, and includes the following software modules: an acquisition module 2331, a dimensionality reduction module 2332, an encoding module 2333, a convolution module 2334, and a determination module 2335. These modules are logically linked and can therefore be arbitrarily combined or further divided according to their implemented functions. The functions of each module will be described below.

[0088] In some embodiments, the terminal or server can implement the information recommendation method provided in this application by running various computer-executable instructions or computer programs. For example, computer-executable instructions can be microprogram-level commands, machine instructions, or software instructions. Computer programs can be native programs or software modules in an operating system; they can be native applications (APPs), i.e., programs that need to be installed in the operating system to run, such as game APPs or instant messaging APPs; or they can be applets that can be embedded in any APP, i.e., programs that only need to be downloaded to a browser environment to run. In summary, the aforementioned computer-executable instructions can be any form of instruction, and the aforementioned computer programs can be any form of application, module, or plugin.

[0089] The information recommendation method provided in this application will be described in conjunction with exemplary applications and implementations of the server provided in the embodiments of this application.

[0090] See Figure 3A , Figure 3A This is a first flowchart illustrating the information recommendation method provided in this application embodiment. Taking the server as the execution entity as an example, it will be combined with... Figure 3A The steps shown are explained.

[0091] In step 101, a vector combination is obtained, wherein the vector combination includes the first multimodal feature vector of the target object and the second multimodal feature vector of the candidate object.

[0092] In some embodiments, see Figure 3B , Figure 3B This is a schematic diagram of the second process of the information recommendation method provided in the embodiments of this application. Figure 3A Step 101, "Obtaining Vector Combinations," can be done through... Figure 3B Steps 1011 to 1014 are implemented, and the details are explained below.

[0093] In step 1011, the first multimodal features of the target object and the second multimodal features of the candidate object are obtained.

[0094] In some embodiments, taking the scenario of in-game friend recommendation as an example, the multimodal features of the target object or candidate object may include at least one of the following: user avatar features, user nickname features, user social graph features, user behavior features, and interaction features.

[0095] For example, user behavioral characteristics may include at least one feature from multiple dimensions such as online time, number of exposures, game level, gender, and age; interaction characteristics may include at least one feature from multiple dimensions such as the number of mutual friends, number of games played together, historical game duration, game affinity, and PageRank score; user social graph characteristics can be extracted from the social graph constructed based on the user's social relationships. Among these, user behavioral and interaction characteristics can be obtained from user logs.

[0096] In step 1012, the first multimodal feature is vectorized to obtain the first multimodal feature vector.

[0097] In some embodiments, the first multimodal features include text features, image features, and social features, see [link to relevant documentation]. Figure 3C , Figure 3C This is a schematic diagram of the third process of the information recommendation method provided in the embodiments of this application. Figure 3B Step 1012 can be achieved through Figure 3C Steps 10121 to 10125 are implemented, and the details are explained below.

[0098] In step 10121, the text of the target object is feature extracted based on the pre-trained language understanding model to obtain text features.

[0099] In some embodiments, the text of the target object can be at least one of a nickname, a personal profile, or a personal tag.

[0100] In some embodiments, see Figure 3D , Figure 3D This is a schematic diagram of the fourth process of the information recommendation method provided in this application embodiment. Before step 10121, the following steps are executed: Figure 3D Steps 201 to 205 are explained in detail below.

[0101] In step 201, the initialized language understanding model and the third training set are obtained. The third training set includes multiple third training samples. Each third training sample includes one positive sample and multiple negative samples. The positive sample is a sentence pair with a similarity greater than the similarity threshold, and the negative sample is a sentence pair with a similarity less than the similarity threshold.

[0102] In some embodiments, the language understanding model includes a Bidirectional Encoder Representation from Transformers (BERT) model, a Bag-of-Words model, a Recursive Neural Network (RNN), a Long Short-Term Memory (LSTM) network, a Generative Adversarial Network (GAN), and a Word Embedding model. Initializing the representation involves randomly assigning values ​​to the parameters of the language understanding model, for example, assigning all parameters to 0 or all parameters to 1.

[0103] In some embodiments, a positive sample refers to a sentence pair consisting of two sentences with a similarity greater than a similarity threshold, and a negative sample refers to a sentence pair consisting of two sentences with a similarity less than a similarity threshold.

[0104] For example, if the similarity threshold is 0.7, sentence A "I like to eat green apples" and sentence B "I like to eat green mangoes" have a similarity of 0.8, so the sentence pair consisting of sentences A and B is a positive sample. Sentence C "I fly a kite in the afternoon" and sentence D "I want to go sing" have a similarity of 0.2, so the sentence pair consisting of sentences C and D is a negative sample.

[0105] In step 202, the positive samples are vectorized to obtain positive sample vectors.

[0106] In some embodiments, one-hot encoding, bag-of-words model, term frequency-inverse document frequency (TF-IDF), or N-gram model can be used to vectorize each sentence in the positive sample to obtain the positive sample vector.

[0107] For example, taking the bag-of-words model, for each sentence in the positive samples, the sentence is decomposed into words or phrases, the frequency of each word in the sentence is counted, and the frequency of each word is combined to form the positive sample vector corresponding to the sentence.

[0108] In step 203, the negative samples are vectorized to obtain negative sample vectors.

[0109] In some embodiments, one-hot encoding, bag-of-words model, term frequency-inverse document frequency, or N-gram model can be used to vectorize each sentence in the negative sample to obtain the negative sample vector.

[0110] For example, taking term frequency-inverse document frequency as an example, for each sentence in the negative samples, the number of times each word appears in the sentence is calculated, the inverse document frequency of each word in the sentence is determined by the inverse proportional function, the product of the number of times each word appears and the inverse document frequency is calculated, and the products corresponding to each word are combined to form the negative sample vector corresponding to the sentence.

[0111] In step 204, a third loss value is determined based on the positive and negative sample vectors.

[0112] In some embodiments, a pre-set similarity threshold is obtained, and the first similarity between two sentence vectors in the positive sample vector and the second similarity between two sentence vectors in the negative sample vector are calculated. The first similarity is greater than the similarity threshold, and the second similarity is less than the similarity threshold. Manually labeled positive and negative ground truth labels are obtained for both the positive and negative sample vectors. Ground truth labels indicate whether two input sentence vectors are similar; positive ground truth labels indicate that two sentence vectors in the positive sample vector are similar, and negative ground truth labels indicate that two sentence vectors in the negative sample vector are dissimilar. Based on the positive and negative sample vectors, the initialized language understanding model is invoked to obtain the positive predicted label for the positive sample vector and the negative predicted label for the negative sample vector. Then, a third loss value is calculated based on the positive and positive predicted labels using a loss function, and the third loss value is also calculated based on the negative and negative predicted labels.

[0113] For example, the loss function can be the mean squared error loss function, the cross-entropy loss function, the multi-label classification loss function, and the triplet loss function.

[0114] In step 205, the parameters of the initialized language understanding model are updated based on the third loss value to obtain the pre-trained language understanding model.

[0115] In some embodiments, the third loss value is backpropagated to update the parameters of the initialized language understanding model. The third loss value is calculated and the parameters are updated iteratively multiple times until the third loss value no longer converges, at which point the iteration process stops, and a pre-trained language understanding model is formed based on the updated parameters.

[0116] For example, backpropagation is implemented using the backpropagation algorithm, which calculates the gradient of each neuron from the output layer to the input layer and updates the neuron's weights and biases based on the gradients. Gradient descent is used to continuously update the parameters, thus reducing the loss value. Various gradient descent algorithms can be used, such as batch gradient descent, stochastic gradient descent, adaptive gradient descent, and momentum gradient descent.

[0117] For an example of steps 201 to 205, see Figure 4 , Figure 4This is a schematic diagram illustrating the training process of the language understanding model provided in an embodiment of this application. Figure 4 In this process, positive and negative samples are first vectorized to obtain positive and negative sample vectors. Then, a third loss value is determined for the positive and negative sample vectors through a loss function to update the parameters of the initialized language understanding model. Finally, the parameters are iteratively updated through backpropagation to form a pre-trained language understanding model.

[0118] This application embodiment trains the language understanding model using positive and negative samples, making the sentence distances in the positive samples closer than those in the negative samples. This allows the extracted text features to better reflect semantic information, improving the accuracy and representativeness of text feature extraction.

[0119] See also Figure 3C In step 10122, feature extraction is performed on the image of the target object to obtain image features.

[0120] In some embodiments, the image of the target object can be a portrait of the target object or a published photo. Features can be extracted from the image of the target object using Transformer encoders, Convolutional Neural Networks (CNNs), Feature Pyramid Networks (FPNs), Attention Models, Graph Neural Networks (GNNs), and Generative Adversarial Networks (GANs) to obtain image features. Among these, Convolutional Neural Networks include Visual Geometry Group Net (VGGNet), Residual Networks (ResNet), and lightweight deep neural networks (MobileNet) designed for embedded devices such as mobile phones.

[0121] For example, using a Transformer encoder, see [link / reference]. Figure 5 , Figure 5This is a schematic diagram of the Transformer encoder provided in this application embodiment. For an image of a target object, the image is first segmented into fixed-size image patches. Then, the segmented image patches are flattened into a sequence, which is a fixed-length vector sequence, where each vector represents an image patch. Next, an embedding layer is applied to each element of the sequence to introduce positional information and other contextual information. For image patches, the embedding layer can include positional embedding and linear embedding. Finally, the Transformer encoder processes the embedded sequence, using a multi-head attention mechanism to assign weights to each element in the embedded sequence and performing a weighted summation to obtain a new sequence representation. Then, the new sequence representation is mapped to a high-dimensional space through fully connected layers and intermediate layers in a feedforward neural network to capture the complex relationships between elements in the sequence, ultimately obtaining image features.

[0122] In step 10123, social features corresponding to the target object are extracted from the pre-constructed social graph.

[0123] In some embodiments, a social graph is constructed based on the target object's friends and friends who have direct or indirect connections with those friends, and social features corresponding to the target object are extracted from the social graph.

[0124] For example, the method for extracting the social features corresponding to the target object can refer to the example in step 10122 above, that is, to extract features from the social graph through the Transformer encoder to obtain the social features corresponding to the target object.

[0125] In step 10124, the text features, image features, and social features are vectorized to obtain text feature vectors, image feature vectors, and social feature vectors, respectively.

[0126] In some embodiments, one-hot encoding, bag-of-words model, term frequency-inverse document frequency, or N-gram model can be used to vectorize text features to obtain text feature vectors. Pre-trained deep learning models, principal component analysis (PCA), and graph neural networks (GNNs) can be used to vectorize image features and social features respectively to obtain image feature vectors and social feature vectors.

[0127] In step 10125, the text feature vector, image feature vector, and social feature vector are combined into a first multimodal feature vector.

[0128] In some embodiments, the text feature vector, image feature vector, and social feature vector are combined sequentially to obtain the first multimodal feature vector.

[0129] For example, if the text feature vector is a, the image feature vector is b, and the social feature vector is c, then [a; b; c] will be used as the first multimodal feature vector.

[0130] In some embodiments, in addition to text features, image features, and social features, behavioral features and interaction features can also be extracted from user logs, and the behavioral features and interaction features can be vectorized to obtain behavioral feature vectors and interaction feature vectors. The text feature vectors, image feature vectors, social feature vectors, behavioral feature vectors, and interaction feature vectors are then combined in sequence to obtain the first multimodal feature vector.

[0131] See also Figure 3B In step 1013, the second multimodal features are vectorized to obtain the second multimodal feature vector.

[0132] In some embodiments, the second multimodal feature also includes text features, image features, and social features. Referring to the process of steps 10121 to 10125 above, the text features, image features, and social features included in the second multimodal feature are vectorized respectively to obtain the text feature vector, image feature vector, and social feature vector corresponding to the second multimodal feature. These vectors are combined to obtain the second multimodal feature vector.

[0133] In step 1014, the first multimodal feature vector and the second multimodal feature vector are combined into a vector combination.

[0134] For example, if the first multimodal feature vector is [a; b; c] and the second multimodal feature vector is [d; e; f], then the vector combination is [[a; b; c]; [d; e; f]].

[0135] This application embodiment extracts features from multiple modalities of the target object and candidate objects respectively, and vectorizes the extracted features to form a vector combination. This reflects the features of the target object and candidate objects from multiple perspectives, making the vector combination used for subsequent operations more representative and diverse, and improving the accuracy of recommendations.

[0136] See also Figure 3A In step 102, dimensionality reduction is performed based on vector combination to obtain the first output vector.

[0137] In some embodiments, the first multimodal feature vector includes a plurality of first feature vectors, the second multimodal feature vector includes a plurality of second feature vectors, the plurality of first feature vectors correspond to a plurality of first modes, the plurality of second feature vectors correspond to a plurality of second modes, and one first mode corresponds to one second mode.

[0138] For example, in a game scenario, the first modality includes the target object's avatar, nickname, pre-built social graph, behaviors, and interaction history. Similarly, the second modality includes the candidate object's avatar, nickname, pre-built social graph, behaviors, and interaction history. One first modality corresponds to one second modality; for example, the target object's avatar corresponds to the candidate object's avatar.

[0139] In some embodiments, see Figure 3E , Figure 3E This is a schematic diagram of the fifth process of the information recommendation method provided in the embodiments of this application. Figure 3A Step 102 can be achieved through Figure 3E Steps 1021 to 1022 are implemented, and the details are explained below.

[0140] In step 1021, for each first feature vector, the following processing is performed: a second candidate mode of the same type as the first mode corresponding to the first feature vector is determined; a second candidate feature vector corresponding to the second candidate mode is determined; a first fusion vector is determined based on the first feature vector and the second candidate feature vector; and pooling is performed based on the first fusion vector to obtain a first output sub-vector.

[0141] Following the example of step 102 above, if the first modality corresponding to the first feature vector is the avatar of the target object, then the second candidate modality of the same type as the first modality is the avatar of the candidate object. The second candidate feature vector corresponding to the avatar of the candidate object is determined. Based on the first feature vector corresponding to the avatar of the target object and the second candidate feature vector corresponding to the avatar of the candidate object, the first fusion vector is determined. Pooling is performed based on the first fusion vector to obtain the first output sub-vector.

[0142] In some embodiments, see Figure 3F , Figure 3F This is a schematic diagram of the sixth process of the information recommendation method provided in the embodiments of this application. Figure 3E Step 1021, "Determine the first fusion vector based on the first feature vector and the second candidate feature vector," can be achieved through... Figure 3F Steps 10211 to 10216 are implemented, and the details are explained below.

[0143] In step 10211, the first feature vector is augmented with channels based on the pre-trained parameter matrix to obtain the first augmented matrix.

[0144] In some embodiments, for each element of the first feature vector, the first feature vector is multiplied and summed element-wise with the pre-trained parameter matrix (i.e., a convolution operation) to obtain a new multidimensional vector, which is then inserted into the corresponding position in the output feature map. This process continues until all elements have been expanded, resulting in the first expanded matrix.

[0145] For example, if the first feature vector a1 is a feature map of size H*W*64, where H represents the height of the feature map, W represents the width of the feature map, and 64 represents the number of channels, and the size of the pre-trained parameter matrix is ​​128*64, then for each element of the feature map, it is regarded as a 64-dimensional vector. This 64-dimensional vector is multiplied by the pre-trained parameter matrix to obtain a new 128-dimensional vector. This new 128-dimensional vector is inserted into the corresponding position of the original feature map to obtain the first expansion matrix. The size of the first expansion matrix is ​​H*W*128, that is, the first feature vector is expanded by the pre-trained parameter matrix to obtain the first expansion matrix b1. Here, the number of columns (64) of the pre-trained parameter matrix is ​​equal to the last dimension (64) of the feature map, and the number of rows (128) of the pre-trained parameter matrix is ​​equal to the last dimension (128) of the first expansion matrix. This means that each 128-dimensional row vector of the pre-trained parameter matrix is ​​used to expand each 64-dimensional vector in the feature map.

[0146] In step 10212, the first extended matrix is ​​nonlinearly mapped to obtain the first mapping matrix.

[0147] Following the example of step 10211 above, an activation function, such as Sigmoid, ReLU, or Tanh, is applied to each element in the first extended matrix b1, and the matrix obtained after nonlinear mapping is used as the first mapping matrix c1.

[0148] In step 10213, the second candidate feature vector is augmented with channels based on the pre-trained parameter matrix to obtain the second augmented matrix.

[0149] Similar to the example in step 10211 above, the second candidate feature vector a2 is augmented with channels based on the pre-trained parameter matrix to obtain the second augmented matrix b2.

[0150] In step 10214, the second extended matrix is ​​nonlinearly mapped to obtain the second mapping matrix.

[0151] Similar to the example in step 10212 above, an activation function, such as Sigmoid, ReLU, or Tanh, is applied to each element in the second extended matrix b2, and the matrix obtained after nonlinear mapping is used as the second mapping matrix c2.

[0152] In step 10215, the first mapping matrix and the second mapping matrix are fused to obtain a fused matrix.

[0153] In some embodiments, fusion can be the addition, combination, or weighted summation of the first mapping matrix and the second mapping matrix.

[0154] Following the examples from steps 10211 to 10214 above, taking addition as an example, the first mapping matrix c1 and the second mapping matrix c2 can be added together to obtain the fusion matrix c1+c2; taking combination as an example, the first mapping matrix c1 and the second mapping matrix c2 can be combined to obtain the fusion matrix [c1; c2]; taking weighted summation as an example, the weight of the first mapping matrix c1 is set to t1, and the weight of the second mapping matrix c2 is set to t2, then the fusion matrix is ​​c1*t1+c2*t2.

[0155] In step 10216, the fusion matrix is ​​nonlinearly mapped to obtain the first fusion vector.

[0156] In some embodiments, an activation function, such as Sigmoid, ReLU, or Tanh, is applied to each element in the fusion matrix, and the vector obtained after nonlinear mapping is used as the first fusion vector.

[0157] This application's embodiments perform channel augmentation on the first feature vector and the second candidate feature vector based on a pre-trained parameter matrix. This introduces more parameter information into the original vectors without adding extra dimensions, helping to increase the model's nonlinear capability and enabling it to learn more complex features. Performing nonlinear mapping after augmentation enhances the model's expressive power. By mapping the original vectors to a new space, the model can capture complex relationships and patterns that may be hidden within the original vectors. Fusing the two nonlinearly mapped vectors integrates their complementary information, helping the model better understand multiple aspects of the data, thereby improving model performance.

[0158] In some embodiments, see Figure 3G , Figure 3G This is a schematic diagram of the seventh process of the information recommendation method provided in the embodiments of this application. Figure 3E Step 1021, "Pooling based on the first fusion vector to obtain the first output sub-vector," can be achieved through... Figure 3G Steps 10217 to 10219 are implemented, and the details are explained below.

[0159] In step 10217, pooling is performed on the rows in the first fusion vector to obtain the row attention vector of the target object for the candidate object.

[0160] In some embodiments, the mean of the elements in each row of the first fusion vector can be calculated as the element at the corresponding position in the row attention vector. For example, the mean of the elements in the first row of the first fusion vector is used as the first element of the row attention vector, the mean of the elements in the second row of the first fusion vector is used as the second element of the row attention vector, and so on.

[0161] For example, the first fusion vector is {[1, 2, 3], [4, 5, 6]}, the mean of the first row is (1+2+3) / 3 = 2, the mean of the second row is (4+5+6) / 3 = 5, then the row attention vector d of the target object for the candidate object is [2, 5].

[0162] In step 10218, pooling is performed on the columns of the first fusion vector to obtain the column attention vector of the candidate object for the target object.

[0163] In some embodiments, the mean of the elements in each column of the first fusion vector can be calculated as the element at the corresponding position in the column attention vector. For example, the mean of the elements in the first column of the first fusion vector is used as the first element of the column attention vector, the mean of the elements in the second column of the first fusion vector is used as the second element of the column attention vector, and so on.

[0164] Following the example of step 10217 above, the mean of the first column is (1+4) / 2 = 2.5, the mean of the second column is (2+5) / 2 = 3.5, and the mean of the third column is (3+6) / 2 = 4.5. Therefore, the column attention vector e of the candidate object for the target object is [2.5, 3.5, 4.5].

[0165] In step 10219, the first feature vector and the second candidate feature vector are fused using the row attention vector and the column attention vector as weights to obtain the first output sub-vector.

[0166] In some embodiments, a first product of the row attention vector and the first feature vector is determined; a second product of the column attention vector and the second candidate feature vector is determined; and the sum of the first product and the second product is determined as a first output sub-vector.

[0167] Following the examples of steps 10211 to 10217 above, determine the first product d*a1 of the row attention vector d and the first feature vector a1; determine the second product e*a2 of the column attention vector e and the second candidate feature vector a2; determine the sum of the first product and the second product d*a1+e*a2 as the first output sub-vector.

[0168] This embodiment of the application obtains a first output sub-vector through mean pooling, which can reduce model parameters and computational load, thereby lowering computational complexity. During mean pooling, important information in the first fused vector can be preserved, while reducing the risk of overfitting, allowing the model to focus more on representing overall features. Simultaneously, noisy features can be deweighted during pooling, effectively improving representational capability.

[0169] In some embodiments, the pre-trained parameter matrix is ​​the parameter of the first machine learning model, see [link to documentation]. Figure 3H , Figure 3H This is a schematic diagram of the eighth process of the information recommendation method provided in this application embodiment. Before step 1021, the following steps are executed: Figure 3H Steps 301 to 308 are explained in detail below.

[0170] In step 301, the first initialized machine learning model is obtained.

[0171] In some embodiments, the first machine learning model can be a Transformer model, a Latent Dirichlet Allocation (LDA) model, or a Multilayer Perceptron (MLP). Initializing the representation involves randomly assigning values ​​to the parameters of the first machine learning model, for example, assigning all parameters of the first machine learning model a value of 0 or all parameters a value of 1.

[0172] In step 302, multiple object combinations and a first true label are obtained. Each object combination includes an object sample and a sample of objects to be recommended. The sample of objects to be recommended is the object to be recommended to the object sample. The first true label represents the probability of recommending the sample of objects to the object sample.

[0173] For example, taking a game scenario, users logged into their game client accounts are considered object samples. The objects recommended to these object samples in the game's friend recommendation interface are considered candidate object samples. An object sample and its corresponding candidate object sample are grouped together. The first real label is calculated for each object combination based on preset criteria, and then manually labeled.

[0174] In step 303, the object sample is vectorized to obtain the object sample vector.

[0175] In some embodiments, one-hot encoding, bag-of-words model, term frequency-inverse document frequency, or N-gram model can be used to vectorize object samples to obtain object sample vectors.

[0176] For example, taking the bag-of-words model, for an object sample, the object sample is decomposed into words or phrases, the frequency of each word in the object sample is counted, and the frequency of each word is combined to form the object sample vector corresponding to that object sample.

[0177] In step 304, the sample of the object to be recommended is vectorized to obtain the sample vector of the object to be recommended.

[0178] In some embodiments, one-hot encoding, bag-of-words model, term frequency-inverse document frequency, or N-gram model can be used to vectorize the sample of the object to be recommended, thereby obtaining the sample vector of the object to be recommended.

[0179] For example, taking term frequency-inverse document frequency as an example, for the sample of objects to be recommended, the number of times each word appears in the sample of objects to be recommended is calculated, the inverse document frequency of each word in the sample of objects to be recommended is determined by the inverse proportional function, the product of the number of times each word appears and the inverse document frequency is calculated, and the products corresponding to each word are combined to form the sample vector of objects to be recommended.

[0180] In step 305, the object sample vector is augmented with channels based on the initialized parameter matrix to obtain the object sample augmented vector.

[0181] For example, if the object sample vector A is a 3D vector, to obtain the expanded object sample vector B, each element of the initialized parameter matrix P can be copied to a new channel. If the dimension of the initialized parameter matrix P is M*N, then the dimension of the expanded vector B will be (M+1)*3, where M+1 represents the new number of channels, that is, the number of channels in the object sample vector A plus the number of channels in the parameter matrix P.

[0182] In step 306, the sample vector of the object to be recommended is expanded based on the initialized parameter matrix to obtain the expanded sample vector of the object to be recommended.

[0183] For example, the sample vector C of the object to be recommended is a 3-dimensional vector. To obtain the expanded sample vector D of the object to be recommended, each element of the initialized parameter matrix P can be copied to a new channel. If the dimension of the initialized parameter matrix P is M*N, then the dimension of the expanded vector D will be (M+1)*3, where M+1 represents the new number of channels, that is, the number of channels in the object sample vector C plus the number of channels in the parameter matrix P.

[0184] In step 307, the first machine learning model initialized based on the vector pair is invoked to obtain the first predicted label, wherein the vector pair includes the object sample augmentation vector and the object sample augmentation vector to be recommended.

[0185] For example, following the examples of steps 305 and 306 above, the object sample augmentation vector B and the object sample augmentation vector D to be recommended are combined into a vector pair [B; D]. Based on the vector pair [B; D], the initialized first machine learning model is called to obtain the first predicted label.

[0186] In step 308, a first loss value is determined based on the first true label and the first predicted label, the parameters of the initialized first machine learning model are updated based on the first loss value, and the updated parameters are combined into a pre-trained parameter matrix.

[0187] In some embodiments, the difference between the first true label and the first predicted label is calculated as the first loss value. The first loss value is backpropagated to update the parameters of the initialized first machine learning model. The process of calculating the first loss value and updating the parameters is repeated multiple times until the first loss value no longer increases or decreases. The iteration process is then stopped, forming the trained first machine learning model. The parameters of the trained first machine learning model are combined into a pre-trained parameter matrix.

[0188] For example, the loss function can be the mean squared error loss function, the cross-entropy loss function, the multi-label classification loss function, and the triplet loss function.

[0189] For example, backpropagation is implemented using the backpropagation algorithm, which calculates the gradient of each neuron from the output layer to the input layer and updates the neuron's weights and biases based on the gradients. Gradient descent is used to continuously update the parameters, thus reducing the loss value. Various gradient descent algorithms can be used, such as batch gradient descent, stochastic gradient descent, adaptive gradient descent, and momentum gradient descent.

[0190] For an example of steps 301 to 308, see Figure 6 , Figure 6 This is a schematic diagram illustrating the principle of training the first machine learning model provided in an embodiment of this application. Figure 6 In this process, the object sample and the object sample to be recommended are vectorized respectively to obtain the object sample vector and the object sample vector to be recommended. Then, the channels of the object sample vector and the object sample vector to be recommended are expanded using the initialized parameter matrix to obtain the object sample expanded vector and the object sample expanded vector to be recommended. The object sample expanded vector and the object sample expanded vector to be recommended are combined into a vector pair. Based on the vector pair, the convolutional layer and fully connected layer of the initialized first machine learning model can be called to obtain the first predicted label. The first loss value is calculated based on the first true label and the first predicted label. The parameters of the initialized first machine learning model are updated based on the first loss value to obtain the trained first machine learning model. The parameters of the trained first machine learning model are combined into a pre-trained parameter matrix.

[0191] See also Figure 3E In step 1022, the multiple first output sub-vectors corresponding to the multiple first feature vectors are fused to obtain the first output vector.

[0192] In some embodiments, a first output vector is obtained by assigning different weights to multiple first sub-vectors using a multi-head attention mechanism and then performing a weighted summation based on the weights.

[0193] For example, if the multiple first feature vectors correspond to multiple first output sub-vectors g1, g2, and g3, with corresponding weights of 1, 2, and 3 respectively, then the first output vector is 1*g1+2*g2+3*g3.

[0194] See also Figure 3A In step 103, attention encoding is performed based on the first multimodal feature vector to obtain the first set of weight coefficients corresponding to the first multimodal vector, and the second output vector representing the individual interest of the target object is determined based on the first set of weight coefficients.

[0195] In some embodiments, see Figure 3I , Figure 3I This is a ninth flowchart illustrating the information recommendation method provided in the embodiments of this application. Figure 3A Step 103, "Perform attention encoding based on the first multimodal feature vector to obtain the first set of weight coefficients corresponding to the first multimodal vector," can be achieved through... Figure 3I Steps 1031 to 1032 are implemented, and the details are explained below.

[0196] In step 1031, the following processing is performed for each first feature vector: the similarity between the first feature vector and other feature vectors is determined, the similarity is normalized to obtain a normalized similarity, and the product of the normalized similarity and the first feature vector is determined as the first weight coefficient of the first feature vector, wherein the other feature vectors are feature vectors other than the first feature vector among multiple first feature vectors.

[0197] In some embodiments, the similarity between the first feature vector and other feature vectors can be calculated using cosine similarity, Euclidean distance, Manhattan distance, or Pearson correlation coefficient. The similarity can be normalized using a loss function to obtain a normalized similarity. The loss function can be a mean squared error loss function, a cross-entropy loss function, a multi-label classification loss function, or a triplet loss function.

[0198] For example, taking cosine similarity as an example, we determine the vector product of the first feature vector and other feature vectors, determine the length product of the lengths of the first feature vector and other feature vectors, and determine the ratio of the vector product to the length product as the similarity.

[0199] For example, if the first feature vector is m and the other feature vectors are n, then the vector product is m*n. The length of the first feature vector is |m|, and the lengths of the other feature vectors are |n|. The product of their lengths is |m|*|n|, so the similarity is (m*n) / (|m|*|n|).

[0200] For example, if the normalized similarity is 0.8 and the first feature vector is a1, then the first weight coefficient of the first feature vector is 0.8*a1.

[0201] In step 1032, the multiple first weight coefficients corresponding to the multiple first feature vectors are combined into a first weight coefficient set.

[0202] For example, if multiple first feature vectors are a1, a2, and a3, and the multiple first weight coefficients corresponding to the multiple first feature vectors are 0.8*a1, 0.6*a2, and 0.9*a3, then the set of first weight coefficients is [0.8*a1; 0.6*a2; 0.9*a3].

[0203] In some embodiments, Figure 3A Step 103, "Determine the second output vector representing the individual interest of the target object based on the first set of weight coefficients", can be achieved by performing the following operations: according to the multiple first weight coefficients corresponding to the multiple first feature vectors in the first set of weight coefficients, the multiple first feature vectors are fused to obtain the second output vector representing the individual interest of the target object.

[0204] Following the example of step 1032 above, the second output vector representing the individual interest of the target object is 0.8*a1*a1+0.6*a2*a2+0.9*a3*a3.

[0205] This application embodiment obtains a second output vector representing the individual interests of the target object by weighted summation of multiple first feature vectors corresponding to the target object, learns the correlation and importance between features of multiple modalities of the target object, reflects the tendency of the target object, and thus reflects the individual interests of the target object.

[0206] See also Figure 3A In step 104, feature extraction is performed on the first multimodal feature vector to obtain a third output vector representing the group interest of the target object's group.

[0207] In some embodiments, the third output vector is obtained by feature extraction from the first multimodal feature vector using a pre-trained set of convolutional kernels. The pre-trained set of convolutional kernels includes row convolutional kernels and column convolutional kernels. (See [link to documentation]). Figure 3J , Figure 3J This is a schematic diagram of the tenth process of the information recommendation method provided in the embodiments of this application. Figure 3A Step 104 can be achieved through Figure 3J Steps 1041 to 1044 are implemented, and the details are explained below.

[0208] In step 1041, feature mapping is performed on each first feature vector of the target object to obtain a mapped feature vector.

[0209] In some embodiments, a mapping function is constructed and the dimension of the mapping is determined. The first feature vector is input into the pre-constructed mapping function to obtain the mapped feature vector. The mapping function can be linear or non-linear. For example, a linear transformation, a polynomial transformation, an exponential transformation, or a neural network can be used.

[0210] In step 1042, the multiple mapping feature vectors corresponding to the multiple first feature vectors are combined according to the row direction to obtain the combined feature vector.

[0211] For example, suppose we have two mapping feature vectors, vector A[1, 2, 3] and vector B[4, 5, 6, 7], with dimensions of 3 and 4 respectively. Vectors A and B have different lengths, so when combining them, we need to pad the shorter vector with zeros to make them the same length. The zero-padding vector A is [1, 2, 3, 0]. Then, we combine the two vectors along the row direction to obtain the combined feature vector C as [[1, 2, 3, 0]; [4, 5, 6, 7]].

[0212] In step 1043, the third product of the combined feature vector and the transpose of the column convolution kernel is determined.

[0213] Following the example from step 1042 above, if the column convolution kernel is K1, then the transpose of the column convolution kernel is... The third product is

[0214] In step 1044, the third product and the fourth product of the row convolution kernel are determined as the third output vector representing the group interest of the target object's group.

[0215] Following the example from step 1043 above. If the row convolution kernel is K2, then the fourth product is... The third output vector is used to represent the group interest of the group to which the target object belongs.

[0216] In some embodiments, the pre-trained convolutional kernel set is the parameter of a second machine learning model, see [link to relevant documentation]. Figure 3K , Figure 3K This is a schematic flowchart of the eleventh step of the information recommendation method provided in this application embodiment. Before step 104, the following steps are executed: Figure 3B Steps 401 to 406 are explained in detail below.

[0217] In step 401, an initialized second machine learning model and training data are obtained, wherein each training data includes initial features of multiple modalities of an object sample.

[0218] In some embodiments, the second machine learning model can be a Transformer model, a topic model (LDA), or an MLP. Initializing the representation involves randomly assigning values ​​to the parameters of the second machine learning model, for example, assigning all parameters of the second machine learning model a value of 0 or all parameters a value of 1.

[0219] In step 402, feature mapping is performed on each initial feature to obtain a training feature vector.

[0220] In some embodiments, a mapping function is constructed and the dimension of the mapping is determined. Initial features are input into the pre-constructed mapping function to obtain a training feature vector. The mapping function can be linear or non-linear, for example, a linear transformation, a polynomial transformation, an exponential transformation, or a neural network can be used.

[0221] In step 403, for each training data, the multiple training feature vectors corresponding to the multiple modalities are combined in the row direction to obtain the sample feature vector.

[0222] For example, for each training data, suppose there are two training feature vectors, namely vector Y[4, 6] and vector Z[2, 9], both of which have a dimension of 2. Combining the two vectors in the row direction, we get the sample feature vector W as [[4, 6]; [2, 9]].

[0223] In step 404, the initialized second machine learning model is invoked based on the sample feature vector to obtain the predicted output.

[0224] In some embodiments, the predicted output is obtained by invoking the convolutional and fully connected layers of a second machine learning model initialized based on the sample feature vector.

[0225] In step 405, the output loss value is determined based on the true output and the predicted output, wherein the true output represents the dot product of the sample feature vector and the initialized convolutional kernel group.

[0226] In some embodiments, the difference between the true output and the predicted output is calculated as the output loss value. The dot product of the sample feature vector and the initialized convolutional kernel group is the sum of the product of the corresponding components of the sample feature vector and the initialized convolutional kernel group.

[0227] Continuing with the example from step 403 above, the sample feature vector W is [[4, 6]; [2, 9]]. The initialized convolutional kernel group has row convolution kernels of [1, 0, 0, 2] and column convolution kernels of [0, 3, 0, 4]. Therefore, the dot product of the sample feature vector W and the convolutional kernel group is 4*1*0 + 6*0*3 + 2*0*0 + 9*2*4 = 72, meaning the true output is 72. If the predicted output is 70, then the output loss value is 2.

[0228] In step 406, the parameters of the initialized second machine learning model are updated based on the output loss value, and the updated parameters are combined into a pre-trained convolutional kernel group.

[0229] In some embodiments, the output loss value is backpropagated to update the parameters of the initialized second machine learning model. The process of calculating the output loss value and updating the parameters is repeated multiple times until the output loss value no longer increases or decreases. The iteration process is then stopped to form a trained second machine learning model. The parameters of the trained second machine learning model are combined into a pre-trained convolutional kernel group.

[0230] For example, the loss function can be the mean squared error loss function, the cross-entropy loss function, the multi-label classification loss function, and the triplet loss function.

[0231] For example, backpropagation is implemented using the backpropagation algorithm, which calculates the gradient of each neuron from the output layer to the input layer and updates the neuron's weights and biases based on the gradients. Gradient descent is used to continuously update the parameters, thus reducing the loss value. Various gradient descent algorithms can be used, such as batch gradient descent, stochastic gradient descent, adaptive gradient descent, and momentum gradient descent.

[0232] For examples of steps 401 to 406, see Figure 7 , Figure 7 This is a schematic diagram illustrating the principle of training a second machine learning model according to an embodiment of this application. Figure 7 In this process, feature mapping is performed on multiple initial features to obtain training feature vectors corresponding to each initial feature. Then, multiple training feature vectors are combined to obtain sample feature vectors. Based on the sample feature vectors, the convolutional and fully connected layers of the initialized second machine learning model can be called to obtain the predicted output. The output loss value is calculated based on the true output and the predicted output. The parameters of the initialized second machine learning model are updated based on the output loss value to obtain the trained second machine learning model. The parameters of the trained second machine learning model are combined into a pre-trained convolutional kernel group.

[0233] This application embodiment determines the modal features that are closer to the convolution kernel group among the multimodal features corresponding to multiple candidate objects by learning the distance (i.e., dot product) between multiple features and the convolution kernel group, thereby reflecting the group interests of the target object's group.

[0234] See also Figure 3A In step 105, recommended parameters are determined based on the first output vector, the second output vector, and the third output vector.

[0235] In some embodiments, see Figure 3L , Figure 3LThis is a schematic diagram of the twelfth step of the information recommendation method provided in the embodiments of this application. Figure 3A Step 105 can be achieved through Figure 3L Steps 1051 to 1052 are implemented, and the details are explained below.

[0236] In step 1051, the first output vector, the second output vector, and the third output vector are combined into an input vector.

[0237] For example, if the first output vector is X1, the second output vector is X2, and the third output vector is X3, then the input vector is [X1; X2; X3].

[0238] In step 1052, a pre-trained third machine learning model is invoked based on the input vector to perform prediction processing and obtain the recommended parameters.

[0239] In some embodiments, taking a game scenario as an example, the recommendation parameters can characterize the degree of matching between the target object and the object to be recommended.

[0240] In some embodiments, the recommended parameters are determined by a pre-trained third machine learning model, see [link to relevant documentation]. Figure 3M , Figure 3M This is a schematic flowchart of the information recommendation method provided in this application embodiment. Before step 1052, the following steps are executed: Figure 3M Steps 501 to 503 are explained in detail below.

[0241] In step 501, an initialized third machine learning model and a second training set are obtained. The second training set includes multiple second training samples, each of which includes a sample input vector and a second true label. The second true label represents the true recommendation parameters.

[0242] In some embodiments, the third machine learning model can be a Transformer model, a topic model (LDA), or an MLP. Initializing the representation involves randomly assigning values ​​to the parameters of the third machine learning model, for example, assigning all parameters of the third machine learning model a value of 0 or all parameters a value of 1.

[0243] In step 502, the initialized third machine learning model is invoked based on the sample input vector to obtain the second predicted label.

[0244] In some embodiments, the second predicted label is obtained by invoking the convolutional and fully connected layers of an initialized third machine learning model based on the sample input vector.

[0245] In step 503, a second loss value is determined based on the second true label and the second predicted label, and the parameters of the initialized third machine learning model are updated based on the second loss value to obtain the pre-trained third machine learning model.

[0246] In some embodiments, the difference between the second true label and the second predicted label is calculated as the second loss value. The second loss value is backpropagated to update the parameters of the initialized third machine learning model. This process of calculating the second loss value and updating the parameters is repeated multiple times until the second loss value no longer increases or decreases. The iteration process is then stopped, and a pre-trained third machine learning model is formed.

[0247] For example, the loss function can be the mean squared error loss function, the cross-entropy loss function, the multi-label classification loss function, and the triplet loss function.

[0248] For example, backpropagation is implemented using the backpropagation algorithm, which calculates the gradient of each neuron from the output layer to the input layer and updates the neuron's weights and biases based on the gradients. Gradient descent is used to continuously update the parameters, thus reducing the loss value. Various gradient descent algorithms can be used, such as batch gradient descent, stochastic gradient descent, adaptive gradient descent, and momentum gradient descent.

[0249] For examples of steps 501 to 503, see Figure 8 , Figure 8 This is a schematic diagram illustrating the principle of training a third machine learning model according to an embodiment of this application. Figure 8 In this process, based on the sample input vector, the convolutional and fully connected layers of the initialized third machine learning model can be called to obtain the second predicted label. Based on the second true label and the second predicted label, the second loss value is calculated. Based on the second loss value, the parameters of the initialized third machine learning model are updated to obtain the pre-trained third machine learning model.

[0250] See also Figure 3A In step 106, a target recommended object for the target object is determined from the multiple candidate objects based on the multiple recommendation parameters corresponding to the multiple candidate objects.

[0251] In some embodiments, see Figure 3N , Figure 3N This is a schematic diagram of the fourteenth process of the information recommendation method provided in the embodiments of this application. Figure 3A Step 106 can be achieved through Figure 3N Steps 1061 to 1063 are implemented, and the details are explained below.

[0252] In step 1061, the multiple recommended parameters are sorted in descending order.

[0253] For example, suppose the recommended parameter for candidate A is 0.3, the recommended parameter for candidate B is 0.8, and the recommended parameter for candidate C is 0.6. The result of sorting in descending order is: 0.8, 0.6, 0.3.

[0254] In step 1062, multiple target recommended parameters are selected from the descending sorted results in descending order.

[0255] In some embodiments, a preset number or a preset proportion of target recommendation parameters can be selected from the descending sorted results in descending order.

[0256] Following the example of step 1061 above, if the preset quantity is 2, then the recommended parameters 0.8 and 0.6 are selected as the target recommended parameters.

[0257] In step 1063, multiple target candidate objects corresponding to multiple target recommendation parameters are determined from multiple candidate objects, and used as target recommendation objects for the target objects.

[0258] Following the examples in steps 1061 and 1062 above, the target recommendation parameters 0.8 and 0.6 correspond to candidate object B and candidate object C, respectively. Therefore, candidate object B and candidate object C are used as the target recommendation objects for the target object.

[0259] This application's embodiments determine recommendation parameters based on the input vector obtained by fusion of three-dimensional vectors, and sort the recommendation parameters in descending order to obtain target recommendation objects. Target recommendation objects with high matching degree with the target objects are determined from multiple dimensions. While ensuring low redundancy, it considers the individual interests of the target objects and the group interests of the target objects' respective groups, ensuring the relevance between the target recommendation objects and the target objects from multiple aspects, thereby improving the accuracy of the recommendations.

[0260] The following will describe an exemplary application of the embodiments of this application in the scenario of information recommendation within a game.

[0261] The embodiments of this application are not limited to the scenario of information recommendation within a game, but are also applicable to other scenarios, such as the scenario of information recommendation in an instant messaging client or the scenario of information recommendation on a social platform.

[0262] Information can be at least one of articles, images, user profiles, or videos. Taking in-game friend recommendations as an example, users in multiplayer cooperative games often need to add friends to play together. Related technologies typically use multimodal features to determine the target recommendation object from multiple candidate objects when making friend recommendations. However, extracting multimodal features of the target object or candidate objects often introduces redundant information. Furthermore, related technologies only consider the individual interests of the target object, limiting the scope of recommendations. The information recommendation method provided in this application determines three different dimensions of output vectors based on the first multimodal feature vector of the target object and the second multimodal feature vector of the candidate objects. These three output vectors are then fused to determine recommendation parameters. Based on these parameters, the target recommendation object is determined. This method not only explores the relationship between the target object and candidate objects but also considers the individual interests of the target object and the group interests of the target object's group. The target recommendation object is determined by multiple dimensions of information, making it more consistent with the target object's interests and improving the accuracy and relevance of the recommendation.

[0263] See Figure 9A , Figure 9A This is a schematic diagram of a user's personal information page provided in an embodiment of this application. Figure 9A In the context of the user's personal information page 901, the user's avatar 902 (i.e., the image of the target object) and name 903 (i.e., the text of the target object) are displayed. See also... Figure 9B , Figure 9B This is a schematic diagram of the user's friend recommendation interface provided in an embodiment of this application. Figure 9B In the friend recommendation interface, a list of people that the user may be interested in is displayed at 904, and sorted by matching degree (i.e. recommendation parameter). For example, user A's matching degree is 86 and user B's matching degree is 83, allowing the user to select the friends they want to add.

[0264] In-game friend recommendation systems refer to the process of identifying a small number of candidates (i.e., target recommended players) from a massive pool of unfamiliar players (i.e., multiple candidate objects) who best match the friend-making preferences of a target player (i.e., the target object) and then ranking them to create a recommendation list. The information recommendation method provided in this application mainly involves two parts: feature extraction and multimodal recommendation. The features to be extracted include user behavior features, interaction features, image features (i.e., picture features), text features, and user social graph features (i.e., social features). Behavioral and interaction features are primarily obtained from the player's (i.e., the target object's) logs. Image and text feature extraction mainly refers to converting a given image or text (here, the user's avatar and nickname) into a feature vector using a feature extractor. Social network feature extraction requires obtaining the graph representation of each player from a pre-constructed social relationship graph using graph neural network technology. These features are then used together in downstream recommendation tasks. For image features, related feature extraction methods require manually designing feature extractors and manually selecting important representation information. To avoid the need for extensive manpower-intensive feature engineering steps, the information recommendation method provided in this application employs deep features. It utilizes a deep neural network, trains a feature extractor based on a large amount of historical data, and automatically learns efficient avatar representations. The model used is a deep convolutional neural network, a type of feedforward neural network that includes convolutional computation and has a deep structure. For text features, game text information is used as corpus to perform a Sentence-BERT pre-training task on the Transformer model, and then the trained feature extractor extracts the user's nickname features. After extracting avatar, nickname, and graph features, information from multiple user modalities is fused to achieve the friend recommendation task. The user's multimodal information (first multimodal features) includes avatar features, nickname features, graph features, behavioral features, and interaction features extracted based on the deep network.

[0265] See Figure 10 , Figure 10 This is a flowchart of the multimodal friend recommendation system provided in an embodiment of this application. Figure 10 In China, the process of a deep learning-based multimodal friend recommendation system includes positive and negative sample sampling, feature extraction, model training, and distributed prediction.

[0266] First, the positive and negative sample sampling process is as follows: Historical data from in-game social recommendations is sampled to obtain multiple positive and negative sample pairs. Positive sample pairs consist of exposed users (i.e., target samples) and candidate friends clicked by the exposed users (i.e., target samples to be recommended). Negative sample pairs consist of exposed users and exposed candidate friends who were not clicked by the exposed users. Because the number of positive sample pairs is far less than that of negative sample pairs, users who have never sent friend requests in the negative sample pairs are filtered out. Simultaneously, the negative sample pairs are randomly downsampled, meaning that historical data from the past week is extracted for model training.

[0267] Secondly, the feature extraction process is as follows: the sample features mainly consist of five parts: user avatar features (i.e., image features), user nickname features (i.e., text features), user social graph features (i.e., social features), user behavior features, and interaction features.

[0268] User behavior characteristics include multiple dimensions such as online time, exposure frequency, game level, gender, and age; interaction characteristics mainly include multiple dimensions such as the number of mutual friends, the number of games played together, the duration of historical games, the overall game affinity, and PageRank score.

[0269] Before extracting user avatar features, an image feature extractor f needs to be obtained, as shown in the following formula (1):

[0270] V=f(I) (1)

[0271] Where I represents the user's avatar, f is the pre-trained image feature model, and V represents the extracted feature vector (i.e., the image feature vector).

[0272] For text features, the Transformer model uses game text as corpus and employs the Sentence-BERT (a language understanding model) method to fine-tune BERT. Sentence-BERT is a Siamese network of pre-trained BERT, capable of obtaining semantically meaningful text vectors. Sentence-BERT was proposed primarily to address the problems of BERT, such as its high time cost in semantic similarity retrieval or the unsuitability of sentence representations obtained by BERT for unsupervised tasks like clustering and sentence similarity calculation. Sentence-BERT uses a Siamese network structure, inputting sentence pairs into two BERT models with shared parameters to obtain character vectors for all words in the sentence. These character vectors are then passed to a pooling layer for average pooling, which calculates the average of all character vectors along the sentence length dimension to obtain the sentence vector representation (i.e., the text feature vector) for each sentence.

[0273] Text or text pairs are fed into the Sentence-BERT model, and the model's output, the embedding vector, is obtained through pre-trained BERT. This embedding vector can be used as a feature of the text for various downstream tasks, such as text classification, sentiment analysis, and similarity calculation.

[0274] To fine-tune BERT, Siamese and triplets networks are used to update parameters, making the generated sentence vectors more semantically informative. The network structure depends on the specific training data and includes the following three structures and objective functions:

[0275] 1. Classification Objective Function: For classification problems, obtain the sentence vectors u and v of the two sentences respectively, combine u and v, and the vector |uv| obtained by subtracting u and v from each other in order, and then multiply the combined vector by a trainable weight W. t ∈R 3n*k Finally, the objective function of formula (2) is optimized as follows:

[0276] o = softmax(W t (u, v, |uv|)) (2)

[0277] Where n is the sentence vector dimension, k is the number of categories, and W t Let be the weights, o be the objective function, u be the sentence vector corresponding to the first sentence, v be the sentence vector corresponding to the second sentence, |uv| be the vector of the difference between u and v, and softmax be the loss function to be calculated. The loss function used here is the cross-entropy loss function.

[0278] 2. Regression Objective Function: This involves calculating the cosine similarity between the sentence vectors of the two input sentences, as shown in formula (3) below. The optimization objective function is the mean squared difference loss (MSE).

[0279] MSE = mean(||V) i -U i || 2 (3)

[0280] Where MSE is the objective function, mean represents the mean calculation, and V i U represents the sentence vector corresponding to the first sentence. i This represents the sentence vector corresponding to the second sentence.

[0281] 3. Triplet Objective Function: The original Siamese network model framework is modified by changing the original two inputs to three sentence inputs. Given an anchor sentence a, an affirmative sentence p, and a negative sentence n, the model optimizes itself by making the distance between a and p less than the distance between a and n, that is, minimizing its objective function o. The objective function is shown in the following formula (4):

[0282] o = max(||S a -S p ||-||S a -S n ||+ε,0) (4)

[0283] Where o represents the objective function, max represents finding the maximum value, a represents the anchoring statement, p represents the affirmative statement, n represents the negative statement, and S represents the target function. a S p and S n Let a, p, and n represent the sentence vectors of sentences a, p, and n respectively, ||·|| represent the distance metric, and ε represent the margin. Here, Euclidean distance is used as the distance metric, and the margin is set to 1.

[0284] Based on the parameters obtained from pre-training, sentence vectors of the input text can be obtained for use in downstream recommendation tasks.

[0285] See Figure 11 , Figure 11 This is an overall framework diagram of the multimodal recommendation model provided in the embodiments of this application. The multimodal recommendation algorithm mainly uses features from five modalities to jointly recommend friends, such as... Figure 1 As shown, the model is mainly divided into three parts: AP-Net, LD-Net, and GD-Net, and their computational details are as follows:

[0286] For AP-Net, an attention pooling mechanism is introduced: to solve the problem of multimodal redundant information, the idea of ​​attention pooling is used to dynamically generate the weights of the corresponding modalities through the pooling mechanism. The advantage is that it can reduce the weights of noisy features during the pooling process, effectively improving the representation ability, and the computational complexity is low because the pooling layer involves fewer learnable parameters. The specific approach is as follows: First, the input user learns a set of extended parameter matrices P and Q (i.e., pre-trained parameter matrices) respectively, and after nonlinear mapping, they are fused to obtain a unified representation G (i.e., the first fusion vector), as shown in formulas (5), (6) and (7):

[0287]

[0288]

[0289]

[0290] Where u and v are the input user pairs, u refers to the user who needs to be recommended to friends, i.e., the target user, v refers to the user recommended to u, and k refers to the k-th modality (i.e., the first modality or the second modality). and It is the parameter matrix used for channel augmentation (i.e., the pre-trained parameter matrix). It is the embedding representation vector of the user's k-th modal information of u and v (i.e., the first feature vector and the second candidate feature vector), tanh represents the nonlinear mapping, and G k It is the unified representation obtained after matrix fusion (i.e., the first fusion vector).

[0291] After obtaining the user's uv fusion matrix, mean pooling operations are performed on the rows and columns respectively to obtain the corresponding attention vector of u to v and the attention vector of v to u, as shown in formulas (8) and (9):

[0292]

[0293]

[0294] Where u and v are the input user pairs, k refers to the k-th modality, and the vector... (i.e., row attention vector) and (i.e., column attention vectors) represent the degree of mutual attention between u and v, and v and u, respectively. `softmax` represents the loss function calculation, `RowMean` represents the row mean pooling operation, `ColMean` represents the column mean pooling operation, and `G`... k This is a unified representation obtained after matrix fusion. Finally, the cross-domain matching information embedding representation vector E of the k-th modality is calculated through attention aggregation. k (i.e., the first output sub-vector), and the final weighted output is obtained, as shown in formula (10):

[0295]

[0296] Where u and v are the input user pairs, and k refers to the k-th modality. It is the embedding representation vector of the user's k-th modal information of u and v, the vector and E represents the degree of mutual interest between u and v, and v and u, respectively. k The information embedding representation vector for cross-domain matching of the k-th modality is represented.

[0297] For LD-Net, since user preferences vary individually, unique individual characteristics should be considered. First, a multi-head self-attention mechanism is used to capture individual preferences. By learning a set of Q, K, and V parameter matrices, the importance of different features for the user's input is captured. Figure 11 The LD-Net part, M i (i∈1, 2, ..., 5) represent the features of different modes. The features of different modes are first combined to obtain X, and X is then combined with the parameter matrix W. Q Wk and W V After performing the calculation as shown in formula (11), E is obtained. local (i.e., the second output vector).

[0298]

[0299] Among them, E local This represents the second input vector, softmax represents the loss function calculation, and W... Q W k and W V Let X represent a three-parameter matrix, where X represents a vector representation of the feature combination of different modalities, and d represents the dimension.

[0300] For GD-Net, a global decision plane (i.e., a pre-trained set of convolutional kernels) is learned in high-dimensional space to learn the global preferences of players in making friends. Since the global decision plane interacts with all training samples, it learns a plane that is closer to the optimal solution for all samples, thus learning a group preference. This global decision plane consists of two sets of learnable convolutional kernels K. row (i.e., row convolution kernel) and K col It consists of (i.e., column convolution kernels). The features of each input modality are mapped by the MLP, then the mapped features of each modality are combined in the row direction, and finally the combined features are subjected to row convolution and column convolution operations respectively to obtain the output feature E. global (i.e., the third output vector), the specific calculation is shown in formula (12):

[0301] E global =∑(∑ k f(M k )·K col T )·K row (12)

[0302] Among them, E global The output features are represented by k, which is the k-th mode, f represents the mapping operation, and M represents the output feature. k K represents the feature vector of the user's k-th modality. row K represents the row convolution kernel of the global decision plane. col Represents the column convolution kernel of the global decision plane.

[0303] During the training phase of GD-Net, convolutional processing is performed on the multimodal features of each training sample to learn the relationship between the multimodal features of each training sample and the global decision plane. If the distance between a certain modality feature of multiple training samples and the global decision plane is less than a distance threshold, it can reflect the group preference of multiple training samples. When using the trained model, the user's multimodal features are directly input to obtain the relationship E between the user's multimodal features and the global decision plane. global This reflects the preferences of the user's group.

[0304] Finally, the three outputs (E) k E local and E global After performing the combination operation, the input is fed into the MLP, and then a one-dimensional value y (i.e., the input vector) is output. The output y is then fed into the activation layer (Sigmoid).

[0305] During model training, to address the imbalanced sample problem and learn difficult samples with greater weight, the loss function focalloss is calculated using the network output and corresponding labels. The classification label y′ represents whether the recommendation is accepted, with 0 indicating no acceptance and 1 indicating acceptance. The training objective is to minimize focalloss, which is defined as shown in formula (13).

[0306]

[0307] Among them, L fl Let f(x) represent the loss function focalloss, α and γ are hyperparameters, which were set to 0.25 and 2 respectively in the experiment, y represents the network output, y′ represents the classification label, and log represents the logarithmic operation.

[0308] During testing, for a user pair, the confidence score of its classification (i.e., the recommendation parameter) is used as the probability of establishing a friendship relationship. Then, for a source user, its target users (i.e., multiple candidate users) are ranked according to the probability of establishing a friendship relationship, and finally, several users with the highest probabilities (i.e., target recommendation objects) are selected for recommendation.

[0309] The embodiments of this application can be applied to recommending social friends of users (i.e. target objects) in game scenarios, especially for users who have content style preferences for avatars when adding friends. By introducing user avatar features for multimodal recommendation, the accuracy of recommendation can be effectively improved and social activity can be enhanced.

[0310] To verify the effectiveness of multimodal features and the recommendation model, the following experiments were conducted:

[0311] The training was performed using exposure and click data from the first 7 days, and the testing was conducted using data from the 8th day. During training, exposure and click pairs were used as positive samples, and non-clicked candidate pairs were used as negative samples. The hit rate was used as the evaluation metric, with a higher hit rate indicating better performance.

[0312] First, ablation experiments were conducted, in which one part of the model was removed each time, and the results were compared with the overall model. The experimental results are shown in Table 1:

[0313]

[0314] Table 1

[0315] It can be seen that the removal Figure 11 Any one of these components can lead to a decrease in effectiveness, demonstrating the effectiveness of each part in a multimodal recommendation model.

[0316] See Figure 12A , Figure 12A This is a schematic diagram illustrating the changes in click-through rate over 7 days, provided in an embodiment of this application. See also... Figure 12B , Figure 12B This is a schematic diagram illustrating the change in click-through rate over 7 days, provided in an embodiment of this application. Figure 12A and Figure 12B In the diagram, the horizontal axis represents dates, specifically data from May 26th to June 1st. The vertical axis represents the click-through rate and click-through rate, respectively. It can be seen that after applying the information recommendation method provided in this application embodiment, the average click-through rate over 7 days increased by 3.19%, and the average click-through rate increased by 5.45%.

[0317] This application's embodiments determine three different dimensional output vectors based on the first multimodal feature vector of the target object and the second multimodal feature vector of the candidate object. These three output vectors are then fused to determine recommendation parameters, which are used to determine the target recommendation object. While exploring the relationship between the target object and the candidate object, the individual interests of the target object and the group interests of the target object's group are also considered. The target recommendation object is determined by information from multiple dimensions, making it more aligned with the target object's interests and improving the accuracy and relevance of the recommendation. Training the language understanding model with positive and negative samples ensures that sentence distances in positive samples are closer than those in negative samples, allowing the extracted text features to better reflect semantic information and improving the accuracy and representativeness of text feature extraction. Channel expansion is performed on the first feature vector and the second candidate feature vector based on a pre-trained parameter matrix. This introduces more parameter information to the original vectors without adding extra dimensions, helping to increase the model's nonlinear capabilities and enabling it to learn more complex features. Nonlinear mapping after expansion enhances the model's expressive power. By mapping the original vectors to a new space, the model can capture complex relationships and patterns that may be hidden within the original vectors. Fusing two vectors that have undergone nonlinear mapping integrates their complementary information, helping the model better understand multiple aspects of the data and thus improving model performance. By weighted summation of multiple first feature vectors corresponding to the target object, a second output vector representing the individual interest of the target object is obtained. The correlation and importance between features of multiple modalities of the target object are learned, reflecting the target object's tendency and thus embodying its individual interest. By learning the distance (i.e., dot product) between multiple features and the convolutional kernel group, the modal features closest to the convolutional kernel group among the multimodal features corresponding to multiple candidate objects are determined, thereby reflecting the group interest of the target object's group.

[0318] The following description continues to illustrate the exemplary structure of the information recommendation device 233 provided in the embodiments of this application as a software module. In some embodiments, such as Figure 2 As shown, the software module stored in the information recommendation device 233 of the memory 230 may include:

[0319] The acquisition module 2331 is used to acquire a vector combination, wherein the vector combination includes the first multimodal feature vector of the target object and the second multimodal feature vector of the candidate object.

[0320] The dimensionality reduction module 2332 is used to perform dimensionality reduction based on vector combination to obtain the first output vector.

[0321] The encoding module 2333 is used to perform attention encoding based on the first multimodal feature vector to obtain the first set of weight coefficients corresponding to the first multimodal vector, and to determine the second output vector representing the individual interest of the target object based on the first set of weight coefficients.

[0322] The convolution module 2334 is used to extract features from the first multimodal feature vector to obtain a third output vector representing the group interest of the target object's group.

[0323] The determination module 2335 is used to determine recommendation parameters based on the first output vector, the second output vector, and the third output vector; and to determine the target recommendation object for the target object from the multiple candidate objects based on the multiple recommendation parameters corresponding to the multiple candidate objects.

[0324] In some embodiments, the acquisition module 2331 is further configured to acquire a first multimodal feature of the target object and a second multimodal feature of the candidate object; vectorize the first multimodal feature to obtain a first multimodal feature vector; vectorize the second multimodal feature to obtain a second multimodal feature vector; and combine the first multimodal feature vector and the second multimodal feature vector into a vector combination.

[0325] In some embodiments, the acquisition module 2331 is further configured to extract features from the text of the target object based on a pre-trained language understanding model to obtain text features; extract features from the image of the target object to obtain image features; extract social features corresponding to the target object from a pre-constructed social graph; vectorize the text features, image features, and social features respectively to obtain text feature vectors, image feature vectors, and social feature vectors; and combine the text feature vectors, image feature vectors, and social feature vectors into a first multimodal feature vector.

[0326] In some embodiments, the acquisition module 2331 is further configured to acquire an initialized language understanding model and a third training set, wherein the third training set includes multiple third training samples, each third training sample including one positive sample and multiple negative samples, the positive sample being a sentence pair with a similarity greater than a similarity threshold, and the negative sample being a sentence pair with a similarity less than a similarity threshold; the positive sample is vectorized to obtain a positive sample vector; the negative sample is vectorized to obtain a negative sample vector; a third loss value is determined based on the positive sample vector and the negative sample vector; the parameters of the initialized language understanding model are updated based on the third loss value to obtain a pre-trained language understanding model.

[0327] In some embodiments, the dimensionality reduction module 2332 is further configured to perform the following processing for each first feature vector: determine a second candidate mode of the same type as the first mode corresponding to the first feature vector; determine a second candidate feature vector corresponding to the second candidate mode; determine a first fusion vector based on the first feature vector and the second candidate feature vector; perform pooling based on the first fusion vector to obtain a first output sub-vector; and fuse multiple first output sub-vectors corresponding to multiple first feature vectors to obtain a first output vector.

[0328] In some embodiments, the dimensionality reduction module 2332 is further configured to perform channel expansion on the first feature vector based on the pre-trained parameter matrix to obtain a first expanded matrix; perform nonlinear mapping on the first expanded matrix to obtain a first mapping matrix; perform channel expansion on the second candidate feature vector based on the pre-trained parameter matrix to obtain a second expanded matrix; perform nonlinear mapping on the second expanded matrix to obtain a second mapping matrix; fuse the first mapping matrix and the second mapping matrix to obtain a fused matrix; and perform nonlinear mapping on the fused matrix to obtain a first fused vector.

[0329] In some embodiments, the dimensionality reduction module 2332 is further configured to perform pooling operation on the rows in the first fusion vector to obtain the row attention vector of the target object for the candidate object; perform pooling operation on the columns in the first fusion vector to obtain the column attention vector of the candidate object for the target object; and fuse the first feature vector and the second candidate feature vector using the row attention vector and the column attention vector as weights to obtain the first output sub-vector.

[0330] In some embodiments, the dimensionality reduction module 2332 is further configured to: obtain an initialized first machine learning model; obtain multiple object combinations and a first true label, wherein each object combination includes an object sample and a sample of objects to be recommended, the sample of objects to be recommended being the object to be recommended, and the first true label representing the probability of recommending the sample of objects to be recommended to the object sample; vectorize the object samples to obtain an object sample vector; vectorize the sample of objects to be recommended to obtain a sample of objects to be recommended; perform channel augmentation on the object sample vector based on the initialized parameter matrix to obtain an augmented object sample vector; augment the sample of objects to be recommended based on the initialized parameter matrix to obtain an augmented sample of objects to be recommended; call the initialized first machine learning model based on the vector pair to obtain a first predicted label, wherein the vector pair includes the augmented object sample vector and the augmented sample of objects to be recommended; determine a first loss value based on the first true label and the first predicted label; update the parameters of the initialized first machine learning model based on the first loss value; and combine the updated parameters into a pre-trained parameter matrix.

[0331] In some embodiments, the encoding module 2333 is further configured to perform the following processing for each first feature vector: determine the similarity between the first feature vector and other feature vectors, normalize the similarity to obtain a normalized similarity, determine the product of the normalized similarity and the first feature vector as a first weight coefficient of the first feature vector, wherein the other feature vectors are feature vectors other than the first feature vector among a plurality of first feature vectors; and combine the plurality of first weight coefficients corresponding to the plurality of first feature vectors into a first weight coefficient set.

[0332] In some embodiments, the encoding module 2333 is further configured to fuse multiple first feature vectors according to multiple first weight coefficients corresponding to multiple first feature vectors in the first weight coefficient set, to obtain a second output vector representing the individual interest of the target object.

[0333] In some embodiments, the convolution module 2334 is further configured to perform feature mapping on each first feature vector of the target object to obtain a mapped feature vector; combine the multiple mapped feature vectors corresponding to the multiple first feature vectors in the row direction to obtain a combined feature vector; determine the third product of the combined feature vector and the transpose of the column convolution kernel; and determine the fourth product of the third product and the row convolution kernel to serve as the third output vector representing the group interest of the group to which the target object belongs.

[0334] In some embodiments, the convolution module 2334 is further configured to acquire an initialized second machine learning model and training data, wherein each training data includes initial features of multiple modalities of an object sample; perform feature mapping on each initial feature to obtain a training feature vector; for each training data, combine the multiple training feature vectors corresponding to the multiple modalities in the row direction to obtain a sample feature vector; call the initialized second machine learning model based on the sample feature vector to obtain a predicted output; determine an output loss value based on the true output and the predicted output, wherein the true output represents the dot product of the sample feature vector and the initialized convolution kernel group; update the parameters of the initialized second machine learning model based on the output loss value, and combine the updated parameters into a pre-trained convolution kernel group.

[0335] In some embodiments, the determining module 2335 is further configured to combine the first output vector, the second output vector, and the third output vector into an input vector; and to call a pre-trained third machine learning model based on the input vector to perform prediction processing to obtain recommended parameters.

[0336] In some embodiments, the determining module 2335 is further configured to obtain an initialized third machine learning model and a second training set, wherein the second training set includes multiple second training samples, each second training sample including a sample input vector and a second true label, the second true label representing the true recommendation parameters; call the initialized third machine learning model based on the sample input vector to obtain a second predicted label; determine a second loss value based on the second true label and the second predicted label, and update the parameters of the initialized third machine learning model based on the second loss value to obtain a pre-trained third machine learning model.

[0337] In some embodiments, the determining module 2335 is further configured to sort the multiple recommendation parameters in descending order; select multiple target recommendation parameters from the result of the descending sort in order from high to low; and determine multiple target candidate objects corresponding to the multiple target recommendation parameters from the multiple candidate objects, so as to serve as target recommendation objects for the target objects.

[0338] This application provides a computer program product comprising a computer program or computer-executable instructions stored in a computer-readable storage medium. The processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the information recommendation method described in this application.

[0339] This application provides a computer-readable storage medium storing computer-executable instructions or a computer program. When the computer-executable instructions or the computer program are executed by a processor, the processor will execute the information recommendation method provided in this application, for example... Figure 3A The information recommendation method is shown.

[0340] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0341] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0342] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0343] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located in one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0344] In summary, this application's embodiments determine three different dimensional output vectors based on the first multimodal feature vector of the target object and the second multimodal feature vector of the candidate object. These three output vectors are then fused to determine recommendation parameters, which are used to identify the target recommendation object. This approach not only explores the relationship between the target object and the candidate object but also considers the individual interests of the target object and the group interests of the target object's group. The combined information from multiple dimensions determines the target recommendation object, making it more aligned with the target object's interests and improving the accuracy and relevance of the recommendation. Training the language understanding model with positive and negative samples ensures that sentence distances in positive samples are closer than those in negative samples, allowing the extracted text features to better reflect semantic information and improving the accuracy and representativeness of text feature extraction. Channel expansion of the first feature vector and the second candidate feature vector based on a pre-trained parameter matrix introduces more parameter information to the original vectors without adding extra dimensions, enhancing the model's nonlinear capabilities and enabling it to learn more complex features. After expansion, nonlinear mapping enhances the model's expressive power. By mapping the original vectors to a new space, the model can capture complex relationships and patterns that may be hidden within the original vectors. Fusing two vectors after nonlinear mapping integrates their complementary information, helping the model better understand multiple aspects of the data and thus improving model performance. By weighted summing of multiple first feature vectors corresponding to the target object, a second output vector representing the individual interest of the target object is obtained. The correlation and importance between features of multiple modalities of the target object are learned, reflecting the target object's tendency and thus embodying its individual interest. By learning the distance (i.e., dot product) between multiple features and the convolutional kernel group, the modal features closest to the convolutional kernel group among the multimodal features corresponding to multiple candidate objects are determined, thereby reflecting the group interest of the target object's group.

[0345] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. An information recommendation method, characterized in that, The method includes: Obtain a vector combination, wherein the vector combination includes a first multimodal feature vector of the target object and a second multimodal feature vector of the candidate object; Dimensionality reduction is performed based on the vector combination to obtain the first output vector; Attention encoding is performed based on the first multimodal feature vector to obtain a first set of weight coefficients corresponding to the first multimodal vector, and a second output vector representing the individual interest of the target object is determined based on the first set of weight coefficients. Feature extraction is performed on the first multimodal feature vector to obtain a third output vector representing the group interest of the group to which the target object belongs; Recommendation parameters are determined based on the first output vector, the second output vector, and the third output vector; Based on the multiple recommendation parameters corresponding to the multiple candidate objects, a target recommendation object for the target object is determined from the multiple candidate objects.

2. The method according to claim 1, characterized in that, The first multimodal feature vector includes multiple first feature vectors, and the second multimodal feature vector includes multiple second feature vectors. The multiple first feature vectors correspond to multiple first modes, and the multiple second feature vectors correspond to multiple second modes. One first mode corresponds to one second mode. The dimensionality reduction based on the vector combination to obtain the first output vector includes: For each of the first feature vectors, the following processing is performed: determine a second candidate mode of the same type as the first mode corresponding to the first feature vector; determine a second candidate feature vector corresponding to the second candidate mode; determine a first fusion vector based on the first feature vector and the second candidate feature vector; and perform pooling based on the first fusion vector to obtain a first output sub-vector. The first output vector is obtained by fusing the multiple first feature vectors corresponding to the multiple first output sub-vectors respectively.

3. The method according to claim 2, characterized in that, The step of determining the first fusion vector based on the first feature vector and the second candidate feature vector includes: The first feature vector is augmented with channels based on the pre-trained parameter matrix to obtain the first augmented matrix; A first mapping matrix is ​​obtained by performing a nonlinear mapping on the first extended matrix; Based on the pre-trained parameter matrix, the second candidate feature vector is augmented with channels to obtain a second augmented matrix; A second mapping matrix is ​​obtained by performing a nonlinear mapping on the second extended matrix; The first mapping matrix and the second mapping matrix are fused to obtain a fused matrix; The fusion matrix is ​​nonlinearly mapped to obtain the first fusion vector.

4. The method according to claim 3, characterized in that, The pre-trained parameter matrix is ​​a set of parameters obtained through a first machine learning model. Before determining the first fusion vector based on the first feature vector and the second candidate feature vector, the method further includes: Obtain the initialized first machine learning model; Obtain multiple object combinations and a first true label, wherein each object combination includes an object sample and a recommended object sample, the recommended object sample being the object to be recommended to the object sample, and the first true label characterizes the probability of recommending the recommended object sample to the object sample; The object sample is vectorized to obtain the object sample vector; The sample of the object to be recommended is vectorized to obtain the sample vector of the object to be recommended; The object sample vector is augmented with channels based on the initialized parameter matrix to obtain the object sample augmented vector. The sample vector of the object to be recommended is expanded based on the initialized parameter matrix to obtain the expanded sample vector of the object to be recommended. The first machine learning model initialized based on vector pairs is used to obtain a first predicted label, wherein the vector pair includes the object sample augmentation vector and the object sample augmentation vector to be recommended. A first loss value is determined based on the first true label and the first predicted label. The parameters of the initialized first machine learning model are updated based on the first loss value, and the updated parameters are combined into the pre-trained parameter matrix.

5. The method according to claim 2, characterized in that, The process of pooling based on the first fusion vector to obtain the first output sub-vector includes: Perform pooling operations on the rows in the first fusion vector to obtain the row attention vector of the target object for the candidate object; The columns in the first fusion vector are pooled to obtain the column attention vector of the candidate object for the target object; Using the row attention vector and the column attention vector as weights, the first feature vector and the second candidate feature vector are fused to obtain the first output sub-vector.

6. The method according to claim 1, characterized in that, The first multimodal feature vector includes multiple first feature vectors, and the second multimodal feature vector includes multiple second feature vectors. The multiple first feature vectors correspond to multiple first modes, and the multiple second feature vectors correspond to multiple second modes. One first mode corresponds to one second mode. The attention encoding based on the first multimodal feature vector yields a first set of weight coefficients corresponding to the first multimodal vector, including: For each of the first feature vectors, the following processing is performed: the similarity between the first feature vector and other feature vectors is determined, the similarity is normalized to obtain a normalized similarity, and the product of the normalized similarity and the first feature vector is determined as the first weight coefficient of the first feature vector, wherein the other feature vectors are feature vectors other than the first feature vector among the plurality of first feature vectors. The multiple first weight coefficients corresponding to the multiple first feature vectors are combined into a first weight coefficient set.

7. The method according to claim 6, characterized in that, The step of determining the second output vector representing the individual interests of the target object based on the first set of weight coefficients includes: Based on the multiple first weight coefficients corresponding to the multiple first feature vectors in the first weight coefficient set, the multiple first feature vectors are fused to obtain a second output vector representing the individual interests of the target object.

8. The method according to claim 2, characterized in that, The third output vector is obtained by extracting features from the first multimodal feature vector using a pre-trained set of convolutional kernels, which includes row convolutional kernels and column convolutional kernels. The step of extracting features from the first multimodal feature vector to obtain a third output vector representing the group interest of the group to which the target object belongs includes: Perform feature mapping on each of the first feature vectors of the target object to obtain a mapped feature vector; The multiple mapping feature vectors corresponding to the multiple first feature vectors are combined in the row direction to obtain a combined feature vector; Determine the third product of the combined feature vector and the transpose of the column convolution kernel; The third product is determined to be the fourth product of the row convolution kernel, which serves as the third output vector representing the group interest of the group to which the target object belongs.

9. The method according to claim 8, characterized in that, The pre-trained convolutional kernel set is the parameter of the second machine learning model; Before performing feature extraction on the first multimodal feature vector to obtain a third output vector representing the group interest of the group to which the target object belongs, the method further includes: Obtain the initialized second machine learning model and training data, wherein each piece of training data includes initial features of multiple modalities of an object sample; Perform feature mapping on each of the initial features to obtain a training feature vector; For each of the training data, the training feature vectors corresponding to the multiple modalities are combined in the row direction to obtain the sample feature vector; Based on the sample feature vector, the initialized second machine learning model is invoked to obtain the prediction output; The output loss value is determined based on the true output and the predicted output, wherein the true output represents the dot product of the sample feature vector and the initialized convolutional kernel group; The parameters of the initialized second machine learning model are updated based on the output loss value, and the updated parameters are combined into the pre-trained convolutional kernel group.

10. The method according to any one of claims 1 to 9, characterized in that, The step of determining the recommendation parameters based on the first output vector, the second output vector, and the third output vector includes: The first output vector, the second output vector, and the third output vector are combined into an input vector; Based on the input vector, a pre-trained third machine learning model is invoked to perform prediction processing to obtain recommended parameters.

11. The method according to claim 10, characterized in that, The recommendation parameters are determined by the pre-trained third machine learning model; Before invoking the pre-trained machine learning model based on the input vector for prediction processing, the method further includes: Obtain an initialized third machine learning model and a second training set, wherein the second training set includes multiple second training samples, each of which includes a sample input vector and a second true label, the second true label representing the true recommendation parameters; The second predicted label is obtained by calling the initialized third machine learning model based on the sample input vector; A second loss value is determined based on the second true label and the second predicted label, and the parameters of the initialized third machine learning model are updated based on the second loss value to obtain a pre-trained third machine learning model.

12. The method according to any one of claims 1 to 9, characterized in that, The step of determining the target recommendation object for the target object from the multiple candidate objects based on the multiple recommendation parameters corresponding to the multiple candidate objects includes: Sort the multiple recommended parameters in descending order; Multiple target recommendation parameters are selected from the descending sort results in descending order; Multiple target candidate objects corresponding to the multiple target recommendation parameters are determined from the multiple candidate objects, and used as target recommendation objects for the target object.

13. The method according to any one of claims 1 to 9, characterized in that, The acquisition of vector combinations includes: Obtain the first multimodal features of the target object and the second multimodal features of the candidate object; The first multimodal feature is vectorized to obtain the first multimodal feature vector; The second multimodal feature is vectorized to obtain the second multimodal feature vector; The first multimodal feature vector and the second multimodal feature vector are combined into a vector combination.

14. The method according to claim 13, characterized in that, The first multimodal feature includes text features, image features, and social features. The step of vectorizing the first multimodal feature to obtain the first multimodal feature vector includes: Based on a pre-trained language understanding model, text features are extracted from the target object's text to obtain text features; Feature extraction is performed on the image of the target object to obtain image features; Extract the social features corresponding to the target object from the pre-constructed social graph; The text features, image features, and social features are vectorized respectively to obtain text feature vectors, image feature vectors, and social feature vectors; The text feature vector, the image feature vector, and the social feature vector are combined to form the first multimodal feature vector.

15. The method according to claim 14, characterized in that, Before the pre-trained language understanding model performs feature extraction on the text of the target object, the method further includes: Obtain an initialized language understanding model and a third training set, wherein the third training set includes multiple third training samples, each of the third training samples includes one positive sample and multiple negative samples, the positive sample is a sentence pair with a similarity greater than a similarity threshold, and the negative sample is a sentence pair with a similarity less than the similarity threshold; The positive samples are vectorized to obtain positive sample vectors; The negative samples are vectorized to obtain negative sample vectors; A third loss value is determined based on the positive sample vector and the negative sample vector; The parameters of the initialized language understanding model are updated based on the third loss value to obtain the pre-trained language understanding model.

16. An information recommendation device, characterized in that, The device includes: An acquisition module is used to acquire a vector combination, wherein the vector combination includes a first multimodal feature vector of the target object and a second multimodal feature vector of the candidate object; The dimension reduction module is used to perform dimension reduction based on the vector combination to obtain a first output vector; The encoding module is used to perform attention encoding based on the first multimodal feature vector to obtain a first set of weight coefficients corresponding to the first multimodal vector, and to determine a second output vector representing individual interest based on the first set of weight coefficients. The convolution module is used to extract features from the first multimodal feature vector to obtain a third output vector representing the group's interest. The determining module is configured to determine recommendation parameters based on the first output vector, the second output vector, and the third output vector; and to determine a target recommendation object for the target object from among the multiple candidate objects based on the multiple recommendation parameters corresponding to the multiple candidate objects respectively.

17. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions for a computer; A processor, when executing computer-executable instructions stored in the memory, implements the information recommendation method according to any one of claims 1 to 15.

18. A computer-readable storage medium storing computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the information recommendation method according to any one of claims 1 to 15.

19. A computer program product comprising computer-executable instructions or a computer program, characterized in that, When the computer-executable instructions or computer program are executed by a processor, they implement the information recommendation method according to any one of claims 1 to 15.

Images