Prediction method, recommendation method, and related devices thereof

Abstract

The present disclosure provides a prediction method, a recommendation method and related devices, wherein the prediction method comprises: obtaining a head portrait image of a social user and a user nickname; performing image feature extraction on the head portrait image by using a convolutional neural network to obtain a head portrait image feature; performing text feature extraction on the user nickname by using a translation model to obtain a nickname text feature; and predicting the gender of the social user according to the head portrait image feature and the nickname text feature. Thus, the gender of the social user is predicted according to the head portrait image feature and the nickname text feature, and the gender prediction accuracy of the social user is improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, and in particular, to a prediction method, a recommendation method, and related devices thereof. BACKGROUND

[0002] Artificial intelligence (AI) is a new technical science of theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. Artificial intelligence is a branch of computer science that aims to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. The research in this field includes robots, speech recognition, image recognition, natural language processing and expert systems, etc. The most important aspect of artificial intelligence is speech recognition technology.

[0003] Currently, some social applications can push personalized images to social users, and the social users can share the personalized images with other users or use the personalized images to post statuses in the social applications.

[0004] In the related art, in a social application, the gender of a social user can be determined according to a social avatar corresponding to the social user, and a personalized image matching the gender can be pushed to the social user according to the gender of the social user. However, the way of determining the gender of the social user according to the social avatar only has low accuracy, which reduces the user experience. SUMMARY

[0005] The present disclosure aims to at least partially solve one of the technical problems in the related art.

[0006] To this end, a first object of the present disclosure is to provide a prediction method to realize gender prediction of a social user by extracting avatar image features corresponding to an avatar image of the social user and nickname text features corresponding to a nickname of the social user, and improving the gender prediction accuracy of the social user.

[0007] A second object of the present disclosure is to provide a recommendation method.

[0008] A third object of the present disclosure is to provide a prediction device.

[0009] A fourth object of the present disclosure is to provide a recommendation device.

[0010] A fifth object of the present disclosure is to provide an electronic device.

[0011] A sixth object of the present disclosure is to provide a non-transitory computer-readable storage medium storing computer instructions.

[0012] A seventh object of the present disclosure is to provide a computer program product.

[0013] To achieve the above object, the first aspect of the present disclosure provides a prediction method, comprising: obtaining a head portrait of a social user and a user nickname; performing image feature extraction on the head portrait by using a convolutional neural network to obtain head portrait image features; performing text feature extraction on the user nickname by using a translation model to obtain nickname text features; and predicting the gender of the social user according to the head portrait image features and the nickname text features.

[0014] The prediction method of the present disclosure improves the prediction accuracy of the gender of the social user by extracting the head portrait image features corresponding to the head portrait of the social user and the nickname text features corresponding to the user nickname, and predicting the gender of the social user according to the head portrait image features and the nickname text features.

[0015] To achieve the above object, the second aspect of the present disclosure provides a recommendation method, comprising: predicting the gender of a social user by using the method of the first aspect of the present disclosure; and recommending image resources matching the predicted gender to the social user.

[0016] To achieve the above object, the third aspect of the present disclosure provides a prediction device, comprising: a first obtaining module configured to obtain a head portrait of a social user and a user nickname; a first extracting module configured to perform image feature extraction on the head portrait by using a convolutional neural network to obtain head portrait image features; a second extracting module configured to perform text feature extraction on the user nickname by using a translation model to obtain nickname text features; and a first predicting module configured to predict the gender of the social user according to the head portrait image features and the nickname text features.

[0017] To achieve the above object, the fourth aspect of the present disclosure provides a recommendation device, comprising: a prediction module configured to predict the gender of a social user by using the device of the third aspect of the present disclosure; and a recommendation module configured to recommend image resources matching the predicted gender to the social user.

[0018] To achieve the above object, the fifth aspect of the present disclosure provides an electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method of the first aspect of the present disclosure or the method of the second aspect of the present disclosure.

[0019] To achieve the above object, the sixth aspect of the present disclosure provides a non-transitory computer readable storage medium storing computer instructions, the computer instructions being used to make the computer execute the method in the first aspect of the present disclosure, or execute the method in the second aspect of the present disclosure.

[0020] To achieve the above object, the seventh aspect of the present disclosure provides a computer program product, comprising a computer program, the computer program being used to implement the method in the first aspect of the present disclosure, or implement the method in the second aspect of the present disclosure when executed by a processor.

[0021] The additional aspects and advantages of the present disclosure will be partially given in the following description, partially become obvious from the following description, or be understood through the practice of the present disclosure. BRIEF DESCRIPTION OF DRAWINGS

[0022] The above described and / or additional aspects and advantages of the present disclosure will become apparent and easily understood from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:

[0023] Figure 1 A structure diagram of a line layer neural network provided by an embodiment of the present disclosure;

[0024] Figure 2 A flow diagram of a prediction method provided by an embodiment of the present disclosure;

[0025] Figure 3 A flow diagram of another prediction method provided by an embodiment of the present disclosure;

[0026] Figure 4 A flow diagram of a prediction method based on a multi-modal base model provided by an embodiment of the present disclosure;

[0027] Figure 5 A head portrait diagram provided by an embodiment of the present disclosure;

[0028] Figure 6 A structure diagram of an attention mechanism provided by an embodiment of the present disclosure;

[0029] Figure 7 A flow diagram of a prediction method provided by an embodiment of the present disclosure;

[0030] Figure 8 A flow diagram of an attention weight of a head portrait image feature and a nickname text feature determined according to an attention network of an embodiment of the present disclosure;

[0031] Figure 9 A flow diagram of another prediction method provided by an embodiment of the present disclosure;

[0032] Figure 10 A flowchart of another prediction method provided by an embodiment of the present disclosure;

[0033] Figure 11 A schematic diagram of the structure of a convolutional neural network provided in an embodiment of the present disclosure;

[0034] Figure 12 A flowchart of another prediction method provided by an embodiment of the present disclosure;

[0035] Figure 13 A schematic diagram of the structure of a translation model provided in an embodiment of the present disclosure;

[0036] Figure 14 A flowchart of another prediction method provided by an embodiment of the present disclosure;

[0037] Figure 15 A flowchart of a recommended method provided in an embodiment of the present disclosure;

[0038] Figure 16 A schematic diagram of the structure of a prediction device provided in an embodiment of the present disclosure;

[0039] Figure 17 A schematic structural diagram of a recommended device provided in an embodiment of the present disclosure;

[0040] Figure 18 It is a block diagram of an electronic device according to a prediction method or a recommendation method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0041] The following describes in detail embodiments of the present disclosure, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0042] Currently, if Figure 1 As shown in the figure, in gender prediction, the user nickname can also be processed into multiple feature vectors through a statistical language model (such as N-Grame), and multiple feature vectors (such as x1, x2, x N-1 and x N However, determining the gender of a social user based solely on their nickname results in low accuracy, which degrades the user experience.

[0043] In response to the above problems, the present disclosure proposes a prediction method, a recommendation method and related devices.

[0044] Before specific description of the embodiments of the present disclosure, first, the commonly used technical words of the present disclosure are introduced for easy understanding.

[0045] User nickname: the name used by the social user on the social application;

[0046] Difference feature: the difference between the avatar image feature and the nickname text feature is obtained by bit-by-bit difference;

[0047] Visual emotional symbol: namely, emoticon;

[0048] First confidence: the probability value obtained by activating the output of the fusion feature through the full connection layer using the activation function;

[0049] Second confidence: the probability value output by the activation function in the prediction network by inputting the avatar image and the user nickname in the training sample into the prediction network.

[0050] The prediction method, the recommendation method and the related devices of the embodiments of the present disclosure are described below with reference to the accompanying drawings.

[0051] Figure 2 The flowchart of a prediction method provided by the embodiments of the present disclosure is shown.

[0052] As shown in Figure 2 , the prediction method comprises the following steps:

[0053] Step 201: obtaining the avatar image of the social user and the user nickname.

[0054] In the embodiments of the present disclosure, the social application page can include the avatar image of the social user and the user nickname. Different users can set different avatar images and user nicknames in the social application page, and the social application can establish an association relationship between the login information and the avatar image and an association relationship between the login information and the user nickname. Therefore, in the embodiments of the present disclosure, the above association relationship can be queried according to the login information to obtain the avatar image of the social user and the user nickname.

[0055] Step 202: performing image feature extraction on the avatar image using a convolutional neural network to obtain an avatar image feature.

[0056] In the embodiments of the present disclosure, the convolutional neural network (CNN) can be a dense convolutional neural network including multiple layers of dense convolutional layers. The avatar image feature is obtained by performing image feature extraction on the avatar image using the dense convolutional neural network including multiple layers of dense convolutional layers.

[0057] Step 203: performing text feature extraction on the user nickname using a translation model to obtain a nickname text feature.

[0058] In the embodiments of the present disclosure, the characters of the user nickname can be taken as the input of a translation model (Transformer), and the encoding layer of the translation model can be used to extract the features of the characters of the user nickname to obtain the nickname text features.

[0059] In step 204, the gender of the social user is predicted according to the avatar image features and the nickname text features.

[0060] Further, the attention weights corresponding to the avatar image features and the nickname text features can be obtained through the attention network, and the fusion features of the avatar image features and the nickname text features can be obtained according to the attention weights, and the gender of the social user can be predicted according to the fusion features.

[0061] In summary, by extracting the avatar image features corresponding to the avatar image of the social user and the nickname text features corresponding to the user nickname, and predicting the gender of the social user according to the avatar image features and the nickname text features, the gender prediction accuracy of the social user is improved.

[0062] In order to accurately predict the gender of the social user according to the avatar image features and the nickname text features, as shown in Figure 3 , the present disclosure provides a prediction method, and the steps of the method are as follows: Figure 3 As shown in FIG. 7, another flowchart of the prediction method provided by the embodiments of the present disclosure is shown, in which embodiments of the present disclosure, the difference features between the avatar image features and the nickname text features can be obtained, the avatar image features, the nickname text features and the difference features are input into an attention network to obtain the attention weights corresponding to the avatar image features and the nickname text features, the fusion features of the avatar image features and the nickname text features are obtained according to the attention weights corresponding to the avatar image features and the nickname text features, and the gender of the social user is predicted according to the fusion features. Figure 3 The embodiments shown in FIG. 7 can include the following steps:

[0063] In step 301, the avatar image of the social user and the user nickname are obtained.

[0064] In step 302, the convolutional neural network is used to extract the image features of the avatar image to obtain the avatar image features.

[0065] In step 303, the translation model is used to extract the text features of the user nickname to obtain the nickname text features.

[0066] In step 304, the difference features between the avatar image features and the nickname text features are obtained.

[0067] It should be noted that, as shown in Figure 4As shown in the multi-modal baseline model, the avatar image is extracted by the CNN to obtain the image feature, the nickname text data is encoded by the translation model Transformer to obtain the text feature, the text feature and the image feature are input into the full connection layer, and the confidence (range [0, 1]) is obtained by the activation function Sigmoid of the full connection layer, and the prediction result is obtained according to the confidence.

[0068] It should be understood that for the neutral avatar image of the social user, the gender can be distinguished by the nickname, such as Figure 5 As shown, Figure 5 The avatar image features obtained by the convolutional neural network from the avatar image corresponding to the left part and the avatar image corresponding to the right part are basically similar, Figure 5 The nickname corresponding to the left part is "X small treasure", Figure 5 The nickname corresponding to the right part is "X strong country", which can be predicted as male, so the attention weight can be used to weaken the avatar image feature of the social user corresponding to the nickname, and the nickname text feature is weighted.

[0069] In the embodiment of the present disclosure, in order to further strengthen the nickname text feature and weaken the avatar image feature, the avatar image feature and the nickname text feature can be subtracted bit by bit to obtain the difference feature between the avatar image feature and the nickname text feature.

[0070] Step 305, input the avatar image feature, the nickname text feature and the difference feature into the attention network to obtain the attention weight of the avatar image feature and the nickname text feature.

[0071] Further, after the avatar image feature, the nickname text feature and the difference feature are spliced, the obtained spliced vector is input into the attention network, and the spliced vector is input into the full connection layer and the nonlinear transformation of the attention network to obtain the attention weight of the avatar image feature and the nickname text feature.

[0072] The structure diagram of the attention mechanism can be as shown in Figure 6 As shown, Figure 6A schematic diagram of the structure of an attention mechanism provided by an embodiment of the present disclosure, which takes into account both the channel attention mechanism and the spatial attention mechanism, and the feature dimensions of the input and output are completely consistent and can be plug-and-play. Among them, the channel attention module can process the input feature map F(H×W×C) through global maximum pooling (Global Max Pooling) and global average pooling (Global Average Pooling) based on width (width) and height (height) respectively to obtain two 1×1×C feature maps, and then send them into a two-layer neural network (MLP) respectively. The number of neurons in the first layer is C / r (r is the reduction rate), the activation function is Relu, and the number of neurons in the second layer is C. The two-layer neural network is shared. Then, the features output by the MLP are subjected to an element-wise addition operation, and then subjected to a Sigmoid activation operation to generate the final channel attention feature (Channel Attention Feature), i.e. Mc. Finally, Mc is element-wise multiplied with the input feature map F to generate the input features required by the spatial attention module. The feature map F' output by the channel attention module is used as the input feature map of the spatial attention module. First, a channel-based Global Max Pooling and Global Average Pooling are performed to obtain two H×W×1 feature maps. These two feature maps are then concatenated based on the channel (channel concatenation). A 7×7 convolution (7×7 is better than 3×3) is then performed to reduce the dimension to one channel, that is, H×W×1. The spatial attention feature, Ms, is then generated through an activation function (Sigmoid). Finally, Ms is multiplied by the input features of the module to obtain the final generated features.

[0073] Step 306: Fusing the avatar image features and the nickname text features according to the attention weight to obtain a fused feature.

[0074] Furthermore, according to the attention weight, the avatar image features and nickname text features are weighted and summed, and the weighted sum result is used as the fusion feature.

[0075] Step 307: predict the gender of the social user based on the fused features.

[0076] Optionally, the fused features are input into a fully connected layer of a prediction network; the output of the fully connected layer is activated using an activation function to obtain a first confidence level; and the gender of the social user is determined based on whether the first confidence level is less than a set threshold.

[0077] That is, to improve the accuracy of predicting the gender of a social user, the fully connected layer of the prediction network can be used to classify the fused features. For example, the fused features are input into the fully connected layer of the prediction network, and the output of the fully connected layer is activated using an activation function to obtain a first confidence score. The first confidence score is then compared with a set threshold. When the first confidence score is less than the set threshold, the gender of the social user can be determined. It should be noted that the gender of the social user corresponding to the first confidence score when it is greater than or equal to the set threshold, and the gender of the social user corresponding to the first confidence score when it is less than the set threshold, can be pre-set. For example, when the first confidence score is less than the set threshold, the gender of the social user can be determined to be female, and when the first confidence score is greater than or equal to the set threshold, the gender of the social user can be determined to be male. For another example, when the first confidence score is less than the set threshold, the gender of the social user can be determined to be male, and when the first confidence score is greater than or equal to the set threshold, the gender of the social user can be determined to be female.

[0078] In order to explain the above embodiment more clearly, an example is given below.

[0079] like Figure 7 As shown in the figure, the Convolutional Neural Network (CNN) is used to extract the image features of the avatar image to obtain the avatar image features, and the Transformer model is used to extract the text features of the user nickname to obtain the nickname text features; according to the attention network ( Figure 7 Attention unit) determines the attention weights of the avatar image features and nickname text features ( Figure 7 Attention weight [1,2]), then, according to the attention weight, the avatar image features and nickname text features are weighted and summed to obtain the fusion feature, the fusion feature is input into the fully connected layer (fully connected network), and the output of the fully connected layer is activated by the activation function (Sigmoid) to obtain the first confidence level (the value range of the first confidence level is [0,1]), and the gender of the social user is determined based on whether the first confidence level is less than the set threshold.

[0080] Among them, the attention weights of the avatar image features and nickname text features are determined according to the attention network as follows: Figure 8 As shown, Figure 8For the flowchart of the embodiment of the present disclosure determining the attention weight of the avatar image feature and the nickname text feature according to the attention network, in order to weaken the avatar image feature and enhance the nickname text feature, the difference feature between the avatar image feature and the nickname text feature can be obtained, the avatar image feature (such as 256 dimensions), the nickname text feature (such as 256 dimensions) and the difference feature (such as 256 dimensions) are input into the attention network, the attention network splices the avatar image feature (such as 256 dimensions), the nickname text feature (such as 256 dimensions) and the difference feature (such as 256 dimensions) to obtain a spliced feature vector (such as 768 dimensions), and the spliced feature vector is input into a full connection network and a nonlinear transformation to obtain two-dimensional activation weights (attention weights) of the avatar image feature and the nickname text feature. The specific implementation can be realized through the following code:

[0081]

[0082] It should be noted that the execution process of steps 301 to 303 can be realized by any one of the embodiments of the present disclosure, and the present disclosure does not limit this and will not be repeated here.

[0083] In summary, by obtaining the difference feature between the avatar image feature and the nickname text feature, inputting the avatar image feature, the nickname text feature and the difference feature into the attention network to obtain the attention weight of the avatar image feature and the nickname text feature, and according to the attention weight, fusing the avatar image feature and the nickname text feature to obtain the fusion feature, the gender of the social user can be predicted. Therefore, by using the avatar image feature, the nickname text feature and the difference feature between the avatar image feature and the nickname text feature, the gender of the social user can be accurately predicted.

[0084] In order to clearly illustrate how to fuse the avatar image feature and the nickname text feature according to the attention weight to obtain the fusion feature, the present embodiment proposes another flowchart of the prediction method. Figure 9 The flowchart of another prediction method provided by the embodiment of the present disclosure.

[0085] As shown in Figure 9 The prediction method can include the following steps:

[0086] Step 901, obtaining an avatar image of a social user and a user nickname.

[0087] Step 902, performing image feature extraction on the avatar image by using a convolutional neural network to obtain an avatar image feature.

[0088] Step 903, performing text feature extraction on the user nickname by using a translation model to obtain a nickname text feature.

[0089] Step 904, obtain a difference feature between the avatar image feature and the nickname text feature.

[0090] Step 905, input the avatar image feature, the nickname text feature and the difference feature into an attention network to obtain an attention weight of the avatar image feature and the nickname text feature.

[0091] Step 906, generate an additional text feature according to at least one of a length of the user nickname, a number of visual emotional symbols contained, a number of digits contained and a number of letters contained.

[0092] It should be understood that, since some characters in the user nickname have relevance to the gender, in the embodiments of the present disclosure, the additional text feature can be generated according to at least one of the length of the user nickname, the number of visual emotional symbols contained, the number of digits contained and the number of letters contained, for example, a four-dimensional additional text feature can be generated according to the length of the user nickname, the number of visual emotional symbols contained, the number of digits contained and the number of letters contained, such as the additional text feature being [length of the user nickname, number of visual emotional symbols, number of digits, number of letters].

[0093] Step 907, splice the additional text feature with the nickname text feature, and fuse the spliced nickname text feature and the avatar image feature according to the attention weight to obtain a fusion feature.

[0094] Further, in order to enhance the nickname text feature, the additional text feature can be spliced with the nickname text feature, and a weighted sum of the spliced nickname text feature and the avatar image feature is obtained according to the attention weight, and the weighted sum result is taken as the fusion feature.

[0095] Step 908, predict the gender of the social user according to the fusion feature.

[0096] It should be noted that the execution processes of steps 901 to 905 and step 908 can be respectively realized by any one of the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this, and will not be repeated here.

[0097] In summary, by generating an additional text feature according to at least one of the length of the user nickname, the number of visual emotional symbols contained, the number of digits contained and the number of letters contained, splicing the additional text feature with the nickname text feature, and fusing the spliced nickname text feature and the avatar image feature according to the attention weight to obtain a fusion feature, the spliced nickname text feature and the avatar image feature are fused according to the attention weight, which can weaken the avatar image feature, enhance the nickname text feature, and improve the accuracy of predicting the gender of the social user.

[0098] In order to accurately obtain the avatar image feature, as shown in Figure 10 ​Figure 10 This is a flow chart of another prediction method provided by an embodiment of the present disclosure. In this embodiment of the present disclosure, multiple dense convolutional layers can be used to extract features from the portrait image to obtain the portrait image features. Figure 10 The illustrated embodiment may include the following steps:

[0099] Step 1001: Obtain a profile picture and nickname of a social user.

[0100] Step 1002: Input the avatar image into the first dense convolutional layer.

[0101] In an embodiment of the present disclosure, the convolutional neural network may be a dense convolutional neural network (e.g., DenseNet121), and the avatar image may be input into the first dense convolution layer of the dense convolutional neural network.

[0102] Step 1003: For each remaining dense convolutional layer except the first dense convolutional layer in the dense convolutional neural network, the output of each dense convolutional layer in the previous order is used as the input of the dense convolutional layer of the current layer to perform image feature extraction to obtain the avatar image features.

[0103] Furthermore, for the remaining dense convolutional layers except the first dense convolutional layer in the dense convolutional neural network, the outputs of the dense convolutional layers in the previous order can be used as the input of the dense convolutional layer in this layer to extract image features to obtain the avatar image features.

[0104] For example, if Figure 11 As shown in the figure, the dense convolution layer x0 is the first dense convolution layer. The portrait image is input into the first dense convolution layer x0, and the output of the first dense convolution layer x0 is used as the input of the dense convolution layer x1. The output of the dense convolution layer x1 and the output of the first dense convolution layer x0 are used as the input of the dense convolution layer x2. Similarly, the outputs of the dense convolution layers x0, x1 and x2 are used as the input of the dense convolution layer x3. The outputs of the dense convolution layers x0, x1, x2 and x3 are used as the input of the dense convolution layer x4. And so on. The output of the last dense convolution layer of the dense convolution neural network is used as the portrait image feature.

[0105] Step 1004 : extract text features from the user nickname using a translation model to obtain nickname text features.

[0106] Step 1005 : predicting the gender of the social user based on the avatar image features and nickname text features.

[0107] It should be noted that the execution process of step 1001 and step 1004 to step 1005 can be implemented in any way in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0108] In summary, the avatar image is input into the first dense convolutional layer; for each remaining dense convolutional layer in the dense convolutional neural network except the first dense convolutional layer, the output of each dense convolutional layer in the front is taken as the input of the dense convolutional layer of the current layer for image feature extraction to obtain the avatar image feature. Thus, the avatar image feature corresponding to the avatar image can be accurately obtained through the dense convolutional neural network.

[0109] In order to accurately obtain the nickname text feature, the nickname text is input into the translation model, and the translation model is used to extract the feature of the input text to obtain the nickname text feature. Figure 12 As shown in the embodiment, Figure 12 The flowchart of another prediction method provided by the embodiment of the present disclosure is shown in the figure. In the embodiment of the present disclosure, the characters in the user nickname can be encoded to obtain an input encoding sequence, and the encoding sequence is input into the translation model for feature extraction to obtain the nickname text feature. Figure 9 The embodiment shown can include the following steps:

[0110] Step 1201, obtaining the avatar image of the social user and the user nickname.

[0111] Step 1202, using the convolutional neural network to extract the image feature of the avatar image to obtain the avatar image feature.

[0112] Step 1203, encoding the characters in the user nickname to obtain an input encoding sequence; wherein the input encoding sequence includes n character positions.

[0113] In the embodiment of the present disclosure, the characters in the user nickname can be encoded to obtain the input encoding sequence corresponding to the user nickname, wherein the encoding sequence includes n character positions, and the maximum value of n can be 32.

[0114] Step 1204, inputting the input encoding sequence into at least one attention encoding layer in the translation model for feature extraction to obtain an m-dimensional encoding vector on the n character positions.

[0115] In the embodiment of the present disclosure, the translation model can include one or more attention encoding layers, and the input encoding sequence is input into at least one attention encoding layer in the translation model for feature extraction to obtain an m-dimensional encoding vector on the n character positions.

[0116] Step 1205, performing vector averaging on the m-dimensional encoding vector on the n character positions to obtain an m-dimensional average encoding vector.

[0117] Further, the feature n x m of the output of the last layer of the translation model is vector-averaged on the n characters to obtain an m-dimensional average encoding vector.

[0118] Step 1206, performing nonlinear mapping according to the m-dimensional average encoding vector to obtain an m'-dimensional text feature, where m' is greater than m.

[0119] Further, the m-dimensional vector is nonlinearly mapped to obtain an m'-dimensional nickname text feature, where m' is greater than m.

[0120] For example, as shown in Figure 13 , a structure diagram of a translation model provided by an embodiment of the present disclosure is shown. Figure 13 The translation model (such as Figure 13 The right part, Multi-Head Attention) can include 4 layers of attention encoding layers, and the encoding sequence (the longest 32 bits) is input into the translation model. The last layer of the translation model can output a 32x128 encoding vector. The vector average is performed on the 32x128 encoding vector on 32 characters, and a 128-dimensional average encoding vector can be obtained. The 128-dimensional average encoding vector is nonlinearly transformed to obtain a 256-dimensional nickname text feature.

[0121] Among them, Multi-Head Attention is to project Q, K, and V through multiple (such as h) different linear transformations, and finally splice different attention results; then, the scaled dot product (Scaled Dot-Product Attention) is used for attention calculation; then, the result of attention calculation is processed by the connection layer (Concat) and the linear layer (Liner) for nonlinear transformation. The structure of Scaled Dot-Product Attention can be as shown in Figure 13 The left part. In Scaled Dot-Product Attention, Q and K are matrix multiplied (MatMul), and the result of matrix multiplication is scaled (Scale) and masked (Mask). The result after masking is processed by the activation function SoftMax, and the processing result of the activation function is matrix multiplied with V, which can be expressed as the following formula:

[0122]

[0123] Among them, QKV are three matrices obtained by performing matrix operations on the input of the attention layer and the model parameters. d k represents a normalization factor, and T represents a transpose operation of a matrix. When d k is small, the dot product result is similar, and when d k is large, it performs better without scaling, but the calculation speed of the dot product is faster, and scaling can reduce the influence after scaling.

[0124] Step 1207, predicting the gender of the social user according to the avatar image feature and the nickname text feature.

[0125] It should be noted that the execution processes of steps 1201 to 1202 and step 1207 can be implemented by any of the embodiments of the present disclosure, and the present disclosure does not limit this, and will not be repeated here.

[0126] In summary, the characters in the user nickname are encoded to obtain an input encoding sequence; wherein the input encoding sequence includes n character positions; the input encoding sequence is input into at least one attention encoding layer in the translation model for feature extraction to obtain an m-dimensional encoding vector at the n character positions; the m-dimensional encoding vector at the n character positions is averaged to obtain an m-dimensional average encoding vector; and the m-dimensional average encoding vector is nonlinearly mapped to obtain an m' dimensional nickname text feature. Thus, the translation model can accurately obtain the nickname text feature.

[0127] In order to accurately predict the gender of the social user according to the fusion feature, as shown in Figure 14 Figure 14 The flowchart of another prediction method provided by the embodiments of the present disclosure is shown in the figure. In the embodiments of the present disclosure, before predicting the gender of the social user according to the fusion feature, the prediction network can be trained, so that the trained prediction network predicts the gender of the social user according to the fusion feature, Figure 14 The embodiments shown can include the following steps:

[0128] Step 1401, obtaining an avatar image of a social user and a user nickname.

[0129] Step 1402, performing image feature extraction on the avatar image by using a convolutional neural network to obtain an avatar image feature.

[0130] Step 1403, performing text feature extraction on the user nickname by using a translation model to obtain a nickname text feature.

[0131] Step 1404, obtaining a plurality of training samples.

[0132] In the embodiments of the present disclosure, each training sample can include an avatar image and a corresponding user nickname, and each training sample can be labeled with an expected gender. The training samples can be collected online in a social software by using a web crawler technology, and the avatar image and the corresponding user nickname are used as the training samples. Alternatively, the avatar image in the training sample can be an offline collected image, or the avatar image in the training sample can be an artificially synthesized image, and the user nickname can be an artificially edited nickname, etc. The embodiments of the present disclosure do not limit this.

[0133] ​At step 1405, the avatar image and the user nickname in the plurality of training samples are input into the prediction network to obtain a second confidence of an output of an activation function in the prediction network.

[0134] Further, the avatar image and the user nickname in the training sample are input into a full connection layer of the prediction network, and an output of the full connection layer is activated by using an activation function, and the activation function can output a second confidence corresponding to the training sample.

[0135] At step 1406, the second confidence in a set interval is filtered out from the second confidence corresponding to each training sample to retain a plurality of target confidences.

[0136] It should be understood that, since some training samples (avatar and nickname) cannot distinguish the gender, the loss value fluctuation in the final stage of training of the prediction network will be more obvious, but the gender prediction of the social user corresponding to these training samples is often inaccurate (sometimes the label of these training samples is male, and sometimes the label is female), and after analysis, it can be determined that the normalized probability of these training samples is not much different, and the confidence is basically distributed in the set interval. Therefore, in the embodiment of the present disclosure, the second confidence of the training sample corresponding to the output of the activation function can be filtered out in the set interval to retain a plurality of target confidences. The example code can be as follows:

[0137]

[0138] / / Mask the idx of sigmoid below the threshold

[0139] eg: threshold = 0.8 / / Need to keep prob > 0.8 and prob < 0.2

[0140] probs = [0.81, 0.7, 0.4, 0.19, 0.3, 0.7]

[0141] Get mask = [1, 0, 0, 1, 0, 0] / /

[0142] def get_mask(self, probs, threshold):

[0143] probs = tf.reshape(probs, [-1])

[0144] mask1 = tf.math.less(probs, 1-threshold)

[0145] mask2 = tf.math.greater(probs, threshold)

[0146] ​return tf.math.logical_or(mask1,mask2)

[0147] def mask_loss(self,loss,probs):

[0148] mask=self.get_mask(probs,self.mask_loss_thres)

[0149] mask=tf.cast(mask,loss.dtype)

[0150] recall = tf.reduce_mean(mask)

[0151] loss = loss * mask

[0152] / / We threw away some of the data loss, so we need to amplify the remaining loss.

[0153] Prevent the model from predicting all results into this interval / /

[0154] loss=tf.math.divide_no_nan(loss,recall)

[0155] return loss”.

[0156] Step 1407 : Determine multiple loss values ​​based on the retained multiple target confidences and the expected genders marked by the corresponding training samples.

[0157] Furthermore, by comparing the multiple retained target confidences with the preset threshold, the predicted genders corresponding to the multiple target confidences can be determined. Based on the predicted gender and the expected gender, combined with the set loss function, multiple loss values ​​can be determined.

[0158] Step 1408: Adjust model parameters of the prediction network according to the multiple loss values.

[0159] Then, according to the multiple loss values, the model parameters of the prediction network are adjusted to minimize the loss value.

[0160] As an example, multiple loss values ​​are adjusted based on the ratio of the total number of multiple training samples to the number of second confidence levels within a set interval; and model parameters of the prediction network are adjusted based on the adjusted loss values.

[0161] That is, in the embodiment of the present disclosure, the total number of the plurality of training samples is compared with the number of the second confidence degrees in the set interval to determine a ratio of the total number of the plurality of training samples and the number of the second confidence degrees in the set interval, and according to the ratio, the plurality of loss values are isometrically amplified, the plurality of loss values after isometric amplification are taken as the loss values after value adjustment, and according to the loss values after value adjustment, the model parameters of the prediction network are adjusted.

[0162] As another example, a gradient centering strategy is adopted to adjust the gradients of the loss values, and according to the adjusted gradients of the loss values, the model parameters of the prediction network are adjusted.

[0163] In the embodiment of the present disclosure, gradient centering refers to centering the gradient vector to zero mean, centering and adjusting the gradient vector of each loss value, and adjusting the model parameters of the prediction network according to the adjusted gradients of each loss value to improve the stability of the training of the prediction network and improve the final result. The code of gradient centering can be as follows:

[0164]

[0165] Step 1409, predicting the gender of the social user according to the avatar image features and the nickname text features.

[0166] It should be noted that the execution processes of steps 1401 to 1403 and step 1409 can be implemented by any one of the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be repeated.

[0167] In summary, by obtaining a plurality of training samples, the avatar image and the user nickname in the plurality of training samples are predicted by using the prediction network to obtain the second confidence degree output by the activation function in the prediction network, the second confidence degrees in the set interval are filtered out from the second confidence degrees corresponding to each training sample to retain a plurality of target confidence degrees, and the plurality of loss values are determined according to the retained plurality of target confidence degrees and the expected gender labeled by the corresponding training sample. Thus, the prediction network can be trained by using the training samples, and the trained prediction network can predict the gender of the social user according to the fused features.

[0168] The prediction method of the embodiment of the present disclosure improves the gender prediction accuracy of the social user by extracting the avatar image features corresponding to the avatar image of the social user and the nickname text features corresponding to the user nickname, and predicting the gender of the social user according to the avatar image features and the nickname text features.

[0169] In order to push personalized images matched with the gender to the social user according to the gender of the social user to improve the user experience, as shown in Figure 15 Figure 15 ​A flowchart of a recommended method provided in the embodiment of the present disclosure is provided. In the embodiment of the present disclosure, Figures 1 to 14 The prediction method described in the embodiment predicts the gender of a social user and recommends image resources that match the gender to the social user based on the predicted gender. Figure 15 The illustrated embodiment may include the following steps:

[0170] Step 1501, adopt Figures 1 to 14 The prediction method described in the embodiment predicts the gender of a social user.

[0171] In the embodiment of the present disclosure, it is possible to use Figures 1 to 14 The prediction method described in the embodiment predicts the gender of a social user based on their profile picture and nickname.

[0172] Step 1502 : Recommending image resources that match the predicted gender to the social user.

[0173] Furthermore, when recommending image resources to social users, image resources that match the predicted gender may be recommended to the social users.

[0174] In summary, by adopting Figures 1 to 14 The prediction method described in the embodiment predicts the gender of a social user and recommends image resources that match the gender to the social user based on the predicted gender. Thus, personalized images that match the gender are pushed to the social user based on the predicted gender of the social user, thereby improving the user experience.

[0175] In order to achieve the above Figures 1 to 14 In an embodiment, the present disclosure further provides a prediction device.

[0176] Figure 16 A schematic diagram of the structure of a prediction device provided in an embodiment of the present disclosure.

[0177] like Figure 16 As shown, the prediction device includes: an acquisition module 1610 , a first extraction module 1620 , a second extraction module 1630 and a prediction module 1640 .

[0178] Among them, the acquisition module 1610 is used to obtain the social user's avatar image and user nickname; the first extraction module 1620 is used to use a convolutional neural network to extract image features of the avatar image to obtain avatar image features; the second extraction module 1630 is used to use a translation model to extract text features of the user nickname to obtain nickname text features; the prediction module 1640 is used to predict the gender of the social user based on the avatar image features and nickname text features.

[0179] As a possible implementation manner of the embodiment of the present disclosure, the prediction module 1640 is configured to: obtain a difference feature between the avatar image feature and the nickname text feature; input the avatar image feature, the nickname text feature and the difference feature into an attention network to obtain an attention weight of the avatar image feature and the nickname text feature; and fuse the avatar image feature and the nickname text feature according to the attention weight to obtain a fused feature, and predict the gender of the social user according to the fused feature.

[0180] As a possible implementation manner of the embodiment of the present disclosure, the prediction module 1640 is further configured to: input the fused feature into a full connection layer of the prediction network; and activate an output of the full connection layer by using an activation function to obtain a first confidence degree, and determine the gender of the social user according to whether the first confidence degree is less than a set threshold.

[0181] As a possible implementation manner of the embodiment of the present disclosure, the prediction module 1640 is further configured to: generate an additional text feature according to at least one of a length of the user nickname, and a number of visual emotional symbols, a number of digits and a number of letters contained in the user nickname; splice the additional text feature with the nickname text feature, and fuse the spliced nickname text feature and the avatar image feature according to the attention weight to obtain the fused feature.

[0182] As a possible implementation manner of the embodiment of the present disclosure, the convolutional neural network is a dense convolutional neural network including multiple dense convolutional layers, and the first extraction module 1620 is configured to: input the avatar image into a first dense convolutional layer; and for each remaining dense convolutional layer in the dense convolutional neural network except the first dense convolutional layer, perform image feature extraction by taking an output of each dense convolutional layer in front as an input of the current dense convolutional layer to obtain the avatar image feature.

[0183] As a possible implementation manner of the embodiment of the present disclosure, the second extraction module 1630 is configured to: encode characters in the user nickname to obtain an input encoding sequence; the input encoding sequence includes n character positions; input the input encoding sequence into at least one attention encoding layer in the translation model to perform feature extraction to obtain m-dimensional encoding vectors at the n character positions; perform vector averaging on the m-dimensional encoding vectors at the n character positions to obtain an m-dimensional average encoding vector; and perform nonlinear mapping on the m-dimensional average encoding vector to obtain an m'-dimensional nickname text feature, where m' is greater than m.

[0184] As a possible implementation manner of the embodiment of the present disclosure, the prediction device 1600 further includes a sample acquisition module, a sample prediction module, a screening module, a determination module and a training module.

[0185] The sample obtaining module is configured to obtain a plurality of training samples; the sample prediction module is configured to predict the avatar image and the user nickname in the plurality of training samples by using the prediction network to obtain a second confidence output by an activation function in the prediction network; the screening module is configured to screen out the second confidence within the set interval from the second confidence corresponding to each of the training samples to retain a plurality of target confidences; the determination module is configured to determine a plurality of loss values according to the retained plurality of target confidences and an expected gender labeled by the corresponding training sample; and the training module is configured to adjust the model parameters of the prediction network according to the plurality of loss values.

[0186] As a possible implementation manner of the embodiments of the present disclosure, the training module is configured to adjust the values of the plurality of loss values according to a ratio of a total number of the plurality of training samples to a number of the second confidences within the set interval, and adjust the model parameters of the prediction network according to the loss values after the value adjustment.

[0187] As a possible implementation manner of the embodiments of the present disclosure, the training module is further configured to adjust the gradient of each loss value by using a gradient centering strategy, and adjust the model parameters of the prediction network according to the gradient of each loss value after the adjustment.

[0188] The prediction device of the embodiments of the present disclosure improves the gender prediction accuracy of the social user by extracting the avatar image features corresponding to the avatar image of the social user and the nickname text features corresponding to the nickname of the social user, and predicting the gender of the social user according to the avatar image features and the nickname text features.

[0189] To achieve the above Figure 15 The present disclosure further provides a recommendation device.

[0190] Figure 17 A structural schematic diagram of a recommendation device according to an embodiment of the present disclosure is shown.

[0191] As Figure 17 shown, the recommendation device 1700 includes a prediction module 1710 and a recommendation module 1720.

[0192] The prediction module 1710 is configured to predict the gender of the social user by using the device as Figure 16 described above; and the recommendation module 1720 is configured to recommend image resources matching the gender to the social user according to the predicted gender.

[0193] The recommendation device of the embodiments of the present disclosure improves the accuracy of the gender prediction of the social user by using the device as Figure 16The prediction device in the embodiment predicts the gender of the social user, and according to the predicted gender, recommends the image resource matched with the gender to the social user, so that personalized images matched with the gender can be pushed to the social user according to the predicted gender of the social user, and user experience is improved.

[0194] To achieve the above-mentioned embodiment, the present disclosure further proposes an electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform Figures 1 to 14 The prediction method in the embodiment, or Figure 15 The recommendation method in the embodiment.

[0195] To achieve the above-mentioned embodiment, the present disclosure further proposes a non-transitory computer-readable storage medium storing computer instructions, the computer instructions being used to make the computer perform Figures 1 to 14 The prediction method in the embodiment, or Figure 15 The recommendation method in the embodiment.

[0196] To achieve the above-mentioned embodiment, the present disclosure further proposes a computer program product, comprising a computer program, the computer program being used to implement Figures 1 to 14 The prediction method in the embodiment, or Figure 15 The recommendation method in the embodiment.

[0197] As Figure 18 shown, Figure 18 is a block diagram of an electronic device according to the prediction method or the recommendation method of the embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown in the figures, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the present disclosure described and / or claimed in this document.

[0198] As Figure 15As shown, the electronic device includes: one or more processors 1801, a memory 1802, and interfaces for connecting various components, including a high-speed interface and a low-speed interface. The various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to an interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 18 A processor 1801 is taken as an example.

[0199] Memory 1802 is a non-transitory computer-readable storage medium provided in the present disclosure. The memory stores instructions executable by at least one processor, causing the at least one processor to perform the prediction method or recommendation method provided in the present disclosure. The non-transitory computer-readable storage medium of the present disclosure stores computer instructions for causing a computer to perform the prediction method or recommendation method provided in the present disclosure.

[0200] The memory 1802 is a non-transitory computer-readable storage medium that can be used to store non-transitory software programs, non-transitory computer executable programs and modules, such as the program instructions / modules corresponding to the prediction method and recommendation method in the embodiments of the present disclosure (for example, the attached Figure 16 The acquisition module 1610, the first extraction module 1620, the second extraction module 1630 and the prediction module 1640 are shown in FIG. Figure 17 The processor 1801 executes the non-transient software programs, instructions, and modules stored in the memory 1802 to execute various functional applications and data processing of the server, that is, to implement the prediction method or recommendation method in the above method embodiment.

[0201] The memory 1802 can include a program region that can store an operating system, at least one application required for a function, and a data region that can store data created according to the generation of the semantic representation model, etc. In addition, the memory 1802 can include a high-speed random access memory, and can further include a non-transitory memory such as at least one disk storage device, a flash memory device, or other non-transitory solid state memory device. In some embodiments, the memory 1802 can optionally include a memory disposed remotely with respect to the processor 1801, and these remote memories can be connected to the electronic device of the prediction method or recommendation method through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0202] The electronic device of the prediction method or recommendation method can further include an input device 1803 and an output device 1804. The processor 1801, the memory 1802, the input device 1803, and the output device 1804 can be connected through a bus or other means, Figure 18 For example, the connection through the bus is exemplified.

[0203] The input device 1803 can receive input digital or character information, and generate key signal input related to a user setting and a function control of the generated electronic device of the prediction method or recommendation method, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, a pointing stick, one or more mouse buttons, a trackball, a joystick, etc. The output device 1804 can include a display device, an auxiliary lighting device (e.g., an LED), a haptic feedback device (e.g., a vibration motor), etc. The display device can include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display, and a plasma display. In some embodiments, the display device can be a touch screen.

[0204] Various embodiments of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0205] These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor, and can be implemented in a high-level procedural and / or object-oriented programming language, and / or in assembly / machine language. As used herein, the terms "machine-readable medium" "computer-readable medium" refers to any computer program product, apparatus and / or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0206] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0207] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0208] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other.

[0209] In addition, the acquisition, storage and application of information involved in the technical solutions of the present disclosure comply with the relevant legal regulations and do not violate public order and good customs.

[0210] It should be understood that the various forms of flow shown above can be used to reorder, add, or delete steps. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, without limitation herein, so long as the desired results of the technology presented by the present disclosure are achieved.

[0211] The specific embodiments discussed above do not constrain the scope of the present disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and alternatives can be made to the specific embodiments without departing from the spirit and principles of the present disclosure. Any such modifications, equivalents, and alternatives should be considered within the scope of the present disclosure.

Claims

1. A prediction method, characterized in that: include: Get the social user's profile picture and nickname; Extracting image features from the portrait image using a convolutional neural network to obtain portrait image features; Extracting text features from the user nickname using a translation model to obtain nickname text features; Predicting the gender of the social user based on the avatar image features and the nickname text features, including: obtaining difference features between the avatar image features and the nickname text features; inputting the avatar image features, the nickname text features, and the difference features into an attention network to obtain attention weights of the avatar image features and the nickname text features; fusing the avatar image features and the nickname text features based on the attention weights to obtain fused features; and predicting the gender of the social user based on the fused features; Before predicting the gender of the social user according to the fusion feature, the method further includes: Acquire multiple training samples; use a prediction network to predict the avatar images and user nicknames in the multiple training samples to obtain a second confidence level output by an activation function in the prediction network; filter out the second confidence levels within a set interval from the second confidence levels corresponding to the training samples to retain multiple target confidence levels; determine multiple loss values ​​based on the retained multiple target confidence levels and the expected genders marked by the corresponding training samples; adjust the multiple loss values ​​based on the ratio between the total number of the multiple training samples and the number of second confidence levels within the set interval; and adjust the model parameters of the prediction network based on the adjusted loss values.

2. The method according to claim 1, characterized in that The predicting the gender of the social user according to the fusion feature includes: Inputting the fused features into the fully connected layer of the prediction network; Activating the output of the fully connected layer using an activation function to obtain a first confidence level; The gender of the social user is determined according to whether the first confidence level is less than a set threshold.

3. The method according to claim 1, characterized in that The step of fusing the avatar image feature and the nickname text feature according to the attention weight to obtain a fused feature includes: generating an additional text feature based on the length of the user nickname and at least one of the number of visual emotional symbols, the number of numbers, and the number of letters contained therein; The additional text feature is spliced ​​with the nickname text feature, and the spliced ​​nickname text feature and the avatar image feature are fused according to the attention weight to obtain a fused feature.

4. The method according to any one of claims 1 to 3, characterized in that The convolutional neural network is a dense convolutional neural network including multiple layers of dense convolutional layers. The image feature extraction of the avatar image using the convolutional neural network to obtain avatar image features includes: Inputting the head portrait image into the first dense convolutional layer; For each remaining dense convolutional layer except the first dense convolutional layer in the dense convolutional neural network, the output of each dense convolutional layer in the previous order is used as the input of the dense convolutional layer of the current layer to extract image features to obtain the avatar image features.

5. The method according to any one of claims 1 to 3, characterized in that The extracting text features of the user nickname using a translation model to obtain nickname text features includes: Encoding the characters in the user nickname to obtain an input code sequence; wherein the input code sequence includes n character positions; Inputting the input code sequence into at least one attention coding layer in the translation model for feature extraction to obtain an m-dimensional code vector at n character positions; Performing vector averaging on the m-dimensional encoding vectors at the n character positions to obtain an m-dimensional average encoding vector; Nonlinear mapping is performed based on the m-dimensional average encoding vector to obtain an m'-dimensional nickname text feature, where m' is greater than m.

6. The method according to claim 1, characterized in that The adjusting model parameters of the prediction network according to the multiple loss values ​​includes: Adopting the gradient centralization strategy to adjust the gradient of each loss value; Adjust the model parameters of the prediction network according to the adjusted gradients of the loss values.

7. A recommendation method, characterized in that: include: Predicting the gender of a social user using the method according to any one of claims 1 to 6; Based on the predicted gender, image resources matching the gender are recommended to the social user.

8. A prediction device, characterized in that: include: The acquisition module is used to obtain the social user's avatar and user nickname; A first extraction module is used to extract image features from the portrait image using a convolutional neural network to obtain portrait image features; A second extraction module is used to extract text features of the user nickname using a translation model to obtain nickname text features; A prediction module, configured to predict the gender of the social user based on the avatar image features and the nickname text features, comprising: obtaining a difference feature between the avatar image features and the nickname text features; inputting the avatar image features, the nickname text features, and the difference feature into an attention network to obtain attention weights of the avatar image features and the nickname text features; fusing the avatar image features and the nickname text features based on the attention weights to obtain a fused feature; and predicting the gender of the social user based on the fused feature; Before predicting the gender of the social user according to the fusion feature, the method further includes: Acquire multiple training samples; use a prediction network to predict the avatar images and user nicknames in the multiple training samples to obtain a second confidence level output by an activation function in the prediction network; filter out the second confidence levels within a set interval from the second confidence levels corresponding to the training samples to retain multiple target confidence levels; determine multiple loss values ​​based on the retained multiple target confidence levels and the expected genders marked by the corresponding training samples; adjust the multiple loss values ​​based on the ratio between the total number of the multiple training samples and the number of second confidence levels within the set interval; and adjust the model parameters of the prediction network based on the adjusted loss values.

9. The device according to claim 8, characterized in that The prediction module is further configured to: Inputting the fused features into the fully connected layer of the prediction network; Activating the output of the fully connected layer using an activation function to obtain a first confidence level; The gender of the social user is determined according to whether the first confidence level is less than a set threshold.

10. The device according to claim 8, characterized in that The prediction module is further configured to: generating an additional text feature based on the length of the user nickname and at least one of the number of visual emotional symbols, the number of numbers, and the number of letters contained therein; The additional text feature is spliced ​​with the nickname text feature, and the spliced ​​nickname text feature and the avatar image feature are fused according to the attention weight to obtain a fused feature.

11. The device according to any one of claims 8 to 10, characterized in that: The convolutional neural network is a dense convolutional neural network including multiple layers of dense convolutional layers, and the first extraction module is used to: Inputting the head portrait image into the first dense convolutional layer; For each remaining dense convolutional layer except the first dense convolutional layer in the dense convolutional neural network, the output of each dense convolutional layer in the previous order is used as the input of the dense convolutional layer of the current layer to extract image features to obtain the avatar image features.

12. A recommendation device, characterized in that: include: A prediction module, configured to predict the gender of a social user using the apparatus according to any one of claims 8 to 11; The recommendation module is configured to recommend image resources matching the predicted gender to the social user based on the predicted gender.

13. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method according to any one of claims 1 to 6, or the method according to claim 7.

14. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable the computer to execute the method according to any one of claims 1 to 6, or to execute the method according to claim 7.

Images