Artificial intelligence-based recommendation method, apparatus, and electronic device

Abstract

The application provides an artificial intelligence-based recommendation method and device, electronic equipment and computer readable storage medium; the method comprises: determining a plurality of historical videos published by an interactive account, wherein the interactive account has an interactive relationship with a target user account; segmenting the plurality of historical videos to obtain a plurality of video segments of each historical video, and extracting frames from the plurality of video segments of each historical video to obtain a representative frame of each video segment; performing feature extraction processing based on the representative frame of each video segment to obtain a first video feature of the interactive account; performing feature enhancement processing based on the attention mechanism on the first video feature of the interactive account to obtain a second video feature of the interactive account; determining a to-be-recommended account based on the second video feature of the interactive account, and performing a recommendation operation on the corresponding target user account based on the to-be-recommended account. Through the application, the account recommendation accuracy can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to artificial intelligence technology, and in particular to a recommendation method and device based on artificial intelligence, an electronic device, and a computer readable storage medium. BACKGROUND

[0002] Artificial intelligence (AI) is the use of digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0003] Account recommendation is an important application of artificial intelligence, and an important part of account recommendation is video account recommendation. In related technologies, only independent video content is used for feature learning, and it is difficult to effectively and accurately learn the features of an account that publishes multiple video contents, so that the account features learned based on independent video content cannot accurately represent the corresponding account, and the account recommended in this way cannot effectively meet the interests of users, causing a poor experience for users. SUMMARY

[0004] The embodiments of the present application provide a recommendation method and device based on artificial intelligence, an electronic device, and a computer readable storage medium, which can improve the accuracy of account recommendation.

[0005] The technical solutions of the embodiments of the present application are as follows:

[0006] The embodiments of the present application provide a recommendation method based on artificial intelligence, comprising:

[0007] determining a plurality of historical videos published by an interactive account, wherein the interactive account has an interactive relationship with a target user account;

[0008] segmenting the plurality of historical videos to obtain a plurality of video segments of each historical video, and performing frame extraction processing on the plurality of video segments of each historical video to obtain a representative frame of each video segment;

[0009] performing feature extraction processing based on the representative frame of each video segment to obtain a feature extraction result of each video segment, and determining a first video feature of the interactive account based on the feature extraction result;

[0010] performing feature enhancement processing based on the attention mechanism on the first video feature of the interactive account to obtain a second video feature of the interactive account;

[0011] determining a to-be-recommended account based on the second video feature of the interactive account, and performing a recommendation operation corresponding to the target user account based on the to-be-recommended account.

[0012] This application provides an artificial intelligence-based recommendation device, comprising:

[0013] The video module is used to identify multiple historical videos posted by an interactive account, wherein the interactive account has an interactive relationship with the target user account;

[0014] The frame extraction module is used to segment the multiple historical videos to obtain multiple video segments of each historical video, and to extract frames from the multiple video segments of each historical video to obtain a representative frame of each video segment.

[0015] The extraction module is used to perform feature extraction processing based on representative frames of each video segment to obtain feature extraction results for each video segment, and to determine the first video feature of the interactive account based on the feature extraction results.

[0016] An attention module is used to perform attention-based feature enhancement processing on the first video features of the interactive account to obtain the second video features of the interactive account.

[0017] The recommendation module is used to determine the account to be recommended based on the second video features of the interactive account, and to perform a recommendation operation corresponding to the target user account based on the account to be recommended.

[0018] In the above scheme, the extraction module is further configured to: when the representative frame is an optical flow field frame corresponding to the video segment, perform optical flow feature extraction processing based on the temporal dimension on the representative frame of each video segment to obtain the temporal features of the video segment; when the representative frame is an image frame corresponding to the video segment, perform content feature extraction processing based on the content dimension on the representative frame of each video segment to obtain the content features of the video segment.

[0019] In the above scheme, when the feature extraction result includes the temporal features and content features of each video segment, the extraction module is further configured to: perform the following processing for each historical video: concatenate multiple temporal features corresponding one-to-one with multiple video segments of the historical video to obtain the temporal features of the historical video; concatenate multiple content features corresponding one-to-one with multiple video segments of the historical video to obtain the content features of the historical video; concatenate the temporal features and the content features of the historical video to obtain the video features of the historical video; and concatenate the video features of the multiple historical videos to obtain the first video features of the interactive account.

[0020] In the scheme, when the feature extraction result comprises content features of each video segment, the extraction module is further configured to perform the following processing on each historical video: performing splicing processing on a plurality of content features corresponding to a plurality of video segments of the historical video to obtain a video feature of the historical video; and performing splicing processing on video features of the plurality of historical videos to obtain a first video feature of the interactive account.

[0021] In the scheme, the extraction module is further configured to perform splicing processing on the video features of the plurality of historical videos to obtain a splicing processing result; and perform full connection processing on the splicing processing result to obtain the first video feature of the interactive account.

[0022] In the scheme, the feature enhancement processing based on the attention mechanism is implemented through an attention network; and the attention module is further configured to perform attention processing on an input of an nth attention network in N cascaded attention networks through the nth attention network to obtain an nth attention result; and transmit the nth attention result to an (n+1)th attention network for further attention processing; wherein N is an integer greater than or equal to 2; n is an integer starting from 1 and increasing, and the value range of n satisfies 1≤n≤N-1; when n is 1, the input of the nth attention network is the first video feature; and when n is 2≤n≤N-1, the input of the nth attention network is an attention result of an (n-1)th attention network.

[0023] In the scheme, the attention module is further configured to perform channel-based average pooling processing on the first video feature of the interactive account through the nth attention network to obtain a global feature corresponding to each channel of the first video feature of the interactive account; perform full connection processing on the global feature corresponding to each channel of the first video feature of the interactive account through the nth attention network to obtain an attention value corresponding to each channel of the first video feature of the interactive account; perform dot multiplication processing on the attention value corresponding to each channel of the first video feature of the interactive account and a feature value of each channel in the first video feature of the interactive account; and update the first video feature of the interactive account based on a dot multiplication processing result.

[0024] In the above scheme, the recommendation module is further configured to: for each candidate account, perform the following processing: obtaining historical videos published by the candidate account; performing feature extraction processing on the historical videos to obtain first video features of the candidate account; performing feature enhancement processing based on an attention mechanism on the first video features of the candidate account to obtain second video features of the candidate account; determining a video similarity between the second video features of the interaction account and the second video features of each of the candidate accounts, and regarding a candidate account with a video similarity greater than a video similarity threshold as a to-be-recommended account.

[0025] In the above scheme, the recommendation module is further configured to: perform at least one of the following processing: sending information for recommending the to-be-recommended account to the target user account; sending information for recommending a plurality of historical videos published by the to-be-recommended account to the target user account.

[0026] In the above scheme, the second video features of the interaction account are determined by an account learning model; the apparatus further comprises a training module configured to: before performing segmentation processing on the plurality of historical videos to obtain a plurality of video segments of each of the historical videos, train the account learning model by: obtaining a plurality of account samples, and constructing a plurality of triple sample based on a number of associated users of the plurality of account samples; substituting second video feature samples corresponding to each account sample in the triple sample into a triple loss function to determine parameters of the account learning model when the triple loss function reaches a minimum value.

[0027] In the above scheme, the training module is further configured to: determine a relevance between any two account samples in the plurality of account samples based on a number of associated users of the plurality of account samples; obtain a plurality of candidate triple samples from the plurality of account samples; wherein each of the candidate triple samples includes three account samples, and there is at least one different account sample between any two of the candidate triple samples; select the triple sample meeting a training condition from the plurality of candidate triple samples; wherein the training condition includes that the relevance between two account samples in the triple sample is greater than a relevance threshold, and the relevance between two account samples is less than the relevance threshold.

[0028] In the above scheme, the training module is further configured to: for the any two account samples, perform the following processing: obtaining a number of associated users of each of the account samples in the any two account samples; obtaining a same number of associated users between the any two account samples; determining a relevance positively correlated with the same number of associated users and negatively correlated with the number of associated users of each of the account samples.

[0029] An electronic device is provided in an embodiment of the application, comprising:

[0030] a memory configured to store executable instructions;

[0031] a processor configured to execute the executable instructions stored in the memory to implement an AI-based recommendation method provided in an embodiment of the application.

[0032] A computer-readable storage medium is provided in an embodiment of the application, storing executable instructions configured to be executed by a processor to implement an AI-based recommendation method provided in an embodiment of the application.

[0033] An embodiment of the application has the following beneficial effects:

[0034] By performing feature extraction processing based on segmented frame extraction on historical videos, the features of each historical video are accurately depicted while effectively avoiding duplicate frames, and then the features of multiple historical videos are converted into the features of an interactive account through an attention mechanism, thereby learning the differences between historical videos and more accurately depicting the features of the account, and thus improving the accuracy of account recommendation based on the similarity of accounts. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figures 1A-1B is a video classification model structure diagram in the related art;

[0036] Figure 2 is a structure diagram of an AI-based recommendation system provided in an embodiment of the application;

[0037] Figure 3 is a structure diagram of an electronic device provided in an embodiment of the application;

[0038] Figures 4A-4C is a flow diagram of an AI-based recommendation method provided in an embodiment of the application;

[0039] Figure 5 is a structure diagram of an AI-based account learning model provided in an embodiment of the application;

[0040] Figure 6 is a feature extraction principle diagram based on AI provided in an embodiment of the application;

[0041] Figures 7A-7B is a product interface diagram of an AI-based recommendation method provided in an embodiment of the application. DETAILED DESCRIPTION

[0042] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings, and the described embodiments should not be regarded as limitations to the present application. All other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0043] In the following description, "some embodiments" are referred to, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0044] In the following description, the terms "first\second\third" are only to distinguish similar objects, and do not represent a specific order of the objects. It can be understood that "first\second\third" can be interchanged in a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0045] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0046] The relevant data collection and processing in the embodiments of the present application should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of authorization of laws and regulations and the personal information subject.

[0047] Before further detailing the embodiments of the present application, the terms and terms involved in the embodiments of the present application are explained, and the terms and terms involved in the embodiments of the present application are applicable to the following explanations.

[0048] 1) Backbone network, specifically refers to a convolutional network for extracting image features. Common backbone networks include BN-Inception network, InceptionV3 network, ResNet network, Xception network, EfficientNet network, etc.

[0049] 2) Account vector, the account vector is a dense vector with fixed dimensions, and its semantic relevance can be calculated by vector distance.

[0050] Referring to Figures 1A-1B , Figures 1A-1Bis a video classification model structure diagram in the related art, and the related work of classifying videos in the related art is to classify images, for example, regarding a video as a collection of image frames, using an image classification model to classify image frames, but there are a large number of repeated frames and useless frames in the conversion process from a video to a frame. Referring to Figure 1A , in the related art, features can be extracted from each image frame (image frame 1-image frame K) of a video through a convolutional network, and then each feature is regarded as a time point and sequentially input into a long short-term memory network to obtain a classification result of the video. Referring to Figure 1B , in the related art, the original two-dimensional convolution kernel can also be expanded to three dimensions based on a three-dimensional convolution, similar to the manner of two-dimensional convolution in the spatial dimension, which can extract features from the bottom up for image frames (image frame 1 to image frame K) in the time dimension to obtain a classification result of the video. The three-dimensional convolution can achieve good classification accuracy.

[0051] The applicant found during the implementation of the present application that since the long short-term memory network does not limit the length of the sequence, it can be used to process videos of any length, but the long short-term memory network itself has the problems of gradient disappearance and explosion, making it difficult to train satisfactory classification results, and since the long short-term memory network needs to be input frame by frame, the classification speed is slow. When the convolution kernel is expanded, since the convolution kernel is expanded from two dimensions to three dimensions, the number of calculation parameters also increases exponentially, so the classification speed also decreases accordingly. Moreover, in the related art, there are only feature learning and classification methods for videos, and there is no neural network specifically for account dimension processing. In the related art, the features of the videos published by an account are directly averaged, and the average processing result is used as the account feature of the account. This way of learning the features of the account will result in a loss of accuracy in the obtained features, especially for accounts with different published texts, i.e., for accounts with large differences in published video content, the learning accuracy of the features is even worse.

[0052] To solve the above problems, the present application provides a recommendation method and device based on artificial intelligence, an electronic device and a computer readable storage medium, which can learn the features of corresponding interactive accounts based on the features of a plurality of historical videos, thereby accurately learning the features of corresponding interactive accounts, thereby effectively improving the recommendation accuracy for all accounts. The following describes an exemplary application of the electronic device provided by the present application, which can be implemented as a server. The following describes an exemplary application when the device is implemented as a server.

[0053] Referring to Figure 2 , Figure 2Fig. 1 is a structural schematic diagram of a recommendation system based on artificial intelligence provided by an embodiment of the present application. The account recommendation system can be used to support various account recommendation scenarios, such as an application scenario of recommending a media account publishing a video, and the like. In the account recommendation system, the terminal 400 is connected to the server 200 through a network, which can be a wide area network or a local area network, or a combination of the two.

[0054] In some embodiments, the functions of the account recommendation system are implemented based on various modules in the server 200. In the process in which the user uses the client, the server 200 takes the collected accounts as training samples, trains the account learning model based on the obtained training samples, integrates the trained account learning model in the server, and in response to the terminal 400 receiving a play operation of a video by the user, the terminal 400 sends a recommendation request instruction to the server 200, the recommendation request instruction carrying an interactive account followed by the user. The server 200 determines the second video features of the interactive account through the account learning model, the server 200 obtains candidate accounts from the database 500, the server 200 determines the second video features of the candidate accounts through the account learning model, determines the similarity between the two second video features, and determines the candidate accounts meeting the similarity condition as candidate accounts to be recommended to the terminal 400, and sends the information of the candidate accounts to be recommended, such as the link of the content published by the candidate accounts, to the terminal 400, so that the terminal 400 adds the candidate accounts to the follow list or directly presents the published content of the candidate accounts.

[0055] In some embodiments, the functions of the account recommendation system are implemented based on various modules in the server 200. In the process in which the user uses the client, the server 200 takes the collected accounts as training samples, trains the account learning model based on the obtained training samples, integrates the trained account learning model in the server, and in response to the terminal 400 receiving a play operation of a video by the user, the terminal 400 sends a recommendation request instruction to the server 200, the recommendation request instruction carrying an interactive account followed by the user. The server 200 determines the second video features of the interactive account through the account learning model, the server 200 obtains candidate accounts from the database 500, the server 200 determines the second video features of the candidate accounts through the account learning model, determines the similarity between the two second video features, and determines the candidate accounts meeting the similarity condition as candidate accounts to be recommended to the terminal 400, and sends the information of the candidate accounts to be recommended, such as the link of the content published by the candidate accounts, to the terminal 400, so that the terminal 400 adds the candidate accounts to the follow list or directly presents the published content of the candidate accounts.

[0056] In some embodiments, the server 200 can be a standalone physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDNs, and basic cloud computing services such as big data and artificial intelligence platforms. The terminal can be a smartphone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, and the like, but is not limited thereto. The terminal and the server can be directly or indirectly connected through wired or wireless communication, which is not limited in the embodiments of the present application.

[0057] Next, the structure of the electronic device for implementing the recommendation method based on artificial intelligence provided by the embodiments of the present application is described. As described above, the electronic device provided by the embodiments of the present application can be the server 200 in Figure 2 . Referring to Figure 3 , Figure 3 , it is a structural schematic diagram of the electronic device provided by the embodiments of the present application, Figure 3 The server 200 shown in FIG. 1 includes at least one processor 210, a memory 250, and at least one network interface 220. The various components in the server 200 are coupled together by a bus system 240. It can be understood that the bus system 240 is used to realize the connection communication between the components. The bus system 240 includes not only a data bus, but also a power bus, a control bus, and a status signal bus. However, for the purpose of clear illustration, all kinds of buses are marked as the bus system 240 in Figure 3 .

[0058] The processor 210 can be an integrated circuit chip with signal processing capability, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor.

[0059] The memory 250 can be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state memory, hard disk drive, optical disk drive, etc. The memory 250 can optionally include one or more storage devices physically located away from the processor 210.

[0060] The memory 250 includes volatile memory or non-volatile memory, and can also include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), and the volatile memory can be random access memory (RAM). The memory 250 described in the embodiments of the present application is intended to include any suitable type of memory.

[0061] In some embodiments, the memory 250 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, which are exemplarily described below.

[0062] The operating system 251 includes system programs for processing various basic system services and performing hardware-related tasks, such as a framework layer, a core library layer, a driver layer, and the like, for implementing various basic services and processing hardware-based tasks; and a network communication module 252 for reaching other computing devices via one or more (wired or wireless) network interfaces 220, such as Bluetooth, Wireless Fidelity (WiFi), and Universal Serial Bus (USB), and the like.

[0063] In some embodiments, the artificial intelligence-based recommendation device provided by the embodiments of the present application can be implemented in a software manner, Figure 3 An artificial intelligence-based recommendation device 255 stored in the memory 250 is shown, which can be software in the form of programs and plug-ins, etc., including the following software modules: a video module 2551, a frame extraction module 2552, an extraction module 2553, an attention module 2554, a recommendation module 2555, and a training module 2556. These modules are logical, and thus can be combined or further split according to the implemented functions. The functions of each module will be described below.

[0064] The artificial intelligence-based recommendation method provided by the embodiments of the present application will be described below in conjunction with exemplary applications and implementations of the server 200 provided by the embodiments of the present application. Referring to Figure 5 , Figure 5 is a structural schematic diagram of an account learning model based on artificial intelligence provided by the embodiments of the present application, which can be applied to an account recommendation system. The account learning model includes a video pulling layer, a segmented random frame extraction layer, a feature extraction layer, and an attention layer. The process of account learning through the account learning model is as follows: the account is input to the video pulling layer of the account learning model, the video pulling layer pulls multiple historical videos published by the account, the segmented random frame extraction layer processes the historical videos in segments to obtain multiple video segments, each video segment is frame-extracted to obtain a representative frame of each video segment, the feature extraction layer performs feature extraction processing on the representative frame of each video segment to obtain the features of the video segments, the features of the multiple video segments are spliced to obtain the frame features of the historical videos, the attention layer splices and fully connects the frame features of the multiple historical videos to determine the first video features of the interactive account, and the attention layer performs feature enhancement processing on the first video features of the interactive account based on the attention mechanism to obtain the second video features of the interactive account as the account features of the account.

[0065] Next, the account recommendation system provided by the embodiments of the present application will be described below in conjunction with an exemplary application and implementation of the account recommendation system. Figure 2Taking the server 200 executing the recommendation system method provided in this application embodiment as an example, the artificial intelligence-based recommendation method provided in this application embodiment is explained. The recommendation system includes a training phase and an application phase. First, the training of the model in the artificial intelligence-based recommendation method provided in this application embodiment is explained.

[0066] See Figure 4A , Figure 4A This is a flowchart illustrating the AI-based recommendation method provided in this application embodiment. The second video feature of the interactive account is determined through an account learning model. Before segmenting multiple historical videos to obtain multiple video segments for each historical video, the method is... Figure 4A The steps 101-102 shown are for training the account learning model.

[0067] In step 101, multiple account samples are obtained, and multiple triplet samples are constructed based on the number of associated users of the multiple account samples.

[0068] In some embodiments, constructing multiple triplet samples based on the number of associated users of multiple account samples in step 101 can be achieved through the following technical solution: determining the correlation between any two account samples in the multiple account samples based on the number of associated users of multiple account samples; obtaining multiple candidate triplet samples from the multiple account samples; wherein each candidate triplet sample includes three account samples, and at least one different account sample exists between any two candidate triplet samples; selecting triplet samples that meet the training conditions from the multiple candidate triplet samples; wherein the training conditions include: there exists a correlation between two account samples in the triplet samples that is greater than a correlation threshold, and there exists a correlation between two account samples that is less than a correlation threshold.

[0069] In some embodiments, the determination of the correlation between any two account samples based on the number of associated users of multiple account samples can be achieved by the following technical solution: performing the following processing on any two account samples: obtaining the number of associated users of each account sample in any two account samples; obtaining the number of identical associated users between any two account samples; determining the correlation that is positively correlated with the number of identical associated users and negatively correlated with the number of associated users of each account sample.

[0070] As an example, the whole network is trained end to end at the training time, the training data is in the form of a triple sample composed of three account samples, the part of the interaction account used to represent the user interest is taken as the account sample constituting the triple sample, for example, the part of the interaction account of the user account is the interaction account followed by the user account within the recent setting time, each triple sample includes three account samples, which are reference samples, positive samples and negative samples, respectively, any one account sample can be taken as a reference sample, under the premise that the reference sample is determined, the positive sample relative to the reference sample and the negative sample relative to the reference sample are determined, the correlation between the positive sample and the reference sample is greater than the correlation threshold, and the correlation between the negative sample and the reference sample is less than the correlation threshold.

[0071] As an example, the correlation R can be calculated by formula (1):

[0072] (1) ;

[0073] Wherein, R is the correlation between the account sample A and the account sample B, fans_A is the number of the account sample A followed by the user account, fans_B is the number of the account sample B followed by the user account, wherein the user account followed the account sample A and the account sample B, and fans_iter represents the number of the account sample A and the account sample B followed by the user account.

[0074] As an example, after obtaining the correlation between the account samples, the positive sample and the negative sample relative to the reference sample are obtained through the correlation threshold (for example, the correlation threshold is 0.1), for example, when the account sample A is taken as the reference sample, the correlation between the account sample A and the account sample B is 0.2, the correlation is greater than the correlation threshold 0.1, the account sample B is taken as the positive sample, and the correlation between the account sample A and the account sample C is 0.08, the correlation is less than the correlation threshold 0.1, the account sample C is taken as the negative sample, if there is no account sample with the correlation less than 0.1 with the account sample A, the account sample is randomly selected as the negative sample, so as to construct the triple (A, B, C) based on the account sample A, the account sample B and the account sample C.

[0075] In step 102, the second video feature sample corresponding to each account sample in the triple sample is substituted into the triple loss function to determine the parameters of the account learning model when the triple loss function reaches the minimum value.

[0076] As an example, the supervised learning is performed in a manner of metric learning, a training target of the metric learning is that spatial distances between accounts simultaneously focused by a user are as close as possible, for example, a distance between account sample A and account sample B is as close as possible, and spatial distances between irrelevant accounts are as far as possible, for example, a distance between account sample A and account sample C is as far as possible, a triple loss function is constructed based on the training target, the triple loss function is composed of two distances, a first distance is a distance between the second video feature of the reference sample and the second video feature of the positive sample, and a second distance is a distance between the second video feature of the reference sample and the second video feature of the negative sample, the triple loss function is positively correlated with the first distance and negatively correlated with the second distance.

[0077] Referring to Figure 4B , Figure 4B is a flowchart of a recommendation method based on artificial intelligence provided by an embodiment of the present application, which will be described in combination with Figure 4B steps 201-205 shown in the figure.

[0078] In step 201, a plurality of historical videos published by an interactive account are determined.

[0079] As an example, the interactive account has an interactive relationship with a target user account.

[0080] As an example, the interactive account can be one or more, and the target user account is associated with at least one interactive account, the interactive account is an account followed by the target user account, or the interactive account is an account with which the target user account has interacted.

[0081] In step 202, the plurality of historical videos are segmented to obtain a plurality of video clips of each historical video, and the plurality of video clips of each historical video are frame extracted to obtain a representative frame of each video clip.

[0082] As an example, referring to Figure 6 , Figure 6 is a schematic diagram of a feature extraction principle based on artificial intelligence provided by an embodiment of the present application, the interactive account publishes a plurality of historical videos, for example, historical video A and historical video B, the historical video A and the historical video B are respectively segmented and frame extracted, taking the historical video A as an example, the historical video A is divided into K video clips, for each video clip in the K video clips, a random frame extraction is performed to obtain an image frame as a representative frame of the corresponding video clip, if the historical video B is also divided into K video clips, an image frame is extracted as a representative frame of the corresponding video clip for each video clip, then a total of 2K representative frames are obtained.

[0083] As an example, the image frame described above is a frame for representing the dimension of video content. When frame extraction is performed, a frame representing the time sequence information of the video can also be extracted. Taking historical video A and historical video B as examples, segmentation and frame extraction are performed on historical video A and historical video B respectively. Taking historical video A as an example, historical video A is divided into K video segments. For each video segment in the K video segments, random frame extraction is performed to obtain a plurality of continuous image frames, and optical flow field mapping is performed on the plurality of continuous image frames to obtain a plurality of optical flow field frames as representative frames of the corresponding video segment. The surrounding continuous frames of the sampling points can be extracted, and the optical flow field frames with optical flow field information can be obtained based on the continuous image frames.

[0084] In step 203, feature extraction is performed based on the representative frame of each video segment to obtain a feature extraction result of each video segment, and a first video feature of the interaction account is determined based on the feature extraction result.

[0085] In some embodiments, the feature extraction result of each video segment can be obtained by performing feature extraction on the representative frame of each video segment in step 203. When the representative frame is an optical flow field frame of the corresponding video segment, time sequence feature extraction is performed on the representative frame of each video segment based on the time sequence dimension to obtain the time sequence feature of the video segment. When the representative frame is an image frame of the corresponding video segment, content feature extraction is performed on the representative frame of each video segment based on the content dimension to obtain the content feature of the video segment.

[0086] As an example, referring to Figure 6 , the feature extraction network includes a spatial flow convolution network and a time flow convolution network. When the representative frame is an optical flow field frame of the corresponding video segment, the time flow convolution network is used to perform time sequence feature extraction on the representative frame of each video segment based on the time sequence dimension to obtain the time sequence feature of the video segment. When the representative frame is an image frame of the corresponding video segment, the spatial flow convolution network is used to perform content feature extraction on the representative frame of each video segment based on the content dimension to obtain the content feature of the video segment.

[0087] In some embodiments, when the feature extraction result includes the time sequence feature of each video segment and the content feature, the determination of the first video feature of the interaction account based on the feature extraction result in step 203 can be implemented by the following technical solution: for each historical video, the following processing is performed: the plurality of time sequence features corresponding to the plurality of video segments of the historical video are spliced to obtain the time sequence feature of the historical video; the plurality of content features corresponding to the plurality of video segments of the historical video are spliced to obtain the content feature of the historical video; the time sequence feature of the historical video and the content feature of the historical video are spliced to obtain the video feature of the historical video; and the video features of the plurality of historical videos are spliced to obtain the first video feature of the interaction account.

[0088] As an example, the splicing processing is performed step by step, first the features corresponding to different video segments are spliced, that is, the plurality of time sequence features corresponding to the plurality of video segments of the historical video are spliced to obtain the time sequence feature of the historical video, and the plurality of content features corresponding to the plurality of video segments of the historical video are spliced to obtain the content feature of the historical video, for example, the time sequence feature of a certain video segment is a 64-dimensional feature, and after splicing the time sequence features of 2 video segments, a 128-dimensional time sequence feature is obtained, after splicing the features corresponding to different video segments, the two types of features corresponding to a certain historical video are spliced, that is, the time sequence feature of the historical video and the content feature of the historical video are spliced to obtain the video feature of the historical video, for example, the 128-dimensional time sequence feature and the 128-dimensional content feature are spliced to obtain a 256-dimensional video feature, and finally the video features corresponding to different historical videos are spliced to obtain the first video feature of the interaction account. Since the content feature and the time sequence feature are extracted separately during feature extraction, the spatial information and the time information of the video can be considered at the same time, so that a video feature with stronger comprehensive representation capability is learned, and an account feature (i.e. the first video feature of the interaction account) with stronger comprehensive representation capability is further learned.

[0089] In some embodiments, when the feature extraction result includes the content feature of each video segment, the determination of the first video feature of the interaction account based on the feature extraction result in step 203 can be implemented by the following technical solution: for each historical video, the following processing is performed: the plurality of content features corresponding to the plurality of video segments of the historical video are spliced to obtain the video feature of the historical video; and the video features of the plurality of historical videos are spliced to obtain the first video feature of the interaction account.

[0090] As an example, the splicing processing is performed in stages, first, the features corresponding to different video clips are spliced, that is, the plurality of content features corresponding to the plurality of video clips of the historical video are spliced to obtain the content features of the historical video, for example, the content features of a video clip are 64-dimensional features, and after splicing the content features of 2 video clips, 128-dimensional content features are obtained, after splicing the features corresponding to different video clips, the video features corresponding to different historical videos are spliced to obtain the first video features of the interactive account, since only the content features are extracted during feature extraction, the spatial information of the video can be learned, thereby learning the video features with stronger content representation capability, and further learning the account features (i.e. the first video features of the interactive account) with stronger content representation capability.

[0091] In some embodiments, the above splicing processing of the video features of the plurality of historical videos to obtain the first video features of the interactive account can be implemented by the following technical solution: splicing processing is performed on the video features of the plurality of historical videos to obtain a splicing processing result; full connection processing is performed on the splicing processing result to obtain the first video features of the interactive account.

[0092] As an example, the splicing processing of the video features of the plurality of historical videos to obtain the splicing processing result, for example, 2 video features of 128 dimensions are spliced to obtain a splicing processing result of 256 dimensions, the full connection processing is performed on the splicing processing result of 256 dimensions by the full connection layer to obtain the first video features of the interactive account, and the correlation between the features can be extracted through the nonlinear change of the full connection layer (also referred to as the dense layer).

[0093] In step 204, the first video features of the interactive account are subjected to feature enhancement processing based on the attention mechanism to obtain the second video features of the interactive account.

[0094] In some embodiments, the feature enhancement processing based on the attention mechanism is implemented by the attention network; the feature enhancement processing based on the attention mechanism of the first video features of the interactive account in step 204 to obtain the second video features of the interactive account can be implemented by the following technical solution: the input of the nth attention network in the N cascaded attention networks is subjected to attention processing by the nth attention network to obtain the nth attention result; the nth attention result is transmitted to the (n+1)th attention network for further attention processing; wherein N is an integer greater than or equal to 2; n is an integer starting from 1 and increasing, and the value range of n satisfies 1≤n≤N-1; when n is 1, the input of the nth attention network is the first video feature, and when n is 2≤n≤N-1, the input of the nth attention network is the attention result of the (n-1)th attention network.

[0095] As an example, the first video feature is subjected to attention processing by the first attention network to obtain a first attention result; the first attention result output by the first attention network is subjected to attention processing by the second attention network to obtain a second attention result; the second attention result output by the second attention network is subjected to attention processing by the third attention network to obtain a third attention result; and so on, the nth attention result output by the nth attention network is subjected to attention processing by the (n+1)th attention network to obtain an (n+1)th attention result, and the result output by the last attention network is the second video feature.

[0096] In some embodiments, the input of the nth attention network is subjected to attention processing to obtain the nth attention result, which can be realized by the following technical solution: the first video feature of the interactive account is subjected to channel-based average pooling processing by the nth attention network to obtain global features corresponding to each channel of the first video feature of the interactive account; the global features corresponding to each channel of the first video feature of the interactive account are subjected to full connection processing by the nth attention network to obtain attention values corresponding to each channel of the first video feature of the interactive account; and the attention values corresponding to each channel of the first video feature of the interactive account are subjected to point multiplication processing with the feature values of each channel of the first video feature of the interactive account to update the first video feature of the interactive account based on the point multiplication processing result.

[0097] As an example, the first video feature has learned the content features of the interactive account, and in order to further optimize the expression effect of the first video feature on the content features of the interactive account, the first video feature needs to be subjected to feature enhancement processing, the first video feature of the interactive account is subjected to channel-based average pooling processing by the squeezing layer of the attention network to obtain global content features corresponding to each channel of the first video feature of the interactive account, the global content features corresponding to each channel of the first video feature of the interactive account are subjected to full connection processing by the attention processing layer of the attention network to obtain attention values corresponding to each channel of the first video feature of the interactive account, the attention values corresponding to each channel of the first video feature of the interactive account are subjected to point multiplication processing with the feature values of each channel of the first video feature of the interactive account to update the first video feature of the interactive account based on the point multiplication processing result, and the updated result output by the last attention network is the second video feature. By subjecting the first video feature to attention processing, the differences between different videos can be effectively learned, and the noise caused by the historical videos of the same interactive account with large differences is weakened.

[0098] In some embodiments, the second video feature of the interactive account is obtained by performing attention-based feature enhancement processing on the first video feature of the interactive account. This can be achieved through the following technical solution: performing channel-based average pooling processing on the first video feature of the interactive account to obtain the global feature corresponding to each channel of the first video feature of the interactive account; performing fully connected processing on the global feature corresponding to each channel of the first video feature of the interactive account to obtain the attention value corresponding to each channel of the first video feature of the interactive account; performing a dot product processing on the attention value corresponding to each channel of the first video feature of the interactive account and the feature value of each channel in the first video feature of the interactive account; updating the first video feature of the interactive account based on the dot product processing result to obtain the second video feature of the interactive account.

[0099] As an example, when performing attention processing on the first video features, it can be done using only one attention network. That is, without using an iterative approach, attention processing on the first video features can be performed directly through one attention network to obtain the second video features. The implementation process can refer to the above embodiment.

[0100] In some embodiments, after obtaining the second video features of an interactive account, the second video features can be extended. Since the second video features only use the content features of the historical videos published by the interactive account, each interactive account has a group of users who like it. For example, most of the users who follow a certain interactive account are women who like cosmetics. By introducing the profile features of this type of user, the generalization of the second video features as an account feature can be enhanced. After obtaining the second video features of the interactive account, the associated user features of the interactive account are obtained, the associated user features and the second video features are fused, and the second video features of the interactive account are updated based on the fusion processing result.

[0101] In step 205, the account to be recommended is determined based on the second video features of the interactive account, and the recommendation operation of the corresponding target user account is performed based on the account to be recommended.

[0102] In some embodiments, see Figure 4C , Figure 4C This is a flowchart illustrating the AI-based recommendation method provided in this application embodiment. In step 205, the account to be recommended is determined based on the second video features of the interactive account. This can be achieved by executing steps 2051-2054 for each candidate account.

[0103] In step 2051, the historical videos posted by the candidate account are obtained.

[0104] In step 2052, feature extraction processing is performed on the historical videos to obtain the first video features of the candidate accounts.

[0105] In step 2053, the first video feature of the candidate account is subjected to attention mechanism-based feature enhancement processing to obtain the second video feature of the candidate account.

[0106] In step 2054, the video similarity between the second video feature of the interaction account and the second video feature of each candidate account is determined, and the candidate account with a video similarity greater than a video similarity threshold is taken as a to-be-recommended account.

[0107] As an example, steps 2051-2054 are performed for the candidate account, wherein the embodiments of steps 2051-2053 can refer to the embodiments of steps 201-204, wherein step 2052 can be implemented by the following technical solution: performing segmentation processing on a plurality of historical videos (historical videos published by the candidate account) to obtain a plurality of video segments of each historical video, performing frame extraction processing on the plurality of video segments of each historical video to obtain a representative frame of each video segment, performing feature extraction processing based on the representative frame of each video segment to obtain a feature extraction result of each video segment, and determining the first video feature of the interaction account based on the feature extraction result.

[0108] As an example, the candidate account includes a cold-start account and a non-cold-start account, the cold-start account belongs to an account without posterior data or with only a small amount of posterior data, and can be defined by account creation time or the number of interaction users, for example, the account creation time is not more than a time threshold (the account creation time is not more than 7 days), for example, the number of interaction users of the account is not more than a user number threshold (the number of interaction users is not more than 10), and the non-cold-start account refers to an account with a creation time exceeding the time threshold or an account with a number of interaction users exceeding the user number threshold.

[0109] In some embodiments, the similarity between each interaction account and the plurality of candidate accounts is sorted in descending order, and at least one candidate account in the front of the descending order sorting result is selected as a to-be-recommended account, and a matching pair of the interaction account and the candidate account is constructed, the matching pair is constructed by any one of the plurality of interaction accounts and any one of the plurality of candidate accounts, the matching pairs are globally sorted in descending order according to the similarity between the interaction account and the candidate account in the matching pair, and the candidate account of the first matching pair in the descending order sorting result is selected as the to-be-recommended account matched with the interaction account, and the following processing is performed for each interaction account: the similarity between the interaction account and the plurality of candidate accounts is sorted in descending order, and at least one candidate account in the front of the descending order sorting result of each interaction account is selected as the to-be-recommended account corresponding to the interaction account.

[0110] As an example, when there are multiple interactive accounts, all similarities can be sorted in descending order when sorted in descending order, for example, the similarities between interactive account A and the two candidate accounts are 0.5 and 0.6 respectively, and the similarities between interactive account B and the two candidate accounts are 0.7 and 0.3 respectively. 0.5, 0.6, 0.3, 0.7 are sorted in descending order, and the two candidate accounts at the top of the ranking are obtained. The two candidate accounts at the top of the ranking are used as the recommended accounts. When sorted in descending order, different interactive accounts can be sorted in descending order, for example, the similarities between interactive account A and the two candidate accounts are 0.5 and 0.6 respectively, and the top 1 candidate account is obtained. The similarity between interactive account B and the two candidate accounts is 0.7 and 0.3 respectively, and the top 1 candidate account is obtained. Then the two candidate accounts are used as the recommended accounts.

[0111] In some embodiments, the recommendation operation of the corresponding target user account based on the recommended account in step 205 can be implemented by the following technical solution: performing at least one of the following processing: sending information for recommending the recommended account to the target user account; sending information for recommending the multiple historical videos published by the recommended account to the target user account.

[0112] As an example, the user can be recommended to the target user account to recommend the video published by the recommended account, or the recommended account can also be directly recommended for attention.

[0113] In the following, an exemplary application of the embodiments of the present application in an actual application scenario will be described.

[0114] With the development of mobile Internet, a large number of users shoot short videos and upload them to short video platforms. Every day, hundreds of millions of videos are generated. When users actually use and watch videos, they often focus on the videos published by specific type accounts, for example, users who like to watch variety shows will focus on variety show accounts as interactive accounts, and users who like game videos will focus on game accounts as interactive accounts. Through the recommendation method based on artificial intelligence provided by the embodiments of the present application, the account characteristics (second video characteristics) capable of expressing the characteristic information of each account are accurately learned. The account includes an interactive account and a candidate account. The accuracy of the account characteristics is crucial for accurate account recommendation and accurate recommendation of videos published by the account. See Figures 7A-7B , Figures 7A-7Bis a product interface diagram of the recommendation method based on artificial intelligence provided by the embodiment of the application. In response to a login operation of a user A of a video recommendation client, a to-be-recommended account 702A and 702B for the user A are displayed in a man-machine interaction interface 701A or 701B, and a historical video 703A published by the to-be-recommended account 702A and a historical video 703B published by the to-be-recommended account 702B are displayed. The to-be-recommended accounts 702A and 702B are determined by the recommendation method based on artificial intelligence provided by the embodiment of the application.

[0115] In some embodiments, referring to Figure 5 , Figure 5 is a model framework diagram of the recommendation method based on artificial intelligence provided by the embodiment of the application. For an interactive account, first, K (K is an integer greater than or equal to 2, for example, K = 10) videos recently published by the interactive account are pulled, then each video is divided into N (N is an integer greater than or equal to 2, for example, N = 10) segments in a segmented frame extraction manner, an image frame is randomly extracted from each video segment, thereby obtaining N image frames of a certain video, then a spatial stream convolution network (for example, an Inception_resnet_v2 network) in a backbone network is used to extract features of the image frames, the obtained features are sequentially spliced, the spliced results are mapped to a first video feature through a full connection layer, and a second video feature is obtained through an attention network to enhance the first video feature, and the second video feature is used as a final account feature.

[0116] In some embodiments, the backbone network can also be an InceptionV4 network, a ResNeXt101 network, an Xception network, and the like. Since the types of video content published by the same account can be different, for example, a game anchor account occasionally publishes a video of the game anchor himself in outdoor sports. In order to reduce the noise caused by the difference of different video contents under the same account, the spliced results are not directly used as the account feature, and the attention network is used to weight the obtained features based on different channels, thereby reducing the noise caused by the difference of different video contents under the same account.

[0117] During training, the entire network is trained end-to-end. For training data and training methods, the complete interactive account sequence of the user's attention interactive account is first pulled. Since the user's interest point can change over time, the user's attention interactive account sequence is cut using a set number of days to obtain a partial interactive account sequence for representing the user's interest. For these partial interactive account sequences, the correlation between the two accounts is calculated according to the correlation formula. Assuming that account A and account B have fan numbers fans_A and fans_B, respectively, and fans_iter represents the user's attention to both accounts, the correlation between the two can be calculated according to the following formula (2):

[0118] (2) ;

[0119] where R is the correlation between account sample A and account sample B, fans_A is the number of user accounts that follow account sample A, fans_B is the number of user accounts that follow account sample B, and fans_iter represents the number of user accounts that follow both account sample A and account sample B.

[0120] After obtaining the correlation between the accounts, the positive and negative examples of the accounts are obtained through a correlation threshold (for example, 0.1). Assuming that the correlation value between account A and account B is 0.2 (greater than 0.1), and the correlation value between account A and account C is 0.08 (less than 0.1), a triple (A, B, C) is constructed, where account A is the reference sample, B is the positive sample, and C is the negative sample. The supervised learning model is obtained through metric learning, and the account features are obtained. If account A does not have a correlation less than 0.1 with other accounts, a random account is selected as the negative sample account. The account features obtained through such a learning method are that the training goal during training is to make the spatial distance between the accounts followed by the user as close as possible (for example, the distance between account A and account B is as close as possible), and the spatial distance between different, unrelated accounts is as far as possible (for example, the distance between account A and account C is as far as possible).

[0121] In some embodiments, since the account features only use the feature information of the video content published by the account, each account has a certain direction or level of favorite user group, such as a user who mostly likes cosmetics. The feature information of this type of user can be introduced to enhance the generalization of the account features learned by the model.

[0122] The frames of the videos published by the account are randomly extracted as the input of the account learning model to obtain the second video features as the account features. In order to avoid the large difference of individual videos published by the account, the attention-based method is used to avoid the influence of noise and obtain more accurate account features. The evaluation set is constructed from the actual user behavior sequence of following the account. Five million users who follow the interactive account are used as the training set, and another five million users who follow the interactive account are used as the test set. In the actual test, the first interactive account followed by the user is used as the retrieval basis to obtain N candidate accounts with high similarity to the interactive account. The hit rate of the N candidate accounts and the actual interactive account followed by the user is calculated. In terms of the effect of the account learning model, the unimproved account learning model currently used online and the account learning model provided in the embodiment are tested. Compared with the unimproved account learning model, the hit rate is increased from 9.21% to 15.66%, about 6 percentage points.

[0123] The following continues to illustrate an exemplary structure of the implementation of the recommendation device 255 based on artificial intelligence provided in the embodiment as a software module. In some embodiments, as shown in FIG. 25, the software modules stored in the recommendation device 255 based on artificial intelligence in the memory 250 can include a video module 2551 configured to determine a plurality of historical videos published by an interactive account, wherein the interactive account has an interaction relationship with a target user account; a frame extraction module 2552 configured to perform segmentation processing on the plurality of historical videos to obtain a plurality of video segments of each historical video, and perform frame extraction processing on the plurality of video segments of each historical video to obtain a representative frame of each video segment; an extraction module 2553 configured to perform feature extraction processing based on the representative frame of each video segment to obtain a feature extraction result of each video segment, and determine a first video feature of the interactive account based on the feature extraction result; an attention module 2554 configured to perform feature enhancement processing based on an attention mechanism on the first video feature of the interactive account to obtain a second video feature of the interactive account; and a recommendation module 2555 configured to determine a to-be-recommended account based on the second video feature of the interactive account, and perform a recommendation operation on the corresponding target user account based on the to-be-recommended account. Figure 3 In some embodiments, the extraction module 2553 is further configured to: when the representative frame is an optical flow field frame corresponding to the video segment, perform optical flow feature extraction processing based on a time sequence dimension on the representative frame of each video segment to obtain a time sequence feature of the video segment; and when the representative frame is an image frame corresponding to the video segment, perform content feature extraction processing based on a content dimension on the representative frame of each video segment to obtain a content feature of the video segment.

[0124]

[0125] ​In some embodiments, when the feature extraction result includes the time sequence feature and the content feature of each video segment, the extraction module 2553 is further configured to: for each historical video, perform the following processing: performing splicing processing on the plurality of time sequence features corresponding to the plurality of video segments of the historical video to obtain the time sequence feature of the historical video; performing splicing processing on the plurality of content features corresponding to the plurality of video segments of the historical video to obtain the content feature of the historical video; performing splicing processing on the time sequence feature of the historical video and the content feature of the historical video to obtain the video feature of the historical video; and performing splicing processing on the video features of the plurality of historical videos to obtain the first video feature of the interaction account.

[0126] In some embodiments, when the feature extraction result includes the content feature of each video segment, the extraction module 2553 is further configured to: for each historical video, perform the following processing: performing splicing processing on the plurality of content features corresponding to the plurality of video segments of the historical video to obtain the video feature of the historical video; and performing splicing processing on the video features of the plurality of historical videos to obtain the first video feature of the interaction account.

[0127] In some embodiments, the extraction module 2553 is further configured to: perform splicing processing on the video features of the plurality of historical videos to obtain a splicing processing result; and perform full connection processing on the splicing processing result to obtain the first video feature of the interaction account.

[0128] In some embodiments, the feature enhancement processing based on the attention mechanism is implemented through an attention network; and the attention module 2554 is further configured to: through the nth attention network in the N cascaded attention networks, perform attention processing on the input of the nth attention network to obtain an nth attention result; and transmit the nth attention result to the (n+1)th attention network for further attention processing; wherein N is an integer greater than or equal to 2; n is an integer starting from 1 and increasing, and the value range of n satisfies 1≤n≤N-1; when n is 1, the input of the nth attention network is the first video feature; and when n is 2≤n≤N-1, the input of the nth attention network is the attention result of the (n-1)th attention network.

[0129] In some embodiments, the attention module 2554 is further configured to: perform channel-based average pooling processing on the first video features of the interaction account by the nth attention network to obtain global features corresponding to each channel of the first video features of the interaction account; perform full connection processing on the global features corresponding to each channel of the first video features of the interaction account by the nth attention network to obtain attention values corresponding to each channel of the first video features of the interaction account; and perform dot multiplication processing on the attention values corresponding to each channel of the first video features of the interaction account and the feature values of each channel in the first video features of the interaction account, and update the first video features of the interaction account based on a result of the dot multiplication processing.

[0130] In some embodiments, the recommendation module 2555 is further configured to: for each candidate account, perform the following processing: obtain historical videos published by the candidate account; perform feature extraction processing on the historical videos to obtain first video features of the candidate account; perform attention mechanism-based feature enhancement processing on the first video features of the candidate account to obtain second video features of the candidate account; determine video similarity between the second video features of the interaction account and the second video features of each candidate account, and take the candidate account with a video similarity greater than a video similarity threshold as a recommended account.

[0131] In some embodiments, the recommendation module 2555 is further configured to: perform at least one of the following: send information for recommending the recommended account to the target user account; and send information for recommending a plurality of historical videos published by the recommended account to the target user account.

[0132] In some embodiments, the second video features of the interaction account are determined by an account learning model; and the apparatus further includes a training module 2556 configured to: before performing the segmentation processing on the plurality of historical videos to obtain a plurality of video segments of each historical video, train the account learning model by: obtaining a plurality of account samples, and constructing a plurality of triple samples based on a number of associated users of the plurality of account samples; and substituting second video feature samples corresponding to each account sample in the triple samples into a triple loss function to determine parameters of the account learning model when the triple loss function reaches a minimum value.

[0133] In some embodiments, the training module 2556 is further configured to determine a relevance between any two account samples in the plurality of account samples based on the number of associated users of the plurality of account samples; and obtain a plurality of candidate triple samples from the plurality of account samples, wherein each candidate triple sample comprises three account samples, and any two candidate triple samples have at least one different account sample; and select a triple sample meeting a training condition from the plurality of candidate triple samples, wherein the training condition comprises that a relevance between two account samples in the triple sample is greater than a relevance threshold, and a relevance between the two account samples is less than the relevance threshold.

[0134] In some embodiments, the training module 2556 is further configured to, for any two account samples, perform the following processing: obtain the number of associated users of each account sample in the any two account samples; obtain a same number of associated users between the any two account samples; and determine a relevance positively correlated with the same number of associated users and negatively correlated with the number of associated users of each account sample.

[0135] The embodiment of the present application provides a computer program product or a computer program, which comprises computer instructions stored in a computer readable storage medium. A processor of a computer device reads the computer instructions from the computer readable storage medium, and the processor executes the computer instructions, so that the computer device executes the recommendation method based on artificial intelligence provided in the embodiment of the present application.

[0136] The embodiment of the present application provides a computer readable storage medium storing executable instructions, wherein the executable instructions are stored in the computer readable storage medium. When the executable instructions are executed by a processor, the processor will execute the recommendation method based on artificial intelligence provided in the embodiment of the present application, for example, as shown in the recommendation method based on artificial intelligence. Figures 4A-4C The embodiment of the present application provides a computer readable storage medium storing executable instructions, wherein the executable instructions are stored in the computer readable storage medium. When the executable instructions are executed by a processor, the processor will execute the recommendation method based on artificial intelligence provided in the embodiment of the present application, for example, as shown in the recommendation method based on artificial intelligence.

[0137] In some embodiments, the computer readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash memory, magnetic surface memory, optical disc, or CD-ROM; or various devices comprising one or any combination of the above memories.

[0138] In some embodiments, the executable instructions can be in the form of a program, software, software module, script or code, written in any form of programming language (including a compiled or interpreted language, or a declarative or procedural language), and can be deployed in any form, including being deployed as a standalone program or as a module, component, subroutine or other unit suitable for use in a computing environment.

[0139] By way of example, executable instructions can correspond to a file in a file system, can be stored in a portion of a file that holds other programs or data, e.g., one or more scripts stored in a markup language document, in a single file dedicated to the program in question, or in multiple coordinated files, e.g., files that store one or more modules, sub programs, or portions of code.

[0140] By way of example, executable instructions can be deployed to be executed on one computer, or on multiple computers of a system of computers in one location, or on multiple computers distributed among multiple locations and interconnected by a communication network.

[0141] To sum up, by performing the feature extraction processing based on the segmented frame extraction on the historical videos, the features of each historical video are accurately depicted while effectively avoiding the repeated frames, and then the features of the multiple historical videos are converted into the features of the interactive account by the attention network, so as to learn the difference between the historical videos, more accurately depict the features of the account, and then improve the account recommendation accuracy based on the similarity of the account.

[0142] The above merely describes the embodiments of the present application, but is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, and improvement within the spirit and scope of the present application shall be included in the protection scope of the present application.

Claims

1. An artificial intelligence-based recommendation method, characterized by, The method comprises the following steps: determining a plurality of historical videos published by an interactive account, wherein the interactive account has an interaction relationship with a target user account; segmenting the plurality of historical videos to obtain a plurality of video segments of each historical video, and frame extraction processing is performed on the plurality of video segments of each historical video to obtain a representative frame of each video segment; performing feature extraction processing based on the representative frame of each video segment to obtain a feature extraction result of each video segment, and determining a first video feature of the interactive account based on the feature extraction result; wherein when the feature extraction result comprises a time sequence feature and a content feature of each video segment, the determination of the first video feature of the interactive account based on the feature extraction result comprises the following processing for each historical video: performing splicing processing on a plurality of time sequence features corresponding to a plurality of video segments of the historical video to obtain a time sequence feature of the historical video; performing splicing processing on a plurality of content features corresponding to the plurality of video segments of the historical video to obtain a content feature of the historical video; and performing splicing processing on the time sequence feature of the historical video and the content feature of the historical video to obtain a video feature of the historical video; and performing splicing processing on the video features of the plurality of historical videos to obtain the first video feature of the interactive account; performing feature enhancement processing based on an attention mechanism on the first video feature of the interactive account to obtain a second video feature of the interactive account; determining a to-be-recommended account based on the second video feature of the interactive account, and performing a recommendation operation corresponding to the target user account based on the to-be-recommended account.

2. The method of claim 1, wherein, The feature extraction processing based on the representative frame of each video segment to obtain a feature extraction result of each video segment comprises: when the representative frame is an optical flow field frame corresponding to the video segment, performing optical flow feature extraction processing based on a time sequence dimension on the representative frame of each video segment to obtain a time sequence feature of the video segment; when the representative frame is an image frame corresponding to the video segment, performing content feature extraction processing based on a content dimension on the representative frame of each video segment to obtain a content feature of the video segment.

3. The method of claim 1, wherein, When the feature extraction result comprises a content feature of each video segment, the determination of the first video feature of the interactive account based on the feature extraction result comprises: performing the following processing for each historical video: performing splicing processing on a plurality of content features corresponding to a plurality of video segments of the historical video to obtain a video feature of the historical video; and performing splicing processing on the video features of the plurality of historical videos to obtain the first video feature of the interactive account. The splicing processing on the video features of the plurality of historical videos to obtain the first video feature of the interactive account comprises:

4. The method according to claim 1 or 3, characterized in that, performing splicing processing on the video features of the plurality of historical videos to obtain a splicing processing result; performing full connection processing on the splicing processing result to obtain the first video feature of the interactive account. ​ 5. The method of claim 1, wherein, The attention mechanism-based feature enhancement processing is implemented through an attention network. The attention mechanism-based feature enhancement processing on the first video feature of the interactive account comprises: The input of the nth attention network is processed through the nth attention network in the N cascaded attention networks to obtain an nth attention result; The nth attention result is transmitted to an (n+1)th attention network for further attention processing; N is an integer greater than or equal to 2; n is an integer starting from 1 and increasing, and the value range of n satisfies 1≤n≤N-1; when n is 1, the input of the nth attention network is the first video feature; when n is 2≤n≤N-1, the input of the nth attention network is the attention result of the (n-1)th attention network, and the output of the nth attention network is the second video feature.

6. The method of claim 5, wherein, The input of the nth attention network is processed through the nth attention network to obtain an nth attention result, comprising: The first video feature of the interactive account is processed through the nth attention network to obtain a global feature corresponding to each channel of the first video feature of the interactive account; The global feature corresponding to each channel of the first video feature of the interactive account is processed through the nth attention network to obtain an attention value corresponding to each channel of the first video feature of the interactive account; The attention value corresponding to each channel of the first video feature of the interactive account is multiplied with the feature value of each channel in the first video feature of the interactive account, and the first video feature of the interactive account is updated based on the multiplication result.

7. The method of claim 1, wherein, The second video feature of the interactive account is determined based on the second video feature of the interactive account, comprising: The following processing is performed for each candidate account: Obtaining a historical video published by the candidate account; The first video feature of the candidate account is obtained by performing feature extraction processing on the historical video; The first video feature of the candidate account is processed through attention mechanism-based feature enhancement processing to obtain a second video feature of the candidate account; The video similarity between the second video feature of the interactive account and the second video feature of each candidate account is determined, and the candidate account with a video similarity greater than a video similarity threshold is taken as the to-be-recommended account.

8. The method of claim 1, wherein, The recommendation operation corresponding to the target user account is performed based on the to-be-recommended account, comprising: At least one of the following processing is performed: Sending information for recommending the to-be-recommended account to the target user account; Sending information for recommending a plurality of historical videos published by the to-be-recommended account to the target user account.

9. The method of claim 1, wherein, The second video feature of the interactive account is determined through an account learning model; Before segmenting the plurality of historical videos to obtain a plurality of video segments of each historical video, the method further comprises: The account learning model is trained in the following manner: Obtaining a plurality of account samples, and constructing a plurality of triplet samples based on the number of associated users of the plurality of account samples; Substitute the second video feature sample corresponding to each account sample in the triplet sample into a triplet loss function to determine the parameters of the account learning model when the triplet loss function reaches a minimum value.

10. The method of claim 9, wherein, The constructing of the plurality of triplet samples based on the number of associated users of the plurality of account samples comprises: Determining the relevance between any two account samples in the plurality of account samples based on the number of associated users of the plurality of account samples; Obtaining a plurality of candidate triplet samples from the plurality of account samples; Each of the candidate triplet samples includes three account samples, and there is at least one different account sample between any two candidate triplet samples; Selecting the triplet samples that meet the training conditions from the plurality of candidate triplet samples; The training conditions include that the relevance between two account samples in the triplet sample is greater than a relevance threshold, and the relevance between two account samples is less than a relevance threshold.

11. The method of claim 10, wherein, The determining of the relevance between any two account samples in the plurality of account samples based on the number of associated users of the plurality of account samples comprises: For the any two account samples, the following processing is performed: Obtaining the number of associated users of each account sample in the any two account samples; Obtaining the number of same associated users between the any two account samples; Determining the relevance that is positively correlated with the number of same associated users and is negatively correlated with the number of associated users of each account sample. 12.A recommendation apparatus based on artificial intelligence, characterized by, Comprise: A video module for determining a plurality of historical videos published by an interactive account, wherein the interactive account has an interactive relationship with a target user account; A frame extraction module for segmenting the plurality of historical videos to obtain a plurality of video segments of each historical video, and extracting frames from the plurality of video segments of each historical video to obtain a representative frame of each video segment; An extraction module for performing feature extraction processing based on the representative frame of each video segment to obtain a feature extraction result of each video segment, and determining a first video feature of the interactive account based on the feature extraction result; wherein when the feature extraction result includes a timing feature and a content feature of each video segment, the determining of the first video feature of the interactive account based on the feature extraction result comprises: for each historical video, performing the following processing: splicing a plurality of timing features corresponding to a plurality of video segments of the historical video to obtain a timing feature of the historical video; splicing a plurality of content features corresponding to the plurality of video segments of the historical video to obtain a content feature of the historical video; splicing the timing feature of the historical video and the content feature of the historical video to obtain a video feature of the historical video; and splicing the video features of the plurality of historical videos to obtain the first video feature of the interactive account; The attention module is configured to perform attention mechanism-based feature enhancement processing on the first video feature of the interaction account to obtain a second video feature of the interaction account. The recommendation module is configured to determine a to-be-recommended account based on the second video feature of the interaction account, and perform a recommendation operation corresponding to the target user account based on the to-be-recommended account.

13. The apparatus of claim 12, wherein, The extraction module is further configured to: perform time sequence dimension-based optical flow feature extraction processing on the representative frame of each video segment to obtain a time sequence feature of the video segment when the representative frame is an optical flow field frame corresponding to the video segment; and perform content dimension-based content feature extraction processing on the representative frame of each video segment to obtain a content feature of the video segment when the representative frame is an image frame corresponding to the video segment.

14. The apparatus of claim 12, wherein, The extraction module is further configured to: perform the following processing on each historical video when the feature extraction result includes the content feature of each video segment: perform splicing processing on a plurality of content features corresponding to a plurality of video segments of the historical video to obtain a video feature of the historical video; and perform splicing processing on the video features of the plurality of historical videos to obtain the first video feature of the interaction account.

15. The apparatus of claim 12 or 14, wherein, The extraction module is further configured to: perform splicing processing on the video features of the plurality of historical videos to obtain a splicing processing result; and perform full connection processing on the splicing processing result to obtain the first video feature of the interaction account.

16. The apparatus of claim 12, wherein, The attention mechanism-based feature enhancement processing is implemented through an attention network; and the attention module is further configured to: perform attention processing on an input of an nth attention network in N cascaded attention networks through the nth attention network to obtain an nth attention result; transmit the nth attention result to an (n+1)th attention network for further attention processing; and N is an integer greater than or equal to 2; n is an integer starting from 1 and increasing by 1, and the value range of n satisfies 1≤n≤N-1; when n is 1, the input of the nth attention network is the first video feature; when n is 2≤n≤N-1, the input of the nth attention network is the attention result of an (n-1)th attention network, and the output of the nth attention network is the second video feature.

17. The apparatus of claim 16, wherein, The attention module is further configured to: perform channel-based average pooling processing on the first video feature of the interaction account through the nth attention network to obtain a global feature corresponding to each channel of the first video feature of the interaction account; perform full connection processing on the global feature corresponding to each channel of the first video feature of the interaction account through the nth attention network to obtain an attention value corresponding to each channel of the first video feature of the interaction account; and perform dot multiplication processing on the attention value corresponding to each channel of the first video feature of the interaction account and the feature value of each channel in the first video feature of the interaction account, and update the first video feature of the interaction account based on a dot multiplication processing result.

18. An electronic device, comprising: The memory is configured to store executable instructions. ​ A processor configured to implement the artificial intelligence based recommendation method of any one of claims 1 to 11 when executing executable instructions stored in the memory.

19. A computer-readable storage medium, characterized in that, A memory storing executable instructions configured to implement the artificial intelligence based recommendation method of any one of claims 1 to 11 when executed by a processor.

20. A computer program product comprising computer instructions, characterized in that, The computer instructions, when executed by a processor, implement the artificial intelligence based recommendation method of any one of claims 1 to 11.

Images